Custom converter not used in aggregation pipeline after $replaceRoot stage

[banhidizoli]

Hi,

We observed this issue after migrating from 3.4.2 to 4.0.1
We are using a custom converter for mapping ZonedDateTime to Date and vice versa. With the old version our aggregation worked fine, after upgrading we received the following exception:

org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for CodecCacheKey{clazz=class java.time.ZonedDateTime, types=null}.

Our aggregation is using a $match stage in the pipeline right after $replaceRoot; in this $match, we compare to a ZonedDateTime vale. This should work properly, as we registered a custom converter for this type.

After some debug, we think that this was introduced by Avoid multiple mapping iterations.

Second mapping iteration is needed, because Aggregation#toPipeline method internally looses the initial RelaxedTypeBasedAggregationOperationContext after Aggregation#replaceRoot (after this stage the NoOpAggregationOperationContext is used, which doesn't seem to pick up the custom converters).

[mp911de]

Would you happen to have an example handy that reproduces your use case?

[banhidizoli]

My domain class looks like

@Document(collection = "entity")
public class Entity {

    private String entityId;
    private ZonedDateTime createdDate;
    private int version;
    
    // Getters and setters
}

And the failing aggregation is

var aggregation = Aggregation.newAggregation(
    Aggregation.sort(Direction.DESC, "version"),
    Aggregation.group("entityId")
        .first(Aggregation.ROOT).as("value"),
    Aggregation.replaceRoot("value"),
    Aggregation.match(Criteria.where("createdDate").lt(ZonedDateTime.now())) // here is the problem
);

return mongoTemplate
    .aggregate(aggregation, Entity.class, MutableObject.class)
    .getMappedResults()
    .stream()
    .map(MutableObject<String>::getValue)
    .collect(Collectors.toSet());

And my custom convertor is

public static class ZonedDateTimeToDateConverter implements Converter<ZonedDateTime, Date> {
    public static final ZonedDateTimeToDateConverter INSTANCE = new ZonedDateTimeToDateConverter();

    private ZonedDateTimeToDateConverter() {
    }

    @Override
    public Date convert(ZonedDateTime source) {
        return source == null ? null : Date.from(source.toInstant());
    }
}

The custom convertor is registered like

@Configuration
@EnableMongoRepositories("my.package")
public class DatabaseConfiguration extends MongoConfigurationSupport {

    @Override
    protected void configureConverters(MongoCustomConversions.MongoConverterConfigurationAdapter converterConfigurationAdapter) {
        var converters = new ArrayList<Converter<?, ?>>();
        converters.add(ZonedDateTimeToDateConverter.INSTANCE);
        converterConfigurationAdapter.registerConverters(converters);
    }

    @Override
    protected String getDatabaseName() {
        return "myDb";
    }
}

[banhidizoli]

We had to downgrade to version 4.0.0 because of this bug. Is there any estimate of when this will be fixed?

[vladpetrican]

Are there any updates regarding this issue ?

[csmager]

I've also come across this issue in as a breaking change in 3.4.7, which is the same change that causes this: #4240.

The first issue I had was because the mapping hasn't happened when this check occurs to see if the last pipeline stage is $out or $merge. I get this stack trace:

org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class XXXX.
        at org.bson.internal.CodecCache.lambda$getOrThrow$1(CodecCache.java:52)
        at java.base/java.util.Optional.orElseThrow(Optional.java:403)
        at org.bson.internal.CodecCache.getOrThrow(CodecCache.java:51)
        at org.bson.internal.OverridableUuidRepresentationCodecRegistry.get(OverridableUuidRepresentationCodecRegistry.java:72)
        at org.bson.internal.ChildCodecRegistry.get(ChildCodecRegistry.java:52)
        at org.bson.codecs.DocumentCodec.writeValue(DocumentCodec.java:209)
        at org.bson.codecs.DocumentCodec.encode(DocumentCodec.java:168)
        at org.bson.codecs.DocumentCodec.encode(DocumentCodec.java:44)
        at org.bson.internal.LazyCodec.encode(LazyCodec.java:38)
        at org.bson.codecs.EncoderContext.encodeWithChildContext(EncoderContext.java:91)
        at org.bson.codecs.DocumentCodec.writeValue(DocumentCodec.java:210)
        at org.bson.codecs.DocumentCodec.encode(DocumentCodec.java:168)
        at org.bson.codecs.DocumentCodec.encode(DocumentCodec.java:44)
        at org.bson.BsonDocumentWrapper.getUnwrapped(BsonDocumentWrapper.java:199)
        at org.bson.BsonDocumentWrapper.containsKey(BsonDocumentWrapper.java:124)
        at com.mongodb.client.internal.AggregateIterableImpl.getOutNamespace(AggregateIterableImpl.java:240)
        at com.mongodb.client.internal.AggregateIterableImpl.asReadOperation(AggregateIterableImpl.java:203)
        at com.mongodb.client.internal.MongoIterableImpl.execute(MongoIterableImpl.java:135)
        at com.mongodb.client.internal.MongoIterableImpl.iterator(MongoIterableImpl.java:92)
        at com.mongodb.client.internal.MongoIterableImpl.forEach(MongoIterableImpl.java:121)
        at com.mongodb.client.internal.MappingIterable.forEach(MappingIterable.java:59)
        at com.mongodb.client.internal.MappingIterable.into(MappingIterable.java:69)
        at org.springframework.data.mongodb.core.MongoTemplate.lambda$doAggregate$24(MongoTemplate.java:2268)
        at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:579)
        at org.springframework.data.mongodb.core.MongoTemplate.doAggregate(MongoTemplate.java:2225)
        at org.springframework.data.mongodb.core.MongoTemplate.doAggregate(MongoTemplate.java:2193)
        at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:2187)
        at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:2081)

I then tried ensuring this stage wasn't last in the pipeline. I then ran into the issue noted here as the previous stage was $replaceRoot.

[banhidizoli]

This problem is stopping us from upgrading to latest version. @mp911de is there any plan to fix this? Or we'll have to adapt to this limitation?

[pbeltechi]

This issue seems to be solved on 4.2.1 version. Just checked it today.