Custom Mongo queries inexplicably return zero results after upgrading to 3.0.8.RELEASE or later (spring-boot dep mgmt: 2.3.10.RELEASE or later)

[oliverlockwood]

I've created a repo to demonstrate the issue, at https://github.com/oliverlockwood/mongodb-issue-demonstration, based on following the guidelines at https://spring.io/guides/gs/accessing-data-mongodb/.

That repo has a clear README but I'll highlight the main points here as well.

• In terms of environment, I have a local Docker container running MongoDB version 4.2.14, mapping ports 27017 and 28017.
• The Customer model class contains a map field, like: public Map<String, DataValue> additionalData;
• There are custom read/write converters configured; we persist the map as an array of (name, value) tuples, viz:
  
• We have a custom repository implementation which allows dynamic queries to be specified (in our real use-case, via REST query payloads; in the example repo, via hard-coding in the run() method)
• The custom Mongo query we generate in the application (regardless of spring-data-mongodb version) is logged out as:
  Query: { "$and" : [{ "additionalData.name" : "a"}, { "additionalData.value" : "b"}]}, Fields: {}, Sort: {}
• With spring-boot version 2.3.9.RELEASE (therefore spring-data-mongodb version 3.0.7.RELEASE) an entry is found as expected by this query
• With spring-boot version 2.3.10.RELEASE or higher (therefore spring-data-mongodb version 3.0.8.RELEASE or higher) no results are found
• Directly querying Mongo via the console, i.e.
  db.getCollection('customer').find({ "$and" : [{ "additionalData.name" : "a"}, { "additionalData.value" : "b"}]})
  finds an entry as expected

I therefore conclude this is a regression in spring-data-mongodb, but I can't for the life of me figure out why it stops returning results.

I hope this in combination with the repo is enough context for you to investigate; please advise if you need me to provide any more information.

Many thanks!

[christophstrobl]

thanks for reporting and the sample - we'll have a look.

[mp911de]

Looks like it is related to #3659.

[oliverlockwood]

@mp911de that's interesting, and slightly surprises me, given the different affected versions (the ticket you refer is noted as a regression in versions after 3.1.9, while this ticket is a regression in versions after 3.0.7).

Still, I look forward to hearing more and will track both issues.

[mp911de]

We had a change where we updated mapping of property paths pointing into collection and map-like structures that maps errornously additionalData.name into additionalData.name.name.

There's quite a bit of a mismatch in the data model. additionalData is defined as Map<String, DataValue> while it actually is List<DataValue>. Changing the data type to additionalData fixes the problem.

We map queries against the domain model so trying to compare additionalData.name=foo against Map<String, DataValue>describes an attempt to query whether the map (subdocument)additionalDatacontains a fieldnamewith the valuefoo while DataValue isn't compatible with that representation.

In any case, trying to query into a map should render the proper field name into the query.

[oliverlockwood]

Thanks for your comment on 18th June, @mp911de. Any news on when this might be fixed? I'm a bit worried about getting stuck on an old version of spring-data-mongodb while we continue to keep updating our platform (next step is to move to Spring Boot 2.5.x).,