Futures not batching correctly in NH 5.2.x

[mattzink]

While attempting to upgrade from NH 5.1.3 to 5.2.5, we have a performance test that started failing relating to batching of .Future() queries. The following test passed as expected in NH 5.1.3, but is now failing in NH 5.2.5 with a statement count of 11(!). I've verified the NH statement statistic count is correct from observing commands issued in the Npgsql driver log.

long preRequestPrepareStatementCount = _sessionFactory.Statistics.PrepareStatementCount;

session.QueryOver<VolumeAttachment>()
    .Where(va => va.Id.IsIn(volumeAttachmentIds))
    .Left.JoinQueryOver(va => va.StatusConditions)
    .Future();

session.QueryOver<VolumeAttachment>()
    .Where(va => va.Id.IsIn(volumeAttachmentIds))
    .Left.JoinQueryOver<VolumeAttachmentPath>(va => va.VolumeAttachmentPaths)
    .Left.JoinQueryOver(vap => vap.StatusConditions)
    .Future();

session.QueryOver<VolumeAttachment>()
    .Where(va => va.Id.IsIn(volumeAttachmentIds))
    .Left.JoinQueryOver<VolumeAttachmentPath>(va => va.VolumeAttachmentPaths)
    .Left.JoinQueryOver(vap => vap.Path)
    .Left.JoinQueryOver(p => p.StatusConditions)
    .Future();

session.QueryOver<VolumeAttachment>()
    .Where(va => va.Id.IsIn(volumeAttachmentIds))
    .Left.JoinQueryOver<VolumeAttachmentPath>(va => va.VolumeAttachmentPaths)
    .Left.JoinQueryOver(vap => vap.Path)
    .Left.JoinQueryOver(p => p.TargetPorts)
    .Future()
    .GetEnumerable();

long prepareStatementDelta = _sessionFactory.Statistics.PrepareStatementCount - preRequestPrepareStatementCount;

Assert.Equals(1, prepareStatementDelta);

[bahusoid]

From the count of executed queries it looks as unrelated to Future queries. It seems to me as some mapping related issues - as non lazy associations initialization queries or existence checks for not-found="ignore" associations. So can you share actual executed queries?

[bahusoid]

Ok. I think I know what it's about. To fix #1293 Loader.InitializeEntitiesAndCollections call was moved to be executed right after each query in batch instead of post processing step (see QueryBatchItemBase.InitializeEntitiesAndCollections). But it seems this call leads to eager loading for all non lazy associations - and so unexpected queries are possible even if other queries in batch are loading those entities.
P.S. Unfortunately I have no time now to work on fix (seems will be busy for the whole summer)

[mattzink]

Thanks for taking a look @bahusoid! Given that this is causing NH 5.2 to issue 11 times the number of queries that it did in NH 5.1, it seems like the cure for #1293 was worse than the disease 😉 I do not know the NH code base very well, but if you could give any guidance I could try and investigate further.

[bahusoid]

it seems like the cure for #1293 was worse than the disease

Maybe. Though If I'm right this issue affects only queries involving lazy="false" entities/associations. Which is not that common usage scenario.

[mattzink]

In our case, I don't see us explicitly calling .Lazy(false) on any mapping. Can this be set implicitly somehow?

[bahusoid]

It's hard too say without looking on entity classes and mappings. Did you check generated queries? All unexpected queries should be related to non lazy entity/association.

You can also verify if my fix works for you - release binaries for pull requests can be found here. But be aware that it's currently done on 5.3 branch.

[hazzik]

@mattzink could you please verify the fix by @bahusoid?

[bahusoid]

5.2 based binaries can be found here

[mattzink]

@hazzik Will try it out today

[mattzink]

Unfortunately, using the bits that @bahusoid linked I get the same results (11 statements executed instead of 1). I was doubtful since we didn't have any eager mappings/queries used in this test. I will try to create a small repro project soon, but we're not doing anything unexpected/advanced.

[hazzik]

Ok, thanks.

[bahusoid]

Strange... But possible as Future in 5.2 is using completely new code behind the scene (#1718). Apparently some other case is missed in new batcher.