NH-3806 - Saving entities with proxy associations leads to fetching associated entities

[nhibernate-bot]

Serhiy Shekhovtsov created an issue — 6th July 2015, 9:26:51:

During batch saving the method IsProcessedAfterAllAssociatedEntities is called and it calls GetHashCode of the associated Entity. At this moment DefaultLazyInitializer intercepts the call and fires ImmediateLoad for the associated proxy entity, which leads to entity fetching.
More details here.

---

Serhiy Shekhovtsov added a comment — 7th July 2015, 12:03:44:

As I figured out this is not related to batching and can be reproduced with a simplest example.

---

Serhiy Shekhovtsov added a comment — 8th July 2015, 9:23:15:

Complete and very basic sample can be downloaded here.

---

Summarizing it:

When using entities which do override the GetHashCode and Equals implementation, and use an identifier generator which does not trigger an immediate flush, saving many entities associated to uninitialized proxies causes the proxies to be loaded.
This reduces the usefulness of ISession.Load.

[johancn87]

Will this ever be fixed?
Is there any good workaround?

[fredericDelaporte]

It may be fixed only once someone decides to have a look at this. It can be you.

[johancn87]

I will give it a try!

(did not mean to sound rude)

[fredericDelaporte]

From the Stack Overflow link provided by the opener, it seems this is bound to ordering of inserts, which is a DML batching optimization. (So it is actually still bound to batching.) But it should happen only with entity classes overriding the default GetHashCode implementation and not using an identity generator triggering immediate flush to database, see below.

A workaround would be to disable ordering of inserts, as proposed on Stack Overflow. There is a configuration setting for this, order_inserts, documented here.

This ordering allows to group inserts together. DML batching only batches together identical operations (identical inserts, updates or deletes), varying only by parameters values. So for batching as most as possible, operations has to be regrouped. It is done by ordering them by entities. But in the case of inserts, the logic has to take care of not attempting to insert an entity before an associated one on which it would have a foreign key.

For this, it has to build dictionaries of these entities mapping to their dependency order. Putting something into a dictionary calls GetHashCode on the object. If it is a proxy built by the default proxy factory, on an entity class overriding the default object.GetHashCode method, it then causes its loading.

This ordering is also triggered for single inserts, but in this case, in short, it does not find anything to check since there is only one entity to insert. This is why the trouble does not occur when flushing after each insert.

(It is to be noted that on the Stack Overflow question, the opener states this is an issue for him because he wants to batch 1 000 inserts together. Well, using an ORM for this is generally not a good idea. They are not meant for usages which looks like ETL needs. That is maybe why there are not so many people having cared for this issue: with usual ORM usages, this issue has likely little performance impacts, especially if batching of lazy loading is enabled.)

From all this, we can infer the following list of work-around:

• Disable ordering of inserts with order_inserts setting.
  This will cause less efficient batching of inserts when different entity types are inserted in the same unit of work.

• Remove overrides of GetHashCode (and Equals) on entities.
  This can cause bugs, especially if working with detached entities.

• Use another proxy factory, able of calling an overridden GetHashCode and Equals without triggering the proxy loading.
  Good luck with this one. Theoretically such a proxy factory can be conceived, provided some restrictions:

  • The entities have to never access their own internal state by something else than a (virtual) property. No access to fields allowed. So by example, the entities would have to use auto-properties only. (Simplest way to guarantee this.)
  • The GetHashCode and Equals overrides would have to only access the id property. (Any other property access would force a proxy load.)

  And then such a factory would have to be coded. Main differences with the NHibernate current one would be the way it handles GetHashCode and Equals. If it can rely on the entity overrides to only access state through the properties, it can then safely cease re-implementing them for forcing loading the state: the property accesses will do it if needed. But the current proxy factory is not easily extensible, so doing this out of NHibernate would likely imply re-coding most of it.

About fixing the issue, it looks likely to me the dependency logic could simply ignore associated entities when they are uninitialized proxies: normally a proxied entity is already persisted, it should not be awaiting to be inserted. Maybe a wicked case exists: loading an entity as a proxy, then deleting it and flushing the session, then re-saving it. But saving it can only work if the proxy has its state loaded, so ignoring uninitialized proxies should be good.

Note that the IsProcessedAfterAllAssociatedEntities method cited by the opener does no more exist. A RequireNewBatch method replaces it (with inverted semantic).
Moreover the insert ordering was having some bugs in some cases (NH-3931), which fix has required to also check associated collections of entities. If such a collection has been initialized with un-initialized entity proxies, these proxies will also be loaded by the ordering. So the UpdateChildrenDependencies method would also have to be fixed. (Also note it would not fix anything for a set mapping, since putting the proxy in a set would already have triggered its loading.)

[johancn87]

As mentioned before the GetHashCode() triggers the load of the proxy. GetHashCode is called in ActionQueue.cs when using order_inserts property in NHibernate to optimize the batching of inserts. In the code below a dictionary is used which will use the hashcode.

InsertActionSorter --> RequireNewBatch(...)

for (var i = 0; i < propertyValues.Length; i++)
{
	var value = propertyValues[i];
	var type = propertyTypes[i];

	if (type.IsEntityType && value != null)
	{
		// find the batch number associated with the current association, if any.
		int associationBatchNumber;
		if (_entityBatchNumber.TryGetValue(value, out associationBatchNumber) &&
			associationBatchNumber > latestBatchNumberForType)
		{
			return true;
		}
	}
}

Anyway when I looked in detail in the documentation it actually explains why I have a problem. In our solution we have a base class for all entities where we override Equals & GetHashCode. When you override those methods proxies will get loaded if those are called.

var proxy = session.Load<MyEntity>(1);
var hash = proxy.GetHashCode(); // <-- Will fetch entity from DB.

http://nhibernate.info/doc/nh/en/index.html#performance-fetching-proxies

Certain operations do not require proxy initialization

• Equals(), if the persistent class does not override Equals()

• GetHashCode(), if the persistent class does not override GetHashCode()

• The identifier getter method

NHibernate will detect persistent classes that override Equals() or GetHashCode().