-
Notifications
You must be signed in to change notification settings - Fork 934
Add support for SAP HANA #1662
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for SAP HANA #1662
Conversation
breglerj
commented
Apr 13, 2018
- add row and column store dialects
- add driver
- add batcher
- add configuration template
- exclude tests that are known not to run on SAP HANA
Related discussion from here:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That looks like a lot of work, and so a lot to review. Changes to test should be minimized as explained in code review comments. I have not reviewed them overall. (That is indeed just a preliminary review.)
using NUnit.Framework; | ||
|
||
namespace NHibernate.Test.Criteria.Lambda | ||
{ | ||
[TestFixture] | ||
public class FunctionsIntegrationFixture : TestCase | ||
{ | ||
protected override bool AppliesTo(Dialect.Dialect dialect) | ||
{ | ||
return !(dialect is AbstractHanaDialect); // HANA does not support inserting a row without specifying any column values |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks to me the same comment has been copied-pasted, without it to stay relevant.
Moreover, we favor a kind of "feature detection" over dialect/driver detection for disabling tests: it tends to bloat far less the history with changes.
By example, for "support inserting a row without specifying any column values", there is TestDialect.SupportsEmptyInserts
. To tell HANA dialect does not support it, add a HanaColumnStoreTestDialect
(that is based on name) in TestDialect
folder of NHibernate.Test, implemented similarly to the other "TestDialect" there, and overriding this property to false
. Then any test relying on this feature will be ignored without needing to change them.
For some other features, there are properties from the base Dialect
class to override.
Some cases are still handled through dialect/driver sniffing, but no new case should be introduced.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comment should be relevant everywhere that I've copied it. There are quite a lot of tests that have a parent entity with only an ID attribute and a child collection. The generated INSERT
statement for such a mapping looks something like INSERT INTO Parent VALUES ()
. The empty parentheses cause a SQL error on HANA, that's why the tests have to be skipped.
I did see TestDialect.SupportsEmptyInserts
. In fact, the PR already includes TestDialect implementations for all HANA dialects. I just wasn't sure that this property covers the exact use case I'm facing. I'll push a new change replacing the explicit checks for the HANA dialect with a check for this property.
/// Each dialect will define a template as a string (exactly like above) marking function | ||
/// parameters with '?' followed by parameter's index (first index is 1). | ||
/// </summary> | ||
[Serializable] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rather than copying comment from base, use <inheritdoc />
. Or even better, document what it changes compared to its base class. Indeed that is the comment from StandardSQLFunctionWithRequiredParameters
which should be copied here, slightly adjusted like A template-based SQL function which substitutes required missing parameters with defaults.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll adjust the documentation.
{ | ||
private readonly object[] _requiredArgs; | ||
|
||
public SQLFunctionTemplateWithRequiredParameters(IType type, string template, object[] requiredArgs) : base(type, template) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Better also add an overload for allowing supplying hasParenthesesIfNoArgs
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do.
src/NHibernate/Dialect/Dialect.cs
Outdated
@@ -2480,6 +2480,14 @@ public virtual string AddColumnString | |||
get { throw new NotSupportedException("No add column syntax supported by Dialect"); } | |||
} | |||
|
|||
/// <summary> | |||
/// The syntax for the suffix used to add a column to a table. Note this is deprecated |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The "Note" copied from AddColumnString
own comment seems to be a note for some deprecation which does not seem to be done NHibernate side. Better remove that note in this new property comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just added the deprecation note because it was already present on AddColumnString
and AddColumnSuffixString
doesn't make sense without AddColumnString
. Do you want me to remove the deprecation note on both properties? Having the deprecation note on only one property seems wrong to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A correct deprecation note is the Obsolete
attribute anyway. Ok for removing both.
- Replace explicit check for AbstractHanaDialect with TestDialect.SupportsEmptyInserts where applicable - Replace copied base class documentation with <inheritdoc /> - Add constructor SQLFunctionTemplateWithRequiredParameters(IType, string, object[], bool) - Improve documentation of SQLFunctionTemplateWithRequiredParameters - Removed deprecation comments from Dialect#AddColumnString and Dialect#AddColumnSuffixString
@@ -22,7 +22,7 @@ public class FunctionsIntegrationFixtureAsync : TestCase | |||
{ | |||
protected override bool AppliesTo(Dialect.Dialect dialect) | |||
{ | |||
return !(dialect is AbstractHanaDialect); // HANA does not support inserting a row without specifying any column values | |||
return TestDialect.SupportsEmptyInserts; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This surprises me, because we already test a database which is not supposed to support this, and which does not fail in all these tests modified by this PR. I have to check what is going on.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, the test uses the native
generator, which resolves to identity
if it is supported, otherwise sequence
if supported, otherwise hilo
. The other db not supporting empty inserts does override NativeIdentifierGeneratorClass
for using hilo
, thus it has a value to insert.
Then the exact condition here should be TestDialect.SupportsEmptyInserts || dialect.NativeIdentifierGeneratorClass != typeof(IdentityGenerator)
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't the check for the NativeIdentifierGeneratorClass
be included in TestDialect.SupportsEmptyInserts
then? It seems strange that you'd have to check for TestDialect.SupportsEmptyInserts
and basically everywhere add the check for the NativeIdentifierGeneratorClass
right after it. If TestDialect.SupportsEmptyInserts
really means something else, would it make sense to introduce a new property like TestDialect.SupportsEmptyInsertsWithNativeIdentifierGenerator
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the mapped class was using identity
directly instead, testing on NativeIdentifierGeneratorClass
inside SupportsEmptyInserts
would defeat its purpose.
The test here does empty inserts only if the native
generator does resolves to identity
, otherwise it does not perform empty inserts. So when the dialect does not support empty inserts but does not resolves native
to identity
, it is irrelevant to disable that test.
SupportsEmptyInsertsWithNativeIdentifierGenerator
is not meaningful. But since the case is very frequent, why not adding a SupportsEmptyInsertsOrHasNonIdentityNativeGenerator
. It could be something like:
/// <summary>
/// Either supports inserting in a table without any column specified in the insert, or has a native
/// generator strategy resolving to something else than identity.
/// </summary>
/// <remarks>This property is useful for cases where empty inserts happens only when the entities
/// generator is <c>native</c> while the dialect uses <c>identity</c> for this generator.</remarks>
public bool SupportsEmptyInsertsOrHasNonIdentityNativeGenerator
=> SupportsEmptyInserts || _dialect.NativeIdentifierGeneratorClass != typeof(IdentityGenerator);
This new property can be used to exclude tests where empty inserts happen only when the entities generator is native while the dialect uses identity for this generator and empty inserts aren't supported by the database.
I am in the process of reviewing this. I have noticed that multi-threaded tests get unexpected failures of the SAP HANA connection opening, like |
That's strange. The multi-threaded tests run fine on my machine. How do you run the tests? What kind of failures are you getting? Which version of the HANA .Net data provider do you use? |
I am running them through JetBrain Rider IDE, with as database a SAP HANA express edition 2.0 SPS03 server-only VM, as client Sap.Data.Hana 2.3.78.0 installed under In the case of NH2192, I have always seen the failure with
In English this would likely be:
The I have tried having the provider files directly loaded instead of relying on having them in the GAC, copying directly files listed in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See breglerj#1 for proposed modifications for handling a part of the review comments.
/// <inheritdoc /> | ||
public override string NoColumnsInsertString | ||
{ | ||
get { throw new MappingException("HANA does not support inserting a row without specifying any column values"); } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removing this override allows numerous otherwise failing tests to pass. The property is called by NHibernate for preparing insert statements in advance, without having already actual inserts to perform. Some tests re-use mapping having entities with just a native id, but without having to insert such entities. With the exception thrown here, it causes such test to fail while they would have succeeded otherwise.
Of course the mapping has still a non-supported part, so it makes sens to reject it that way. But this represents a discrepancy in behavior with other dialects not supporting empty inserts, and causes many additional changes in tests. Moreover if some future version of HANA starts supporting empty inserts with the default syntax generated by NHibernate, this will prevents the use of the feature till the dialect is changed.
So I think it is better here to not override the property and let SAP HANA handle the case (raising an error currently) itself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll remove the override. Although even if HANA supports this in a future version it's not guaranteed that it'll use the syntax values( )
it still makes sense for the tests.
namespace NHibernate.AdoNet | ||
{ | ||
/// <summary> | ||
/// Summary description for HanaBatchingBatcher. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even if it may be copied from another batcher, please put a more meaningful xml comment, like "Batcher for SAP HANA."
|
||
public override void AddToBatch(IExpectation expectation) | ||
{ | ||
Debug.Assert(CurrentCommand is ICloneable); // HanaCommands are cloneable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using Debug.Assert
is not part of NHibernate practices. Throw an InvalidOperationException
instead for this case (if the command is not cloneable), if you want this to be checked here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, there are some other cases, but there are likewise many things we are changing/modernizing/fixing, and so we avoid new occurrences to be added. Another case is the Java naming, we try to avoid having more of it to be added and ask for things to be renamed according to .Net usual conventions. (It appears private field are no more covered by public guidelines, but previously they were.)
if (_currentBatch == null) | ||
{ | ||
// use first command as the batching command | ||
_currentBatch = (batchUpdate as ICloneable).Clone() as DbCommand; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not use the as
cast when the cast is never supposed to fail: it results in a null reference exception when it fails, instead of throwing a way more obvious invalid cast exception. Use ((ICloneable)batchUpdate) instead.
_currentBatch = (batchUpdate as ICloneable).Clone() as DbCommand; | ||
} | ||
|
||
_currentBatchCommands.Add((batchUpdate as ICloneable).Clone() as DbCommand); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as above. But rather than casting at three places (including the initial check), better do that once in the initial check, like:
if (!(CurrentCommand is ICloneable cloneableCurrentCommand))
throw new InvalidOperationException("Current command is not an ICloneable");
And then just use cloneableCurrentCommand
without any cast anymore.
public override bool SupportsRowValueConstructorSyntaxInInList => true; | ||
|
||
/// <inheritdoc /> | ||
public override bool SupportsCircularCascadeDeleteConstraints => false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only test relying on this succeeds when this is set as true
. Am I missing something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I copied this from the Hibernate dialect. This was the case in older releases, but apparently it is supported now.
public override bool SupportsCircularCascadeDeleteConstraints => false; | ||
|
||
/// <inheritdoc /> | ||
public override bool SupportsExpectedLobUsagePattern => false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems many overrides have been taken from Hibernate. NHibernate side, this one does not change anything.
|
||
#endregion | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Double blank line.
/// <inheritdoc /> | ||
public override string SelectGUIDString | ||
{ | ||
get { return "select sysuuid from dummy"; } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding sys.
would be safer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the same reasons as with sequence values, it doesn't matter here because there isn't actually anything being selected from the table.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does still matter. If there is no row in the table, it will not return anything.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like you're right. It should be changed to sys.dummy
.
public override bool SupportsExistsInSelect => false; | ||
|
||
/// <inheritdoc /> | ||
public override bool SupportsScalarSubSelects => false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems to support them indeed. But some tests relying on this fail for other reasons. This needs more investigations. It looks like this is ordering by an aggregating scalar sub-selecting which are not supported (and it looks like a HANA bug).
Partially address the review.
Do some more reformatting, fix some more tab/spaces mixup.
Adjust code according to review
I have now added a commit for handling the remaining scalar sub-selects failures.
|
HANA doesn't support returning values from
I would suggest leaving it set to Regarding the test failures:
I'll have to investigate further.
These test fail because there is no ordering specified and HANA doesn't return the results in the expected order. Adding an
I still can't reproduce the failure. I'll have to investigate further.
I can't really explain this one. The failure originates in the SqlGenerator. The
Same root cause as
This seems to be related to the data provider. I'll have to check.
Foreign key constraints over unique columns weren't supported until HANA 2 SP3. I'll have to investigate why the tests still fail. |
Here is an update regarding the failing tests:
This failure only happens when using the HANA batcher. When using the
These tests pass with the latest commit.
This seems to be a bug in the HANA data provider. I've filed a bug report for this.
I still can't really explain this one. The failure originates in the SqlGenerator. The
Same root cause as
This seems to be a bug in the HANA data provider. I've filed a bug report for this.
This is most likely a bug in HANA. The error happens when defining foreign key constraints to unique columns involving a |
If possible, it would be great to have public link to the HANA bug reports causing each failing test. It would allow to check if the bug status has evolved. About NH2294 and NH2394, their failure appears to be due to overriding I think |
The row-store case needs more work too: more than hundred tests fail for it, because row-store seems to support less features than column store. By example: |
Unfortunately the HANA bug reports aren't publically available. NHibernate performs backtracking for parameter specifications which is where the error in NH2294 and NH2394 originates. Hibernate doesn't do the backtracking which is why it works there. I'm fine with removing the override of |
Yes, better add an issue for that trouble with |
- Add separate drivers for row and column stores since the row store doesn't support system transactions - Disable support for distributed transactions and select for update on multiple tabled in the row store dialect - Skip NHibernate.Test.ReadOnly.TextHolder because the HANA row store doesn't support the TEXT data type
I've added issue #1676 |
I've pushed another change which should fix the HANA row store errors. It looks like the HANA row store doesn't support system transactions at all, so I've created a separate driver disabling system transaction support. In addition I've disabled distributed transaction support in the HANA row store dialect as well as support for outer join for update. With these changes alltests pass with the exception of a known few related to bugs in the HANA data provider or server. |
The querying bug with |
That's interesting. I'll add it to the bug report.
The error is caused by the column named |
@fredericDelaporte Thanks for merging and thanks for all your help. I really appreciate it. |
Thanks for your sustained effort with this PR too. Not every contributor are ready to this. |
@fredericDelaporte Do you have an estimate when there is going to be an official release (5.2 ?) containing this PR? |