-
Notifications
You must be signed in to change notification settings - Fork 934
NH-3919 - Clean up and harmonize datetime types with regards to different dialects #703
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NH-3919 - Clean up and harmonize datetime types with regards to different dialects #703
Conversation
fcd9da3
to
67f96e7
Compare
@@ -21,11 +21,11 @@ public void ForeignKeyNames() | |||
|
|||
|
|||
string script = string.Join("\n", | |||
cfg.GenerateSchemaCreationScript(new MsSql2000Dialect())); | |||
cfg.GenerateSchemaCreationScript(new MsSql2008Dialect())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SQL 2000 dialect was failing due to "unknow type datetime2"...
@@ -1,26 +1,21 @@ | |||
<Project Sdk="Microsoft.NET.Sdk"> | |||
<Import Project="../../build-common/NHibernate.props" /> | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should I revert those riders glitches, or just let it do that once and hopefully for all?
@@ -36,5 +74,26 @@ protected override void RegisterDefaultProperties() | |||
base.RegisterDefaultProperties(); | |||
DefaultProperties[Environment.ConnectionDriver] = typeof(Sql2008ClientDriver).AssemblyQualifiedName; | |||
} | |||
|
|||
public override string CurrentTimestampSQLFunctionName => | |||
"SYSDATETIME()"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Previously it was using CURRENT_TIMESTAMP
, calling indeed getdate()
, so yielding datetime
instead of datetime2
.
@@ -39,7 +39,7 @@ public override IType DataType | |||
return fe.DataType; | |||
} | |||
ISQLFunction sf = Walker.SessionFactoryHelper.FindSQLFunction(Text); | |||
return sf == null ? null : sf.ReturnType(null, null); | |||
return sf?.ReturnType(null, Walker.SessionFactoryHelper.Factory); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unique case I have spotted where a methods chain of calls ending by calling IType.SqlTypes
was not supplying an IMapping
(the factory in this case).
src/NHibernate/Type/ClassMetaType.cs
Outdated
@@ -16,7 +16,8 @@ public partial class ClassMetaType : AbstractType | |||
{ | |||
public override SqlType[] SqlTypes(IMapping mapping) | |||
{ | |||
return new SqlType[] { NHibernateUtil.String.SqlType }; | |||
var types = new[] { NHibernateUtil.String.SqlType }; | |||
return mapping?.Dialect.RefineSqlTypes(this, types) ?? types; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have added that logic for each type, excepted those yielding no types and those building them from other types.
@@ -34,9 +34,9 @@ private IInternalLogger Log | |||
|
|||
/// <summary> | |||
/// Initialize a new instance of the NullableType class using a | |||
/// <see cref="SqlType"/>. | |||
/// <see cref="NHibernate.SqlTypes.SqlType"/>. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was referencing the NullabeType.SqType
property otherwise.
src/NHibernate/Type/NullableType.cs
Outdated
{ | ||
return new SqlType[] {SqlType}; | ||
var types = new[] { SqlType }; | ||
return mapping?.Dialect.RefineSqlTypes(this, types) ?? types; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That the main change for supporting the feature: this allow switching the NHibernate type default corresponding sql type for another.
Hibernate does not do that. They do not need. Java does not know DbType.DateTime and DbType.DateTime2, it only knows Types.TIMESTAMP
. Hibernate just need to tell this one should be mapped to sql type datetime2
in SQL 2008 dialect.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They call it on nullSafeGet/nullSafeSet/extract(no equivalent on our side)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok. It does not seems near to be ported currently, since it depends on their type descriptor which is not ported currently, and uses instance overrides for doBind
which relies on members of the calling class, pattern not doable that way in .Net. (We could likely use method delegates instead.) And I have not checked if their override is taken into account also for hbm2ddl, as mine.
{ | ||
var validator = new SchemaValidator(cfg); | ||
validator.Validate(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Delegating to Validate
for checking we have the expected type in database. Which means being confident Validate
correctly knows if it should be DateTime
or DateTime2
.
It appeared to me overly complicated to test this manually for all dialects, so I delegated to Validate
. I have furthermore checked in a debug session directly in the db what type it was to ascertain we were now having datetime2 with SQL2008.
67f96e7
to
f9cfbac
Compare
} | ||
|
||
private void AssertSqlType(ClientDriverWithParamsStats driver) | ||
private void AssertSqlType(ClientDriverWithParamsStats driver, int expectedCount) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Checking the exact usage count by the way.
} | ||
|
||
[Test] | ||
public void QueryUseExpectedSqlType() | ||
{ | ||
if (!TestDialect.SupportsNonDataBoundCondition) | ||
Assert.Ignore("Dialect does not support the test query"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Debuging the failure, the expected DbType and SqlType usages count is found (5), but Firebird does not accept the last condition. Removing it allows the query to be executed, but I want this condition for avoiding having the Hql parser determining the type for all parameters from the properties they are compared to, and instead use the type supplied by the parameter. So I ignore this test for Firebird.
a64b86d
to
4572948
Compare
Last commit is part 2. I intend to add more tests though. |
var grandchild = new Node {Name = "grandchild"}; | ||
var root = new Node | ||
{ | ||
Created = RoundForDialect(DateTime.Now), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just fixing the trouble in test. It could be fixed at the type level by overriding IsEqual
(and preferably setter and getter too). But this would be a possible breaking change for timestamp
when used a as simple property, because this will cause it to use the dialect declared accuracy instead of the actual database accuracy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Odbc causes issue with milliseconds due to NH-3895 which does not seem actually fixed when the Loader
is used.
@@ -6,23 +6,23 @@ | |||
<id name="Id" type="Int32"> | |||
<generator class="assigned" /> | |||
</id> | |||
<property name="Name" length="2000" /> | |||
<property name="Name" length="1999" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With below change to expand parameters, the size defined by OdbcDriver now actually ends in final parameters, and it causes issues for nvarchar(2000) and above: they get converted to NText by ODBC, causing:
ERROR [42000] [Microsoft][SQL Server Native Client 11.0][SQL Server]The data types nvarchar and ntext are incompatible in the equal to operator.
So here this is just a workaround for this test which was not about long string. But I fear there is a serious trouble with Odbc, and we should consider removing that size specification which was not actually taken into accont until below change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have searched about this trouble of ODBC switching to ntext for nvarchar >= 2000. Only to find someone confirming it here, but no one seem to have a clue about why is it like this.
@@ -231,7 +231,7 @@ public void RemoveUnusedCommandParameters(DbCommand cmd, SqlString sqlString) | |||
.ForEach(unusedParameterName => cmd.Parameters.RemoveAt(unusedParameterName)); | |||
} | |||
|
|||
public virtual void ExpandQueryParameters(DbCommand cmd, SqlString sqlString) | |||
public virtual void ExpandQueryParameters(DbCommand cmd, SqlString sqlString, SqlType[] parameterTypes) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This method in the case of ODBC was replacing the original parameters by new one, partially clone, leaving aside Scale
, Precision
and Size
set by OdbcDriver.InitializeParamater
. So I wonder if those changes in InitializeParameter
were tested. It does not seem to be the case. When looking at NH-3895 which was supposed to be fixed by one of those changes, there are no test in the commit.
Due to ExpandQueryParameters
, Scale
is lost. Due to the change in DateTimeType
, we now have milliseconds, and this has triggered the NH-3895 bug for numerous tests. So I have changed ExpandQueryParameters
for it to create parameters the same way as original one, calling InitializeParamater
. This fixes NH-3895. But it has triggered the trouble explained in previous comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Having dug into files history, the size setting has been added with 03eb58a in January 2010, while the ExpandQueryParameters
was not yet there. This one has been added in July 2010 with 131b97e. So, not having Size/Precision/Scale actually specified for Odbc parameters (and all other drivers not handling named parameters - DB2, Ifx, Ingres, OleDb, OracelLite, SybaseAsa - if they do set them in InitializeParameter
,) is an old regression.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But later on NH-3036 reverted the size setting for sql client due to issues with like
(which may requires larger size than the actual column size), so I think we should remove this .Size
setting from Odbc parameters too
In fact Odbc seems to re-adjust size anyway according to some comment in Set of string type.
//TODO: Fix me after 4.6.2 update. Size and Precision has been added to DbParameter | ||
var p = dataParameter as IDbDataParameter; | ||
if (p != null) | ||
return p.DbType + " (" + p.Size + ":" + p.Scale + ":" + p.Precision + ")"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have I missed something or this todo was wrong on the version number? Precision
was added in 4.5.1, and others are older.
c859197
to
fe35045
Compare
@@ -29,14 +29,14 @@ public void SimpleSelectTest() | |||
long simple1Key = 15; | |||
Simple simple1 = new Simple(); | |||
simple1.Address = "Street 12"; | |||
simple1.Date = DateTime.Now; | |||
simple1.Date = RoundForDialect(DateTime.Now); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Many tests comparing date before and after saving are changed to round them to the dialect known accuracy. Previously they were rounded to seconds due to DateTimeType
Hibernate like semantic. Now that NHibernate does no more round to second for this type, we need to take into account the database accuracy.
@@ -40,31 +38,33 @@ public void SelectConditionalValuesTest() | |||
.Select(o => new | |||
{ | |||
o.Color, | |||
AliveDays = (int)(DateTime.Now - o.Birthdate).TotalDays, | |||
AliveDays = (now - o.Birthdate).TotalDays, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Test had start being flaky with DateTimeType
no more truncating to seconds. The diff followed by TotalDays
does not count boundaries but a fractional days count. We should not truncate it to int but better instead compare with a tolerance.
@@ -23,14 +24,14 @@ public void Next() | |||
[Test] | |||
public void Seed() | |||
{ | |||
DateTimeType type = NHibernateUtil.DateTime; | |||
var type = NHibernateUtil.DateTime2; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yuck. Was not testing what it was supposed to.
/// TestFixtures for the <see cref="DateTimeNoMsType"/>. | ||
/// </summary> | ||
[TestFixture] | ||
public class DateTimeNoMsTypeFixture |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In fact this is what was DateTimeTypeFixture
.
// This test fails against the ODBC driver. The driver would need to be override to allow longer parameter sizes than the column. | ||
AssertExpectedFailureOrNoException( | ||
exception, | ||
(Sfi.ConnectionProvider.Driver is OdbcDriver)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was no more failing since 131b97e which was losing the Size
set on parameter. But since "expand parameter" is now fixed, it would fail again if we do not do NH-4083.
src/NHibernate/Type/TimestampType.cs
Outdated
|
||
namespace NHibernate.Type | ||
{ | ||
/// <summary> | ||
/// This is almost the exact same type as the DateTime except it can be used | ||
/// in the version column, stores it to the accuracy the database supports, | ||
/// and will default to the value of DateTime.Now if the value is null. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
default to the value of DateTime.Now if the value is null
: wrong, since it was "implemented" into Set
which is not called when the value is null
.
Moreover since when this comment was written, it seems DateTimeType
has gained the ability to serve as version too.
src/NHibernate/Type/TimestampType.cs
Outdated
/// and will default to the value of DateTime.Now if the value is null. | ||
/// This is almost the exact same type as the <see cref="DateTimeType" /> excepted it can be used | ||
/// in the version column, stores it to the accuracy the database supports | ||
/// and will default to the value of <see cref="TimestampType.Now" /> if the value is not valid. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We may remove that since I do not think it was intended, the aim for the code being to replace null
values, but in a method which is never called on null
. It coud be a breaking change for edge cases, where a timestamp
is mapped to an object
property in which the app may put non DateTime
data...
I have left the accuracy part in case for some reason we were to reduce DateTimeType
one again.
src/NHibernate/Type/TimestampType.cs
Outdated
} | ||
/// <inheritdoc /> | ||
protected override DateTime GetDateTimeToSet(object value, ISessionImplementor session) => | ||
(value as DateTime?) ?? Now; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This override should be removed if we want to clean the failed attempt at replacing null values by Now
. By the way if keeping it, we should probably call Seed
instead of Now
.
return '\'' + value.ToString() + '\''; | ||
} | ||
public override string ToString(object val) => | ||
((DateTime) val).ToString("O"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe the only significant change of Timestamp
compared to the new DateTimeType
.
var dbValue = base.GetDateTimeToSet(value, session); | ||
if (dbValue.Kind != DateTimeKind.Utc) | ||
throw new ArgumentException("Kind is NOT Utc", nameof(value)); | ||
return dbValue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With Timestamp.ToString
, that is all the differences with UtcDateTimeType
. UtcDateTimeType
does not throw if it receives a non UTC value: it silently switch its kind to UTC.
src/NHibernate/Cfg/Environment.cs
Outdated
/// for dialects supporting datetime2. | ||
/// </summary> | ||
[Obsolete("Migrate SQL datetime type to datetime2 instead.")] | ||
public const string SqlTypesKeepDateTime = "sql_types.keep_datetime"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DateTime2
being unsupported by OdbcDriver, we may use it for allowing using 2008 dialect with Odbc. So I think I will remove the obsolete attribute.
The subject is a bit deeper since Odbc does not understand Time
either, and likely Date
too: we should have a mean to switch those types too. So we can also decide that only dialects up to 2005 are compatible with ODBC.
(By the way this page states ODBC does support SQL2008 types, but how are we supposed to do in a .Net context, mystery.)
An other option would be to try moving the type override from dialect to driver. But currently IMapping
interface which is where I pull the dialect from for refining types is not much suitable for having the driver, unless we add the ability for Configuration.Mapping
to instantiate a driver.
I think I will just remove the Obsolete
attribute.
509b41d
to
ee12f78
Compare
All date time types are now tested. One more bug to fix by the way. |
ee12f78
to
03879e2
Compare
Such a Pandora's box. |
If it is not 5.0 it should be 6, too much changes for a minor. |
I think this is better to leave it for 6. A good exercise for a types refactoring. At least much better than trying to rush in 5.0 |
* NHibernate date time types (datetime, timestamp, localdatetime, utcdatetime, dbtimestamp) use db type datetime2 if dialect supports it. * NHibernate type datetime2 is obsolete. * Add test for NH-3984 by Ralf Kornelius
* NHibernate DateTime type does not cuts milliseconds anymore * Add new types which cut milliseconds * Refactor everything together
case DbType.Time: | ||
return true; | ||
} | ||
return false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, I have been a bit quick with the "refactor extract method here".
I just realized that "NoMs" types are not needed anymore. |
Why, because we have scale? Scale is not understood and handled by every database. |
NH-3919 - Clean up and harmonize datetime types with regards to different dialects.
Possible breaking changes: