Skip to content

Add a generic batcher for insert/update/delete statements, usable with PostgreSQL and others #1588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Mar 10, 2018
Merged

Conversation

maca88
Copy link
Contributor

@maca88 maca88 commented Mar 1, 2018

The batcher is working by concatenating multiple statements with a semicolon. In order to avoid having duplicate parameter names the IDriver.GenerateCommand method is used to generate a new command from the current one, where the passed SqlString is altered so that the parameter names will start from the latest parameter name of the batch command.

Update:
As the batcher does not use any PostgeSQL specifics and neither uses any reflection to do its job, it could be renamed to something more generic if desired. The batcher works on SQL Server (tested on 2012), so it can be used where SqlClientBatchingBatcher is not available with the limitation of 2100 parameters.

@maca88 maca88 changed the title Added batching support for insert/update/delete statements on PostgreSQL WIP Added batching support for insert/update/delete statements on PostgreSQL Mar 1, 2018
@maca88 maca88 changed the title WIP Added batching support for insert/update/delete statements on PostgreSQL Added batching support for insert/update/delete statements on PostgreSQL Mar 2, 2018
@hazzik
Copy link
Member

hazzik commented Mar 3, 2018

The batcher works on SQL Server (tested on 2012), so it can be used where SqlClientBatchingBatcher is not available with the limitation of 2100 parameters.

Can you add automatic flush if the limit is going to be reached?

@fredericDelaporte
Copy link
Member

As the batcher does not use any PostgeSQL specifics and neither uses any reflection to do its job, it could be renamed to something more generic if desired.

This sounds as a good idea. According to wikipedia, the semi-colon is in the SQL standard (quite lame this standard is not public and forces to find "second hand" sources), so why not naming it GenericBatchingBatcher. (Drop the Client, it looks to me as a remnant of SqlClient naming, used by the SQL-Server client code.) Since at least some databases allow to redefine the statement terminaison character to something else than a semi-colon, maybe this character should be configurable. But well, it would be just a small nice to have.

But for this PR I think we should then not go more far than using it for PostgreSQL. Further enhancements for using it with other DB should in my opinion be done with another PR.

Can you add automatic flush if the limit is going to be reached?

That 2100 limit is likely specific to SQL-Server, so it would have to be something applied only when this batcher is used for SQL-Server. I think it is a good idea, but for a complementary PR aiming at re-using this generic batcher for SQL-Server.

…d and renamed the batcher to a more universal name
@maca88
Copy link
Contributor Author

maca88 commented Mar 3, 2018

Can you add automatic flush if the limit is going to be reached?

Done with a test added.

so why not naming it GenericBatchingBatcher

Renamed.

Since at least some databases allow to redefine the statement terminaison character to something else than a semi-colon, maybe this character should be configurable

I added the character to be configurable on the generic batcher factory.

But for this PR I think we should then not go more far than using it for PostgreSQL

As I did the proposed changes we could edit the title of this PR or create a new PR and close this one.

@maca88
Copy link
Contributor Author

maca88 commented Mar 3, 2018

Unfortunately FirebirdSql.Data.FirebirdClient, Oracle.ManagedDataAccess and System.Data.SqlServerCe clients do not support concatenating commands with a semicolon.
I did some research to see if there is something that we could do but I didn't found anything that would work correctly.
With FirebirdClient, the only way to execute multiple statements is via EXECUTE BLOCK command, but the problem is that it is not possible to pass parameters as far as I tested.

With Oracle.ManagedDataAccess I was able to execute multiple statements by wrapping them into a BEGIN END block, but the problem is that the returned value of the modified rows is always -1.

With SqlServerCe I tried to add a GO command after each command as proposed here, but I didn't worked.

I've removed those three from tests and added a list of unsupported clients in the summary of the batcher.

@fredericDelaporte fredericDelaporte changed the title Added batching support for insert/update/delete statements on PostgreSQL Add a generic batcher for insert/update/delete statements, usable with PostgreSQL and others Mar 8, 2018
@fredericDelaporte
Copy link
Member

fredericDelaporte commented Mar 8, 2018

With SqlServerCe I tried to add a GO command after each command as proposed here, but I didn't worked.

Indeed it can not, because the question is about "SQL Compact 4.0 Toolbox", which is a client tool (the question title is misleading about this). The GO statement is a client command, I have never seen it understood "server" side.

@fredericDelaporte fredericDelaporte self-assigned this Mar 8, 2018
@fredericDelaporte fredericDelaporte added this to the 5.1 milestone Mar 8, 2018
@hazzik hazzik merged commit 05264ba into nhibernate:master Mar 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants