Skip to content

Feature/import documents batch threads #276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jul 24, 2019
Merged

Feature/import documents batch threads #276

merged 9 commits into from
Jul 24, 2019

Conversation

rkhaja
Copy link
Contributor

@rkhaja rkhaja commented Jul 16, 2019

This pull request addresses enhancements requested in #273
Please review.

@hkernbach
Copy link
Member

Hey @rkhaja, first of all, thank you very much for contributing. The code itself looks good to me. We need a JUnit test to test that feature, then we're able to merge.

If you could also add a test, it would be great. Otherwise, we will add one as soon as we can.

@rkhaja
Copy link
Contributor Author

rkhaja commented Jul 22, 2019

@hkernbach Thanks! I will of course write the JUnit test(s) (hopefully very soon) ... I wanted to be sure that the this code met your coding standards before I spent effort to write the tests.

@rkhaja
Copy link
Contributor Author

rkhaja commented Jul 23, 2019

I've added the following test:

	@Test
	public void importDocumentsBatchSizeNumThreads() {
		final Collection<BaseDocument> values = new ArrayList<BaseDocument>();
		for (int i = 1; i <= 100; i++) {
			values.add(new BaseDocument(String.valueOf(i)));
		}
		int batchSize = 5;
		int numThreads = 4;
		final Collection<DocumentImportEntity> docsList = db.collection(COLLECTION_NAME).importDocuments(values,
				new DocumentImportOptions(), batchSize, numThreads);
                assertThat(docsList.size(), is(values.size() / batchSize));
		for (final DocumentImportEntity docs : docsList) {
			assertThat(docs, is(notNullValue()));
			assertThat(docs.getCreated(), is(batchSize));
			assertThat(docs.getEmpty(), is(0));
			assertThat(docs.getErrors(), is(0));
			assertThat(docs.getIgnored(), is(0));
			assertThat(docs.getUpdated(), is(0));
			assertThat(docs.getDetails(), is(empty()));
		}
	}

Essentially it is a copy of the basic importDocuments() test except that iterates through individual DocumentImportEntity objects.

	@Test
	public void importDocuments() {
		final Collection<BaseDocument> values = new ArrayList<BaseDocument>();
		values.add(new BaseDocument());
		values.add(new BaseDocument());
		values.add(new BaseDocument());
		final DocumentImportEntity docs = db.collection(COLLECTION_NAME).importDocuments(values);
		assertThat(docs, is(notNullValue()));
		assertThat(docs.getCreated(), is(values.size()));
		assertThat(docs.getEmpty(), is(0));
		assertThat(docs.getErrors(), is(0));
		assertThat(docs.getIgnored(), is(0));
		assertThat(docs.getUpdated(), is(0));
		assertThat(docs.getDetails(), is(empty()));
	}

I hope the test is sufficient but if it is not please let me know.

Copy link
Member

@hkernbach hkernbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hkernbach
Copy link
Member

Thank you for contributing, will merge your PR now. This will be then part of our next ArangoDB Java Driver release (Version: 5.0.8).

@rkhaja
Copy link
Contributor Author

rkhaja commented Jul 24, 2019

Thank you!

@rkhaja rkhaja deleted the feature/import-documents-batch-threads branch July 24, 2019 18:40
@rkhaja rkhaja restored the feature/import-documents-batch-threads branch July 25, 2019 20:23
@rkhaja rkhaja deleted the feature/import-documents-batch-threads branch July 25, 2019 20:24
throw new ArangoDBException(e);
}
documentImportEntityList.add(documentImportEntity);
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hkernbach I noticed a small problem.
RIght before the return statement on this line
we need to shutdown the executorService.

        executorService.shutdown();

If another pull request is needed, please let me know.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix that, thanks for spotting :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants