java.lang.UnsatisfiedLinkError: no sqlite4java-osx-x86_64-1.0.392 in java.library.path

[SachinRaghunathan]

Stack overflow question:
https://stackoverflow.com/questions/34137043/amazon-dynamodb-local-unknown-error-exception-or-failure/35353377#35353377

Anyone know a way to fix this?

[erfangc]

👍 this makes writing unit tests using DynamoDB local pretty much impossible without resorting to convoluted mocking ... it would be nice if the Jar shipped with the necessary binaries or avoided using sqlite as a underlying implementation and opted for something like h2 instead?

[tekumara]

sqlite4java will look for the native binaries in the same directory as its jar. So for now, I've placed the following in a lib dir:

lib/sqlite4java-1.0.392.jar
lib/libsqlite4java-osx-1.0.392.dylib

And then via my build tool I manually add lib/sqlite4java-1.0.392.jar to the classpath.

sqlite4java and its native libraries are available here: http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22com.almworks.sqlite4java%22

[Tony080]

The basic idea is the same as @tekumara claims. But in my scenario, it can also work with maven/gradle across the version control.

First, you need to download the zip file from offcial website. Unzip the file, copy all the *.dll, *.dylib, *.so to a folder under your project root. Say, src/test/resources/libs.
Then, add the code

System.setProperty("sqlite4java.library.path", "src/test/resources/libs/");

before you initialize a local instance of AmazonDynamoDB.
Then it should work!

[gremsam]

Ok, I dug into this (because this was bothering me) and here is what I found (for the next dev to find this ticket):

The important function in question is loadLibraryX (You can see it on this mirror). This function is going to look for a OS/Arch appropriate binary in 3 locations. If it finds it in none of the locations you get the above error.

One of the locations is the forcedPath. This is set by the system property sqlite4java.library.path (what you see Tony doing in their answer). The next location is the "default" path which defaults to the location of the jar (this is why tekumara's answer works as their setting the binary to be in that location). Finally it falls back on looking a System path (using the java internal System function System#loadLibrary).

Unfortunately in order to just add a jar to your pom or gradle config there needs to be a distributed jar that includes the binaries in the same package. In particular com.almworks.sqlite4java:sqlite4java:1.0.392 depends on other jars which include the OS/Arch specific binaries. Since the logic ONLY looks in the same path as it's loading from, if you don't explicitly force the transitive dependency's path it can't find the binary (as they aren't bundled together). This means it won't work without one of the hacks listed above.

This feels like a real drawback of this particular dependency. I might recommend either publishing a fat jar (for testing) that bundled this together, or picking a different dependency which has better resolution for this problem

[fxiong1]

I had a similar problem running local dynamodb under Windows. After making all the files under DynamoDBLocal_lib as "executable" (chmod u+x * under cygwin), this problem went away.

[superdurszlak]

Is there any progress on this issue? We're facing the same problem and currently we consider giving up on local DynamoDB altogether and resorting to docker containers instead. Native dependencies are being lost both on Windows and MacOS.

Also, solutions like setting path to DLLs in Gradle scripts work in some cases only (as long as you use Gradle or Gradle-based run configs in IDEs), and mingling with permissions in FS doesn't sound like something you want to incorporate into your workflow.

[bernard01]

Not an example. Just bad quality code, AWS marketing bullshit not even able to re-use 3rd party code, total waste of time. Scammers.

[wolkenschieber]

Running test DynamoDBEmbeddedTest.java throws following exception for me:

ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
Dez. 04, 2020 9:40:19 VORM. com.almworks.sqlite4java.Internal log
WARNUNG: [sqlite] cannot open DB[1]: com.almworks.sqlite4java.SQLiteException: [-91] cannot load library: java.lang.UnsatisfiedLinkError: no sqlite4java-linux-amd64-1.0.392 in java.library.path: [/usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib]
Dez. 04, 2020 9:40:19 VORM. com.almworks.sqlite4java.Internal log
SCHWERWIEGEND: [sqlite] SQLiteQueue[]: error running job queue
com.almworks.sqlite4java.SQLiteException: [-91] cannot load library: java.lang.UnsatisfiedLinkError: no sqlite4java-linux-amd64-1.0.392 in java.library.path: [/usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib]
	at com.almworks.sqlite4java.SQLite.loadLibrary(SQLite.java:97)
	at com.almworks.sqlite4java.SQLiteConnection.open0(SQLiteConnection.java:1441)
	at com.almworks.sqlite4java.SQLiteConnection.open(SQLiteConnection.java:282)
	at com.almworks.sqlite4java.SQLiteConnection.open(SQLiteConnection.java:293)
	at com.almworks.sqlite4java.SQLiteQueue.openConnection(SQLiteQueue.java:464)
	at com.almworks.sqlite4java.SQLiteQueue.queueFunction(SQLiteQueue.java:641)
	at com.almworks.sqlite4java.SQLiteQueue.runQueue(SQLiteQueue.java:623)
	at com.almworks.sqlite4java.SQLiteQueue.access$000(SQLiteQueue.java:77)
	at com.almworks.sqlite4java.SQLiteQueue$1.run(SQLiteQueue.java:205)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.UnsatisfiedLinkError: no sqlite4java-linux-amd64-1.0.392 in java.library.path: [/usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib]
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2670)
	at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:830)
	at java.base/java.lang.System.loadLibrary(System.java:1873)
	at com.almworks.sqlite4java.Internal.tryLoadFromSystemPath(Internal.java:352)
	at com.almworks.sqlite4java.Internal.loadLibraryX(Internal.java:124)
	at com.almworks.sqlite4java.SQLite.loadLibrary(SQLite.java:95)
	... 9 more

Dez. 04, 2020 9:40:19 VORM. com.almworks.sqlite4java.Internal log
SCHWERWIEGEND: [sqlite] SQLiteQueue[]: stopped abnormally, reincarnation is not possible for in-memory database

[wolkenschieber]

Seems like there are some dependencies missing in the examples. Here's what made it work for me:

1. Extract libraries in build. Add to pom.xml

	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-dependency-plugin</artifactId>
				<version>3.1.2</version>
				<executions>
					<execution>
						<id>copy</id>
						<phase>test-compile</phase>
						<goals>
							<goal>copy-dependencies</goal>
						</goals>
						<configuration>
							<includeScope>test</includeScope>
							<includeTypes>so,dll,dylib</includeTypes>
							<outputDirectory>${project.basedir}/src/test/resources/native-libs</outputDirectory>
						</configuration>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>

2. Set system property sqlite4java.library.path in test case

public class DynamoDBEmbeddedTest {
	@Test
	public void createTableTest() {
		System.setProperty("sqlite4java.library.path", "src/test/resources/native-libs");
		AmazonDynamoDB ddb = DynamoDBEmbedded.create().amazonDynamoDB();

More details can be found in this StackOverflow answer.

[adrian-baker]

https://github.com/cashapp/tempest/blob/master/tempest2-testing-jvm/src/main/kotlin/app/cash/tempest2/testing/JvmDynamoDbServer.kt shows how this can be done programmatically, without needing additional build time steps to assemble binaries into extra native lib directories.

I added a SO answer adapting this same approach - https://stackoverflow.com/questions/26901613/easier-dynamodb-local-testing/69034936#69034936.

It's a pity this isn't better handled in the library itself.