ConcurrentJarCreator.createEntry leaks file descriptors

[piotrp]

On Windows a plugin running in package phase (reproducible-build-maven-plugin) fails because JAR in target has on open file descriptor, opened via:

	at java.util.zip.ZipFile.<init>(ZipFile.java:156)
	at java.util.jar.JarFile.<init>(JarFile.java:166)
	at java.util.jar.JarFile.<init>(JarFile.java:103)
	at sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:930)
	at sun.misc.URLClassPath$JarLoader.access$800(URLClassPath.java:791)
	at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:876)
	at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:869)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.misc.URLClassPath$JarLoader.ensureOpen(URLClassPath.java:868)
	at sun.misc.URLClassPath$JarLoader.<init>(URLClassPath.java:841)
	at sun.misc.URLClassPath$3.run(URLClassPath.java:565)
	at sun.misc.URLClassPath$3.run(URLClassPath.java:555)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.misc.URLClassPath.getLoader(URLClassPath.java:554)
	at sun.misc.URLClassPath.getLoader(URLClassPath.java:519)
	at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:484)
	at sun.misc.URLClassPath.findResource(URLClassPath.java:214)
	at java.net.URLClassLoader$2.run(URLClassLoader.java:569)
	at java.net.URLClassLoader$2.run(URLClassLoader.java:567)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findResource(URLClassLoader.java:566)
	at org.codehaus.plexus.archiver.zip.PlexusIoZipFileResourceCollection$1.getResource(PlexusIoZipFileResourceCollection.java:81)
	at org.codehaus.plexus.archiver.zip.PlexusIoZipFileResourceCollection$ZipFileResourceIterator$ZipFileResource.getURL(PlexusIoZipFileResourceCollection.java:122)
	at org.codehaus.plexus.components.io.resources.PlexusIoURLResource.getContents(PlexusIoURLResource.java:38)
	at org.codehaus.plexus.components.io.resources.Deferred.getContents(Deferred.java:60)
	at org.codehaus.plexus.components.io.resources.proxy.ResourceInvocationHandler.invoke(ResourceInvocationHandler.java:66)
	at com.sun.proxy.$Proxy24.getContents(Unknown Source)
	at org.codehaus.plexus.archiver.ArchiveEntry.getInputStream(ArchiveEntry.java:137)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver$1.get(AbstractZipArchiver.java:550)
	at org.codehaus.plexus.archiver.zip.ConcurrentJarCreator.createEntry(ConcurrentJarCreator.java:276)
	at org.codehaus.plexus.archiver.zip.ConcurrentJarCreator.addArchiveEntry(ConcurrentJarCreator.java:196)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.zipFile(AbstractZipArchiver.java:512)
	at org.codehaus.plexus.archiver.jar.JarArchiver.zipFile(JarArchiver.java:501)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.zipFile(AbstractZipArchiver.java:561)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.addResources(AbstractZipArchiver.java:421)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.createArchiveMain(AbstractZipArchiver.java:331)
	at org.codehaus.plexus.archiver.zip.AbstractZipArchiver.execute(AbstractZipArchiver.java:234)
	at org.codehaus.plexus.archiver.AbstractArchiver.createArchive(AbstractArchiver.java:987)
	at org.apache.maven.plugins.assembly.archive.archiver.AssemblyProxyArchiver.createArchive(AssemblyProxyArchiver.java:445)
	at org.apache.maven.plugins.assembly.archive.DefaultAssemblyArchiver.createArchive(DefaultAssemblyArchiver.java:184)
	at org.apache.maven.plugins.assembly.mojos.AbstractAssemblyMojo.execute(AbstractAssemblyMojo.java:478)
	at org.apache.maven.plugins.assembly.mojos.SingleAssemblyMojo.execute(SingleAssemblyMojo.java:61)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:956)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:290)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:194)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

I started looking for leaks and found that stream opened in ConcurrentJarCreator.createEntry is closed only when an exception occurs, but not after successful execution.

[plamentotev]

Hi,

Thank you for reporting this.

Could you please provide a project that reproduces the issue.

While I don't exclude the possibility of leaks, createEntry does not close the stream on normal completion because the stream is used by the caller and it is the caller responsibility to make sure it is closed.

[piotrp]

After looking at it a bit more it may be an issue with how maven-assembly-plugin uses plexus-archiver to create fat JARs. I managed to reproduce it with an empty project: https://gist.github.com/Crack/01cbbc4561f75616a63242f30677bf54
To fail it must be built on Windows.

To find the stacktrace I provided above:

1. Download http://search.maven.org/remotecontent?filepath=org/kohsuke/file-leak-detector/1.11/file-leak-detector-1.11-jar-with-dependencies.jar , save as fld.jar next to pom.xml from my project.
2. Download https://github.com/Zlika/reproducible-build-maven-plugin/archive/v0.5.2.tar.gz , introduce sleep on exception to let maven linger after failure by changing DefaultZipStripper.strip to:

    @Override
    public void strip(final File zip, final File stripped) throws IOException
    {
        this.stripper.strip(zip, stripped);
        if (this.overwrite)
        {
            try {
                Files.move(stripped.toPath(), zip.toPath(), StandardCopyOption.REPLACE_EXISTING);
            } catch (FileSystemException e) {
                System.out.println("CATCH");
                try {Thread.sleep(1000 * 60 * 5);} catch (InterruptedException e1) {}
            }
        }
    }

and install it locally.
3. Build my project:

export MAVEN_OPTS=-javaagent:fld.jar=http=19999
mvn install

4. Go to http://localhost:19999 to get information on all open files

[plamentotev]

Thanks for the project and the detailed explanation.

I've found two leaks:

• PlexusIoZipFileResourceCollection does not close the URLClassLoader
• JarArchiver opens an InputStream but does not closes it.

I'll prepare fix tomorrow.