Reading large file from ReactiveGridFsTemplate causes a stackoverflow and the code to hang [DATAMONGO-2393]

[spring-projects-issues]

Nick Stolwijk opened DATAMONGO-2393 and commented

When I try to read a bigger file from GridFs with the ReactiveGridFsTemplate the test fails and hangs with a StackOverflow exception.

Drop-in test for ReactiveGridFsTemplateTests.java

@Test // DATAMONGO-2393
public void getResourceShouldRetrieveLargeContentByIdentity() throws IOException {
   byte[] content = new byte[64_000];
   ThreadLocalRandom.current().nextBytes(content);
   AsyncInputStream upload = AsyncStreamHelper.toAsyncInputStream(new ByteArrayInputStream(content));

   ObjectId reference = operations.store(upload, "foo.xml", null, null).block();

   operations.findOne(query(where("_id").is(reference))).flatMap(operations::getResource)
         .flatMapMany(ReactiveGridFsResource::getDownloadStream) //
         .transform(DataBufferUtils::join) //
         .as(StepVerifier::create) //
         .consumeNextWith(dataBuffer -> {

            byte[] actual = new byte[dataBuffer.readableByteCount()];
            dataBuffer.read(actual);

            assertThat(actual).isEqualTo(content);
         }) //
         .verifyComplete();
}

---

Affects: 2.2 GA (Moore)

Reference URL: https://stackoverflow.com/questions/58335826/stackoverflowerror-while-reading-file-from-reactive-gridfs

Referenced from: pull request #799

Backported to: 2.2.1 (Moore SR1)

[spring-projects-issues]

Mark Paluch commented

The buffer size in conjunction with the number of roundtrips leads to the stack overflow. Reactive programming is prone to stack overflows due to long operator chains.

The default chunk size is 256 bytes which is controlled by DataBufferFactory. As workaround, construct ReactiveGridFsTemplate using a DataBufferFactory that uses larger buffers (4kb, 8kb) to reduce the number of recursive reads

[spring-projects-issues]

Nick Stolwijk commented

I thought I had been playing with de DataBufferFactory and its initial size. Due to my hack for DATAMONGO-2393, subclassing the ReactiveGridFsTemplate it never used the bigger databuffers but always used the default databuffer. This problem exists between my keyboard and my chair. ;-)

public class HackedReactiveGridFsTemplate extends ReactiveGridFsTemplate {
    private final DataBufferFactory dataBufferFactory = new DefaultDataBufferFactory();

    public HackedReactiveGridFsTemplate(ReactiveMongoDatabaseFactory dbFactory, MongoConverter converter, String bucket, int defaultInitialCapacity) {
        super(new DefaultDataBufferFactory(false, defaultInitialCapacity), dbFactory, converter, bucket);
    }

    public Mono<ReactiveGridFsResource> getResource(GridFSFile file) {
        Assert.notNull(file, "GridFSFile must not be null!");
        return Mono.fromSupplier(() -> {
            GridFSDownloadStream stream = this.getGridFs().openDownloadStream(file.getId());
            return new ReactiveGridFsResource(file, BinaryStreamAdapters.toPublisher(stream, this.dataBufferFactory));
        });
    }
}

[spring-projects-issues]

Nick Stolwijk commented

Should the ReactiveGridFsTemplate use a bigger databuffer by default? 

[spring-projects-issues]

Mark Paluch commented

Let's keep the issue at critical priority since a larger buffer softens the impact but it does not fix the stack overflow issue.

For now, it would make sense to use a bigger read chunk size and ideally a chunk size that can be configured externally. A default of 4kb will work for files up to 1MB size. We need to investigate how we can rewrite the code from recursive processing to drain loop processing

[spring-projects-issues]

Mark Paluch commented

I pushed a change to our feature branch for this ticket and we have a branch build that has produced version 2.3.0.DATAMONGO-2393-SNAPSHOT. Care to upgrade and see whether the change fixes your issue?

Make sure to include our snapshot repository:

<dependencies>
  <dependency>
    <groupId>org.springframework.data</groupId>
    <artifactId>spring-data-mongodb</artifactId>
    <version>2.3.0.DATAMONGO-2393-SNAPSHOT</version>
  </dependency>
</dependencies>

<repositories>
  <repository>
    <id>spring-libs-snapshot</id>
    <url>https://repo.spring.io/libs-snapshot</url>
    <snapshots>
      <enabled>true</enabled>
    </snapshots>
  </repository>
</repositories>

[spring-projects-issues]

Mark Paluch commented

Nick Stolwijk did you get a chance to check whether the fix addresses your issue?