Description
The RequestBody.fromByteBuffer() (and AsyncRequsetBody.fromByteBuffer()) method reads the contents of the ByteBuffer starting from position 0 to the ByteBuffer's limit. This is incorrect usage of ByteBuffers.
Expected Behavior
When passing RequestBody.fromByteBuffer() a ByteBuffer with a non-zero position, the contents of the request body should be the contents of the ByteBuffer from it's position to it's limit.
Current Behavior
When passing RequestBody.fromByteBuffer() a ByteBuffer with a non-zero position, the contents of the request body are the contents of the ByteBuffer from position zero to it's limit.
Possible Solution
RequestBody.fromByteBuffer() is currently defined as:
public static RequestBody fromByteBuffer(ByteBuffer byteBuffer) {
return fromBytesDirect(BinaryUtils.copyAllBytesFrom(byteBuffer));
}
The bug can be fixed by changing it to:
public static RequestBody fromByteBuffer(ByteBuffer byteBuffer) {
return fromBytesDirect(BinaryUtils.copyBytesFrom(byteBuffer));
}
Unfortunately, this is not a backward compatible change, if people are relying on this behavior, changing it would break them.
There are a couple of easy workarounds.
// fromByteBuffer makes a copy anyways, just read the contents yourself and use fromBytes
byte[] bytes = new byte[byteBuffer.remaining()];
byteBuffer.get(bytes);
RequestBody requsetBody = RequsetBody.fromBytes(bytes);
// slice() creates a new ByteBuffer with position = 0
RequestBody requestBody = RequestBody.fromByteBuffer(byteBuffer.slice());
Steps to Reproduce (for bugs)
JUnit 5 unit test:
@Test
void requestBody_fromByteBuffer_positionAtOne() throws IOException {
byte[] expected = "this is a string".getBytes();
// create a byte buffer with expected as it's contents, starting at position 1
ByteBuffer byteBuffer = ByteBuffer.allocate(expected.length + 1);
byteBuffer.position(1);
byteBuffer.duplicate().put(expected);
// double check it's correct
assert byteBuffer.equals(ByteBuffer.wrap(expected));
RequestBody body = RequestBody.fromByteBuffer(byteBuffer);
try (InputStream is = body.contentStreamProvider().newStream()) {
byte[] allBytes = is.readAllBytes();
assert Arrays.equals(allBytes, expected);
}
}
Context
I ran into this when trying to do a multi-part upload to S3 from sections of the same ByteBuffer. Since the position is non-zero for all but the first part, each of the parts contained all the previous parts, making the uploaded object incorrect.
Your Environment
- AWS Java SDK version used: 2.10.22, I think it affects all 2.x versions
- JDK version used: 13, but it is relevant to all versions.
- Operating System and version: Windows 10, should affect all operating systems.