[Bug] Storage GetBytesAsync crash for gzip-encoded file

[vnorilo]

[REQUIRED] Please fill in the following fields:

• Unity editor version: 2019.4.14f1
• Firebase Unity SDK version: 6.15.2 (also tried 7.1.0)
• Source you installed the SDK: Unity Package Manager
• Problematic Firebase Component: Storage
• Other Firebase Components in use: Auth, Database, RemoteConfig, Messaging
• Additional SDKs you are using: none
• Platform you are using the Unity editor on: Mac
• Platform you are targeting: Android
• Scripting Runtime:IL2CPP

[REQUIRED] Please describe the issue here:

Android client crashes when calling GetBytesAsync for a Storage reference that points to a gzip-encoded file. In editor, the method works fine and provides the compressed stream bytes.

Steps to reproduce:

What happened? How can we make the problem occur?

• Upload and gzip-encode a file to Firebase storage with e.g. gsutil cp -Z <local-file> <gs://remote-file>
• Call FirebaseStorage.DefaultInstance.GetReference(<remote-file>).GetBytesAsync(1000000)

On Android device, the call will always crash, even when the file is much, much smaller than the provided limit:

2021/03/09 10:16:11.359 6904 7586 Error StorageException StorageException has occurred.
2021/03/09 10:16:11.359 6904 7586 Error StorageException An unknown error occurred, please check the HTTP result code and inner exception for server response.
2021/03/09 10:16:11.359 6904 7586 Error StorageException  Code: -13000 HttpResult: 200
2021/03/09 10:16:11.359 6904 7586 Error StorageException The maximum allowed buffer size was exceeded.
2021/03/09 10:16:11.359 6904 7586 Error StorageException java.lang.IndexOutOfBoundsException: The maximum allowed buffer size was exceeded.
2021/03/09 10:16:11.359 6904 7586 Error StorageException 	at com.google.firebase.storage.internal.cpp.CppByteDownloader.doInBackground(CppByteDownloader.java:55)
2021/03/09 10:16:11.359 6904 7586 Error StorageException 	at com.google.firebase.storage.StreamDownloadTask.run(com.google.firebase:firebase-storage@@19.1.1:179)
2021/03/09 10:16:11.359 6904 7586 Error StorageException 	at com.google.firebase.storage.StorageTask.lambda$getRunnable$7(com.google.firebase:firebase-storage@@19.1.1:1072)
2021/03/09 10:16:11.359 6904 7586 Error StorageException 	at com.google.firebase.storage.StorageTask$$Lambda$12.run(Unknown Source:2)
2021/03/09 10:16:11.359 6904 7586 Error StorageException 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
2021/03/09 10:16:11.359 6904 7586 Error StorageException 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
2021/03/09 10:16:11.359 6904 7586 Error StorageException 	at java.lang.Thread.run(Thread.java:784)

My suspicion is that the Android client allocates the byte buffer based on file size in storage. However, the fetch fails to send accepts: gzip to the server, which should cause automatic decompression in Google Cloud Storage and the resulting uncompressed stream is too big to fit the buffer, whose size may be based on the compressed size.

[vimanyu]

Hi,
Thanks for creating this issue. I just tried this real quick in our Unity storage quickstart example and could not reproduce it.
Here are the steps I tried,

• Unity 2020.1.7f1 and FirebaseStorage 7.1.0
• Uploaded "test.txt", "test.zip" and "test.gz" (created with gzip -c test.txt > test.gz) to storage bucket via the Firebase console web interface.
• Downloaded bytes for each of them in Editor and on my Android 8.0 device and I didn't see any crashes.

Could you verify if our quickstart example crashes for you on Android too? If yes, could you try enabling debug logging and running once again,

Firebase.LogLevel logLevel = Firebase.LogLevel.Debug;

[vnorilo]

Thank you for the reply! Did you also set the content encoding metadata to gzip for the gz stream? To be clear, it is the (correct) metadata that causes SDK to crash, there is no problem per se in storing gzipped streams without encoding metadata. I'm not sure if that can be done via the console web interface, hence I used gsutil cp -Z, which is the documented way of uploading gzip-encoded content to cloud storage. I will check the quickstart next week. Thanks again!

[vimanyu]

I see. I didn't try setting the content encoding metadata though the file type is identified as "application/x-gzip
" in the console. It will be quicker for us to debug if we can reproduce this easily in our quickstarts.