Skip to content

rustdoc-json: Output is is zstd compressed twice. #2833

Closed
@aDotInTheVoid

Description

@aDotInTheVoid

The new rustdoc json enpoints (https://docs.rs/about/rustdoc-json) return files that need to be decompressed twice before getting the json:

alona@Ashtabula:~/tmp/docsrszstd$ curl --location  https://docs.rs/crate/clap/latest/json -o ./docs-rs-output
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 68106  100 68106    0     0   241k      0 --:--:-- --:--:-- --:--:--  241k
alona@Ashtabula:~/tmp/docsrszstd$ file ./docs-rs-output
./docs-rs-output: Zstandard compressed data (v0.8+), Dictionary ID: None
alona@Ashtabula:~/tmp/docsrszstd$ unzstd ./docs-rs-output -o decompressed-once
./docs-rs-output    : 68097 bytes
alona@Ashtabula:~/tmp/docsrszstd$ file ./decompressed-once
./decompressed-once: Zstandard compressed data (v0.8+), Dictionary ID: None
alona@Ashtabula:~/tmp/docsrszstd$ unzstd ./decompressed-once -o decompressed-twice
./decompressed-once : 422580 bytes
alona@Ashtabula:~/tmp/docsrszstd$ file ./decompressed-twice
./decompressed-twice: JSON text data
alona@Ashtabula:~/tmp/docsrszstd$ du -sh * | sort -h
68K     decompressed-once
68K     docs-rs-output
416K    decompressed-twice

This doesn't save any space, but makes using this more cumbersome and slower.

The files served from docs.rs should be the json compressed once, not twice.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions