x/image/bmp: out of memory

[dvyukov]

Run the following program on the following input:

package main

import (
    "bytes"
    "io/ioutil"
    "os"
    "golang.org/x/image/bmp"
)

func main() {
    data, _ := ioutil.ReadFile(os.Args[1])
    img, err := bmp.Decode(bytes.NewReader(data))
    if err != nil {
        return
    }
    var w bytes.Buffer
    err = bmp.Encode(&w, img)
    if err != nil {
        panic(err)
    }
}

https://drive.google.com/file/d/0B20Uwp8Hs1oCTzFrMndWMjNtTHM/view?usp=sharing

It crashes as:

fatal error: runtime: out of memory

goroutine 1 [running]:
runtime.systemstack_switch()
    src/runtime/asm_amd64.s:216 fp=0xc208041b58 sp=0xc208041b50
runtime.mallocgc(0x1000000300, 0x4b27c0, 0x1, 0x0)
    src/runtime/malloc.go:625 +0x8f1 fp=0xc208041c28 sp=0xc208041b58
runtime.newarray(0x4b27c0, 0x1000000300, 0x1)
    src/runtime/malloc.go:736 +0xce fp=0xc208041c68 sp=0xc208041c28
runtime.makeslice(0x4ac5a0, 0x1000000300, 0x1000000300, 0x0, 0x0, 0x0)
    src/runtime/slice.go:32 +0x178 fp=0xc208041cb8 sp=0xc208041c68
golang.org/x/image/bmp.decodeNRGBA(0x7f87732092d8, 0xc208014420, 0x7f87732090d0, 0xc20802e020, 0x10, 0x4000000c, 0x1, 0x0, 0x0, 0x0, ...)
    src/golang.org/x/image/bmp/reader.go:83 +0x185 fp=0xc208041e30 sp=0xc208041cb8
golang.org/x/image/bmp.Decode(0x7f87732092d8, 0xc208014420, 0x0, 0x0, 0x0, 0x0)
    src/golang.org/x/image/bmp/reader.go:114 +0x1be fp=0xc208041ed0 sp=0xc208041e30
main.main()
    /tmp/bmp.go:12 +0x132 fp=0xc208041f90 sp=0xc208041ed0

I am on commit 65a798f031fd31a65574938bed2ec44c2bcba496

[chai2010]

The test bmp size is 16x1073741836, bpp is 32.
This cause the image.NewNRGBA(image.Rect(0, 0, c.Width, c.Height)) panic.
I think this is not a bug.

[dvyukov]

My understanding is that BMP is not compressed format. So decoding must try to allocate 16x1073741836x4 only if input size is larger than that. In this case input is few bytes, so an attempt to allocate 64GB of memory does look like a bug.
FWIW, encoding/gob has such logic that detects when declared data size is larger than input.
Ideally we have such logic for compressed image formats as well. But it is not directly possible. Probably we can introduce MaxMem option for decoding of compressed formats, i.e. "decode this blob using no more than 100MB of memory".

[minux]

BMP does support RLE compression. I'm not sure x/image/bmp supports RLE though.

[chai2010]

@minux x/image/bmp does not supports any compression.

[rsc]

@dvyukov, use Go 1.5 milestone for bugs in released code. Subrepos are not released. The code in x/image/bmp and Go 1.5 are completely unrelated. The milestone for subrepos is Unreleased.

[nigeltao]

BMP is not a compressed format, but the structure of the file format is first header, then payload. The image dimensions are in the header, and the decoder allocates a pixel buffer after the header is processed but before the payload is processed, because processing the payload involves writing the pixels to somewhere: the pixel buffer.

Decode takes an io.Reader, not an io.ReadSeeker, so at the time it allocates the pixel buffer, it cannot know whether or not there's 'enough input' remaining. Conversely, we may be decoding an image downloaded from a socket, where we cannot know how much input is remaining. So I think this is all WAI.

For decoding images in general, with compression, knowing the total file size doesn't necessarily mean that you know whether or not there's enough pixel data. LZW-style length-difference or RLE encoding means that very large uncompressed data can compress very, very well.

That the various image libraries should reject an 16x1073741836 early, checking some sort of MaxMem option before allocating a buffer, is issue #5050.