BUG: read_csv segfault

[dsm054]

Came across this when trying to parse a poorly-formatted csv file and got the number of columns wrong. Mostly it segfaults; sometimes it produces a broken last row.

>>> import pandas as pd
>>> from StringIO import StringIO
>>> pd.__version__
'0.13.0rc1-43-g4f9fefc'
>>> 
>>> f = StringIO('1,1,1,1,0\n'*2 + '\n'*2)
>>> df = pd.read_csv(f,names=list("abcd"))
Segmentation fault (core dumped)

When it doesn't segfault it tends to produce

      a   b   c    d
 1    1   1   1    0
 1    1   1   1    0
NaN NaN NaN NaN  NaN
NaN NaN NaN NaN  [unicode box: 0 0 0 1]

[guyrt]

This is a C parser problem.

Python parser appears to correctly handle names of lengths 1 and up.
C parser chokes on len(names) in [1, 2] and also 3 or 4 if any names are strings.

[jreback]

gr8

care to submit a PR?

[guyrt]

Yeah, I'm working on it. Isolated problem (#tokens < expected causing read from out of bounds) and working on fix.

[jreback]

@guyrt progress on this?

[jreback]

@dsm054 you want to do a PR for this?

@guyrt ?

[guyrt]

My fix hit a road block. I'll try to return to this once I free up some time.

[jreback]

@guyrt ? or cc @mcwitt

[jreback]

cc @mcwitt
cc @guyrt

[cstenac]

Hi,

Not totally sure, but I think this one is related too:

import pandas as pd, StringIO

pd.read_csv(StringIO.StringIO("a,b\nc"), skiprows=0, names = ["a"])

Crashes about 50-80% of time, gives some NaN in other cases.

Reproduced on OS X and Ubuntu, Pandas 0.13.1 and 0.14.1

[guyrt]

Reproduced @cstenac 's example on windows as well. Note first example, where we just loaded in whatever from memory. That's concerning.

 In [3]: pandas.read_csv(StringIO.StringIO("a,b\nc"), skiprows=0, names = ["a"])
Out[3]:
    a
a   b
c  2♫

In [4]: pandas.read_csv(StringIO.StringIO("a,b\nc"), skiprows=0, names = ["a"])
<failed>

That does appear to be same issue: the C engine computes an expected size and reads until it fills that size.

[tommyengstrom]

If there are two implementations of this and one is broken, shouldn't the other one be the default?