BUG: Crash using to_csv with QUOTE_NONE

[adamaaaa]

Using release 0.11.0 with Python 2.7.1, but it doesn't look like anything has changed in git since.

I can write DataFrames to a file using to_csv fine unless I pass the option: quoting=pandas.io.parsers.csv.QUOTE_NONE in which case I get exceptions of the form:

packages/pandas/core/format.py", line 1054, in _save_chunk
lib.write_csv_rows(self.data, ix, self.nlevels, self.cols, self.writer)
File "lib.pyx", line 832, in pandas.lib.write_csv_rows (pandas/lib.c:13466)
_csv.Error: need to escape, but no escapechar set

A short search reveals links like http://www.velocityreviews.com/forums/t612166-how-can-i-use-quotes-without-escaping-them-using-csv.html implying that due to some csv oddities we should be setting something like quotechar=None when calling csv.writer.

I tried modifying pandas/core/format.py to add quotechar=None to the call to csv.writer and this does indeed seem to fix the problem. Unfortunately I don't have everything set up to build from git so I can't submit a patch.

Thanks.

[jreback]

seems to work fine in 0.11 (fyi QUOTE_MINIMAL is the default), what exactly are you trying to format

pls post your code

In [1]: pd.__version__
Out[1]: '0.11.0'

In [1]: df = DataFrame({'A': [1, 2, 3], 'B': ['foo', 'bar', 'baz']})

In [2]: buf = StringIO()

In [4]: import csv

In [8]: df.to_csv(buf, index=False, quoting=csv.QUOTE_NONE)

In [9]: buf.getvalue()
Out[9]: 'A,B\n1,foo\n2,bar\n3,baz\n'

[adamaaaa]

I tried to get you a minimal reproduction of the issue. It seems to occur if a string in the DataFrame contains the quote character " but quoting is set to QUOTE_NONE:

import pandas
import StringIO
df = pandas.DataFrame({'A': ['hello', '{"hello"}']})
buf = StringIO.StringIO()
import csv
df.to_csv(buf, index=False, quoting=csv.QUOTE_NONE)

Running this in ipython causes a crash for me: "Error: need to escape, but no escapechar set"

And by way of validation:

In [7]: pandas.version
Out[7]: '0.11.0'

Thanks,

Adam

[jreback]

thanks for the report....and good example...marked as a bug for 0.13

[jtratner]

@adamaaaa this should work now.

[agora123]

Hello adamaaaa, I am having some issues in my code and I was hoping you can point me in the right direction.
I have this code giving me an error: '_csv.Error: need to escape, but no escapechar set'
I have tried to set quotechar = None, but I get this 'TypeError: quotechar must be set if quoting enabled'
Can you tell me what I can possibly modify to get this problem. I need this data field : "LTE,eHRPD" to go into csv just like shown here, and I am struggling to make it work. Thank you!

import pandas
import csv

df = pandas.DataFrame({'A': ['1.0627', '0625', '"LTE,eHRPD"']})
print(df)

df.to_csv(quoting=csv.QUOTE_NONE)