ENH/DOC/CLN: Document arguments and reconcile C and Python engines for read_csv

[kawochen]

Known differences between Python & C engines

Update here

Features supported in the Python engine only

• skipfooter / skip_footer (API: skipfooter or skip_footer? read_csv can't seem to decide #13349) - num of lines at the bottom of the file to skip
• sniffing (sep=None) - deduce the sep ENH: add read_csv sniffing (sep=None) for C engine #9645
• regex sep - regular expression/multicharacter seperator

Features supported in the C engine only

• dtype - specify dtype for providing dtype or {column_name: dtype} (related as this is a conflicting option: read_csv dtype argument not working when there is a footer #5232) (done in API: add dtype= option to python parser #14295)
• warn_bad_lines - issue a warnings for each bad line (ENH: Support malformed row handling in Python engine #15925)
• error_bad_lines - if False, drop bad lines instead of raising (ENH: Support malformed row handling in Python engine #15925)
• lineterminator - specify the line terminating character
• C engine accepts float for nrows but Python engine raises read_csv python engine errors #10476 (closed by BUG: Properly validate and parse nrows in read_csv #13275)
• decimal option, ENH: support decimal option in PythonParser #12933 (closed by ENH: support decimal option in PythonParser #12933 #13189)
• delim_whitespace ENH: Python parser now accepts delim_whitespace=True #12958
• na_filter ENH: add support for na_filter in Python engine #13321
• float_precision, documented here and here, DOC: actually document float_precision in read_csv #13377

  In C engine only (but undocumented)

  • low_memory (PR DOC: low_memory in read_csv #13293)

marked as internal on C engine only (maybe be a bit louder about this in the internal code)

• buffer_lines DEPR, DOC: Deprecate buffer_lines in read_csv #13360

Undocumented arguments to read_csv

• doublequote DOC: document doublequote in read_csv #13368
• compact_ints API: Deprecate compact_ints and use_unsigned in read_csv #13323
• use_unsigned API: Deprecate compact_ints and use_unsigned in read_csv #13323
• as_recarray #(DEPR: Deprecate as_recarray in read_csv #13373)
• memory_map, IO: memory_map kw in read_csv #7477, DOC, ENH: Support memory_map for Python engine #13381

Differences

• validity of names and its length with respect to usecols API/DOC: Specification for names parameter in read_csv #16469
• different handling of na_values when converters is also present. Inconsistent Handling of na_values and converters in read_csv #13302
• different handling of columns aggregated to create date columns API: Inconsistent handling of columns aggregated to create date columns #23845

[kawochen]

I think error_bad_lines and warn_bad_lines should be one argument. They seem too intertwined now.

[jreback]

thanks for the list @kawochen. There are some issues which are relevant for some of the points. Can you link them when you have a chance (put next to the check boxes)

[gfyoung]

@kawochen : add as_recarray to the undocumented options. This one returns a np.recarray instead of a DataFrame of the data if set to True.

[gfyoung]

nrows issue has been fixed! Please check it off @jreback @kawochen