Skip to content

BUG: read_fwf() does not obey skip_blank_lines #37758

Closed
@dnicolodi

Description

@dnicolodi

The read_fwf() function does not obey the skip_blank_lines argument:

data = io.StringIO("""\
 1 2 3
 4 5 6
""")

d = pandas.read_fwf(data, header=None, colspec=[(0, 2), (2, 4), (4, 6)])

results in:

   0  1  2
0  1  2  3
1  4  5  6

while

data = io.StringIO("""\
 1 2 3

 4 5 6
""")

d = pandas.read_fwf(data, header=None, colspec=[(0, 2), (2, 4), (4, 6)])

results in

     0    1    2
0  1.0  2.0  3.0
1  NaN  NaN  NaN
2  4.0  5.0  6.0

The skip_blank_lines argument defaults to True, but also explicitly setting it results in the same behavior. The read_fwf() function documentation says that it takes all arguments supported by TextFileReader thus this is unexpected.

I have tested this with Pandas 1.1.3 as distributed by conda, however, the relevant code does not seem to have changed on master.

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugNeeds TriageIssue that has not been reviewed by a pandas team member

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions