Skip to content

BUG: pandas.read_fwf doesn't correctly works with skiprows as list and nrows #44021

Closed
@anmyachev

Description

@anmyachev

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the master branch of pandas.

Reproducible Example

import io
import pandas

fwf_data = (
    "a       b           c          d\n"
    "id8141  360.242940  149.910199 11950.7\n"
    "id1594  444.953632  166.985655 11788.4\n"
    "id1849  364.136849  183.628767 11806.2\n"
    "id1230  413.836124  184.375703 11916.8\n"
    "id1948  502.953953  173.237159 12468.3\n"
    "id8141  360.242940  149.910199 11950.7\n"
    "id1594  444.953632  166.985655 11788.4\n"
    "id1849  364.136849  183.628767 11806.2\n"
)

df1 = pandas.read_fwf(filepath_or_buffer=io.StringIO(fwf_data), skiprows=[2, 5])
print(df1)
'''Output
        a           b           c        d
0  id8141  360.242940  149.910199  11950.7
1  id1849  364.136849  183.628767  11806.2
2  id1230  413.836124  184.375703  11916.8
3  id8141  360.242940  149.910199  11950.7
4  id1594  444.953632  166.985655  11788.4
5  id1849  364.136849  183.628767  11806.2
'''

df2 = pandas.read_fwf(filepath_or_buffer=io.StringIO(fwf_data), skiprows=[2, 5], nrows=7)
print(df2)
'''Output
        a           b           c        d
0  id8141  360.242940  149.910199  11950.7
1  id1849  364.136849  183.628767  11806.2
2  id1230  413.836124  184.375703  11916.8
3  id1948  502.953953  173.237159  12468.3
4  id8141  360.242940  149.910199  11950.7
5  id1594  444.953632  166.985655  11788.4
6  id1849  364.136849  183.628767  11806.2
'''

Issue Description

For df2, not all specified lines are skipped.

Expected Behavior

df2 = pandas.read_fwf(filepath_or_buffer=io.StringIO(fwf_data), skiprows=[2, 5], nrows=7)
print(df2)
'''Expected output
        a           b           c        d
0  id8141  360.242940  149.910199  11950.7
1  id1849  364.136849  183.628767  11806.2
2  id1230  413.836124  184.375703  11916.8
3  id8141  360.242940  149.910199  11950.7
4  id1594  444.953632  166.985655  11788.4
'''

Installed Versions

INSTALLED VERSIONS

commit : 73c6825
python : 3.8.10.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19042
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 12, GenuineIntel
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : English_United States.1252

pandas : 1.3.3
numpy : 1.21.1
pytz : 2020.1
dateutil : 2.8.2
pip : 21.2.2
setuptools : 57.4.0
Cython : None
pytest : 6.2.4
hypothesis : None
sphinx : 4.1.2
blosc : None
feather : 0.4.1
xlsxwriter : None
lxml.etree : 4.6.3
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.0.1
IPython : 7.27.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : 2021.07.0
fastparquet : None
gcsfs : None
matplotlib : 3.2.2
numexpr : 2.7.3
odfpy : None
openpyxl : 3.0.7
pandas_gbq : 0.15.0
pyarrow : 3.0.0
pyxlsb : None
s3fs : 0.4.2
scipy : 1.7.0
sqlalchemy : 1.4.22
tables : 3.6.1
tabulate : None
xarray : 0.19.0
xlrd : 2.0.1
xlwt : None
numba : None

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions