Skip to content

pd.read_html() convert <br> to space #29528

Closed
@abitrolly

Description

@abitrolly

Code Sample, a copy-pastable example if possible

import pandas as pd
pd.read_html('http://vybary2019.by/regions/49.html',header=0)[0]

Problem description

In original web page the first cell of table contains three separate words.

image

In pandas table the first cell contains only two words, with first word being the join of the former two first.

image

[why the current behaviour is a problem]
Because during pandas parsing there is a data loss that is non-trivial to fix.

The reason for this behaviour is the presence of <br> tag between first two words in original content.

image

<br> should be converted to space or at least to a new line when converting to text representation.

I found the similar reference in #14608 (comment) but no separate GitHub issue.

Expected Output

image

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit : None python : 3.6.8.final.0 python-bits : 64 OS : Linux OS-release : 4.14.137+ machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : en_US.UTF-8

pandas : 0.25.3
numpy : 1.17.3
pytz : 2018.9
dateutil : 2.6.1
pip : 19.3.1
setuptools : 41.4.0
Cython : 0.29.13
pytest : 3.6.4
hypothesis : None
sphinx : 1.8.5
blosc : None
feather : 0.4.0
xlsxwriter : None
lxml.etree : 4.2.6
html5lib : 1.0.1
pymysql : None
psycopg2 : 2.7.6.1 (dt dec pq3 ext lo64)
jinja2 : 2.10.3
IPython : 5.5.0
pandas_datareader: 0.7.4
bs4 : 4.6.3
bottleneck : 1.2.1
fastparquet : None
gcsfs : None
lxml.etree : 4.2.6
matplotlib : 3.1.1
numexpr : 2.7.0
odfpy : None
openpyxl : 2.5.9
pandas_gbq : 0.11.0
pyarrow : 0.14.1
pytables : None
s3fs : 0.3.5
scipy : 1.3.1
sqlalchemy : 1.3.10
tables : 3.4.4
xarray : 0.11.3
xlrd : 1.1.0
xlwt : 1.3.0
xlsxwriter : None

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugIO HTMLread_html, to_html, Styler.apply, Styler.applymap

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions