pandas.DataFrame.apply runs twice if an exception is raised

[rdelassus]

Code Sample, a copy-pastable example if possible

import pandas as pd

# create a dataframe containing a nan value
inputs = [{'a': i} for i in range(2)]
inputs.append({'a':None})
df = pd.DataFrame(inputs)

def buggy_function(row):
    if not pd.isnull(row.a):
        print(f'foo {row.a}')
    else:
        raise Exception()

df.apply(buggy_function, axis=1)

Problem description

Before the exception is raised, buggy_function is applied a second time. Not just on the first row, as described in the documentation, but to all rows until the exception.

Actual output:
foo 0.0
foo 1.0
foo 0.0
foo 1.0
-> exception raised

Expected Output

foo 0.0
foo 1.0
-> exception raised

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]

INSTALLED VERSIONS

commit: None
python: 3.7.1.final.0
python-bits: 64
OS: Linux
OS-release: 4.15.0-34-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.4
pytest: None
pip: 18.1
setuptools: 40.6.3
Cython: None
numpy: 1.15.4
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 7.2.0
sphinx: None
patsy: 0.5.1
dateutil: 2.7.5
pytz: 2018.9
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 3.0.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: 1.2.16
pymysql: None
psycopg2: 2.7.6.1 (dt dec pq3 ext lo64)
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

[WillAyd]

I don't think this is a bug. Generally raising an Exception in a UDF can interfere with the error handling of pandas (or any application)

[TomAugspurger]

This is a documented shortcoming of the current DataFrame.apply implementation.

#24748 did some work to reduce the cases where the function is run twice, but your example (returning None) isn't one of them.