NotImplementedError: method backfill not yet implemented for IntervalIndex

[ValueRaider]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

import pandas as pd
dt = pd.Timestamp.now()
h = pd.Timedelta("1h")

# 3 simple interval ranges, with a big gap between 2nd and 3rd
intervals = pd.IntervalIndex.from_arrays([dt, dt+h, dt+10*h], [dt+h, dt+2*h, dt+11*h], closed="left")

intervals.get_indexer([dt + 5*h], method="bfill")

Expectation: the timestamp in get_indexer is mapped to the 3rd interval range, but instead this happens:

NotImplementedError: method backfill not yet implemented for IntervalIndex

[hamedgibago]

Hi, would you explain expected result?

[ValueRaider]

The result should be [2]. Behaviour similar to pd.Index.get_indexer()

[hamedgibago]

@ValueRaider When you use it intervals.get_indexer([dt + 5*h], method="bfill") without method="bfill", it returns -1. So I think the error is not because of method="bfill" . dt+5*h is not in none of these three ranges. As you can see first one is now and one hour later, second is one hour later and 2 hour later and third is 10 hour later and 11 hour later. So 5 hours later is not in the range.

As a sample suppose that now is 11 A.M. Ranges are below:
[11,12] [12,13] [21,22]
So you when you calling intervals.get_indexer([dt + 5*h] it wants to get index of 16 which is not in none of ranges.

[ValueRaider]

So I think the error is not because of method="bfill" . dt+5*h is not in none of these three ranges

Wrong. Read the Pandas documentation for Index.get_indexer:

backfill / bfill: use NEXT index value if no exact match

So bfill should return the next range that includes value.

[hamedgibago]

Ok thats correct. Do you have another working example with other data? Or it's not totally implemented?

[ValueRaider]

NotImplementedError: method backfill not yet implemented for IntervalIndex

NotImplementedError = not implemented

[hamedgibago]

take

[hamedgibago]

I have implemented code with testing. All checks are ok but Doc Build and Upload workflow results error. In log there is nothing to find the root of error. would you help @mroeschke @jorisvandenbossche @jreback?
https://github.com/hamedgibago/pandas/actions/runs/4459701029/jobs/7837174530

waiting for workers...
In file included from /home/runner/micromamba/envs/test/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1948,
                 from /home/runner/micromamba/envs/test/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /home/runner/micromamba/envs/test/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:5,
                 from /home/runner/.cache/ipython/cython/_cython_magic_da5cd844e719547b088d83e81faa82ac.c:769:
/home/runner/micromamba/envs/test/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~
In file included from /home/runner/micromamba/envs/test/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1948,
                 from /home/runner/micromamba/envs/test/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /home/runner/micromamba/envs/test/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:5,
                 from /home/runner/.cache/ipython/cython/_cython_magic_14ff65f3705192578d563d86399e4562.c:770:
/home/runner/micromamba/envs/test/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~
/home/runner/micromamba/envs/test/lib/python3.8/site-packages/nbsphinx/__init__.py:1043: RuntimeWarning: You are using an unsupported version of pandoc (3.1.1).
Your version must be at least (1.12.1) but less than (3.0.0).
Refer to https://pandoc.org/installing.html.
Continuing with doubts...
  nbconvert.utils.pandoc.check_pandoc_version()
/home/runner/work/pandas/pandas/doc/source/whatsnew/v1.4.3.rst:86: WARNING: Cannot find contributors for range 'v1.4.2..v1.4.3': Cmd('git') failed due to: exit code(128)
  cmdline: git log --grep=Co-authored --pretty=%b v1.4.2..v1.4.3
  stderr: 'fatal: ambiguous argument 'v1.4.2..v1.4.3': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]''
/home/runner/work/pandas/pandas/doc/source/whatsnew/v1.4.4.rst:79: WARNING: Cannot find contributors for range 'v1.4.3..v1.4.4|HEAD': Cmd('git') failed due to: exit code(128)
  cmdline: git log --grep=Co-authored --pretty=%b v1.4.3..HEAD
  stderr: 'fatal: ambiguous argument 'v1.4.3..HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]''
/home/runner/work/pandas/pandas/doc/source/whatsnew/v1.5.0.rst:1299: WARNING: Cannot find contributors for range 'v1.4.4..v1.5.0': Cmd('git') failed due to: exit code(128)
  cmdline: git log --grep=Co-authored --pretty=%b v1.4.4..v1.5.0
  stderr: 'fatal: ambiguous argument 'v1.4.4..v1.5.0': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]''
/home/runner/work/pandas/pandas/doc/source/whatsnew/v1.5.1.rst:136: WARNING: Cannot find contributors for range 'v1.5.0..v1.5.1': Cmd('git') failed due to: exit code(128)
  cmdline: git log --grep=Co-authored --pretty=%b v1.5.0..v1.5.1
  stderr: 'fatal: ambiguous argument 'v1.5.0..v1.5.1': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]''
/home/runner/work/pandas/pandas/doc/source/whatsnew/v1.5.2.rst:60: WARNING: Cannot find contributors for range 'v1.5.1..v1.5.2': Cmd('git') failed due to: exit code(128)
  cmdline: git log --grep=Co-authored --pretty=%b v1.5.1..v1.5.2
  stderr: 'fatal: ambiguous argument 'v1.5.1..v1.5.2': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]''
/home/runner/work/pandas/pandas/doc/source/whatsnew/v1.5.3.rst:72: WARNING: Cannot find contributors for range 'v1.5.2..v1.5.3': Cmd('git') failed due to: exit code(128)
  cmdline: git log --grep=Co-authored --pretty=%b v1.5.2..v1.5.3
  stderr: 'fatal: ambiguous argument 'v1.5.2..v1.5.3': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]''
/home/runner/work/pandas/pandas/doc/source/whatsnew/v2.0.0.rst:1422: WARNING: Cannot find contributors for range 'v1.5.0rc0..v2.0.0|HEAD': Cmd('git') failed due to: exit code(128)
  cmdline: git log --grep=Co-authored --pretty=%b v1.5.0rc0..HEAD
  stderr: 'fatal: ambiguous argument 'v1.5.0rc0..HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]''
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done

[mroeschke]

For that build, doc build warnings are errors

2023-03-19T20:41:02.5284569Z /home/runner/work/pandas/pandas/doc/source/whatsnew/v1.4.3.rst:86: WARNING: Cannot find contributors for range 'v1.4.2..v1.4.3': Cmd('git') failed due to: exit code(128)
2023-03-19T20:41:02.5285193Z   cmdline: git log --grep=Co-authored --pretty=%b v1.4.2..v1.4.3
2023-03-19T20:41:02.5285708Z   stderr: 'fatal: ambiguous argument 'v1.4.2..v1.4.3': unknown revision or path not in the working tree.
2023-03-19T20:41:02.5286152Z Use '--' to separate paths from revisions, like this:
2023-03-19T20:41:02.5286519Z 'git <command> [<revision>...] -- [<file>...]''
2023-03-19T20:41:02.5287137Z /home/runner/work/pandas/pandas/doc/source/whatsnew/v1.4.4.rst:79: WARNING: Cannot find contributors for range 'v1.4.3..v1.4.4|HEAD': Cmd('git') failed due to: exit code(128)
2023-03-19T20:41:02.5289695Z   cmdline: git log --grep=Co-authored --pretty=%b v1.4.3..HEAD
2023-03-19T20:41:02.5290205Z   stderr: 'fatal: ambiguous argument 'v1.4.3..HEAD': unknown revision or path not in the working tree.
2023-03-19T20:41:02.5290642Z Use '--' to separate paths from revisions, like this:
2023-03-19T20:41:02.5291007Z 'git <command> [<revision>...] -- [<file>...]''
2023-03-19T20:41:02.5291629Z /home/runner/work/pandas/pandas/doc/source/whatsnew/v1.5.0.rst:1299: WARNING: Cannot find contributors for range 'v1.4.4..v1.5.0': Cmd('git') failed due to: exit code(128)
2023-03-19T20:41:02.5292164Z   cmdline: git log --grep=Co-authored --pretty=%b v1.4.4..v1.5.0
2023-03-19T20:41:02.5292666Z   stderr: 'fatal: ambiguous argument 'v1.4.4..v1.5.0': unknown revision or path not in the working tree.
2023-03-19T20:41:02.5293103Z Use '--' to separate paths from revisions, like this:
2023-03-19T20:41:02.5293465Z 'git <command> [<revision>...] -- [<file>...]''
2023-03-19T20:41:02.5294068Z /home/runner/work/pandas/pandas/doc/source/whatsnew/v1.5.1.rst:136: WARNING: Cannot find contributors for range 'v1.5.0..v1.5.1': Cmd('git') failed due to: exit code(128)
2023-03-19T20:41:02.5294600Z   cmdline: git log --grep=Co-authored --pretty=%b v1.5.0..v1.5.1
2023-03-19T20:41:02.5295100Z   stderr: 'fatal: ambiguous argument 'v1.5.0..v1.5.1': unknown revision or path not in the working tree.
2023-03-19T20:41:02.5295537Z Use '--' to separate paths from revisions, like this:
2023-03-19T20:41:02.5295896Z 'git <command> [<revision>...] -- [<file>...]''
2023-03-19T20:41:02.5297711Z /home/runner/work/pandas/pandas/doc/source/whatsnew/v1.5.2.rst:60: WARNING: Cannot find contributors for range 'v1.5.1..v1.5.2': Cmd('git') failed due to: exit code(128)
2023-03-19T20:41:02.5298246Z   cmdline: git log --grep=Co-authored --pretty=%b v1.5.1..v1.5.2
2023-03-19T20:41:02.5298751Z   stderr: 'fatal: ambiguous argument 'v1.5.1..v1.5.2': unknown revision or path not in the working tree.
2023-03-19T20:41:02.5299187Z Use '--' to separate paths from revisions, like this:
2023-03-19T20:41:02.5299549Z 'git <command> [<revision>...] -- [<file>...]''
2023-03-19T20:41:02.5301385Z /home/runner/work/pandas/pandas/doc/source/whatsnew/v1.5.3.rst:72: WARNING: Cannot find contributors for range 'v1.5.2..v1.5.3': Cmd('git') failed due to: exit code(128)
2023-03-19T20:41:02.5302188Z   cmdline: git log --grep=Co-authored --pretty=%b v1.5.2..v1.5.3
2023-03-19T20:41:02.5302704Z   stderr: 'fatal: ambiguous argument 'v1.5.2..v1.5.3': unknown revision or path not in the working tree.
2023-03-19T20:41:02.5303231Z Use '--' to separate paths from revisions, like this:
2023-03-19T20:41:02.5303592Z 'git <command> [<revision>...] -- [<file>...]''
2023-03-19T20:41:02.5305509Z /home/runner/work/pandas/pandas/doc/source/whatsnew/v2.0.0.rst:1422: WARNING: Cannot find contributors for range 'v1.5.0rc0..v2.0.0|HEAD': Cmd('git') failed due to: exit code(128)

[hamedgibago]

Thank you. I'll check it.