ENH: cumcount() for DataFrames and Series

[johne13]

Unless I am missing something obvious, we can do this:
df.groupby('x').cumcount()
but not:
df.cumcount()

Obviously, there are workarounds, but seems like we should be able to do on a dataframe/series if we can do on a groupby object?

[jreback]

an Index already does the

pls show a usecase

[jorisvandenbossche]

@johne13 you aren't confusing it with cumsum? Besides obtaining it from an index, you can also just do range(len(df))

[johne13]

It is not a super-common event, to be sure, but there are cases like this: that I bump into somewhat regularly:
http://stackoverflow.com/questions/36044890/how-to-speed-up-a-very-slow-pandas-apply-function/36047836#36047836

Simpler example:

df = pd.DataFrame({ 'x':[1,2,1,2] })
#df[ df.x == 2 ].cumcount()    # doesn't work of course

df.loc[ df.x == 2, 'y'] = 1    # so I do a two step with cumsum as 2nd step
df.y.cumsum()

Out[428]: 
0    NaN
1    1.0
2    NaN
3    2.0

So yeah, lots of easy ways to do this on ENTIRE dataset but as soon as you want to do a selection, it gets a little messier. Or again, maybe I am missing something obvious.

Also, I would think whatever justification there is for having this as a groupby method applies just as much to having it work for a dataframe or series, no?

[jreback]

the common idiom is

.reset_index()

[jreback]

no for a groupby this is expensive if u don't have an array impl (which is why it exists)

[johne13]

I guess I am missing something. I couldn't do this, for example,
df['new'] = df[ df.x == 2 ].reset_index()
because obviously I have reset the index so can't correctly align it with the original dataset.

I should have been more clear about that aspect. I am generally trying to create a new column when I mistakenly attempt to use the nonexistent cumcount method.

[jreback]

ok not averse to adding this method for consistency (among other cum. methods) but should return a Series

pretty trivial impl if u would like to submit a PR

[jreback]

signature would be identical to .cum* (at end of core/generic.py)

In [1]: pd.DataFrame.cumsum?
Signature: pd.DataFrame.cumsum(self, axis=None, dtype=None, out=None, skipna=True, **kwargs)
Docstring:
Return cumulative sum over requested axis.

Parameters
----------
axis : {index (0), columns (1)}
skipna : boolean, default True
    Exclude NA/null values. If an entire row/column is NA, the result
    will be NA

Returns
-------
sum : Series
File:      ~/pandas/pandas/core/generic.py
Type:      instancemethod

[max-sixty]

Potentially not as fast (?), and handles NAs differently from the case above, but very idiomatic:

In [12]: df.where(df==2).expanding().count()
Out[12]: 
     x
0  0.0
1  1.0
2  1.0
3  2.0