groupby enumerate method

[hayd]

I'm not sure what a good word for this is (count is taken, order means sort)!

But it's quite an often used thing to create a column which enumerates the items in each group / counts their occurrences.

You can hack it:

In [1]: df = pd.DataFrame([[1, 2], [2, 3], [1, 4], [1, 5], [2, 6]])

In [2]: g = df.groupby(0)

In [3]: g.apply(lambda x: pd.Series(np.arange(len(x)), x.index))
Out[3]:
0    0
1    0
2    1
3    2
4    1
dtype: int64

In [5]: df['order'] = _

In [6]: df
Out[6]:
   0  1  order
0  1  2      0
1  2  3      0
2  1  4      1
3  1  5      2
4  2  6      1

I've seen this in a few SO questions, here's just one.

cc @cpcloud (and I've seen @jreback answer a question with this)

[jreback]

counter() ?

[hayd]

I kind of favour enumerate() as it's more in line with python (and it reminds me of Demolition Man).

I was wrong: count isn't taken for groupby, it's size, but I'm -1 on same/similarly named methods with different meaning across groupby/DataFrame.

[cpcloud]

plus with enumerate maybe you could provide an iterator or column to cycle through..

[hayd]

@cpcloud Interesting idea, not sure how it'd work (if you pass in an iterator it'd be eaten by the first thing/start part way through?). But perhaps mapping after the enumeration would be the way to go (mod and apply a getitem)?

In [21]: df['order'].apply(list('abc').__getitem__)  # continued from above
Out[21]:
0    a
1    a
2    b
3    c
4    b
Name: order, dtype: object

In [22]: (df['order'] % 2).apply(list('ab').__getitem__)  # I guess this is what you mean by cycle
Out[22]:
0    a
1    a
2    b
3    a
4    b
Name: order, dtype: object

To implement is there a more efficient way? Or should it just be a shortcut:

g.apply(lambda x: pd.Series(np.arange(len(x)), x.index))

[hayd]

I wonder if enumerate could also mean enumerate by the group it's in (or does that not make sense?) maybe that could be a possible kwarg

[jtratner]

enumerate is something specific in Python (iterator with index), might want
to pick a different name.

As I read through I had this question: how is this different from
value_counts()?

[cpcloud]

it's enumerating the integer index of the elements in a group

[cpcloud]

sort of like a cumvalue_counts()

[cpcloud]

(which is a horrible name for this)

[hayd]

I think enumerate is the right name for this because it's something specific to python... I really like this name.

Was wondering if another potentially useful/expected thing for enumerate to do is enumerate the groups (I'm not sure), a little bit like this:

from itertools import count
c = count(-1)
df = pd.DataFrame([['a', 2], ['b', 3], ['a', 4], ['a', 5], ['b', 6]])

In [4]: df
Out[4]: 
   0  1
0  a  2
1  b  3
2  a  4
3  a  5
4  b  6

In [5]: g = df.groupby(0)

In [6]: g.apply(lambda x: pd.Series(c.next(), x.index))
Out[23]: 
0    0
1    1
2    0
3    0
4    1
dtype: int64

[hayd]

Although potentially the groups are not well ordered anyway....

[hayd]

(I think they are ordered, so this should be ok, I wonder if these two alternatives could be distinguished with a well chosen kwarg...)

[ifmihai]

My point of view,
as an average python/pandas user,

a name with the verb 'to count' is the most intuitive
because you count the apparitions
I take the first value in a Series, let's say value is 6,
and then I parse visually the Series counting the apparitions of 6

enumerate in python is not counting
it gets every element with its index
so it's not intuitive at all from my point of view
it's quite confusing actually

[hayd]

I take the point that it is somewhat confusing, but thing that any other option will be significantly more so.

This is enumerating each of the "apparitions". count is the wrong word since it usually has the meaning (elsewhere in pandas, e.g. DataFrame .count) of counting the total occurrences (i.e. groupby .size). We are not counting, we are enumerating.

enumerate in python is not counting it gets every element with its index

If you replace index with occurrence, this is exactly what we are doing.