Inappropriate/inconsistent behavior in data assignment (v. 0.13)

[cyrusmaher]

I find it convenient to store lists or series as elements in a dataframe. However, it seems panda gets confused if the iterable is the same length as the number of rows in the dataframe. Here's a minimal example:

matchmat_test = pd.DataFrame(index=[0,1], 
                             columns=[1], dtype=list)
matchmat_test.ix[0,0] = [1,2,3]

Returns:

           0
0  [1, 2, 3]
1        NaN

If we then assign to the same element again:

matchmat_test.ix[0,0] = [1,2]

we get:

 0
0  1
1  2

[jreback]

@cyrusmaher I fixed this, but keep in mind that storing anything object-like a list (or you can see my example, I created an object that is really a wrapped scalar). is very inefficient as the resultant dtype is object. You are basically losing all of the power of vectorized computation.

Furthermore you can only do very limited setting operations, e.g. only set via a single indexer, anything else won't work.

You should look at using Panels / Multi-indexed frames or multiple structures (e.g. frames) to avoid using lists at all costs.

[cyrusmaher]

Thank you! For what I'm working on, the lists (really np arrays) are all of
different sizes and all operations are elementwise. The dataframe indexing
is useful and applymap is nice too. Thanks again for your hard work!

On Jan 22, 2014, at 5:18 PM, jreback notifications@github.com wrote:

@cyrusmaher https://github.com/cyrusmaher I fixed this, but keep in mind
that storing anything object-like a list (or you can see my example, I
created an object that is really a wrapped scalar). is very inefficient as
the resultant dtype is object. You are basically losing all of the power of
vectorized computation.

Furthermore you can only do very limited setting operations, e.g. only set
via a single indexer, anything else won't work.

You should look at using Panels / Multi-indexed frames or multiple
structures (e.g. frames) to avoid using lists at all costs.

—
Reply to this email directly or view it on
GitHubhttps://github.com//issues/6043#issuecomment-33088291
.