Series Changes breaks rpy2?

[jankatins]

It seems that the changes to to Series break the data conversion to R: running this Notebook doesn't work anymore with some dev version from last week:
https://gist.github.com/kevindavenport/7771325/raw/87ab5603f406729c6a3866f95af9a1ebfedcf619/Mahalanobis_Outliers.ipynb

The resulting error is this:

#xydata=pandas.DataFrame(...)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-38-74fcaa767ca0> in <module>()
----> 1 get_ipython().run_cell_magic(u'R', u'-i xydata,xycols # list object to be transferred to python here', u'install.packages("ggplot2") # Had to add this for some reason, shouldn\'t be necessary\nlibrary(ggplot2)\ndf = data.frame(xydata)\nnames(df) <- c(xycols)\nprint(head(df))\nplot = ggplot(df, aes(x = X, y = Y)) + \ngeom_point(alpha = .8, color = \'dodgerblue\',size = 5) +\ngeom_point(data=subset(df, Y >= 6.7 | X >= 4), color = \'red\',size = 6) +\ntheme(axis.text.x = element_text(size= rel(1.5),angle=90, hjust=1)) +\nggtitle(\'Distance Pairs with outliers highlighted in red\')\nprint(plot)')

C:\portabel\Python27\lib\site-packages\IPython\core\interactiveshell.pyc in run_cell_magic(self, magic_name, line, cell)
   2141             magic_arg_s = self.var_expand(line, stack_depth)
   2142             with self.builtin_trap:
-> 2143                 result = fn(magic_arg_s, cell)
   2144             return result
   2145 

C:\portabel\Python27\lib\site-packages\IPython\extensions\rmagic.py in R(self, line, cell, local_ns)

C:\portabel\Python27\lib\site-packages\IPython\core\magic.pyc in <lambda>(f, *a, **k)
    191     # but it's overkill for just that one bit of state.
    192     def magic_deco(arg):
--> 193         call = lambda f, *a, **k: f(*a, **k)
    194 
    195         if callable(arg):

C:\portabel\Python27\lib\site-packages\IPython\extensions\rmagic.py in R(self, line, cell, local_ns)
    585                     except KeyError:
    586                         raise NameError("name '%s' is not defined" % input)
--> 587                 self.r.assign(input, self.pyconverter(val))
    588 
    589         if getattr(args, 'units') is not None:

C:\portabel\Python27\lib\site-packages\rpy2\robjects\functions.pyc in __call__(self, *args, **kwargs)
     84                 v = kwargs.pop(k)
     85                 kwargs[r_k] = v
---> 86         return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs)

C:\portabel\Python27\lib\site-packages\rpy2\robjects\functions.pyc in __call__(self, *args, **kwargs)
     29 
     30     def __call__(self, *args, **kwargs):
---> 31         new_args = [conversion.py2ri(a) for a in args]
     32         new_kwargs = {}
     33         for k, v in kwargs.iteritems():

C:\portabel\Python27\lib\site-packages\rpy2\robjects\pandas2ri.pyc in pandas2ri(obj)
     26                 od[name] = StrVector(values)
     27             else:
---> 28                 od[name] = ro.conversion.py2ri(values)
     29         return DataFrame(od)
     30     elif isinstance(obj, PandasIndex):

C:\portabel\Python27\lib\site-packages\rpy2\robjects\pandas2ri.pyc in pandas2ri(obj)
     49         else:
     50             # converted as a numpy array
---> 51             res = original_conversion(obj)
     52         # "index" is equivalent to "names" in R
     53         if obj.ndim == 1:

C:\portabel\Python27\lib\site-packages\rpy2\robjects\numpy2ri.pyc in numpy2ri(o)
     56             raise(ValueError("Unknown numpy array type."))
     57     else:
---> 58         res = ro.default_py2ri(o)
     59     return res
     60 

C:\portabel\Python27\lib\site-packages\rpy2\robjects\__init__.pyc in default_py2ri(o)
    146         res = rinterface.SexpVector([o, ], rinterface.CPLXSXP)
    147     else:
--> 148         raise(ValueError("Nothing can be done for the type %s at the moment." %(type(o))))
    149     return res
    150 

ValueError: Nothing can be done for the type <class 'pandas.core.series.Series'> at the moment.

I'm not sure if this is something pandas cares, but even if not it would be nice to mention it in the release notes.

[jreback]

I think they might need to have a slightly different conversion function, can you post to their dev site?

[jankatins]

https://bitbucket.org/lgautier/rpy2/issue/173/pandasseries-changes-breaks-rpy2

[jreback]

ok...well make this an API issue here to 'track' it.

@jreback, this could be a big deal once 0.13.0 final is released. No word from rpy2?

Moved to 0.13 to make final decision right before release is due. Would be good to preempt
a backlash upon release in one way or another.

This also breaks the docs.

[jreback]

@JanSchulz can u ping rpy2?
see what their schedule is?

cc @lgautier, we're anxious to avoid releasing a point release with no mitigation for this.

[jtratner]

could be as simple as checking for __array__() method if necessary.

Hope it is, but it needs to happen on rpy2's side, no?

[jtratner]

@jreback why doesn't Series support the __array_interface__? Was that an explicit choice?

I think we might need to do that to be able to actually work with rpy2.

[jreback]

not necessary as array is called

but if u need it explicitly then go for it
didn't seem that hard

I don't have rpy so really can't even try it

[jreback]

here is an example

emcconville/wand#65

[jtratner]

@JanSchulz could you post a really simple example? I can't seem to get your notebook to work. I think the fix for rpy2 is actually really simple.

[jtratner]

@jreback could we just do:

def __array_interface__(self):
    return self.__array__().__array_interface__()

?

[jreback]

it's a property (don't need function call)
but might work

[jankatins]

@jtratner I also only worked with rpy2 the first time with that notebook. :-) The notebook is from @kevindavenport
@kevindavenport: do you have some more experience with rpy?

Here is a small notebook with three cells:

--- setup ---
%load_ext rmagic
import pandas as pd
df = pd.DataFrame({"x":[1,2,3,4,5], "y":[1,2,3,2,1]})
vals = df.values
cols = df.columns
---
--- using df.values: works ---
%%R -i vals,cols # list object to be transferred to python here
install.packages("ggplot2") # Had to add this for some reason, shouldn't be necessary
library(ggplot2)
df = data.frame(vals)
names(df) <- c(cols)
plot = ggplot(df, aes(x = x, y = y)) + 
geom_point(alpha = .8, color = 'dodgerblue',size = 5)
print(plot)
---
--- df directly: does not work ---
%%R -i df # list object to be transferred to python here
install.packages("ggplot2") # Had to add this for some reason, shouldn't be necessary
library(ggplot2)
df = data.frame(df)
plot = ggplot(df, aes(x = x, y = y)) + 
geom_point(alpha = .8, color = 'dodgerblue',size = 5)
print(plot)
---