SparseSeries.__array__ only returns non-fills

[jnothman]

Code Sample, a copy-pastable example if possible

>>> np.array(pd.SparseSeries([0, 1], fill_value=0))
array([1])

Expected Output

array([0, 1])

this should really be consistent with Series rather than just returning the non-fill values (i.e. rather than being equivalent to np.array(ps.SparseArray([np.nan, 1])).

output of pd.show_versions()

Pandas 0.18.1

should alone be relevant.

Apologies I've not checked if this is fixed in master. Just passing on issues from
scikit-learn/scikit-learn#7352.

[jreback]

__array__ densifying is not what you would want. It must be explict.

In [7]: pd.SparseSeries([0, 1], fill_value=0).get_values()
Out[7]: array([0, 1])

In [8]: np.array(pd.SparseSeries([0, 1], fill_value=0).to_dense())
Out[8]: array([0, 1])

[shoyer]

I agree with @jnothman -- I think this is a bug with SparseSeries. It should not implement a __array__ method like this.

[jnothman]

I think you're going to confuse users when they pass things to objects that
expect array-likes. If you want to raise NotImplemented in array that's
fine; your library should be explicit. Returning something semantically
obscure, as you do, is not helpful.

[sinhrks]

I understand current impl is for applying ufunc little efficient. I agree that np.array(SparseSeries) returns dense is more natural from end user's point of view.

[jnothman]

but correctness/appropriateness depends on which ufunc anyway, I think.

[shoyer]

@sinhrks are you saying you did this intentionally for ufunc support? This is not the right way to do that (though the right way may not be possible until/if __numpy_ufunc__).

[sinhrks]

@shoyer No, the spec hasn't changed for quite a while. ufunc is supported prior to #13853.

[shoyer]

OK, I am going to reopen this as something we should discuss fixing for pandas 2.0.

If it isn't possible to implement ufuncs on sparse objects with an incorrect implementation of __array__, then IMO we shouldn't implement the ufunc interface at all. This is a variation on the theme of pandas as a leaky abstraction for numpy that is the main motivation for pandas 2.0 in the first place.