Error with Pipeline and imblearn.over_sampling.RandomOverSampler

[skatsaounis]

Description

When running the method fit() on the following Pipeline the code fails with TypeError. If I remove the RandomOverSampler from the Pipeline then I face no error. Is it a bug or a wrong initialization?

Steps/Code to Reproduce

from imblearn.pipeline import Pipeline
from sklearn.feature_selection import VarianceThreshold
from sklearn.preprocessing import StandardScaler
from imblearn.over_sampling import RandomOverSampler
from sklearn.decomposition import PCA
from sklearn.neighbors import KNeighborsClassifier
from sklearn.utils import shuffle
import numpy as np
import pandas as pd

mnist_train = pd.read_csv("https://www.python-course.eu/data/mnist/mnist_train.csv", header=None).values
mnist_test = pd.read_csv("https://www.python-course.eu/data/mnist/mnist_test.csv", header=None).values
mnist = np.concatenate((mnist_train, mnist_test), axis=0)

xfull = mnist[:, 1:]
yfull = mnist[:, :1]
sdata, starget = shuffle(xfull, yfull, random_state=36)
samples = 1000
X = sdata[0:samples-1,:]
y = starget[0:samples-1]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.33, random_state=20176)
selector = VarianceThreshold()
scaler = StandardScaler()
ros = RandomOverSampler()
pca = PCA()
clf = KNeighborsClassifier(n_jobs=-1)
pipe = Pipeline(
    steps=[
           ('selector', selector), ('scaler', scaler), ('sampler', ros),
           ('pca', pca), ('kNN', clf)])
# X_train.shape is (669, 784)
# Y_train.shape is (669, 1)
pipe.fit(X_train, y_train)

Expected Results

No error is thrown.

Actual Results

TypeError                                 Traceback (most recent call last)
<ipython-input-51-da2a32d7cb52> in <module>()
----> pipe.fit(X_train, y_train)

10 frames
/usr/local/lib/python3.6/dist-packages/sklearn/model_selection/_search.py in fit(self, X, y, groups, **fit_params)
    737             refit_start_time = time.time()
    738             if y is not None:
--> 739                 self.best_estimator_.fit(X, y, **fit_params)
    740             else:
    741                 self.best_estimator_.fit(X, **fit_params)

/usr/local/lib/python3.6/dist-packages/imblearn/pipeline.py in fit(self, X, y, **fit_params)
    285 
    286         """
--> 287         Xt, yt, fit_params = self._fit(X, y, **fit_params)
    288         with _print_elapsed_time('Pipeline',
    289                                  self._log_message(len(self.steps) - 1)):

/usr/local/lib/python3.6/dist-packages/imblearn/pipeline.py in _fit(self, X, y, **fit_params)
    247                     message_clsname='Pipeline',
    248                     message=self._log_message(step_idx),
--> 249                     **fit_params_steps[name]
    250                 )
    251             # Replace the transformer of the step with the fitted

/usr/local/lib/python3.6/dist-packages/joblib/memory.py in __call__(self, *args, **kwargs)
    566 
    567     def __call__(self, *args, **kwargs):
--> 568         return self._cached_call(args, kwargs)[0]
    569 
    570     def __getstate__(self):

/usr/local/lib/python3.6/dist-packages/joblib/memory.py in _cached_call(self, args, kwargs, shelving)
    532 
    533         if must_call:
--> 534             out, metadata = self.call(*args, **kwargs)
    535             if self.mmap_mode is not None:
    536                 # Memmap the output at the first call to be consistent with

/usr/local/lib/python3.6/dist-packages/joblib/memory.py in call(self, *args, **kwargs)
    732         if self._verbose > 0:
    733             print(format_call(self.func, args, kwargs))
--> 734         output = self.func(*args, **kwargs)
    735         self.store_backend.dump_item(
    736             [func_id, args_id], output, verbose=self._verbose)

/usr/local/lib/python3.6/dist-packages/imblearn/pipeline.py in _fit_resample_one(sampler, X, y, message_clsname, message, **fit_params)
    412                       **fit_params):
    413     with _print_elapsed_time(message_clsname, message):
--> 414         X_res, y_res = sampler.fit_resample(X, y, **fit_params)
    415 
    416         return X_res, y_res, sampler

/usr/local/lib/python3.6/dist-packages/imblearn/base.py in fit_resample(self, X, y)
     79         )
     80 
---> 81         output = self._fit_resample(X, y)
     82 
     83         if self._X_columns is not None or self._y_name is not None:

/usr/local/lib/python3.6/dist-packages/imblearn/over_sampling/_random_over_sampler.py in _fit_resample(self, X, y)
    102     def _fit_resample(self, X, y):
    103         random_state = check_random_state(self.random_state)
--> 104         target_stats = Counter(y)
    105 
    106         sample_indices = range(X.shape[0])

/usr/lib/python3.6/collections/__init__.py in __init__(*args, **kwds)
    533             raise TypeError('expected at most 1 arguments, got %d' % len(args))
    534         super(Counter, self).__init__()
--> 535         self.update(*args, **kwds)
    536 
    537     def __missing__(self, key):

/usr/lib/python3.6/collections/__init__.py in update(*args, **kwds)
    620                     super(Counter, self).update(iterable) # fast path when counter is empty
    621             else:
--> 622                 _count_elements(self, iterable)
    623         if kwds:
    624             self.update(kwds)

TypeError: unhashable type: 'numpy.ndarray'

Versions

Linux-4.14.137+-x86_64-with-Ubuntu-18.04-bionic
Python 3.6.9 (default, Nov 7 2019, 10:44:02)
[GCC 8.3.0]
NumPy 1.18.0
SciPy 1.3.3
Scikit-Learn 0.22.1
Imbalanced-Learn 0.6.1

[chkoar]

Hi Stamatis, that is interesting. I don't have access currently to a pc but could you try this one?

[skatsaounis]

@chkoar Ok it seems that if I use pipe.fit(X_train, y_train.reshape(-1)) instead of pipe.fit(X_train, y_train) the problem goes away.

I have a question as I am new to the field. Is this transformation of y_train legit?

[chkoar]

I don't know if legit is the word that you want to use but to me, yes. Probably this might needs fix and test.

[glemaitre]

Basically we expect y_train to be 1D. If it is 2D it should be one-hot encoded classes.

This said the error intrigued me. Basically, joblib wants to pickle the y_train which should be possible. We do that to cache intermediate steps I think.

[glemaitre]

Oh wait Counter is the issue here.

[glemaitre]

>>> from collections import Counter
>>> import numpy as np
>>> X = np.array([1, 2, 3, 4])
>>> Counter(X)
Counter({1: 1, 2: 1, 3: 1, 4: 1})
>>> Counter(X.reshape(-1, 1))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/lemaitre/miniconda3/lib/python3.7/collections/__init__.py", line 566, in __init__
    self.update(*args, **kwds)
  File "/home/lemaitre/miniconda3/lib/python3.7/collections/__init__.py", line 653, in update
    _count_elements(self, iterable)
TypeError: unhashable type: 'numpy.ndarray'
>>> 

[skatsaounis]

@glemaitre indeed. RandomOverSampler expects to find a (1, X) y_train but shouldn't be good with (X,1) y_train as well? I mean that StandardScaler and KNeighborsClassifier can work with the same pipeline and y_train. This question is code specific.

The other question is regarding results, as I do not have any knowledge on the field (this is part of an academic course I participate). If y_train is (X, 1) or (1, X) is something that produces different results or it should be the same?

[chkoar]

@glemaitre should we try to reshape y in that case? It makes sense.

import numpy as np
from collections import Counter

y = np.array([0, 1])  # Shape: (2,)
Counter(y)

y = np.array([[0], [1]])  # Shape: (2, 1)
Counter(y)  # This will fail

[glemaitre]

should we try to reshape y in that case? It make sense

We should have a check_array that should convert to as 1-dimensional array. Somehow, it could also solve the issue passing a column dataframe.

[glemaitre]

RandomOverSampler expects to find a (1, X) y_train but shouldn't be good with (X,1) y_train as well? I mean that StandardScaler and KNeighborsClassifier can work with the same pipeline and y_train. This question is code specific.

you should want to have an array of size (X,) and not (X, 1) or (1, X).

If y_train is (X, 1) or (1, X) is something that produces different results or it should be the same?

Both should not work :)

[chkoar]

should we try to reshape y in that case? It make sense

We should have a check_array that should convert to as 1-dimensional array. Somehow, it could also solve the issue passing a column dataframe.

Exactly.

[glemaitre]

@skatsaounis basically the work-around is to either ravel or reshape as you did for your specific case. But we should solve the issue.

[chkoar]

@glemaitre will you work on this case or will you let me to fix it?

[glemaitre]

If you have a bit of time, I can make the fix and I will review.

…

On Tue, 7 Jan 2020 at 14:27, Christos Aridas ***@***.***> wrote: @glemaitre <https://github.com/glemaitre> will you work on this case or will you let me to fix it? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#671?email_source=notifications&email_token=ABY32P556GSZ5SFVALQ3TZLQ4R7KXA5CNFSM4KCRTDQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEII3CHY#issuecomment-571584799>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABY32P34YK24ECGPSQRQ44DQ4R7KXANCNFSM4KCRTDQA> .

-- Guillaume Lemaitre Scikit-learn @ Inria Foundation https://glemaitre.github.io/