Numpy is not available error

[lauredecaudin]

Hello !

I tried to use the package with the code found in the README,

import numpy as np
from sklearn.preprocessing import StandardScaler
from pypots.data import load_specific_dataset, mcar, masked_fill
from pypots.imputation import SAITS
from pypots.utils.metrics import cal_mae
# Data preprocessing. Tedious, but PyPOTS can help. 🤓
data = load_specific_dataset('physionet_2012')  # PyPOTS will automatically download and extract it.
X = data['X']
num_samples = len(X['RecordID'].unique())
X = X.drop('RecordID', axis = 1)
X = StandardScaler().fit_transform(X.to_numpy())
X = X.reshape(num_samples, 48, -1)
X_intact, X, missing_mask, indicating_mask = mcar(X, 0.1) # hold out 10% observed values as ground truth
X = masked_fill(X, 1 - missing_mask, np.nan)
# Model training. This is PyPOTS showtime. 💪
saits = SAITS(n_steps=48, n_features=37, n_layers=2, d_model=256, d_inner=128, n_head=4, d_k=64, d_v=64, dropout=0.1, epochs=10)
saits.fit(X)  # train the model. Here I use the whole dataset as the training set, because ground truth is not visible to the model.
imputation = saits.impute(X)  # impute the originally-missing values and artificially-missing values
mae = cal_mae(imputation, X_intact, indicating_mask)

And I get this error :

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<command-282728034361361> in <module>
     15 # Model training. This is PyPOTS showtime. 💪
     16 saits = SAITS(n_steps=48, n_features=37, n_layers=2, d_model=256, d_inner=128, n_head=4, d_k=64, d_v=64, dropout=0.1, epochs=10)
---> 17 saits.fit(X)  # train the model. Here I use the whole dataset as the training set, because ground truth is not visible to the model.
     18 imputation = saits.impute(X)  # impute the originally-missing values and artificially-missing values
     19 mae = cal_mae(imputation, X_intact, indicating_mask)

/databricks/python/lib/python3.8/site-packages/pypots/imputation/saits.py in fit(self, train_X, val_X)
    216 
    217     def fit(self, train_X, val_X=None):
--> 218         train_X = self.check_input(self.n_steps, self.n_features, train_X)
    219         if val_X is not None:
    220             val_X = self.check_input(self.n_steps, self.n_features, val_X)

/databricks/python/lib/python3.8/site-packages/pypots/base.py in check_input(self, expected_n_steps, expected_n_features, X, y, out_dtype)
     76                 X = torch.tensor(X).to(self.device)
     77             elif is_array:
---> 78                 X = torch.from_numpy(X).to(self.device)
     79             else:  # is tensor
     80                 X = X.to(self.device)

RuntimeError: Numpy is not available

Anyone knows what's happening ?

I use Databricks with a cluster 10.4 LTS https://docs.databricks.com/release-notes/runtime/10.4.html
I installed PyPots with pip install pypots

Thanks

[WenjieDu]

Hi Laure, thank you for starting this discussion! Could you please provide the version numbers of your pytorch and numpy?

[WenjieDu]

No problem. My pleasure 😃

[lauredecaudin]

I'm trying to use your package for an electricity dataset, for my company, so I'm trying to understand how your dataprep is working, do you think I should use only my electricity consumption column, or should I add some features related to time and weather ?

[WenjieDu]

It depends on your application scenario. Could you please provide more details about your application? For example, what is your task? Imputation/classification, etc. What features do you have in your dataset?

[lauredecaudin]

This is imputation, for an energy consumption time series,
I have 2 columns : datetime (timsestamp) and consumption (float)
I could have more features but I would like to try without

[WenjieDu]

Usually, given more task-related features, the model will perform better. For example, in your scenario, the temperature is definitely related to energy consumption. Generally, too hot or too cold will result in more consumption. You can run a set of experiments to make comparisons.