valueError: y must be a structured array with the first field being a binary class event indicator and the second field the time of the event/censorin

Question

It appears that I have my code in the same form as the scikit-survival documentation.

data_y = df[['sensored', 'sensored_2']].to_numpy()
data_x = df.drop(['sensored', 'sensored_2'], axis = 1)
data_y
array([[True, 481],
   [True, 424],
   [True, 519],
   ...,
   [True, 13],
   [True, 96],
   [True, 6]], dtype=object)

From the scikit-survial documentation the array was created from the dataset upon loading. I am trying to create my array from a dataframe, but continue to get the error in the title when I try to fit the array to the model.

sksurv.linear_model import CoxPHSurvivalAnalysis
estimator = CoxPHSurvivalAnalysis()
estimator.fit(df_dummy_3, data_y)
ValueError: y must be a structured array with the first field being a binary 
class event indicator and the second field the time of the event/censoring

Documentation:

from sksurv.datasets import load_veterans_lung_cancer

data_x, data_y = load_veterans_lung_cancer()
data_y
array([( True,  72.), ( True, 411.), ( True, 228.), ( True, 126.),
       ( True, 118.), ( True,  10.), ( True,  82.), ( True, 110.),
       ( True, 314.), (False, 100.), ( True,  42.), ( True,   8.),
       ( True, 144.), (False,  25.), ( True,  11.), ( True,  30.),
       ( True, 384.), ( True,   4.), ( True,  54.), ( True,  13.),
       (False, 123.), (False,  97.), ( True, 153.), ( True,  59.),
       ( True, 117.), ( True,  16.), ( True, 151.), ( True,  22.),
       ( True,  56.), ( True,  21.), ( True,  18.), ( True, 139.),
       ( True,  20.), ( True,  31.), ( True,  52.), ( True, 287.),
       ( True,  18.), ( True,  51.), ( True, 122.), ( True,  27.),
       ( True,  54.), ( True,   7.), ( True,  63.), ( True, 392.),
       ( True,  10.), ( True,   8.), ( True,  92.), ( True,  35.),
       ( True, 117.), ( True, 132.), ( True,  12.), ( True, 162.),
       ( True,   3.), ( True,  95.), ( True, 177.), ( True, 162.),
       ( True, 216.), ( True, 553.), ( True, 278.), ( True,  12.),
       ( True, 260.), ( True, 200.), ( True, 156.), (False, 182.),
       ( True, 143.), ( True, 105.), ( True, 103.), ( True, 250.),
       ( True, 100.), ( True, 999.), ( True, 112.), (False,  87.),
       (False, 231.), ( True, 242.), ( True, 991.), ( True, 111.),
       ( True,   1.), ( True, 587.), ( True, 389.), ( True,  33.),
       ( True,  25.), ( True, 357.), ( True, 467.), ( True, 201.),
       ( True,   1.), ( True,  30.), ( True,  44.), ( True, 283.),
       ( True,  15.), ( True,  25.), (False, 103.), ( True,  21.),
       ( True,  13.), ( True,  87.), ( True,   2.), ( True,  20.),
       ( True,   7.), ( True,  24.), ( True,  99.), ( True,   8.),
       ( True,  99.), ( True,  61.), ( True,  25.), ( True,  95.),
       ( True,  80.), ( True,  51.), ( True,  29.), ( True,  24.),
       ( True,  18.), (False,  83.), ( True,  31.), ( True,  51.),
       ( True,  90.), ( True,  52.), ( True,  73.), ( True,   8.),
       ( True,  36.), ( True,  48.), ( True,   7.), ( True, 140.),
       ( True, 186.), ( True,  84.), ( True,  19.), ( True,  45.),
       ( True,  80.), ( True,  52.), ( True, 164.), ( True,  19.),
       ( True,  53.), ( True,  15.), ( True,  43.), ( True, 340.),
       ( True, 133.), ( True, 111.), ( True, 231.), ( True, 378.),
       ( True,  49.)],
      dtype=[('Status', '?'), ('Survival_in_days', '


I have been trying to change the datatype of my array above without luck. Any help would be greatly appreciated.

user_13700190 · Answer

The fit method expects the y data to be a structured array. In our case, this is an array of Tuples, where the first element is the status and second one is the survival in days.

To put our data in the format the fit method expects, we need first transform the elements of the array from lists (e.g. [True, 424]) to tuples (e.g. (True, 424)). After that, we can group our tuples in the structured array. Below I will show an example:

Suppose we have the following dataframe:

df = pd.DataFrame([[True, 123.],[True, 481.], [True, 424.], [True, 519.]], columns=['sensored', 'sensored_2'])

And we get y as you did:

data_y = df[['sensored', 'sensored_2']].to_numpy()
data_y
array([[True, 123.0],
   [True, 481.0],
   [True, 424.0],
   [True, 519.0]], dtype=object)

One way of putting the data_y in the format we expect is to create a list of tuples using its elements and, after that, create a structured array:

#List of tuples
aux = [(e1,e2) for e1,e2 in data_y]

#Structured array
new_data_y = np.array(aux, dtype=[('Status', '?'), ('Survival_in_days', '


Then, you can use that data to fit your model:
from sksurv.linear_model import CoxPHSurvivalAnalysis
estimator = CoxPHSurvivalAnalysis()
estimator.fit(df_dummy, new_data_y)

user_901925 · Answer

Your array is 2d object dtype:

In [237]: arr = np.array([[True, 481],
     ...:    [True, 424],
     ...:    [True, 519],
     ...:    [True, 13],
     ...:    [True, 96],
     ...:    [True, 6]], dtype=object)
In [238]: arr
Out[238]: 
array([[True, 481],
       [True, 424],
       [True, 519],
       [True, 13],
       [True, 96],
       [True, 6]], dtype=object)

Assuming the documentation show the desired structured array dtype:

In [239]: dt=dtype=[('Status', '?'), ('Survival_in_days', '


As you can see from the documentation, a structured array is displayed (and input) as list of tuples.
In [240]: [tuple(row) for row in arr]
Out[240]: [(True, 481), (True, 424), (True, 519), (True, 13), (True, 96), (True, 6)]

Using that with dtype:
In [241]: data=np.array([tuple(row) for row in arr], dtype=dt)
In [242]: data
Out[242]: 
array([( True, 481.), ( True, 424.), ( True, 519.), ( True,  13.),
       ( True,  96.), ( True,   6.)],
      dtype=[('Status', '?'), ('Survival_in_days', '

If any of this unclear, you may need to read the structured array docs
https://numpy.org/doc/stable/user/basics.rec.html

valueError: y must be a structured array with the first field being a binary class event indicator and the second field the time of the event/censorin

2 Answers