Python:IndexingError: Unalignable boolean Series provided as indexer (index of the boolean Series and of the indexed object do not match)

user_18090706 0
Viewed 476

I want to create clinical dataframe which contains a column last_fu that retains the Last Alive Less Initial Pathologic Diagnosis Date Calculated Day Value row in the raw_clinical_patient dataframe if the value is more than 2. The first four columns are header.

import pandas as pd

clinical = pd.DataFrame()
clinical["last_fu"] = raw_clinical_patient[raw_clinical_patient["Last Alive Less Initial Pathologic Diagnosis Date Calculated Day Value"].loc[4:].astype(int)>=2]

Traceback:

/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:5: UserWarning: Boolean Series key will be reindexed to match DataFrame index.
  """
---------------------------------------------------------------------------
IndexingError                             Traceback (most recent call last)
<ipython-input-55-e1f5dd93f92a> in <module>()
      3 # Read from the fifth row onwards
      4 clinical = pd.DataFrame()
----> 5 clinical["last_fu"] = raw_clinical_patient[raw_clinical_patient["Last Alive Less Initial Pathologic Diagnosis Date Calculated Day Value"].loc[4:].astype(int)>=2]

2 frames
/usr/local/lib/python3.7/dist-packages/pandas/core/indexing.py in check_bool_indexer(index, key)
   2387         if mask.any():
   2388             raise IndexingError(
-> 2389                 "Unalignable boolean Series provided as "
   2390                 "indexer (index of the boolean Series and of "
   2391                 "the indexed object do not match)."

IndexingError: Unalignable boolean Series provided as indexer (index of the boolean Series and of the indexed object do not match).


raw_clinical_patient[["#Other Patient ID", "Last Alive Less Initial Pathologic Diagnosis Date Calculated Day Value"]].head(10)
#Other Patient ID Last Alive Less Initial Pathologic Diagnosis Date Calculated Day Value
#Legacy DMP patient identifier (DMPnnnn) Time interval from the last day on which a per...
#STRING NUMBER
#1 1
OTHER_PATIENT_ID DAYS_TO_INITIAL_PATHOLOGIC_DIAGNOSIS
30a1fe5e-5b12-472c-aa86-c2db8167ab23 0
df3c1d61-79c1-43e9-971a-8029497ffeab 0
3ac8f759-2df0-4ef5-913f-ce05b8dd8937 0
98714d95-b62e-4f34-9cd1-91542da463eb 0
fb75eb9d-05be-4e6b-a515-f39efb15e66c 0
c1f590eb-21ed-4939-adc5-0f7e6f485b56 0

Edit:

clinical["last_fu"] = (raw_clinical_patient
            .loc[4:]
            .copy()
            .astype(int)
            .query(f'{"Last Alive Less Initial Pathologic Diagnosis Date Calculated Day Value"}>=2'))

Also raises an error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-63-fc3f116ef0fb> in <module>()
      6             .loc[4:]
      7             .copy()
----> 8             .astype(int)
      9             .query(f'{"Last Alive Less Initial Pathologic Diagnosis Date Calculated Day Value"}>=2'))

8 frames
/usr/local/lib/python3.7/dist-packages/pandas/_libs/lib.pyx in pandas._libs.lib.astype_intsafe()

ValueError: invalid literal for int() with base 10: '30a1fe5e-5b12-472c-aa86-c2db8167ab23'
pandas python
1 Answers

Problem: Filtering with a boolean array with different size from the original dataframe. (As the error says)

Solution: Filter after subset.

long_name = 'Last Alive Less Initial Pathologic Diagnosis Date Calculated Day Value'
clinical = raw_clinical_patient.loc[4:].copy()
clinical = clinical.astype(int)
clinical = clinical[clinical[long_name]>=2]

or use chaining

clinical = (raw_clinical_patient
            .loc[4:]
            .copy()
            .astype(int)
            .query("'Last Alive Less Initial Pathologic Diagnosis Date Calculated Day Value'>=2"))

Becareful replacing strings: For example

name = 'hello'
print(name) -> 'hello'
print(f'{name} world') -> 'hello world'
print(f'{hello} world') -> '{hello} world'
print('{name} world') -> '{name} world'
user_126373740
Related