Having trouble with - class 'pandas.core.indexing._AtIndexer'

user_14112764 0
Viewed 167

I'm working on a ML project to predict answer times in stack overflow based on tags. Sample data:

Unnamed: 0  qid i   qs  qt  tags    qvc qac aid j   as  at
0   1   563355  62701.0 0   1235000081  php,error,gd,image-processing   220 2   563372  67183.0 2   1235000501
1   2   563355  62701.0 0   1235000081  php,error,gd,image-processing   220 2   563374  66554.0 0   1235000551
2   3   563356  15842.0 10  1235000140  lisp,scheme,subjective,clojure  1047    16  563358  15842.0 3   1235000177
3   4   563356  15842.0 10  1235000140  lisp,scheme,subjective,clojure  1047    16  563413  893.0   18  1235001545
4   5   563356  15842.0 10  1235000140  lisp,scheme,subjective,clojure  1047    16  563454  11649.0 4   1235002457

I'm stuck at the data cleaning process. I intend to create a new column named 'time_taken' which stores the difference between the at and qt columns. Code:

import pandas as pd
import numpy as np
df = pd.read_csv("answers.csv")
df['time_taken'] = 0
print(type(df.time_taken))
for i in range(0,263541):
    val = df.qt[i]
    qtval = val.item()
    val = df.at[i]
    atval = val.item()
    df.time_taken[i] = qtval - atval

I'm getting this error:

Traceback (most recent call last):

  File "<ipython-input-39-9384be9e5531>", line 1, in <module>
    val = df.at[0]

  File "D:\Softwares\Anaconda\lib\site-packages\pandas\core\indexing.py", line 2080, in __getitem__
    return super().__getitem__(key)

  File "D:\Softwares\Anaconda\lib\site-packages\pandas\core\indexing.py", line 2027, in __getitem__
    return self.obj._get_value(*key, takeable=self._takeable)

TypeError: _get_value() missing 1 required positional argument: 'col'

The problem here lies in the indexing of df.at Types of both df.qt and df.at are

<class 'pandas.core.indexing._AtIndexer'> 
<class 'pandas.core.series.Series'> respectively.

I'm an absolute beginner in data science and do not have enough experience with pandas and numpy.

pandas data-cleaning dataframe indexing python
1 Answers

There is, to put it mildly, an easier way to do this.

df['time_taken'] = df['at'] - df.qt

The AtIndexer issue comes up because .at is a pandas method. You want to make sure to not name columns any names that are the same as a Python/Pandas method for this reason. You can get around it just by indexing with df['at'] instead of df.at.

Besides that, this operation — if I'm understanding it — can be done with one short line vs. a long for loop.

user_123994090
Related