I have a large dataframe containing, amongst other things, a (Norwegian) social security number. It is possible to get the date of birth out of this number via a special algorithm. However, every now and then an illegal social security number creeps into the database corrupting the calculation.
What I would like to do is to tag every line having an illegal social security number, along with a log message showing the error raised.
Consider the following, constructed, example
import pandas as pd
from datetime import date
sample_data = pd.DataFrame({'id' : [1, 2, 3], \
'sec_num' : [19790116, 19480631, 19861220]})
# The actual algorithm transforming the sec number is more complicated
# this is just for illustration purposes
def int2date(argdate: int):
try:
year = int(argdate / 10000)
month = int((argdate % 10000) / 100)
day = int(argdate % 100)
return date(year, month, day)
except ValueError:
raise ValueError("Value:{0} not a legal date.".format(argdate))
I want to create the following output
id sec_num date_of_birth is_in_error error_msg
0 1 19790116 1979-01-16 False
1 2 19480631 None True 19480631 is not a legal date
2 3 19861220 1986-12-20 False
I have tried
try:
sample_data['date_of_birth'] = [int2date(sec_num) for \
sec_num in sample_data['sec_num']]
sample_data['is_in_error'] = False
sample_data['error_msg'] = ''
except ValueError as e:
sample_data['is_in_error'] = True
sample_data['error_msg'] = str(e)
but this produces this
id sec_num is_in_error error_msg
0 1 19790116 True Value:19480631 not a legal date.
1 2 19480631 True Value:19480631 not a legal date.
2 3 19861220 True Value:19480631 not a legal date.
I guess the problem is that I assign the date_of_birth-column in one operation and the errors in another. I am not sure how to catch and create the is_in_error and error_msg columns simulationously.