Setting the data model in pandas

user_6054404 0
Viewed 100

So I'm used to database ETL. In SQL I Create the table and set the char lengths, data types etc. As I understand it pandas uses the max length of whatever is put into the dataframe. Fine if you're staying in python, but I need to specify these things explicitly.

Here's some base code to work from, pointers welcome:

df=pd.Dataframe()
df['ID'] = some data probably i + 1
df['text'] = some text length set to max 255
pandas etl
1 Answers

Here is an informative article on pandas datatypes:
https://pbpython.com/pandas_dtypes.html

If you want to see the datatypes of your dataframe, you can do:

df.info()

You can set datatypes explicitly where you can choose to ignore or raise errors with .astype():

df['ID'] = df['ID'].astype(int, errors='raise')
df['ID'] = df['ID'].astype(int, errors='ignore')

For strings you can set the datatype as follows:

df['text'] = df['text'].astype('string')

Or if you're using an older version of pandas < 1.0 then do:

df['text'] = df['text'].astype(str)

If you want to set a max length for your string, you could do:

df['text'] = df['text'].str.slice(0, 255)
user_34891550
Related