How to pass in the f expression column to a function when extending the dataframe in pydatatable?

user_9479925 0
Viewed 159

I'm trying to generate some random data and keep it in a datatable, for this reason I have created a custom function as:

def make_data(nrows):

    DT = dt.Frame({'x': 5*np.random.normal(size=nrows)})

    DT_EX = DT[:,f[:].extend({'y': 0.01*f['x'] + 0.1*np.random.normal(size=nrows)})]

    return DT_EX

On executing this function it gives back a DT as:

In [3]: make_data(5)                                                                                                                                                                                        
Out[3]: 
   |         x           y
-- + ---------  ----------
 0 | -0.486592   0.227217 
 1 | -1.90302   -0.0509506
 2 |  4.69407    0.0758279
 3 | -7.08778   -0.152139 
 4 |  0.917043  -0.204939

I would like to add on a np.sin function to the y column expression as:

def make_data_version_two(nrows):

    DT = dt.Frame({'x': 5*np.random.normal(size=nrows)})

    DT_EX = DT[:,f[:].extend({'y': np.sin(f.x) + 0.01*f.x + 0.1*np.random.normal(size=nrows)})]

    return DT_EX

On executing this function it throws out an error:

AttributeError                            Traceback (most recent call last)
<ipython-input-5-81a67037f9f1> in <module>
----> 1 make_data_version_two(5)

<ipython-input-4-e532306680bb> in make_data_version_two(nrows)
      3     DT = dt.Frame({'x': 5*np.random.normal(size=nrows)})
      4 
----> 5     DT_EX = DT[:,f[:].extend({'y': np.sin(f.x) + 0.01*f['x'] + 0.1*np.random.normal(size=nrows)})]
      6 
      7     return DT_EX

AttributeError: 'Expr' object has no attribute 'sin'

My question is how to pass datatable column to any other functions like np.sin(f.x) or np.round(f.x) etc.

I have tried these variants as well, none of them worked.

  • np.sin(f['x'])

  • np.sin(['x'])

py-datatable python
2 Answers

Yup, got it. i have reviewed the datatable math module fucntions, and created a new function as 

def make_data_version_three(nrows):

    DT = dt.Frame({'x': 5*np.random.normal(size=nrows)})

    DT_EX = DT[:,f[:].extend({'y': dt.math.sin(f.x) + 0.01*f['x'] + 0.1*np.random.normal(size=nrows)})]

    return DT_EX

So here i have imported a math module from datatable, from math i called sin function as

dt.math.sin(f.x)

Output:

In [8]: make_data_version_three(5)                                                                                                                                                                          
Out[8]: 
   |         x          y
-- + ---------  ---------
 0 | -13.8865   -1.11732 
 1 |  -2.21624  -0.809127
 2 |  -1.84779  -1.0217  
 3 |   5.42131  -0.659641
 4 |   4.77623  -1.04619 

[5 rows x 2 columns]

Here recommended to go through the math module of datatable

user_94799250

Numpy has a some functions you can use to generate random data. You could try something like this.

import pandas as pd
import numpy as np

df = pd.DataFrame(data=np.random.rand(10), columns=['x'] )

df['y']  = np.sin(df['x'])
print(df)
user_50582300
Related