How to filter timestamp column in Data Flow of Azure Data Factory

user_4359039 0
Viewed 2038

I have timestamp column where I have written following expression to filter the column:

contact_date  >= toTimestamp('2020-01-01') && 
contact_date  <= toTimestamp('2020-12-31')

It doesn't complain about syntax but after run it doesn't filter based on date specified. Simply to say logic doesn't work. Any idea?

Date Column in Dataset:

enter image description here

azure-data-factory-2 azure-data-factory dataflow azure-devops
3 Answers

Please don't use toTimestamp() function. I tested and you will get null output.

I use a Filter active to filter the data. Please use the toString() and change the expression like bellow:

toString({contact_date })>= toString('2020-01-01') && toString({contact_date })<= toString('2020-12-31')

Ref my example, contact_date and output: enter image description here

enter image description here

user_105492810

I think you need to either add the hour, minute and second portions of your dates or specify an appropriate format string, as per the docs, "If the timestamp is omitted the default pattern yyyy-[M]M-[d]d hh:mm:ss[.f...] is used."

user_13509870

Hi put in and operation or in () ie

    (contact_date  >= toTimestamp('2020-01-01') &&  contact_date  <= toTimestamp('2020-12-31'))

    or

     AND(contact_date  >= toTimestamp('2020-01-01') , contact_date  <= toTimestamp('2020-12-31'))
user_60858030
Related