While pushing data to HDFS from Visual Studio, I want to add a timestamp to the HDFS file name but I am only able to add filename and cannot generate timestamp automatically.
Is there any way to give timestamp to file while pushing to hdfs in SSIS using HDFS File destination
Viewed 118
2 Answers
If it can be done (don't have HDFS stuff installed), the approach will be the same we use when we want a table name from variable with an ADO.NET destination.
Click out of the Data Flow and go to the Control Flow. Right click on the Data Flow and select Properties. In the Properties Menu, find Expressions... and then scroll through the available properties in the first column. If there's an HDFS section, it would have something like HDFS File Destination path or something similarly named.
This can be set from the [HDFS File Destination].[FilePath] property as an expression from the Data Flow Task properties. You can find useful information in the following article which I have published three months ago (check the HDFS File Destination > Expression section):
You should use an expression like:
"/user/sysusr/system/Program/program_" + (DT_WSTR,10)DATEPART("yyyy",GETDATE()) + (DT_WSTR,10)DATEPART("mm",GETDATE()) + (DT_WSTR,10)DATEPART("dd",GETDATE()) + (DT_WSTR,10)DATEPART("Hh",GETDATE()) + (DT_WSTR,10)DATEPART("mi",GETDATE()) + (DT_WSTR,10)DATEPART("ss",GETDATE()) + (DT_WSTR,10)DATEPART("Ms",GETDATE()) + ".csv"
References
Related