Is it possible to save a CSV file in Foundry Code Repositories transforms-python language rather than saving them in the Parquet format?
1 Answers
Yes, this can be done by specifying the output dataset format when calling the write_dataset function. You can include compression options as well when making the call. For example:
@transform(
my_input = Input('/path/to/input/dataset'),
my_output = Output('/path/to/output/dataset')
)
def compute_function(my_input, my_output):
my_output.write_dataframe(
my_input.dataframe(),
output_format = "csv",
options = {
"compression": "gzip"
}
)
Related