MATLAB - concatenating a datetime array to an empty array

czar1249 0
Viewed 40

I have a project which outputs data into many files. These files look something like this as interpreted by readmatrix:

ans =

  8×8 table

         Date_Time         EZOO2Con___    EZOCO2Con_ppm_    SGPCO2Con_ppm_    SGPTVOC_ppb_    BMEHumidity___    BMEPressure_Pa_    BMETemp_DegC_
    ___________________    ___________    ______________    ______________    ____________    ______________    _______________    _____________

    09/06/2022 11:55:17       19.16            419               400               0               48.5              95948             22.57    
    09/06/2022 11:55:18       19.16            419               400               0              48.89              99577             22.58    
    09/06/2022 11:55:19       19.16            419               400               0              48.89              99578             22.58    
    09/06/2022 11:55:20       19.15            420               400               0              48.84              99584             22.57    
    09/06/2022 11:55:21       19.15            420               400               0              48.95              99574             22.58    
    09/06/2022 11:55:22       19.15            421               400               0              49.15              99578             22.57    
    09/06/2022 11:55:23       19.15            421               400               0               48.9              99577             22.56    
    09/06/2022 11:55:24       19.15            422               400               0               48.9              99573             22.57

For my previous test, I have approx. 289 separate files of this format which I'd like to combine together in 8 arrays for plotting.

The Date_Time column is a string with MM/DD/YYYY and HH:MM:SS separated by a space. When using the table2array command, I am able to convert the date&time column of each file's data into a datetime array. However, I am unable to use the cat or vertcat functions to append the date&time column to my "combined" array. Below is the code that's giving me trouble:

for k = 1:length(fileList)
    baseFileName = fileList(k).name;
    fullFileName = fullfile(fileList(k).folder, baseFileName);
    fprintf(1, 'Now reading %s\n', fullFileName);
    data = readtable(fullFileName);
    timecol = data(:,1);
    EZOCO2col = data(:,2);
    EZOO2col = data(:, 3);
    SGP30CO2col = data(:, 4);
    SGP30TVOCcol = data(:, 5);
    BME280Humcol = data(:, 6);
    BME280Presscol = data(:, 7);
    BME280Tempcol = data(:, 8);
    
    timecol_array = table2array(timecol);
    EZOCO2col_array = table2array(EZOCO2col);
    EZOO2col_array = table2array(EZOO2col);
    SGP30CO2col_array = table2array(SGP30CO2col);
    SGP30TVOCcol_array = table2array(SGP30TVOCcol);
    BME280Humcol_array = table2array(BME280Humcol);
    BME280Presscol_array = table2array(BME280Presscol);
    BME280Tempcol_array = table2array(BME280Tempcol);
    
    timecol_tot  = cat(1, timecol_tot, timecol_array);
    EZOCO2col_tot = cat(1, EZOCO2col_tot, EZOCO2col_array);
    EZOO2col_tot = cat(1, EZOO2col_tot, EZOO2col_array);
    SGP30CO2col_tot = cat(1, SGP30CO2col_tot, SGP30CO2col_array);
    SGP30TVOCcol_tot = cat(1, SGP30TVOCcol_tot, SGP30TVOCcol_array);
    BME280Humcol_tot = cat(1, BME280Humcol_tot, BME280Humcol_array);
    BME280Presscol_tot = cat(1, BME280Presscol_tot, BME280Presscol_array);
    BME280Tempcol_tot = cat(1, BME280Tempcol_tot, BME280Tempcol_array);

    
end

I receive this error each time:

Error using datetime/cat (line 1376)
All inputs must be datetimes or date/time character vectors or date/time strings.

Error in Plot_attempt_9_6_22_1 (line 66)
    timecol_tot  = cat(1, timecol_tot, timecol_array);

As per How to preallocate a datetime array in matlab, I have tried:

timecol_tot = [];,
timecol_tot = datetime([],[],[],[],[],[]);, and
timecol_tot = NaT(1,1); to no avail.

Because the length of each of these files may vary, I didn't try pre-allocating the datetime array in the size of the incoming data, since that may not work across different datasets. However, it does work if I have only one file.

Is there a way to do this that would allow me to just initialize an empty array and concatenate datetime arrays to it without defining the size of the first datetime set to add?

matlab datetime arrays
2 Answers

You are asking 2 questions, I am new here but some moderators may tell you break it down into 2 separate questions.

I am here answering both anyway :

  1. The error doesn't seem to be coming from cat

With the data you have provided I have reproduced cat output and cat works fine; it concatenates DATATIME when required and floats told to.

The problem has to be somewhere else ; as long as you feed cat with same format there shouldn't be any error.

The error comes from a function called Plot_attempt_9_6_22_1 right?

Either

  • you are sending to plot rows instead of columns therefore plot input vectors contain mixed formats, a transpose or more than one missing somewhere,

or

  • somewhere your are mixing date/time formats with something that is not date/time type.

2.- Do you really need to preallocate memory?

In any case MATLAB suggests to preallocate tables with the following expression :

T = table('Size',sz,'VariableTypes',varTypes)

not with the C/C++ style mentioned in question

John BG0

I ended up fixing this problem by adding an if statement, and now, it looks something like this:

if k == 1
    timecol_tot  = timecol_array;
    EZOCO2col_tot = cat(1, EZOCO2col_tot, EZOCO2col_double);
    EZOO2col_tot = cat(1, EZOO2col_tot, EZOO2col_double);
    SGP30CO2col_tot = cat(1, SGP30CO2col_tot, SGP30CO2col_array);
    SGP30TVOCcol_tot = cat(1, SGP30TVOCcol_tot, SGP30TVOCcol_array);
    BME280Humcol_tot = cat(1, BME280Humcol_tot, BME280Humcol_array);
    BME280Presscol_tot = cat(1, BME280Presscol_tot, BME280Presscol_array);
    BME280Tempcol_tot = cat(1, BME280Tempcol_tot, BME280Tempcol_array);
else
    timecol_tot  = cat(1, timecol_tot, timecol_array);
    EZOCO2col_tot = cat(1, EZOCO2col_tot, EZOCO2col_double);
    EZOO2col_tot = cat(1, EZOO2col_tot, EZOO2col_double);
    SGP30CO2col_tot = cat(1, SGP30CO2col_tot, SGP30CO2col_array);
    SGP30TVOCcol_tot = cat(1, SGP30TVOCcol_tot, SGP30TVOCcol_array);
    BME280Humcol_tot = cat(1, BME280Humcol_tot, BME280Humcol_array);
    BME280Presscol_tot = cat(1, BME280Presscol_tot, BME280Presscol_array);
    BME280Tempcol_tot = cat(1, BME280Tempcol_tot, BME280Tempcol_array);
end

Another reason that I was getting a type error was because some of my files were empty or corrupted. If you're having similar issues, check to make sure that you're not working with empty or corrupted csv's!

czar12490
Related