How to auto detect and parse a date format in Julia?

user_14847557 0
Viewed 110

I am trying to build an algorithm to perform auto detection of date time formats and parse them accordingly. And I would like to seek some advice on improving and enhancing my algorithm.

In the following code, I tried a simple approach to build all the possible date formats and then iterate over them to match the string to dateformat, once matched it will parse the date.

Code:

using Dates
function createDateFormats()
    sep = [",",".","-","/",":"]
    dateFormatComb = []
    for i in sep
        vals = [string("dd",i,"mm",i,"yyy"),string("mm",i,"dd",i,"yyy"),
                string("yyy",i,"mm",i,"dd"),string("yyy",i,"dd",i,"mm"),
                string("mm",i,"yyy",i,"dd"),string("dd",i,"yyy",i,"mm")
                ]
        push!(dateFormatComb, vals)
    end
    return vcat(dateFormatComb...)
end

function parse(x)
    dateFormat = createDateFormats()
    try
        for i in 1:size(dateFormat,1)
             try
                 val = Date.(x, dateFormat[i])
                 yearCol = Dates.year.(val)
                 monthCol = Dates.month(val)
                 dayCol = Dates.day.(val) 
                 dayofweekCol = Dates.dayofweek.(dayCol)
                 return yearCol, monthCol, dayCol, dayofweekCol
             catch
                    continue
             end
         end
     catch 
         throw(ArgumentError("Invalid date object"))
     end
end

However, this is quite limited and not efficient. Also, once the time is involved the complexity increases furthermore. May I ask, if someone has a better approach to perform such operations? Thanks, would appreciate all the suggestions and advice.

julia date datetime
1 Answers

Here is a way to do it that also outlines the ambiguity:

using Dates

const text = "2022-3-5 is one and 2021/12.14 5:30:00 is another"
const datetimeregex = 
   Regex(raw"(\d+)[:.\-/,](\d+)[:.\-,](\d+)(?:\s?T?\s?(\d+):(\d+):(\d+))?")

dt = [DateTime([parse(Int, s) for s in m.captures if s isa AbstractString]...)
                            for m in eachmatch(datetimeregex, text)]

@show dt # [DateTime("2022-03-05T00:00:00"), DateTime("2021-12-14T05:30:00")]

Now, what happens if the year is 2 digits? What happens if the order is mm, dd, yy versus dd, mm, yy? How do you tell whether 10-11-2021 is October 11 or November 10? In those cases you have to know what convention was used or errors will occur.

user_42828470
Related