How to parse log file to csv file in Java

Following will do the transformation.

Answer 01

You can use the following solution if the structure of the logs are fixed.

while ((str = in.readLine()) != null) {
    str = str.replaceAll(" wt.nm=|&wt.ti=| ", ",").replace("&", "");
    System.out.println(str);
    wb.write(str);
}

Answer 02

If the structure is not fixed. You can use pattern matching to extract the relevant strings. Please refer to the following.

while ((str = in.readLine()) != null) {
    
    String nm = "";
    String ti = "";
    // First Assuming Date and time willmbe always there.
    String [] parts =  str.split(" ");

    String tokenizingString = parts[2];
    String pattern1 = "wt.nm=";
    String pattern2 = "wt.ti=";

    Pattern p = Pattern.compile(Pattern.quote(pattern1) + "(.*?)" + Pattern.quote("&"));
    Matcher m = p.matcher(str);
    // Assumes the pattern occurs once, hence only getting the first matcher
    if (m.find()) {
        nm = m.group(1);
    }
    
    Pattern p2 = Pattern.compile(Pattern.quote(pattern2) + "(.*?)" + Pattern.quote("&"));
    Matcher m2 = p2.matcher(str);
    
    if (m2.find()) {
        ti = m2.group(1);
    }
    
    String logEntry = parts[0] + "," + parts[1] + "," + nm + "," + ti;
    System.out.println(logEntry);
}

The format in your data (ignoring the leading date and time) looks like:

• wt.nm is always present
• wt.ti is always present
• wt.nm always appears before wt.ti
• there are never additional name+value pairs (beyond wt.nm and wt.ti)

However, the data has a more general pattern to it:

• one (or more) name+value pairs
• for each name+value pair, the name is separated from the value by =
• each name+value pair is separated from other pairs by &

Also, it's easy to imagine encountering additional name+value pairs (who says it will always be those same two forever? why not five name+values in a single line?), or perhaps the ordering changes (could wt.ti show up before wt.nm?).

The code below takes a general approach, working with the general pattern of your input data. I included some sample data to show how it works.

• the first input line – "2022-06-01 11:00:00 wt.nm=aa&wt.ti=t1&" – one of your original inputs
• the second line – "2022-06-01 11:00:00 xxxxxxxxxxx=bb&" – uses a different "name" altogether, and only has a single name+value pair (not two): xxxxxxxxxxx=bb
• the third line – "2022-06-01 11:00:00 xxxxxxxxxxx=cc&yyyyyyyyyyyy=t3&zzzzzzzz=99&" – includes a third name+value pair: zzzzzzzz=99. who says there will always be just two name+value pairs?

String[] lines = {
        "2022-06-01 11:00:00 wt.nm=aa&wt.ti=t1&",
        "2022-06-01 11:00:00 xxxxxxxxxxx=bb&",
        "2022-06-01 11:00:00 xxxxxxxxxxx=cc&yyyyyyyyyyyy=t3&zzzzzzzz=99&"
};
for (String line : lines) {
    System.out.println("original: " + line);
    String edit1 = line.replaceAll("&", " ");
    System.out.println("          " + edit1);

    StringTokenizer tokenizer = new StringTokenizer(edit1);
    StringBuilder finalLine = new StringBuilder();
    while (tokenizer.hasMoreTokens()) {
        String token = tokenizer.nextToken();
        System.out.println("   token: " + token);
        if (token.contains("=")) {
            int positionOfEqualsSign = token.indexOf("=");
            String value = token.substring(positionOfEqualsSign + 1);
            finalLine.append(value);
        } else {
            finalLine.append(token);
        }
        if (tokenizer.hasMoreTokens()) {
            finalLine.append(",");
        }
    }
    System.out.println("   final: " + finalLine);
    System.out.println();
}

Here's the output from that code, which includes a lot of extra output in order to be easier to follow:

original: 2022-06-01 11:00:00 wt.nm=aa&wt.ti=t1&
          2022-06-01 11:00:00 wt.nm=aa wt.ti=t1 
   token: 2022-06-01
   token: 11:00:00
   token: wt.nm=aa
   token: wt.ti=t1
   final: 2022-06-01,11:00:00,aa,t1

original: 2022-06-01 11:00:00 xxxxxxxxxxx=bb&
          2022-06-01 11:00:00 xxxxxxxxxxx=bb 
   token: 2022-06-01
   token: 11:00:00
   token: xxxxxxxxxxx=bb
   final: 2022-06-01,11:00:00,bb

original: 2022-06-01 11:00:00 xxxxxxxxxxx=cc&yyyyyyyyyyyy=t3&zzzzzzzz=99&
          2022-06-01 11:00:00 xxxxxxxxxxx=cc yyyyyyyyyyyy=t3 zzzzzzzz=99 
   token: 2022-06-01
   token: 11:00:00
   token: xxxxxxxxxxx=cc
   token: yyyyyyyyyyyy=t3
   token: zzzzzzzz=99
   final: 2022-06-01,11:00:00,cc,t3,99