How get only token using sed, grep or awk?

Question

The Token is on middle of lines. I tried some possibilities, but not match, like:

sed -n '/matched/,$p' file
sed -n '/token=/,$p' file > out.txt
grep -oP '(?<=token: ).*' file
grep -Po 'token\s\K.*' file

I have this:

207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "GET /api_producao.php/procedimento?filter%5B%5D=status,eq,SIM&filter%5B%5D=clinicaId,eq,50&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 71194 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "GET /api_producao.php/salas?filter=clinicaId,eq,50&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 303 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "PUT /api_producao.php/sala_horario/14 HTTP/1.1" 200 1 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "GET /api_producao.php/sala_horario?filter=salaID,eq,15&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 141 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:33 -0300] "GET /api_producao.php/aux_clinica_profissional?filter=clinicaId,eq,42&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4 HTTP/1.1" 200 2730 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:33 -0300] "GET /api_producao.php/profissional?filter=id,in,5,0,9,7,51,10,25,24,33,34,48,0,45,27,105,130,115,118,119,124,0,127&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4 HTTP/1.1" 200 10827 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:33 -0300] "GET /api_producao.php/sala_horario?filter=salaID,eq,15&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 141 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"

And need this:


eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE

user_3832970 · Answer

You can use

grep -Po 'token=\K\S+' file

Sed solution:

sed -nE 's/.*token=([^[:space:]]+).*/\1/p' file

See the regex demo. Details:

token= - a fixed substring
\K - match reset operator that discards the text matched so far from the current match memory buffer
\S+ - one or more non-whitespace chars.

See the online demo:

#!/bin/bash
s='207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "GET /api_producao.php/procedimento?filter%5B%5D=status,eq,SIM&filter%5B%5D=clinicaId,eq,50&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 71194 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "GET /api_producao.php/salas?filter=clinicaId,eq,50&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 303 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "PUT /api_producao.php/sala_horario/14 HTTP/1.1" 200 1 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "GET /api_producao.php/sala_horario?filter=salaID,eq,15&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 141 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:33 -0300] "GET /api_producao.php/aux_clinica_profissional?filter=clinicaId,eq,42&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4 HTTP/1.1" 200 2730 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:33 -0300] "GET /api_producao.php/profissional?filter=id,in,5,0,9,7,51,10,25,24,33,34,48,0,45,27,105,130,115,118,119,124,0,127&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4 HTTP/1.1" 200 10827 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:33 -0300] "GET /api_producao.php/sala_horario?filter=salaID,eq,15&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 141 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"'
grep -Po 'token=\K\S+' <<< "$s"

Output:

eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE

The sed command means

-n - suppress default line output
E - enable POSIX ERE syntax
.*token=([^[:space:]]+).* - find any text, then token=, then capture any one or more non-whitespace chars into Group 1 and then match the rest of the text on the line
\1 - replace the match with Group 1 value
p - print the result of the substitution.

NOTE:

Whenever the value you need to extract has no space + protocol name right after it, and there can be other query params after the token one, you need to negate & as well as whitespace. Then just use grep -Po 'token=\K[^&\s]+' file and sed -nE 's/.*token=([^&[:space:]]+).*/\1/p' file.

user_874188 · Answer

Looks like you want

sed -n 's/.*token=$[^ &]*$.*/\1/p' file

or

grep -Po 'token=\K[^ &]*' file

user_16372109 · Answer

Using sed

$ sed -n '/GET/{s/.*=//;s/ .*//p}' input_file
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE

user_6266192 · Answer

Suggesting:

 grep -oP '(?<=&token)[^ ]*' input.1.txt

user_14672114 · Answer

A slightly more verbose but also more portable version - It's basically doing the good ole' $1=$1

after using FS itself to locate where the token is,
use 1 call to sub() to clean the tail end of anything that isn't part of the JSON token
force a comparison of "less than empty string", which is always false (0), and anything raised to zero-th power in awk becomes a numeric one (1),

thus allowing the entirety of the stuff inside

`$( )`

to become $(1), and using one more time of $1=$1 to auto-trim blanks at both ends

[m/n/g]awk '

 BEGIN { _=(___=index(FS="^.+[&]"\
            "token=","."))~(__="")
 } +_



eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE

eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4
.
.
.


Tested and confirmed to work on gawk 5.1.1, mawk 1.3.4, mawk 1.9.9.6, and macOS nawk
— The 4Chan Teller

user_5424988 · Answer

With awk you can use the match function, and take the 6 characters of token= into account when printing.

The [^[:space:]]+ part in the pattern matches 1 or more occurrences of any character except whitespace characters

awk 'match($0, /token=[^[:space:]]+/) {
  print substr($0, RSTART+6, RLENGTH-6)
}' file

Output

eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE

user_10785975 · Answer

I noticed jwt tag in your question, JWT uses base64 encoding, it starts with header which must have 2 keys: alg and typ, order is not important so prefix might be either {"alg" or {"typ" which after base64-encodings gives eyJhbGci or eyJ0eXAi. Taking above considerations in account I would use GNU AWK following way, let file.txt content be

207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "GET /api_producao.php/procedimento?filter%5B%5D=status,eq,SIM&filter%5B%5D=clinicaId,eq,50&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 71194 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "GET /api_producao.php/salas?filter=clinicaId,eq,50&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 303 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "PUT /api_producao.php/sala_horario/14 HTTP/1.1" 200 1 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:32 -0300] "GET /api_producao.php/sala_horario?filter=salaID,eq,15&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 141 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:33 -0300] "GET /api_producao.php/aux_clinica_profissional?filter=clinicaId,eq,42&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4 HTTP/1.1" 200 2730 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:33 -0300] "GET /api_producao.php/profissional?filter=id,in,5,0,9,7,51,10,25,24,33,34,48,0,45,27,105,130,115,118,119,124,0,127&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMiwiaXNzIjoiaHR0cHM6Ly93d3cuc2RycG9ydG8uY29tLmJyL2JhY2svb2RvbnRvbG9naWEvcHVibGljL2FwaS92MS9zaWduaW4iLCJpYXQiOjE2NDg0ODc1OTksImV4cCI6MTY0ODQ5MTE5OSwibmJmIjoxNjQ4NDg3NTk5LCJqdGkiOiJUZEtqTnlnWWxiZkJmVEpWIn0.ZNIZJtJQS8BbITnkgGL8j-8g7q47nZ7TTdIA2-6s2k4 HTTP/1.1" 200 10827 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"
207.244.229.206 - - [28/Mar/2022:14:35:33 -0300] "GET /api_producao.php/sala_horario?filter=salaID,eq,15&transform=1&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjExMSwiaXNzIjoiaHR0cHM6Ly9zZHJwb3J0by5jb20uYnIvYmFjay9vZG9udG9sb2dpYS9wdWJsaWMvYXBpL3YxL3NpZ25pbiIsImlhdCI6MTY0ODQ4Njk4MCwiZXhwIjoxNjQ4NDkwNTgwLCJuYmYiOjE2NDg0ODY5ODAsImp0aSI6IndsZVNhZ1ppb3VWVUFRb2cifQ.gnAzxxtWNdUY_Ech0Lj5v8TNMvSTOnjLWLHYhY3eMvE HTTP/1.1" 200 141 "-" "GuzzleHttp/6.5.5 curl/7.81.0 PHP/7.4.28"

then

awk 'BEGIN{FPAT="(eyJhbGci|eyJ0eXAi)[-_=A-Za-z0-9]+"}NF{print $1}' file.txt

output

eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9

Explanation: I use FPAT (Field Pattern) to inform GNU AWK that field starts with eyJhbGci or eyJ0eXAi which is then followed by 1 or more of characters from Base 64 Encoding with URL and Filename Safe Alphabet as described in RFC3548 (ASCII letters, ASCII digits, minus, understrike, equal sign)

(tested in gawk 4.2.1)

How get only token using sed, grep or awk?

7 Answers