sed - remove multiple lines if contains ends in 'can't find label'

user_466227 0
Viewed 89

I have a very large log file (more than 2GB) and would like to remove all debug logs if they contain 'EntityFramework'.

info: Microsoft.EntityFrameworkCore.Migrations[20405]
      No migrations were applied. The database is already up to date.
dbug: Microsoft.EntityFrameworkCore.Infrastructure[10407]
      'IDCDbContext' disposed.
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
      Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.
info: Hangfire.PostgreSql.PostgreSqlStorage[0]
      Start installing Hangfire SQL objects...

Here I would like to remove the log below and keep the others

dbug: Microsoft.EntityFrameworkCore.Infrastructure[10407]
      'IDCDbContext' disposed.

What I've tried so far:

sed -i '/^dbug/{:b,N;/^[[:lower:]]/!bb};/.*EntityFramework.*/d' logs

However it results in sed: can't find label for jump to b'

Any idea?

sed regex
3 Answers

This job is better suited for awk:

awk '!p || !/^[[:blank:]]/ {p = /^dbug:/} !p' file

info: Microsoft.EntityFrameworkCore.Migrations[20405]
      No migrations were applied. The database is already up to date.
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
      Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.
info: Hangfire.PostgreSql.PostgreSqlStorage[0]
      Start installing Hangfire SQL objects...

We keep a flag p to control whether to print or not. p is set to 1 when a line starts with /dbug:/ and remains set for lines following dbug: that start with whitespaces.

user_5482250

The current error is due to the comma after the b label, there must be a semi-colon. Also, you should include /.*EntityFramework.*/d (or better, /EntityFramework/d) into the command block so that it is executed only inside it:

sed -i '/^dbug/{:b;N;/^[[:lower:]]/!bb;/EntityFramework/d}' logs

See the online demo.

user_38329700

1st solution: With your shown samples, please try following awk program, written and tested in GNU awk. Simple explanation would be, using awk's match function to match regex \ndbug:[^\n]*\n[^\n]* and printing only those lines which are required by OP's shown output(not matched lines only).

awk -v RS= 'match($0,/\ndbug:[^\n]*\n[^\n]*/){print substr($0,1,RSTART-1) substr($0,RSTART+RLENGTH)}' Input_file

2nd solution: Using record separator capability of awk and print the appropriate values needed by OP.

awk -v RS='\ndbug:[^\n]*\n[^\n]*\n' '{gsub(/\n+$/,"")}1' Input_file
user_58665800
Related