Parse text with uncertain number of fields

This solution is a bit long, but should be easy to work with:

#!/usr/bin/env python3

# main.py

import csv
import fileinput
import re


def filter_fields(row):
    output = []
    for field_number, field in enumerate(row, 1):
        if field_number == 1:
            output.append(field)
        elif "DD:" in field or "GG:" in field:
            output.append(field.split()[0])
        elif "transcript=" in field:
            # Remove stuff from after the colon to the last space
            output.append(re.sub(r":.* ", ":", field))

    return "\t".join(output)


reader = csv.reader(fileinput.input(), delimiter="\t")
for row  in reader:
    print(filter_fields(row))

How to run it:

# Output to the screen
python3 main.py text.txt

# Output to a file
python3 main.py text.txt > out.txt

# Use as a filter
cat text.txt | python3 main.py

Notes

• In this solution, each line of text is broken into a row of fields.
• The function filter_fields will take each row, decide what field to to keep and reformat. It then return those fields, tab separated.
• The re.sub(...) call says: Delete everything after the colon, up to the last space.

Assuming those \t in your sample text are real tabs, this Perl one liner will do it. If they're literal \t text then this needs to be tweaked a tad. Put each field you want to grab in the regex alternation after GG:.

perl -lne '@wanted = $_ =~ m{(^Gene[ABCD]|(?:transcript=|DD:|GG:)\S+)}g; print join "\t", @wanted if @wanted; ' inputfile.txt > outputfile.txt

Output:

GeneA   transcript=aaabb.1      DD:hudnf.1      GG:jdubf.1
GeneB   transcript=sdfs.1       DD:sdfsw.1      GG:fghfg.1
GeneC   DD:hudnf.1      GG:jdubf.1
GeneD   transcript=asdasd.2     transcript=aaabb.1      DD:hudnf.1      GG:jdubf.1

With regard to your clarification, this one liner will give you very fine control over what gets captured. This is more of a full parse approach.

perl -lne '($gene, @parts) = split/\t/,$_; my @wanted; foreach $p (@parts) { $p =~ m{(?|^([A-Z]{2}:).*?(transcript=\S+)|(DD:\S+)|(GG:\S+))} and push @wanted, "$1$2"; }; print join "\t", $gene, @wanted if @wanted; ' inputfile.txt > outputfile.txt

Output:

GeneA   CA:transcript=aaabb.1   DD:hudnf.1      GG:jdubf.1
GeneB   CA:transcript=sdfs.1    DD:sdfsw.1      GG:fghfg.1
GeneC   DD:hudnf.1      GG:jdubf.1
GeneD   AB:transcript=asdasd.2  CA:transcript=aaabb.1   DD:hudnf.1      GG:jdubf.1

This gets all DD and GG first components as well as any ID with a transcript= in it.

It works like this.

1. split out the gene and break up each tab seperated part
2. within each part grab the bits we care about with regex
3. save the bits that matched and push it into @wanted only if the regex matched
4. the (?| regex term causes the numbered capture variables to remain the same across alternations rather than keep going up. i.e. we only have $1 and $2 instead of $1, $2, $3, $4.
5. print out the line if there is anything @wanted

Bear in mind that with each "simplification" of the task you risk losing data that you actually want. You might end up with a full fledged parser if the data is complex enough.

Click here to read more about how to use Perl for pipelining. https://perldoc.perl.org/perlrun

The task is trivial and can be solved in one line of code by defining regular expression specifying information of interest.

To test on input data substitute while <DATA> with while <> and give filename as an argument to the script.

use strict;
use warnings;
use feature 'say';

my $re = qr/(Gene.|[A-Z]{2}:(?=[^:]*transcript=)|transcript=\S+|GG:\S+|DD:\S+)/;

say join("\t",/$re/g) while <DATA>;

exit 0;

__DATA__
GeneA   AB:xrbyk | jdnif | otherText    BB:abdf | jdhfkc | otherDifferentText   CA:bdmf | nfjvks | transcript=aaabb.1   DD:hudnf.1 type=cds GG:jdubf.1 type=cds
GeneB   BB:dfsfg | dfsfvdf | otherDifferent CA:zdcfsdf | xfgdfgs | transcript=sdfs.1    DD:sdfsw.1 type=cds GG:fghfg.1 type=cds
GeneC   AB:dsfsdf | xdvv | otherText1   BB:xdsd | sdfsdf | otherDifferentText2  DD:hudnf.1 type=cds GG:jdubf.1 type=cds
GeneD   AB:dfsdf | Asda | transcript=asdasd.2   CA:bdmf | nfjvks | transcript=aaabb.1   DD:hudnf.1 type=cds GG:jdubf.1 type=cds

Output

GeneA   CA:     transcript=aaabb.1      DD:hudnf.1      GG:jdubf.1
GeneB   CA:     transcript=sdfs.1       DD:sdfsw.1      GG:fghfg.1
GeneC   DD:hudnf.1      GG:jdubf.1
GeneD   AB:     transcript=asdasd.2     CA:     transcript=aaabb.1      DD:hudnf.1      GG:jdubf.1

With a small code change the exact desired output can be achieved

use strict;
use warnings;
use feature 'say';

my $re = qr/(Gene.|[A-Z]{2}:transcript=\S+|GG:\S+|DD:\S+)/;

while( <DATA> ) {
    s/[^:]*(?=transcript=)//g;
    say join("\t",/$re/g);
}

exit 0;

__DATA__
GeneA   AB:xrbyk | jdnif | otherText    BB:abdf | jdhfkc | otherDifferentText   CA:bdmf | nfjvks | transcript=aaabb.1   DD:hudnf.1 type=cds GG:jdubf.1 type=cds
GeneB   BB:dfsfg | dfsfvdf | otherDifferent CA:zdcfsdf | xfgdfgs | transcript=sdfs.1    DD:sdfsw.1 type=cds GG:fghfg.1 type=cds
GeneC   AB:dsfsdf | xdvv | otherText1   BB:xdsd | sdfsdf | otherDifferentText2  DD:hudnf.1 type=cds GG:jdubf.1 type=cds
GeneD   AB:dfsdf | Asda | transcript=asdasd.2   CA:bdmf | nfjvks | transcript=aaabb.1   DD:hudnf.1 type=cds GG:jdubf.1 type=cds

Output

GeneA   CA:transcript=aaabb.1   DD:hudnf.1      GG:jdubf.1
GeneB   CA:transcript=sdfs.1    DD:sdfsw.1      GG:fghfg.1
GeneC   DD:hudnf.1      GG:jdubf.1
GeneD   AB:transcript=asdasd.2  CA:transcript=aaabb.1   DD:hudnf.1      GG:jdubf.1