How grab urls from text file which have .com .org .net domain with unix command

1st solution: With your shown samples, please try following awk code. Simple explanation would be, setting field separator as space OR / for all the lines and in main block of awk program checking if line starts with HTTP: then print 3rd field else print 1st field to get required values as per requirements.

awk -F' |/' '/^HTTP:/{print $3;next} {print $1}' Input_file

2nd solution: Using sed please try following code. Using -E option of -E to enable ERE(extended regular expressions) and capturing group capability of sed here. Here is the Online demo for used regex in sed code.

sed -E 's/^(HTTP:\/\/)?([^[:space:]]+).*$/\2/'  Input_file

3rd solution: Using GNU grep here with its \K option which allows us to match things with regex and forget/neglect them while printing. Here is the Online demo for used regex in grep solution.

grep -oP '^(HTTP:\/\/)?\K([^[:space:]]+)'  Input_file

I would harness GNU sed for this task following way, let file.txt content be

site.com   - org name - title
site.net   - other name - another title
HTTP://target.ca - ca site - ca title

then

sed -e 's|^http://||i' -e 's|[[:space:]].*||' file.txt

gives output

site.com
site.net
target.ca

Explanation: I inform GNU sed to apply two changes, 1st is to replace http:// at begin of line (^) with empty line acting in case-insensitive way (i), 2nd is to replace first whitespace character and everything beyond it with empty string. Observe that I use | as delimiters and therefore I do no need to escape / character.

(tested in GNU sed 4.2.2) (tested in

Another simple awk solution:

awk '{sub(/^HTTP:\/\//, ""); print $1}' file

site.com
site.net
target.ca