Create a column with the domain name of each line from a URL in Python

user_16419508 0
Viewed 198

I'm beginning with Python and I'm having trouble creating a new column that gets the domain name from the URL column.

Could anyone explain to me how to do this ?

import pandas as pd

concu = pd.read_csv(r'/Desktop/Concu.csv')

df = pd.DataFrame(concu, columns= ['URL'])

#df3['Domain'] = #GET THE DOMAIN NAME (NO HTTP/HTTPS/WWW. AND without everything 
after .com, .io...)

print(df)




                                                   URL
0     https://smartkeyword.io/seo-outils-google-util...
1     https://smartkeyword.io/referencement-naturel-...
2     https://smartkeyword.io/seo-on-page-urls-optim...
3     https://smartkeyword.io/seo-outils-google-util...
4     https://smartkeyword.io/seo-outils-google-cach...
...                                                 ...
6758           https://www.primelis.com/consultant-seo/
6759       https://www.primelis.com/expertises/seo/aso/
6760  https://www.primelis.com/labo/consideration-li...
6761  https://www.primelis.com/faq/fonctionnement-re...
6762  https://www.primelis.com/blog/comment-est-ne-l...

[6763 rows x 1 columns]

Thank you very much !

pandas dataframe python regex
1 Answers

You can try to extract anything between // and / with www. as optional, and take the second group.

>>> df['URL'].str.extract('//(www\.){0,1}(.*?)/')[1]

0       smartkeyword.io
1       smartkeyword.io
2       smartkeyword.io
3       smartkeyword.io
4       smartkeyword.io
6758       primelis.com
6759       primelis.com
6760       primelis.com
6761       primelis.com
6762       primelis.com
Name: 1, dtype: object

As asked in comment, if you don't want to use group captures, you can look for zero or exact once occurrence of www. using (?...)? and extract the domain.

>>> df['URL'].str.extract('//(?:www\.)?(.*?)/')

                    0
0     smartkeyword.io
1     smartkeyword.io
2     smartkeyword.io
3     smartkeyword.io
4     smartkeyword.io
6758     primelis.com
6759     primelis.com
6760     primelis.com
6761     primelis.com
6762     primelis.com

NOTE: ?: indicates that the subpattern is a non-capture subpattern. That means whatever is matched in (?:www\.), even though it's enclosed by () it won't appear in the list of matches, only (.*?) will.

Since you mentioned in comment that there are sub-domains as well, if so try following:

df['URL'].str.extract('//(?:\w+\.)?(.*?\..*?)/')

The above regex will just capture the required text skipping www. or any other sub-domains.

user_91363480
Related