Object of type 'Response' has no len() when trying to scrape a website

user_10130455 0
Viewed 740

I'm trying to run this for loop to scrape some data from a website.

from bs4 import BeautifulSoup
import requests

source = requests.get('https://atp-ejendomme.dk/find-lejemaal/christiansbro/')

soup = BeautifulSoup(source, 'lxml')

for div in soup.find_all('div', class_='fact no-hover'):
    fact_title = div.find('p', class_='fact_title tag visible')
    print(fact_title)

    fact_answer = div.find('p', class_='fact_answer tag visible')
    print(fact_answer)

    print()

But when I run it I get this error

Traceback (most recent call last):
  File "/Users/officehub/Desktop/Scrapers/atpScraper.py", line 21, in <module>
    soup = BeautifulSoup(source, 'lxml')
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/bs4/__init__.py", line 307, in __init__
    elif len(markup) <= 256 and (
TypeError: object of type 'Response' has no len()
web-scraping beautifulsoup python
1 Answers

Your code is almost correct, but some CSS classes are added dynamically via Javascript. Change the selectors to get the information, for example:

import requests
from bs4 import BeautifulSoup


url = 'https://atp-ejendomme.dk/find-lejemaal/christiansbro/'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

for k, v in zip(soup.select('.fact_title'), soup.select('.fact_answer')):
    print('{:<25} {}'.format(k.text, v.text))

Prints:

Basisleje                 1.850 kr./m²/år
Skatter og afgifter       333 kr./m²/år
Drift                     406 kr./m²/år
Vand og varme             186 kr./m²/år
Fordeling                 3.386 m²
Benyttelse                Kontor
Energimærke               C
Sagsnummer                2601
user_100359850
Related