I'm making a scraping project with bs4 and I simply cannot see a way to make it less eye-bleeding. Everything looks like this:
try:
max_tweet_date = max(
[tweet_date_to_datetime(timeline_item.find('span', {'class': 'tweet-date'}).find('a')['title'])
for timeline_item in tweets_or_replies_soup.find_all('div', {'class': 'timeline-item'})]
)
except AttributeError:
print('No tweets or replies')
print()
neo4j_database.set_user_updated_now(current_user)
except IndexError:
print('Not enough tweets or no tweets')
print()
neo4j_database.set_user_updated_now(current_user)
except ValueError:
print("Cannot recognise dates, skipping")
print()
neo4j_database.set_user_updated_now(current_user)
if is_tweet_too_old(max_tweet_date):
print('User is inactive, deleting')
print()
neo4j_database.delete_user(current_user)
There are always tons of exceptions that need to be handled and the way it handles attributes is begging to be contained in a separate function. Are there any helpful patterns or maybe other parsing libraries that can help clean up this mess?