Skip to content
Advertisement

soup.select() returns an empty list

I have an issue with .select which always returns an empty list while practicing webscraping. I am working on the following page: https://presse.ania.net/news/?page=1 using BeautifulSoup.

I am getting and parsing HTML as following:

JavaScript

I try to retrieve the urls of each articles displayed on the page, under class “title row-space-1” (I use developer tools of chrome to find class, disabled JavaScript like suggested in other posts), and put them in a list called “news”

JavaScript

However I keep having an empty list when I print ‘news’

JavaScript

Searching on Stackoverflow I tried:

  • Disabling JavaScript on website
  • Adding a time sleep to let the page download
  • Using .find_all, . find and .select, tried with CSS selectors first then kwargs (all return empty list or NoneType object).

None of these worked and I am stuck with my mistake. I think there is something specific in my way of understanding this HTML and selecting class with CSS but I can’t find what (partly because I successfully used this code for other websites earlier.).

Could you please educate me on what I am missing?

I appreciate your help!

Advertisement

Answer

Try this:

JavaScript

Output:

JavaScript
User contributions licensed under: CC BY-SA
8 People found this is helpful
Advertisement