Skip to content

Tag: web-scraping

Calendar Data Picker in Selenium

Hi I am trying to figure it out how to pick day from the calendar in Python. I would like to select the date 04/27/2022 under requested loading date. I read some post related to this but I could not figure it out. This is what I did so far but not working. enter image description here Answer You do

How to use playwright and beautifulsoup on web page which has pagination?

Locked for 342 days. There are disputes about this question’s content being resolved at this time. It is not currently accepting new answers or interactions. I am new to web scraping. I want to scrape the data (comments and respective dates) from this web page https://forum.donanimhaber.com/turkcell-25-000-dk-12-000-sms-70-gb-internet-12-ay-320-tl-ana-konu–151777938 It has pagination for pages…. This is the way I am doing This code

How to go to the next page using BeautifulSoup?

I am trying to scrape data from all the 37 web pages from this website. The website I am scrapping doesn’t allow going to the next page through the search bar. This is the HTML written for the next button. I know that this can be done with Selenium, but is there any way to do this with BeautifulSoup? Is

Python helium get contents of table after click

I am using helium to scrape a webpage. After the click action i am presented with a table and i need to scrape the contents of the table but How do i select the table after the click ? Answer You would need to use find_elements_… to get all <table>, and use for-loop to work with every table separatelly amd

Selenium element is not attached to the page document

I am trying to scrape this particular site with Python: https://www.milanofinanza.it/quotazioni/ricerca/listino-completo-2ae?refresh_cens. I need to get all the isin codes and the names. My idea was to get them all in 2 separated lists, to do that I try to get the entire column (by changing the Xpath to tr rather than tr1) and then add it to the list. My

scraping multiple tags at once

I’m trying to scrape the imdb top 250 movies and I want to get all the links of those movies from this page https://www.imdb.com/chart/top/ I tried but I’m only getting the first link only, so my question is how to scale this code to include the whole list of 250 movies? Answer bs.find(‘td’,{‘class’:’titleColumn’}) gives you the first entry, and find_all(‘a’)