A div of class=”tableBody” has many divs as children. I want to get all its div child and get the string which I have highlighted in this picture. the above code returns me a empty list. I am trying to learn BS4. I appreciate it if you could help me with the code. Answer The data you see on the
Tag: beautifulsoup
How to extract the URL of a webpage without knowing beforehand?
I’m trying to make an iterative web search that pulls up a google search page ONLY when it needs to. Therefore, I don’t know the URLs ahead of time. I am aware of the .current_url argument from Selenium but it does not give me what I want. When I do print(driver.current_url) I only get https://www…
Beautiful Soup findAll() finds half of them
I’m trying to scrape information on the price of offices in France and I successfully developed the code to scrape all the information I needed. Though, I quickly noticed that something was wrong with the number of outputs and more precisely that my algorithm was returning only half of the occurences pr…
Extract individual links from a single youtube playlist link using python
I need a python script that takes link to a single youtube playlist and then gives out a list containing the links to individual videos in the playlist. I realize that same question was asked few years ago, but it was asked for python2.x and the codes in the answer don’t work properly. They are very wei…
A problem with web scraping using python ,BeautifulSoup and pandas ‘read_html’
Thank you for the helpers ! I am scraping a table of data about covid19 and push it into a pandas data frame , it was working until this morning . That the code : This morning I starting to get the next error : Can you please help me figure it out ? Answer Try changing the last line
unable to scrape status of product
I want to scrape price and status of website. I am able to scrape price but unable to scrape status. Couldn’t find in JSON as well. here is link: https://www.zoro.com/jonard-tools-diagonal-cutting-plier-8-l-jic-2488/i/G2736212/?recommended=true Answer You can use Json microformat embedded inside the pag…
Need help parsing link from iframe using BeautifulSoup and Python3
I have this url here, and I’m trying to get the video’s source link, but it’s located within an iframe. The video url is https://ndisk.cizgifilmlerizle.com… inside an iframe called vjs_iframe. My code is below: I don’t know how to get the url within the iframe, since not even the…
web-scraping error message: ‘int’ object has no attribute ‘get’
Hello Stack Overflow contributors! I want to scrape multiple pages of a news website; it shows an error message during this step The error message is The lines of code are More specifically, this page and pages next to it are what I want to scrape: https://nypost.com/search/China+COVID-19/page/1/?orderby=rele…
Properly formatting the table after scraped by BeautifulSoup
I am new to Python. I have been trying to scrape a table from http://www.phc4.org/reports/utilization/inpatient/CountyReport20192C001.htm. The targeted table is titled as “Utilization by Body System”. I was able to capture the table by using BeautifulSoup; however, the scraped dataframe has been d…
Python POST request for web scraping
I am using beautifulsoup and requests to scrape html contents for this webpage. Based on the selection made in the page — a list of stations is populated in the page. Clicking on any one station renders an html page with td values. For e.g. My objective is to get data for each station from the list. I a…