I am trying to scrape shopee.co.id using beautifulsoup and selenium. There are 60 product in a single search results page. At the end of the code, I checked the extracted data using len() and it shows that I only extracted 42 of them. How should I fix the code to obtain all the search results? Here is the cod…
Tag: beautifulsoup
Syntax error in For loop because of searching for a “:” in Python
This is probably a really easy fix but I don’t know how to do it. I’m using Python. I just want to search for og:title in description but the : in og:title keeps giving me a syntax error. I tried quotation marks but that didn’t work. I had the same problem with a – sign in another thin…
page scraping using beautiful soup, without links
I am using the following code to extract text from a web page: The problem is, when I open text, I get all the links from the bottoms that exist at the top of the page, which I don’t want. How can i modify the above code to do so? I also gets the footnotes, which i may want, but
How to scrape billboard using find_all?
I have a pretty specific question if anyone could help that would be awesome. I am trying to scrape the songs from https://www.billboard.com/charts/hot-100/ and I am stuck on this code. What should I put for tag = soup.find_all(‘???’) to get the title ‘About Damn Time. Answer Try to select t…
python beautifulsoup duplicating results
I’m trying to learn beatifulsoup (and python as a whole, pretty much still a beginner) and playing around with how to use it properly. I notice that when I scrape the website I’m testing for data from the search results, it lists it 3 times. Specifically, I’m trying to output the title, link…
Extracting JSON from HTML with BeautifulSoup
I’ve now tried everything for the past few hours but I can’t extract a specific thing from the HTML below. I want to grab the “sessionCartId” but I can’t figure out how…. Thats what i tried so far : ^^ This gives me the correct script tag but i cant transform it into a json…
Cannot Scrape All Links in Webpage Python BeautifulSoup
I am trying to use beautifulsoup to get the links off of this webpage: https://nfdc.faa.gov/nfdcApps/services/ajv5/fixes.jsp I need the links to all of the fixes in Arizona (AZ), so I search for AZ, and when I start by hitting ‘A’ under ‘View fixes in alphabetical order:’, I am not abl…
BeautifulSoup – can’t find attribute
I’m trying to scrape this link. I want to get to this part here: I can see where this part of the website is when I inspect the page: But I can’t get to it from BeautifulSoup. Here is the code that I’m using and all the ways I’ve tried to access it: Also, when I just look at the
Save Images with orginal name
I have script to download images from website. But it’s saves name with ‘images1, images2, images3, images4 etc’ I need to save images with orginal name. If the images name 43343.jpg i need to save with 43343.jpg I use beautifulsoup and requests for this case. Sorry my english. It’s no…
nonetype object has no attribute text error while scraping data
when i try to scrap data from this amazon link. I got AttributeError: ‘NoneType’ object has no attribute ‘text’ My Code: I think that the laptop_rating variable store the content in string format even if we not include .text. I’m thinking that might be the reason for getting None…