How would I increment the numerical portion of a specific parameter in a URL string?

Here’s my specific example: param1 = ‘XXXXXXXXXXX’ param2 = ‘XXXX’ param3 = ‘XXXX’ param4 = ‘XXXX’ param5 = ‘XXXX’ param6 = ‘XXXX’ #param7 below holds the (always numerical) value that needs to be …

Python Scrape Forum for Title for each Post

I’m new to Web scraping and new to Python. I want to scrape for the title of each Posting at the Forum of the URL, so then a new Post is created with 1 of the Titles below i’d like to receive a Mail …

Python – Scraping a PDF file from a URL

I want to scrape pdf files from this site https://www.sigmaths.net/Reader.php?var=manuels/ph/physique_pilote_7b.pdf I tried this code for that but it doesn’t work. Can anybody tell me why, please? res …

Delete unwanted elements of python webscaping loop results

I’m currently trying to extract text and labels (Topics) from a webpage with the following code : Texts = [] Topics = [] url = ‘https://www.unep.org/news-and-stories/story/yes-climate-change-driving-…

Can’t get tags when scraping data

I am trying to scrape all tr tags using BeautifulSoup, but it returns none. Code: from urllib.request import urlopen from bs4 import BeautifulSoup url = ‘https://www.pro-football-reference.com/years/…

Why does linkextractor skip link?

I am Scraping some pages and am trying to use the LinkExtractor to get the URLs from the response. In general that is going quite ok, but the LinkExtractor is not able to extract the relative link to …

Webscraping Dynamic Website to Pull Recent News Article URLs

I am attempting to pull investing news articles from a dynamic website using Python. I have tried a couple of tutorials that worked for static websites, but I have had issues pulling the URL to a …

Stale element: iterating through webelement list python

I’m using selenium for the first time to get some information about a fantasy soccer game I play with my friends (we have a competition). I’m facing issues iterating through a list of webelements. …

Getting no data when scraping a table

I am trying to scrape historical data from a table in coinmarketcap. However, the code that I run gives back “no data.” I thought it would be fairly easy, but not sure what I am missing. url …

using Selenium to get texts inside ‘ul’ tag?

Please help me to find the solution to get the text inside ‘ul’ tag. I want to get the information which is separated with commas like: ‘Contains Enzymatically Active B-Vitamins, Dietary Supplement, …