I am trying to scrape a website like this: https://seeksophie.com/options/1-5hr-basic-candle-workshop. From this website, I’d like to get all date schedules (for 1 year) for the activity, and all of dates in the website are in form of span component. It is important for me to get notAllowed and flatpickr-disabled class from the component as I will have to filter available
Tag: web-scraping
How to extract the following src (iframe) from the code using python (BeautifulSoup)
I’m trying to extract the ‘src’ from this , but I’m not succeeding. This page’s is dynamic, it only appears if I search. Site: http://191.253.16.180:8080/ConsultaLei/Default.aspx?numero=3001 view-source:http://191.253.16.180:8080/ConsultaLei/Default.aspx?numero=3001 Answer To simulate POST on this site request you can use this example: Prints: EDIT: To get multiple pages: Prints:
Finding button by XPath with Selenium
I have been trying to locate the following button using Selenium WebDriver: I used css selector, xpath, class and nothing seems to work (even by just copy pasting the one given by the inspector. The closest I’ve been is locating the div with the class _1ibtygfe) Here is everything I tried, I’m desperate I don’t understand why it does not
Selenium error: Message: Select only works on elements, not on
I know that I should find “select” element so I can choose from drop-down list, But here google trends don’t provide “select” element and I want to choose any value from data and time list and When I try to do this I got this error ‘Message: Select only works on elements, not on ‘…. I found a solution but
Get element text with a partial string match using Selenium (Python)
I am trying to extract the text from within a <strong> tag that is deeply nested in the HTML content of this webpage: https://www.marinetraffic.com/en/ais/details/ships/imo:9854612 For example: The strong tag is the only one on the webpage that will contain the string ‘cubic meters’. My objective is to extract the entire text, i.e., “138124 cubic meters Liquid Gas”. When I try
Scraping dynamic dropdown with Selenium in Python
I woud like a bit of help with the following. I am trying to scrape the elements of the tickers’ dropdown on this website: https://live.hxro.io/tixwix My code is as follow using selenium This will only return As it is an Ajax call I am not sure how to retrieve the other tickers in an efficient way. I thought the function
Scraping etoro with python
I’m trying to use Selenium to automatically connect to my etoro account and get some figures from my portfolio. I work on Google Colab and from now, here is what I have: However, I have this error message I have tried to change and use find_element_by_class, by_xpath, etc but I couldn’t find how to do it. Could you give me
How to change names of scraped images with Python?
So I need to download the images of every coin on the list on CoinGecko, so I wrote the following code: However, I need to save the images with their names being the same as the ticker of the coin of that list from CoinGecko (rename bitcoin.png?1547033579 to BTC.png, ethereum.png?1595348880 to ETH.png, and so forth). There are over 7000 images
Get embedded JSON data in div using selenium
I am new to scraping with Selenium and am stumped on how to extract a JSON that is conveniently available directly within a div. The div even contains a tag named data-json The basic code I have so far is: The JSON that I am looking for is a div within a div, where the outermost div has a tag
Format the extracted covid vaccine data from website
Trying to format the “Vaccine data” from URL to pandas dataframe https://www.mygov.in/sites/default/files/covid/vaccine/covid_vaccine_timeline.json Here is the parent website https://www.mygov.in/ Sample output I am trying to extract the data in the below format in my data frame Answer