I am trying to find menu prices for certain fast food restaurants from this website by different state. There is a dropdown menu where different states are the options. After I select a state (for example, California), I want to web scrape the different prices of their Ice Cream. However, I keep getting the same error message preventing me from getting the data. I think it works fine until going to the website, but it can’t seem to select it by state. What do I need to do in order to:
- Get selenium to accurately select a certain state (California)
- Get the menu prices for just the “Ice Cream” category?
The Code is below:
!pip3 install selenium !pip3 install webdriver-manager from selenium import webdriver import time from webdriver_manager.chrome import ChromeDriverManager from selenium.webdriver.support.ui import Select from selenium.webdriver.common.keys import Keys import pandas as pd path = ("C:/path/to/chromedriver.exe") driver = webdriver.Chrome(executable_path = path) url = "https://www.fastfoodmenuprices.com/baskin-robbins-prices/" driver.get(url) select_state = driver.find_element_by_xpath("//select[@class='tp-variation']").click ## select_state = driver.find_element_by_id("variation-tablepress-34") select_state = driver.find_element_by_id("variation-tablepress-34") select_state.select_by_value("MS4yOA==") time.sleep(3)
The Error I get is:
AttributeError: 'WebElement' object has no attribute 'select_by_value'
A photo of the code + error is below:
select_by_value() is a method from select class and to use it you have to use the Select(webelement) method.
To select the option California you can use the following locator strategy:
driver.get("https://www.fastfoodmenuprices.com/baskin-robbins-prices") Select(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//select[@class='tp-variation']")))).select_by_value("MS4yOA==")