I’m trying to get all the names of the perfumes on this page
But when I try to get the text of the web element attribute I get nothing. In fact, I tried en colaboratory:
!pip install selenium !apt-get update # to update ubuntu to correctly run apt install !apt install chromium-chromedriver !cp /usr/lib/chromium-browser/chromedriver /usr/bin import sys sys.path.insert(0,'/usr/lib/chromium-browser/chromedriver') from selenium import webdriver chrome_options = webdriver.ChromeOptions() chrome_options.add_argument('--headless') chrome_options.add_argument('--no-sandbox') chrome_options.add_argument('--disable-dev-shm-usage') wd = webdriver.Chrome('chromedriver',chrome_options=chrome_options) wd.get("https://www.beaute-test.com/parfums_femme.php") media_content = wd.find_elements_by_class_name("bt__media__content")
En media_content
we have several elements:
[<selenium.webdriver.remote.webelement.WebElement (session="18f3eed8343ccf0c22fc5c7bae48cff7", element="3123e486-a978-4113-b359-c3e9a89c7e95")>, <selenium.webdriver.remote.webelement.WebElement (session="18f3eed8343ccf0c22fc5c7bae48cff7", element="4a7b2d4e-0179-4957-ab32-8f37470bc694")>, <selenium.webdriver.remote.webelement.WebElement (session="18f3eed8343ccf0c22fc5c7bae48cff7", element="a7fb478d-aa55-46b8-8953-a7b4dd84220e")>,... ]
But this one does not return anything:
>>>media_content[1].get_attribute('text') '' >>>media_content[1].text ''
I also tried using the xpath directly on one name I wass looking for but it returns nothing as well.
So how do I get the text of a WebElement ?
Advertisement
Answer
To display all 25 of the names do the following.
media_content=wd.find_elements_by_css_selector("div[class='bt__media__content']>p>a>b") for media in media_content: print (media.text)
I also used to click on the agree popup.
WebDriverWait(wd, 10).until(EC.element_to_be_clickable((By.ID, "didomi-notice-agree-button"))).click()
Outputs
Libre L'Interdit Madame Glamour Suddenly Gabrielle My Way Eau de Parfum - Alien Idôle Good Girl Coco Noir Scandal Mon Guerlain Mon Rouge! Black Opium Eau de Parfum Mauboussin à La Folie Black Orchid Voce Viva La Nuit Trésor Eau de Parfum - La Vie est Belle Chloé Eau de Parfum Eau de Parfum - La petite Robe noire Suddenly Diamonds Mon Paris Eau de Parfum La Belle Hypnotic Poison Eau de Toilette Allure