I’m trying to get all the names of the perfumes on this page
But when I try to get the text of the web element attribute I get nothing. In fact, I tried en colaboratory:
JavaScript
x
15
15
1
!pip install selenium
2
!apt-get update # to update ubuntu to correctly run apt install
3
!apt install chromium-chromedriver
4
!cp /usr/lib/chromium-browser/chromedriver /usr/bin
5
import sys
6
sys.path.insert(0,'/usr/lib/chromium-browser/chromedriver')
7
from selenium import webdriver
8
chrome_options = webdriver.ChromeOptions()
9
chrome_options.add_argument('--headless')
10
chrome_options.add_argument('--no-sandbox')
11
chrome_options.add_argument('--disable-dev-shm-usage')
12
wd = webdriver.Chrome('chromedriver',chrome_options=chrome_options)
13
wd.get("https://www.beaute-test.com/parfums_femme.php")
14
media_content = wd.find_elements_by_class_name("bt__media__content")
15
En media_content
we have several elements:
JavaScript
1
5
1
[<selenium.webdriver.remote.webelement.WebElement (session="18f3eed8343ccf0c22fc5c7bae48cff7", element="3123e486-a978-4113-b359-c3e9a89c7e95")>,
2
<selenium.webdriver.remote.webelement.WebElement (session="18f3eed8343ccf0c22fc5c7bae48cff7", element="4a7b2d4e-0179-4957-ab32-8f37470bc694")>,
3
<selenium.webdriver.remote.webelement.WebElement (session="18f3eed8343ccf0c22fc5c7bae48cff7", element="a7fb478d-aa55-46b8-8953-a7b4dd84220e")>,
4
]
5
But this one does not return anything:
JavaScript
1
5
1
>>>media_content[1].get_attribute('text')
2
''
3
>>>media_content[1].text
4
''
5
I also tried using the xpath directly on one name I wass looking for but it returns nothing as well.
So how do I get the text of a WebElement ?
Advertisement
Answer
To display all 25 of the names do the following.
JavaScript
1
4
1
media_content=wd.find_elements_by_css_selector("div[class='bt__media__content']>p>a>b")
2
for media in media_content:
3
print (media.text)
4
I also used to click on the agree popup.
JavaScript
1
2
1
WebDriverWait(wd, 10).until(EC.element_to_be_clickable((By.ID, "didomi-notice-agree-button"))).click()
2
Outputs
JavaScript
1
26
26
1
Libre
2
L'Interdit
3
Madame Glamour Suddenly
4
Gabrielle
5
My Way
6
Eau de Parfum - Alien
7
Idôle
8
Good Girl
9
Coco Noir
10
Scandal
11
Mon Guerlain
12
Mon Rouge!
13
Black Opium Eau de Parfum
14
Mauboussin à La Folie
15
Black Orchid
16
Voce Viva
17
La Nuit Trésor
18
Eau de Parfum - La Vie est Belle
19
Chloé Eau de Parfum
20
Eau de Parfum - La petite Robe noire
21
Suddenly Diamonds
22
Mon Paris Eau de Parfum
23
La Belle
24
Hypnotic Poison Eau de Toilette
25
Allure
26