Selenium is unable to get element.text while using python and web driver

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
chrome_driver_path = "/Users/greta/Development/chromedriver"
driver = webdriver.Chrome(executable_path=chrome_driver_path)

driver.get("https://en.wikipedia.org/wiki/Main_Page")

articles = driver.find_element(By.XPATH, "/html/body/div[3]/div[3]/div[5]/div[1]/div[1]/div/div[3]/a[1]")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[3]/div[3]/div[5]/div[1]/div[1]/div/div[3]/a[1]"))).click()
try:
    number_of_articles = articles.get_attribute("textContent")
    print(number_of_articles)
except:
    print("Unable to find")
finally:
    driver.quit()

I am learning Selenium with Python. Above is my code. I want to retrieve this circled element in Wikipedia, as we can see it is an anchor tag<a> that is within a <div> that has an id of “articlecount”. I’ve tried the By.CSS_SELECTOR which didn’t work, so I use XPATH instead as shown in the code. But it still doesn’t work.

Then I find out that if I print the articles variable alone, the code won’t raise an error. However, as soon as I print articles.text or articles.get_attribute("textContent"), the error still occurs.

Would be great if anybody can give me a helping hand. Thanks in advance.

enter image description here

Answer

You have a small bug. In this line, you are clicking on the element after finding it, which causes the page to change and you cant find the article count.

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[3]/div[3]/div[5]/div[1]/div[1]/div/div[3]/a[1]"))).click()

Just remove the click() and it will work

Advertisement

Answer