Skip to content

How can I print more links from the HTML DOM using Python Selenium?

undetected Selenium
ADAM

Html: 

<div class="xxxx">
<a href="ooooo.pdf"></a>
</div>

Python selenium code trials:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
print(wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.xxxx a"))).get_attribute('href'))

Output:

aaaaa.pdf

How print ooooo.pdf and aaaaa.pdf?

I want to print more links, what should I do?

Advertisement

Answer

element_to_be_clickable() returns a single WebElement hence href attribute of only the first matching element is printed.

Solution

To extract all the href attribute values you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following locator strategies:

  • Using CSS_SELECTOR:

    print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.xxxx a")))])

  • Using XPATH:

    print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='xxxx']//a")))])

Alternative

As an alternative you can also try:

  • Using CSS_SELECTOR:

    print([my_elem.get_attribute("href") for my_elem in driver.find_elements(By.CSS_SELECTOR, "div.xxxx a")])

  • Using XPATH:

    print([my_elem.get_attribute("href") for my_elem in driver.find_elements(By.XPATH, "//div[@class='xxxx']//a")])