Selenium – couldn’t click next page

I’m trying to click the next button (>) and then repeat until the last page is reached. I’ve tried searching for solutions from similar questions but I couldn’t figure out what’s wrong/make it work.

Here is my code, thank you!

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as Wait
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import TimeoutException

driver = webdriver.Chrome("C:/Users/krish/Desktop/chromedriver_win32/chromedriver.exe")

driver.get('https://www.cnbcindonesia.com/tag/pasar-modal')
wait = WebDriverWait(driver,30)


while True:
 try:
    element = wait.until(EC.element_to_be_clickable((By.XPATH, "//a[@class = 'icon icon-angle-right']/i")))
    if (element != 0):
        element.click()
 except TimeoutException as ex:
        break

Answer

Your xpath looks incorrect, please try the below

Xpath

//i[@class='icon icon-angle-right']

CSS Selector

.icon.icon-angle-right

There are multiple approaches to address this issue and a couple of them are as follows:

As you intent to invoke click() you need to induce WebDriverWait in conjunction with the WebDriverWaitWebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".icon.icon-angle-right"))).click()

Using XPATH:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//i[@class='icon icon-angle-right']"))).click()

Incase the error …another element obscures it… still persists first you need to induce WebDriverWait inconjunction with the expected_conditions for the invisibility_of_element() of the blocking element as follows:

Using CSS_SELECTOR:

WebDriverWait(driver, 20).until(EC.invisibility_of_element((By.CSS_SELECTOR, ".icon.icon-angle-right")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".icon.icon-angle-right"))).click()

Using XPATH:

WebDriverWait(driver, 20).until(EC.invisibility_of_element((By.XPATH, "//i[@class='icon icon-angle-right']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//i[@class='icon icon-angle-right']"))).click()

If the issue still persists you can use the execute_script() method as follows:

Using CSS_SELECTOR:

WebDriverWait(driver, 20).until(EC.invisibility_of_element((By.CSS_SELECTOR, ".icon.icon-angle-right")))
driver.execute_script("arguments[0].click();", WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".icon.icon-angle-right"))))

Using XPATH:

WebDriverWait(driver, 20).until(EC.invisibility_of_element((By.XPATH, "//i[@class='icon icon-angle-right']")))
driver.execute_script("arguments[0].click();", WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//i[@class='icon icon-angle-right']"))))

Note: : You have to import below

from selenium.webdriver.support.ui import WebDriverWait       
from selenium.webdriver.common.by import By       
from selenium.webdriver.support import expected_conditions as EC