Skip to content
Advertisement

How do I convert this selenium next button clicking code into loop so I can get url of all the pages until next button disappears

Hello I wrote this selenium code to click Next button and give me url of the next page.

The Question is

  1. I want to convert this code in a loop so I can click on the next button & collect all URLs until next button disappears.

  2. How do I out all the collected URLs in a list?

next = driver.find_element(By.LINK_TEXT, "Next")
next.click()
urrl = driver.current_url
print(urrl)
driver.quit()

I tried While True loop for this.

while True:
    try:
        urrl = driver.current_url    **## I tried this line after clicking the next button as well**                       
        next = driver.find_element(By.LINK_TEXT,"Next")
        next.click()
    except:
        break

I was able to click on the next button until the end but I can not figure out how to collect url of the webpage and how to append them into a list.

Tried append but I think I am doing something wrong.

Advertisement

Answer

You can write a function to test if the element exists:

def is_element_exists(xpath, id_flag=False):
    try:
        if id_flag:
            driver.find_element_by_id(xpath)
        else:
            driver.find_element_by_xpath(xpath)
        return True
    except Exception as e:
        # print("Excpetion:[%s][%s]" % (e, traceback.format_exc()))
        print('do not find the node')
    return False
User contributions licensed under: CC BY-SA
3 People found this is helpful
Advertisement