Skip to content
Advertisement

Selenium element is not attached to the page document

I am trying to scrape this particular site with Python: https://www.milanofinanza.it/quotazioni/ricerca/listino-completo-2ae?refresh_cens.

I need to get all the isin codes and the names. My idea was to get them all in 2 separated lists, to do that I try to get the entire column (by changing the Xpath to tr rather than tr1) and then add it to the list.

My code goes through the pages but at a certain point just stop working (even if I add time.sleep(10) to be sure that the code starts scraping when the site is fully loaded).

My code looks like this:

wd = wd.Chrome()
wd.implicitly_wait(10)

wd.get('https://www.milanofinanza.it/quotazioni/ricerca/listino-completo-2ae')

company_name = []
isin = []
for n in range(0,15):
time.sleep(10)
tickers = wd.find_elements(By.XPATH,"//*[@id='mainbox']/div[2]/div[2]/div[4]/div/table/tbody/tr/td[1]")
isin = wd.find_elements(By.XPATH,"/html/body/div[3]/div/div/div/main/div[1]/div[2]/div[2]/div[4]/div/table/tbody/tr/td[10]/span")
for el in tickers:
    company_name.append(el.text)
for i in isin:
    isin.append(i.text)

l=wd.find_element(By.XPATH,"/html/body/div[2]/div/div/div/main/div[1]/div[2]/div[2]/div[4]/div/div[1]/div/button[4]")
wd.execute_script("arguments[0].click();",l)
print("data collected")

How can I solve this problem? Here some pictures to better understand:

Name:

enter image description here

Isin:

enter image description here

Advertisement

Answer

I’m sorry, but I can’t see how your existing code is working.
On my side I see the locator you are using for isin matching nothing.
I have updated locators here.
I would advice you never using automatically created locators.
Also the code you presenting here is missing indentations. I hope your actual code having proper indentations.
Please see if this will work better:

wd = wd.Chrome()

wd.get('https://www.milanofinanza.it/quotazioni/ricerca/listino-completo-2ae')

company_name = []
isin = []
for n in range(0,15):
    time.sleep(10)
    tickers = wd.find_elements(By.XPATH,"//table[contains(@class,'celled')]//tbody//tr//td[1]")
    isins = wd.find_elements(By.XPATH,"//table[contains(@class,'celled')]//tbody//tr//td[10]")
    for el in tickers:
        company_name.append(el.text)
    for is_el in isins:
        isin.append(is_el.text)
 
    l=wd.find_element(By.XPATH,'//nav//button[@ng-click="getDataTableNextClick()"]')
    wd.execute_script("arguments[0].click();",l)
User contributions licensed under: CC BY-SA
10 People found this is helpful
Advertisement