I am currently using Selenium for a webcrawling application, and after several successful modules, the following situation left me stuck: I’m trying to locate an element of the class “menu” with the text “Reports” that is located inside a frame named “frame_applic”.
Pretty straightforward, right? Should be as simple as:
browser = webdriver.Chrome()
#Do the necessary crawling to get to this specific menu...
browser.switch_to_frame('frame_applic')
for obj in browser.find_elements_by_class_name('menu'):
if obj.text == 'Reports':
bt_reports = obj
break
#The variable bt_reports should be the one requested.
Here is where it gets interesting: although no error is raised when I attempt to switch to the desired frame, no elements are found with a class name “menu”. Surely I must have misspelled the frame or the class name, right? So after deciding to inspect the HTML and doing no other change whatsoever, the elements are there, as if the mere act of looking at the source code of the webpage changed the outcome of the result. What could possibly be happening here?
I am not allowed to post the full HTML, but the following structure encompasses the question:
<html>
<head>
<TITLE>Page</TITLE>
</head>
<frameset ...>
<frame name="frame_menu" src="https:..." >
<frame name="frame_applic" src="menu.asp?...">
#document
<table width="100%" border="0" cellpadding="5" cellspacing="0">
<tr>
<td colspan="2" align="right">
<table width="95%" border="0" cellpadding="5" cellspacing="0">
<tr><td><a href="https:..." class="menu">Reports</a></td></tr>
<tr><td><a href="https:..." class="menu">Change Password</a></td></tr>
<tr><td><a href="https:..." class="menu">Change Secret Phrase</a></td></tr>
</table>
</td>
</tr>
</table>
</body>
<frame name="frame_bottom" src="https:...">
</frameset>
</html>
Advertisement
Answer
After some closer inspection, I found out that the underlying problem is that the source information from the desired frame is in an Active Server Page (asp), as seen under:
<frame name="frame_applic" src="menu.asp?...">
And it is not directly accessible from the page I’m on. The act of inspecting the elements on the browser apparently forces the elements to coexist in the webdriver instance, allowing me to fetch the elements inside the frame, whereas they wouldn’t be previously accessible.
I was able to solve this issue by first finding the requested frame, and then pointing the webdriver to the source of it, with:
browser.get(browser.find_element_by_xpath("//frame[@name='frame_applic']")
.get_attribute('src'))
And then simply getting the elements as I was doing previously, with:
menu_list = browser.find_elements_by_class_name('menu')