I am using helium to scrape a webpage.
JavaScript
x
6
1
result = start_firefox(
2
"https://www.medtronic.com/covidien/en-us/products/brain-monitoring/bis-monitoring-system.html",
3
)
4
5
result.find_element_by_class_name("js-open-table-overlay").click()
6
After the click action i am presented with a table and i need to scrape the contents of the table but How do i select the table after the click ?
Advertisement
Answer
You would need to use find_elements_...
to get all <table>
, and use for
-loop to work with every table separatelly amd use (nested) for
-loop to get <tr>
(row) and <th>
(header) in table, and use (nested) for
-loop to get <td>
(cell) in row. And you would have to add elements to correct (nested) lists.
But this page uses normal <table>
and this table is all time in HTML (not added by javaScript) so it can be simpler to use pandas.read_html()
to get all <table>
JavaScript
1
9
1
import pandas as pd
2
3
url = "https://www.medtronic.com/covidien/en-us/products/brain-monitoring/bis-monitoring-system.html"
4
5
all_tables = pd.read_html(url)
6
7
for table in all_tables:
8
print(table.to_string())
9
Result
JavaScript
1
12
12
1
ORDER CODE DESCRIPTION UNIT OF MEASURE QUANTITY
2
0 186-1014 BIS™ Complete 4-Channel Monitoring System Each 1
3
1 186-0210 BIS™ Complete 2-Channel Monitor Each 1
4
2 186-0224-AMS BIS™ LOC 4-Channel Monitor with Patient Interface Cable (PIC-4) Each 1
5
3 186-0195-AMS BIS™ LOC 2-Channel Monitor with Patient Interface Cable Each 1
6
4 186-0212 BIS™ Bilateral Sensor Each 1
7
8
DISPLAY BIS™ COMPLETE 2-CHANNEL MONITOR BIS™ COMPLETE 4-CHANNEL MONITORING SYSTEM
9
0 Parameters BIS, SQI, EMG, SR, BC, EEG BIS, SQI, EMG, SR, BC, TP, SEF, EEG
10
1 Trended parameters BIS, SQI, EMG, SR, BC BIS, SQI, EMG, SR, BC, SEF
11
2 BIS™ alarm Upper and lower limit, results in visual and audible alert when out of range Upper and lower limit, results in visual and audible alert when out of range
12