I am using helium to scrape a webpage.
result = start_firefox( "https://www.medtronic.com/covidien/en-us/products/brain-monitoring/bis-monitoring-system.html", ) result.find_element_by_class_name("js-open-table-overlay").click()
After the click action i am presented with a table and i need to scrape the contents of the table but How do i select the table after the click ?
Advertisement
Answer
You would need to use find_elements_...
to get all <table>
, and use for
-loop to work with every table separatelly amd use (nested) for
-loop to get <tr>
(row) and <th>
(header) in table, and use (nested) for
-loop to get <td>
(cell) in row. And you would have to add elements to correct (nested) lists.
But this page uses normal <table>
and this table is all time in HTML (not added by javaScript) so it can be simpler to use pandas.read_html()
to get all <table>
import pandas as pd url = "https://www.medtronic.com/covidien/en-us/products/brain-monitoring/bis-monitoring-system.html" all_tables = pd.read_html(url) for table in all_tables: print(table.to_string())
Result
ORDER CODE DESCRIPTION UNIT OF MEASURE QUANTITY 0 186-1014 BIS™ Complete 4-Channel Monitoring System Each 1 1 186-0210 BIS™ Complete 2-Channel Monitor Each 1 2 186-0224-AMS BIS™ LOC 4-Channel Monitor with Patient Interface Cable (PIC-4) Each 1 3 186-0195-AMS BIS™ LOC 2-Channel Monitor with Patient Interface Cable Each 1 4 186-0212 BIS™ Bilateral Sensor Each 1 DISPLAY BIS™ COMPLETE 2-CHANNEL MONITOR BIS™ COMPLETE 4-CHANNEL MONITORING SYSTEM 0 Parameters BIS, SQI, EMG, SR, BC, EEG BIS, SQI, EMG, SR, BC, TP, SEF, EEG 1 Trended parameters BIS, SQI, EMG, SR, BC BIS, SQI, EMG, SR, BC, SEF 2 BIS™ alarm Upper and lower limit, results in visual and audible alert when out of range Upper and lower limit, results in visual and audible alert when out of range