Skip to content
Advertisement

Unable to locate text from web page using Selenium

I am trying to scrape amazon reviews for a certain product, but I am unable to locate the text for the ratings using selenium. But the same thing is easily scraped using soup.

Link to page: https://www.amazon.in/BenQ-inch-Bezel-Monitor-Built/product-reviews/B073NTCT4R/ref=cm_cr_arp_d_paging_btm_next_2?ie=UTF8&reviewerType=all_reviews&pageNumber=39

Here is my code using Soup:

JavaScript

##Output 4.3 5.0 1.0 5.0 2.0 4.0 1.0 5.0 5.0 5.0 5.0 5.0 5.0

Here is my code using Selenium:

JavaScript

I am unable to get the same results with Selenium, all I get are blanks equivalent to the number of items present on that page. I have also tried using XPath and class_name but didn’t get the required response.

Advertisement

Answer

To get the review ratings Induce WebDriverWait and wait for presence_of_all_elements_located() and use get_attribute("innerHTML") instead of text

Code:

JavaScript

Output on console :

JavaScript
User contributions licensed under: CC BY-SA
10 People found this is helpful
Advertisement