I just need a little help finding an element in my python script with Beautiful Soup.
Below is the html:
<div class="lg-7 lg-offset-1 md-24 sm-24 cols"> <div class="row pr__prices"> <div class="lg-24 md-12 cols"> <input id="analytics_prodPrice_47187" type="hidden" value="2.91"> <div class="pr__pricepoint"> <div id="product_price" class="pr__price"> <span class="pound"> £</span>3<span class="pence">.49<span class="incvat">INC VAT</span></span> <span class="price__extra">(<span id="unit_price">£11.26</span>/<span id="unit_price_measure">Ltr</span>)</span> </div> </div>
What I am trying to do is get the product price, and looking at the html above, it looks like it is found within this section from the html above (price is £3.49):
<div id="product_price" class="pr__price"> <span class="pound"> £</span>3<span class="pence">.49<span class="incvat">INC VAT</span></span> <span class="price__extra">(<span id="unit_price">£11.26</span>/<span id="unit_price_measure">Ltr</span>)</span> </div>
My issue is that even though I use Beautiful Soup to try and get the price like so:
pound = soup.find('span',attrs={'class':'pound'}) pence = soup.find('span',attrs={'class':'pence'}) prices.append(pound.text + pence.text)
I get this exception stating:
prices.append(pound.text + pence.text) AttributeError: 'NoneType' object has no attribute 'text'
So to me it looks like it’s returning a None or null. Does anybody have an idea on how I can get to the element?
EDIT
Looking at the answers below, I tried to replicate them but instead of using a static HTML, I call on the website url. What I noticed is that even though the code works for a static html, it doesn’t work when I call on the url that contains the page that contains that html.
CODE:
from bs4 import BeautifulSoup import pandas as pd import requests data = requests.get('https://www.screwfix.com/p/no-nonsense-sanitary-silicone-white-310ml/47187').text soup = BeautifulSoup(data, 'html.parser') currency = soup.select_one('span.pound') currency_str = next(currency.strings).strip() pound_str = currency.nextSibling pence = soup.select_one('span.pence') pence_str = next(pence.strings).strip() print(f"{currency_str}{pound_str}{pence_str}") # £3.49
Error:
currency_str = next(currency.strings).strip() AttributeError: 'NoneType' object has no attribute 'strings'
Advertisement
Answer
Here’s a different approach.
from bs4 import BeautifulSoup data = ''' <div class="lg-7 lg-offset-1 md-24 sm-24 cols"> <div class="row pr__prices"> <div class="lg-24 md-12 cols"> <input id="analytics_prodPrice_47187" type="hidden" value="2.91"> <div class="pr__pricepoint"> <div id="product_price" class="pr__price"> <span class="pound"> £</span>3<span class="pence">.49<span class="incvat">INC VAT</span></span> <span class="price__extra">(<span id="unit_price">£11.26</span>/<span id="unit_price_measure">Ltr</span>)</span> </div> </div> ''' soup = BeautifulSoup(data, 'html.parser') currency = soup.select_one('span.pound') currency_str = next(currency.strings).strip() pound_str = currency.nextSibling pence = soup.select_one('span.pence') pence_str = next(pence.strings).strip() print(f"{currency_str}{pound_str}{pence_str}") # £3.49