Python

I just need a little help finding an element in my python script with Beautiful Soup.

Below is the html:

<div class="lg-7 lg-offset-1 md-24 sm-24 cols">
<div class="row pr__prices">
<div class="lg-24 md-12 cols">
<input id="analytics_prodPrice_47187" type="hidden" value="2.91">
<div class="pr__pricepoint">
   <div id="product_price" class="pr__price">
      <span class="pound">
      £</span>3<span class="pence">.49<span class="incvat">INC VAT</span></span>
      <span class="price__extra">(<span id="unit_price">£11.26</span>/<span id="unit_price_measure">Ltr</span>)</span>
   </div>
</div>

What I am trying to do is get the product price, and looking at the html above, it looks like it is found within this section from the html above (price is £3.49):

   <div id="product_price" class="pr__price">
      <span class="pound">
      £</span>3<span class="pence">.49<span class="incvat">INC VAT</span></span>
      <span class="price__extra">(<span id="unit_price">£11.26</span>/<span id="unit_price_measure">Ltr</span>)</span>
   </div>

My issue is that even though I use Beautiful Soup to try and get the price like so:

pound = soup.find('span',attrs={'class':'pound'})
pence = soup.find('span',attrs={'class':'pence'})
prices.append(pound.text + pence.text)

I get this exception stating:

 prices.append(pound.text + pence.text)
AttributeError: 'NoneType' object has no attribute 'text'

So to me it looks like it’s returning a None or null. Does anybody have an idea on how I can get to the element?

EDIT

Looking at the answers below, I tried to replicate them but instead of using a static HTML, I call on the website url. What I noticed is that even though the code works for a static html, it doesn’t work when I call on the url that contains the page that contains that html.

CODE:

from bs4 import BeautifulSoup
import pandas as pd
import requests

data = requests.get('https://www.screwfix.com/p/no-nonsense-sanitary-silicone-white-310ml/47187').text

soup = BeautifulSoup(data, 'html.parser')
currency = soup.select_one('span.pound')
currency_str = next(currency.strings).strip()

pound_str = currency.nextSibling

pence = soup.select_one('span.pence')
pence_str = next(pence.strings).strip()

print(f"{currency_str}{pound_str}{pence_str}")  # £3.49

Error:

 currency_str = next(currency.strings).strip()
AttributeError: 'NoneType' object has no attribute 'strings'

Answer

Here’s a different approach.

from bs4 import BeautifulSoup

data = '''
<div class="lg-7 lg-offset-1 md-24 sm-24 cols">
<div class="row pr__prices">
<div class="lg-24 md-12 cols">
<input id="analytics_prodPrice_47187" type="hidden" value="2.91">
<div class="pr__pricepoint">
   <div id="product_price" class="pr__price">
      <span class="pound">
      £</span>3<span class="pence">.49<span class="incvat">INC VAT</span></span>
      <span class="price__extra">(<span id="unit_price">£11.26</span>/<span id="unit_price_measure">Ltr</span>)</span>
   </div>
</div>
'''

soup = BeautifulSoup(data, 'html.parser')
currency = soup.select_one('span.pound')
currency_str = next(currency.strings).strip()

pound_str = currency.nextSibling

pence = soup.select_one('span.pence')
pence_str = next(pence.strings).strip()

print(f"{currency_str}{pound_str}{pence_str}")  # £3.49

BeautifulSoup – how to call on a nested element

Advertisement

Answer