i’m trying to scrape and list 24 items from a certain website. instead, it shows the same item 24 times.
to get the first price i did this:
JavaScript
x
20
20
1
gamma_html = uClient.read()2
uClient.close()3
page_soup = soup(gamma_html, "html.parser")4
hout = page_soup.findAll ("article",{"class":"mc-product-tile product-online-state-for_sale_not_orderable_in_stock product-state-for_sale"})5
print len(hout)6
i = 07
contain = hout[i]8
brand = contain.div.img["alt"]9
price = contain.findAll("div",{"class":"product-price-current"})10
price = str(price).split("<div class="product-price-current">")[1]11
price = price.split(".</div>")[0]12
price_2 = contain.findAll("span",{"class":"product-price-decimal"})13
price_2 = str(price_2).split("<span class="product-price-decimal">")[1]14
price_2 = price_2.split("</span>")[0]15
price_3 = (price + "," + price_2)16
product = brand + price_317
print(product)18
for i in range(24):19
print(product)20
as said, it does print 24 lines. but instead of listing all the different ones, it just lists the first result (probably because the i = 0). i’m very bad at coding so I don’t know what to change to get the wanted result. any thoughts?
Advertisement
Answer
Try the below code.
JavaScript
1
10
10
1
from bs4 import BeautifulSoup2
import requests3
url = "https://www.gamma.nl/assortiment/l/hout/bouwhout/hardhout"4
html = requests.get(url)5
soup = BeautifulSoup(html.text, 'html.parser')6
7
for item in soup.select("article.mc-product-tile"):8
print("title : " + item.select_one("div.product-name a").text)9
print("Price : " + item.select_one("div.wishlist-icon")['data-productprice'].strip())10
Output:
JavaScript
1
49
49
1
title : Lat hardhout geschaafd wit gegrond 9x116 mm 210 cm2
Price : 16.993
title : Lat geschaafd hardhout 12x55 mm 210 cm4
Price : 8.495
title : Lat hardhout geschaafd 20x28 mm 210 cm6
Price : 9.597
title : Plank eiken 19x195 mm 250 cm8
Price : 29.499
title : Lat hardhout geschaafd 28x44 mm 210 cm10
Price : 14.9911
title : Lat hardhout geschaafd 9x116 mm 210 cm12
Price : 14.9913
title : Lat hardhout geschaafd wit gegrond 12x68 mm 210 cm14
Price : 14.9915
title : Lat hardhout geschaafd 20x44 mm 210 cm16
Price : 12.4917
title : Glaslat hardhout wit gegrond 17x28 mm 270 cm18
Price : 10.9919
title : Lat hardhout geschaafd wit gegrond 12x44 mm 210 cm20
Price : 9.9921
title : Lat hardhout geschaafd 12x68 mm 210 cm22
Price : 12.9923
title : Lat hardhout geschaafd 20x68 mm 210 cm24
Price : 15.9925
title : Lat hardhout geschaafd 9x35 mm 210 cm26
Price : 6.7927
title : Lat geschaafd hardhout 12x12 mm 270 cm28
Price : 3.2929
title : Lat hardhout geschaafd 12x35 mm 210 cm30
Price : 8.2931
title : Lat geschaafd hardhout 9x44 mm 270 cm32
Price : 7.4933
title : Lat geschaafd hardhout 12x27 mm 270 cm34
Price : 6.9935
title : Lat hardhout geschaafd 12x116 mm 210 cm36
Price : 19.4937
title : Lat hardhout geschaafd 9x28 mm 210 cm38
Price : 4.4939
title : Lat hardhout geschaafd 44x44 mm 210 cm40
Price : 24.9941
title : Lat geschaafd hardhout 12x44 mm 270 cm42
Price : 8.9943
title : Lat geschaafd hardhout 9x27 mm 270 cm44
Price : 5.1945
title : Lat hardhout geschaafd 44 x 68 mm 210 cm46
Price : 43.9847
title : Lat hardhout geschaafd 20x116 mm 210 cm48
Price : 34.9549