Extract content of with python inf Beautiful Soup [duplicate]

Question

This question already has answers here: How can I parse (read) and use JSON? (5 answers) Closed 24 days ago. 1/ I get my data with this code and then im trying to extract the value of the attribute "sku": 2/ this is the data that i get. my goal is to extract the value of the attribute "sku" but

Accepted Answer

There are different things to point out:Instead of find_all() that would return a resultset use find(), cause there is only one script tag the would match.Intstead of converting the tag and its content into a str() use .text to extract the content.Main issue is that you have to replace all &quot; in your extracted text, cause per se it is no valid json if you print, it would look like:'{&quot;@context&quot;:&quot;http://schema.org&quot;,&quot;@type&quot;:&quot;Product&quot;,&quot;aggregateRating&quot;...'Fix:paragraphs = json.loads(soup.find('script', type='application/ld+json').text.replace("&quot;", """))Short code:url = 'https://www.zalando.de/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36'}req = requests.get(url, headers=headers)soup = BeautifulSoup(req.text, "html.parser")jsonData =  json.loads(soup.find('script', type='application/ld+json').text.replace("&quot;", """))JSON:{'@context': 'http://schema.org', '@type': 'Product', 'aggregateRating': {'@type': 'AggregateRating', 'bestRating': 5, 'ratingValue': '4.4', 'ratingCount': 32, 'worstRating': 1, 'reviewCount': 20}, 'brand': {'@type': 'Brand', 'name': 'Nike Sportswear'}, 'color': 'white/black/sail/team orange/weiß', 'image': ['https://img01.ztat.net/article/spp-media-p1/5aab2cb0e8de49eba3a3a13b58e97639/f183297517d64ec9b36f25f93c92d336.jpg?imwidth=103', 'https://img01.ztat.net/article/spp-media-p1/dd5d970c70c34b5fb78d96dbba4682c8/56ffd527ef7b479bb4dccd0c4d950816.jpg?imwidth=103&amp;filter=packshot', 'https://img01.ztat.net/article/spp-media-p1/0fdf0aede23e4440ad555e07b39ba659/60ad2f546cf94245abf63ad95170b4db.jpg?imwidth=103', 'https://img01.ztat.net/article/spp-media-p1/6ceb2462605e4272af66895a2267e619/4a6b815ae0e44d8b9cdc3d5cdd871d79.jpg?imwidth=103', 'https://img01.ztat.net/article/spp-media-p1/cea4e4de28144aca96579f6e5c390196/1fac999507a84688a2299c8ca0af11bd.jpg?imwidth=103', 'https://img01.ztat.net/article/spp-media-p1/ef435f76c0934820875e930679b1d176/ac3547823f4b458ba1243f1338265394.jpg?imwidth=103', 'https://img01.ztat.net/article/spp-media-p1/f93228b4eacb48f88c4b781d30495120/f75ce631aa3141aaaea0590d82fb4993.jpg?imwidth=103', 'https://img01.ztat.net/article/spp-media-p1/a753a62466c34f029c8b677e50025ef4/88b91b412cb74fe7a6b524ab022668a3.jpg?imwidth=103'], 'itemCondition': 'http://schema.org/NewCondition', 'manufacturer': 'Nike Sportswear', 'name': 'BLAZER MID &#x27;77 JUMBO UNISEX - Sneaker high - white/black/sail/team orange', 'offers': [{'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110035000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110040000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110045000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110050000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110055000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110060000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110065000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110070000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110075000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110080000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/OutOfStock', 'price': '99.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110085000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110090000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/OutOfStock', 'price': '99.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110095000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110100000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110105000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110110000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110115000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110120000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110125000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110130000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110140000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}, {'@type': 'Offer', 'availability': 'http://schema.org/InStock', 'price': '109.95', 'priceCurrency': 'EUR', 'sku': 'NI112N02S-A110150000', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html'}], 'review': [{'@type': 'Review', 'author': {'@type': 'Person', 'name': 'Joel'}, 'datePublished': '2022-01-19', 'description': 'Sind nicht schön.', 'reviewRating': {'@type': 'Rating', 'bestRating': 5, 'worstRating': 1, 'ratingValue': '1'}}, {'@type': 'Review', 'author': {'@type': 'Person', 'name': 'Laura'}, 'datePublished': '2022-01-15', 'description': 'Tolle Passform! Sehr gute Qualität!', 'reviewRating': {'@type': 'Rating', 'bestRating': 5, 'worstRating': 1, 'ratingValue': '5'}}, {'@type': 'Review', 'author': {'@type': 'Person', 'name': 'Fabienne'}, 'datePublished': '2022-01-13', 'description': 'super geiler Sneaker. Dieses Modell ist viel bequemer als der normale Blazer wegen des weichen Abschlusses oben. Ist gemacht wie eine Socke.. Top...', 'reviewRating': {'@type': 'Rating', 'bestRating': 5, 'worstRating': 1, 'ratingValue': '5'}}, {'@type': 'Review', 'author': {'@type': 'Person', 'name': 'Fouad'}, 'datePublished': '2022-01-07', 'description': 'Sehr chillig und sehr stylisch auf locker', 'reviewRating': {'@type': 'Rating', 'bestRating': 5, 'worstRating': 1, 'ratingValue': '5'}}, {'@type': 'Review', 'author': {'@type': 'Person', 'name': 'Zalando Kunde'}, 'datePublished': '2022-01-03', 'description': 'Sieht top aus und ist sehr bequem.', 'reviewRating': {'@type': 'Rating', 'bestRating': 5, 'worstRating': 1, 'ratingValue': '5'}}, {'@type': 'Review', 'author': {'@type': 'Person', 'name': 'Dzafer'}, 'datePublished': '2021-12-26', 'description': 'Leider Kleberreste am Schuh gehabt, sonst war alles top.', 'reviewRating': {'@type': 'Rating', 'bestRating': 5, 'worstRating': 1, 'ratingValue': '3'}}, {'@type': 'Review', 'author': {'@type': 'Person', 'name': 'anna'}, 'datePublished': '2021-12-22', 'description': 'alles perfekt', 'reviewRating': {'@type': 'Rating', 'bestRating': 5, 'worstRating': 1, 'ratingValue': '5'}}, {'@type': 'Review', 'author': {'@type': 'Person', 'name': 'Sara'}, 'datePublished': '2021-12-18', 'description': 'Super bequemer Sneaker, sieht dabei noch schick aus. Durch den elastischen Schaft viel einfacher reinzuschlüpfen, als bei dem normalen Modell!', 'reviewRating': {'@type': 'Rating', 'bestRating': 5, 'worstRating': 1, 'ratingValue': '5'}}, {'@type': 'Review', 'author': {'@type': 'Person', 'name': 'Zalando Kunde'}, 'datePublished': '2021-12-16', 'description': 'Stylischer, bequemer schuh', 'reviewRating': {'@type': 'Rating', 'bestRating': 5, 'worstRating': 1, 'ratingValue': '5'}}, {'@type': 'Review', 'author': {'@type': 'Person', 'name': 'Asli'}, 'datePublished': '2022-01-04', 'description': 'Einfach nur Bomben Qualität sieht  getragen super aus  gr stimmt ich musste mir sonst immer eine nr größer nehmen bei Nike  bei diesem Model passte meine Schuhgröße. 👍👍👍👍', 'reviewRating': {'@type': 'Rating', 'bestRating': 5, 'worstRating': 1, 'ratingValue': '5'}}], 'sku': 'NI112N02S-A11', 'url': '/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html', 'description': ' Nike Sportswear BLAZER MID &#x27;77 JUMBO UNISEX - Sneaker high - white/black/sail/team orange für 109,95xa0€ (2022-01-20) versandkostenfrei bei Zalando bestellen.'}ExampleNote: changed behavior of defs a bit, cause fetch_info should reference to request, &#8230;import requests, jsonfrom bs4 import BeautifulSoupdef request(url):    headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36'}    req = requests.get(url, headers=headers)    return reqdef fetch_info(url):    soup = BeautifulSoup(request(url).text, "html.parser")    paragraphs = json.loads(soup.find('script', type='application/ld+json').text.replace("&quot;", """))    return paragraphsfor item in fetch_info('https://www.zalando.de/nike-sportswear-blazer-mid-77-jumbo-unisex-sneaker-high-whiteblacksailteam-orange-ni112n02s-a11.html')['offers']:    print(item['sku'])OutputNI112N02S-A110035000NI112N02S-A110040000NI112N02S-A110045000NI112N02S-A110050000NI112N02S-A110055000NI112N02S-A110060000NI112N02S-A110065000NI112N02S-A110070000NI112N02S-A110075000NI112N02S-A110080000NI112N02S-A110085000NI112N02S-A110090000NI112N02S-A110095000...

Advertisement

Answer

Example

Output