How to Get all Product detail it prints the same things but I want others products to detail also here is the link from where I want to fetch the data of all product:https://www.nike.com/gb/w/womens-lifestyle-shoes-13jrmz5e1x6zy7ok
JavaScript
x
22
22
1
import requests
2
from bs4 import BeautifulSoup
3
import pandas as pd
4
import numpy as np
5
from selenium import webdriver
6
7
8
url= "https://www.nike.com/gb/w/womens-lifestyle-shoes-13jrmz5e1x6zy7ok"
9
driver = webdriver.Chrome('D:/chromedriver')
10
driver.get(url)
11
pageSource = driver.page_source
12
for n in range(10):
13
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
14
soup = BeautifulSoup(pageSource, 'lxml')
15
content= soup.findAll('div',class_='product-grid')
16
content
17
for item in content:
18
title= item.find('div',class_ = 'product-card__title').text
19
link = item.find('a', {'class': 'product-card__link-overlay'})['href']
20
price=item.find('div',class_ ='product-price css-11s12ax is--current-price').text
21
print(title,price,link)
22
Advertisement
Answer
What happens?
- Their is a wrong indent with your
print
- Their is only one element with
class
ofproduct-grid
How to fix?
Check the indent of your
print
, it should be in the loop to print the items.Change your selection to:
JavaScript121content= soup.find_all('div',class_='product-card')
2
Bonus: Instead
findAll()
better use the new syntaxfind_all()
Example
JavaScript
1
22
22
1
import requests
2
from bs4 import BeautifulSoup
3
import pandas as pd
4
import numpy as np
5
from selenium import webdriver
6
7
8
url= "https://www.nike.com/gb/w/womens-lifestyle-shoes-13jrmz5e1x6zy7ok"
9
driver = webdriver.Chrome(executable_path=r'C:Program FilesChromeDriverchromedriver.exe')
10
driver.get(url)
11
pageSource = driver.page_source
12
for n in range(10):
13
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
14
soup = BeautifulSoup(pageSource, 'lxml')
15
content= soup.find_all('div',class_='product-card')
16
content
17
for item in content:
18
title= item.find('div',class_ = 'product-card__title').text
19
link = item.find('a', {'class': 'product-card__link-overlay'})['href']
20
price=item.find('div',class_ ='product-price css-11s12ax is--current-price').text
21
print(title,price,link)
22