How to Get all Product detail it prints the same things but I want others products to detail also here is the link from where I want to fetch the data of all product:https://www.nike.com/gb/w/womens-lifestyle-shoes-13jrmz5e1x6zy7ok
import requests from bs4 import BeautifulSoup import pandas as pd import numpy as np from selenium import webdriver url= "https://www.nike.com/gb/w/womens-lifestyle-shoes-13jrmz5e1x6zy7ok" driver = webdriver.Chrome('D:/chromedriver') driver.get(url) pageSource = driver.page_source for n in range(10): driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") soup = BeautifulSoup(pageSource, 'lxml') content= soup.findAll('div',class_='product-grid') content for item in content: title= item.find('div',class_ = 'product-card__title').text link = item.find('a', {'class': 'product-card__link-overlay'})['href'] price=item.find('div',class_ ='product-price css-11s12ax is--current-price').text print(title,price,link)
Advertisement
Answer
What happens?
- Their is a wrong indent with your
print
- Their is only one element with
class
ofproduct-grid
How to fix?
Check the indent of your
print
, it should be in the loop to print the items.Change your selection to:
content= soup.find_all('div',class_='product-card')
Bonus: Instead
findAll()
better use the new syntaxfind_all()
Example
import requests from bs4 import BeautifulSoup import pandas as pd import numpy as np from selenium import webdriver url= "https://www.nike.com/gb/w/womens-lifestyle-shoes-13jrmz5e1x6zy7ok" driver = webdriver.Chrome(executable_path=r'C:Program FilesChromeDriverchromedriver.exe') driver.get(url) pageSource = driver.page_source for n in range(10): driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") soup = BeautifulSoup(pageSource, 'lxml') content= soup.find_all('div',class_='product-card') content for item in content: title= item.find('div',class_ = 'product-card__title').text link = item.find('a', {'class': 'product-card__link-overlay'})['href'] price=item.find('div',class_ ='product-price css-11s12ax is--current-price').text print(title,price,link)