How to Get all Product detail it prints the same things but I want others products to detail also here is the link from where I want to fetch the data of all product:https://www.nike.com/gb/w/womens-lifestyle-shoes-13jrmz5e1x6zy7ok
import requests
from bs4 import BeautifulSoup
import pandas as pd
import numpy as np
from selenium import webdriver
url= "https://www.nike.com/gb/w/womens-lifestyle-shoes-13jrmz5e1x6zy7ok"
driver = webdriver.Chrome('D:/chromedriver')
driver.get(url)
pageSource = driver.page_source
for n in range(10):
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
soup = BeautifulSoup(pageSource, 'lxml')
content= soup.findAll('div',class_='product-grid')
content
for item in content:
title= item.find('div',class_ = 'product-card__title').text
link = item.find('a', {'class': 'product-card__link-overlay'})['href']
price=item.find('div',class_ ='product-price css-11s12ax is--current-price').text
print(title,price,link)
Advertisement
Answer
What happens?
- Their is a wrong indent with your
print - Their is only one element with
classofproduct-grid
How to fix?
Check the indent of your
print, it should be in the loop to print the items.Change your selection to:
content= soup.find_all('div',class_='product-card')Bonus: Instead
findAll()better use the new syntaxfind_all()
Example
import requests
from bs4 import BeautifulSoup
import pandas as pd
import numpy as np
from selenium import webdriver
url= "https://www.nike.com/gb/w/womens-lifestyle-shoes-13jrmz5e1x6zy7ok"
driver = webdriver.Chrome(executable_path=r'C:Program FilesChromeDriverchromedriver.exe')
driver.get(url)
pageSource = driver.page_source
for n in range(10):
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
soup = BeautifulSoup(pageSource, 'lxml')
content= soup.find_all('div',class_='product-card')
content
for item in content:
title= item.find('div',class_ = 'product-card__title').text
link = item.find('a', {'class': 'product-card__link-overlay'})['href']
price=item.find('div',class_ ='product-price css-11s12ax is--current-price').text
print(title,price,link)