Skip to content
Advertisement

Python Scraping: Trying to scrape a specific data (phone details) according to user input

I am doing web scraping from www.gsmarena.com . i want to extract a specific data according to the user input. this code returns all the phone models and names, i want to extract just the samsung phones details with a specific input suck as the RAM,ROM,CPU and COLOR. help me please. thanks in advance.

import requests
from bs4 import BeautifulSoup


def link_scan(link_url):
    c = 1
    source_code=requests.get(link_url)
    plain_text=source_code.text
    soup = BeautifulSoup(plain_text)
    for link in soup.find_all('div',{'class':'brandmenu-v2 light l-box clearfix'}):
       for li in link.find_all('li'):
           for anc in li.find_all('a'):
               anc_src = r'http://www.gsmarena.com/' + anc.get('href')
               anc_name = anc.string
               print(c, anc_name,"n", anc_src, "n")
               c += 1
               inside_scan(anc_name, anc_src)


def inside_scan(name, hrefs):
    i = 1
    source_code=requests.get(hrefs)
    plain_text=source_code.text
    soup = BeautifulSoup(plain_text)
    for link in soup.find_all('div',{'class':'makers'}):
       for li in link.find_all('li'):
           for anc in li.find_all('a'):
               for nam in (sp.find('span') for sp in anc.find_all('strong')):
                   modal_name = nam.string
                   print("t", i, "t", name, modal_name)
                   i += 1

link_scan(r'http://www.gsmarena.com/')

Advertisement

Answer

I would advise you to play sometime with urls. In your case user may ask for a specific mobile phone manufacturer and target url would look like this:

https://www.gsmarena.com/samsung-phones-9.php

Moreover, you are quite lucky because you can fetch a certain cell phone details without redirecting to it’s page. In your case each cell phone refers to anchor tag with a class name like this:

<a href="samsung_galaxy_m31s-10333.php">

It means you can parse links that start from “Samsung” in order to filter the query according to the user’s needs:

https://www.gsmarena.com/samsung

To fetch CPU, RAM, e.t.c info you have to refer anchor tags:

<a href="samsung_galaxy_m31s-10333.php"><img src="https://fdn2.gsmarena.com/vv/bigpic/samsung-galaxy-m31s.jpg" title="Samsung Galaxy M31s Android smartphone. Announced Jul 2020. Features 6.5″ Super AMOLED display, Exynos 9611 chipset, 6000 mAh battery, 128 GB storage, 8 GB RAM, Corning Gorilla Glass 3."><strong><span>Galaxy M31s</span></strong></a>
User contributions licensed under: CC BY-SA
5 People found this is helpful
Advertisement