My goal is to scrape, the name and location of the Cannabis retailer in Manitoba Canada.Then place all the data onto a excel file.From this website: https://www.google.com/maps/d/u/0/viewer?mid=1jTYtNQgB0CVC27Bn_xIsFGeVuCh_KCdR&ll=49.671544600000004%2C-96.65402250000001&z=8.
I am not sure if this is the right direction, use to just scraping request with json and not using b4. However any advice would be appreciated.
import requests
from bs4 import BeautifulSoup as bs
import pandas
r = requests.get("https://www.google.com/maps/d/u/0/viewer?mid=1jTYtNQgB0CVC27Bn_xIsFGeVuCh_KCdR&ll=49.85504530000007%2C-97.11150419999998&z=8")
soup = bs(r.content)
Address = soup.find_all("div",{"class": "fO2voc-jRmmHf-MZArnb-Q7Zjwb"})
Name = soup.find_all("div",{"class": "qqvbed-p83tee-lTBxed"})
print(Address,Name)
output:
[] []
Advertisement
Answer
I thought it would be cool to try, so I spent a bit of time. It doesn’t work.
The google page is built up dynamically as you click on things and get adresses. There are no addresses in the file until you start clicking and expanding on them or hovering over things. If you use the google developer tools and click on things to inspect them you will seem to think that it will work because you can find div tags in there with Manitoba Canada in them like below:
<div class="fO2voc-jRmmHf-MZArnb-Q7Zjwb">MB-6 & Twin Beach Rd, Manitoba R0C 2S0, Canada</div>
But if you search the doc loaded none of that exists on the initial load.
If you were able to get the complete file somehow, this would find all the addresses (maybe among other things):
strings_of_interest = soup.findAll('div', text = re.compile('Manitoba'))