Objective: I am trying to pull data from the Detailed forecast section on this weather forecast website. Then I am trying to put that data in a tabular data frame using pandas
Question: I get an error in the last line – could I get some advice, please?
This is my code thus far:
import csv
import requests
from bs4 import BeautifulSoup
from IPython.display import HTML
!pip install BS4
!pip install Requests
!pip install lxml
page = requests.get("https://forecast.weather.gov/MapClick.php?
lat=37.7772&lon=-122.4168#.Xx5gsZ5Kj51")
soup = BeautifulSoup(page.content, 'html.parser')
seven_day = soup.find(id="detailed-forecast")
forecast_items = seven_day.find_all(class_="panel-body")
tonight = forecast_items[0]
print(tonight.prettify())
period = tonight.find(class_="col-sm-2 forecast-label").get_text()
short_desc = tonight.find(class_="col-sm-10 forecast-text").get_text()
print(period)
print(short_desc)
period_tags = seven_day.select(".panel-body.col-sm-10 forecast-text")
periods = [pt.get_text() for pt in period_tags]
short_descs = [sd.get_text() for sd in seven_day.select(".panel-body .col-smtemps
= [t.get_text() for t in seven_day.select(".tombstone-container .temp")
import pandas as pd
weather = pd.DataFrame({
"period": period,
"short_desc": desc,
})
weather
Advertisement
Answer
import requests
import pandas as pd
from bs4 import BeautifulSoup
page = requests.get("https://forecast.weather.gov/MapClick.php?lat=37.7772&lon=-122.4168#.Xx5gsZ5Kj51")
soup = BeautifulSoup(page.content, 'html.parser')
divs = soup.find("div", {"id":"detailed-forecast-body"})
period = []
desc = []
for div in divs.find_all("div", class_="row-forecast"):
period.append(div.find("div", class_="forecast-label").get_text(strip=True))
desc.append(div.find("div", class_="forecast-text").get_text(strip=True))
weather = pd.DataFrame({
"period": period,
"short_desc": desc,
})
print(weather)
Output:
period short_desc 0 Tonight Increasing clouds, with a low around 56. West ... 1 Monday Cloudy through mid morning, then gradual clear... 2 Monday Night Increasing clouds, with a low around 56. West ... 3 Tuesday Cloudy through mid morning, then gradual clear... 4 Tuesday Night Mostly cloudy, with a low around 56. West sout... 5 Wednesday Mostly cloudy, with a high near 67. 6 Wednesday Night Mostly cloudy, with a low around 56. 7 Thursday Partly sunny, with a high near 70. 8 Thursday Night Partly cloudy, with a low around 58. 9 Friday Sunny, with a high near 73. 10 Friday Night Mostly clear, with a low around 57. 11 Saturday Sunny, with a high near 74. 12 Saturday Night Mostly clear, with a low around 57. 13 Sunday Sunny, with a high near 73.