Get meta tag content by name, beautiful soup and python

Tags: , , , ,



I’m trying to get the meta data from this website(here’s the code).

import requests
from bs4 import BeautifulSoup

source = requests.get('https://www.svpboston.com/').text

soup = BeautifulSoup(source, features="html.parser")

title = soup.find("meta", name="description")
image = soup.find("meta", name="og:image")

print(title["content"] if title else "No meta title given")
print(image["content"]if title else "No meta title given")

however I get this error.

Traceback (most recent call last):
  File "C:/Users/User/PycharmProjects/Work/Web Scraping/Selenium/sadsaddas.py", line 9, in <module>
    title = soup.find("meta", name="description")
TypeError: find() got multiple values for argument 'name'

Any ideas?

Answer

From bs4 docs:

You can’t use a keyword argument to search for HTML’s name element, because Beautiful Soup uses the name argument to contain the name of the tag itself. Instead, you can give a value to ‘name’ in the attrs argument

To grab tag by specific attribute, I recommend you putting it into dictionary and pass that dictionary into .find() as the attrs argument. But you passed wrong attribute to grab title and image as well. You should be grabbing meta tag with property=<...> rather than name=<...>. Below is the final code to get what you need:

import requests
import requests
from bs4 import BeautifulSoup

source = requests.get('https://www.svpboston.com/').text

soup = BeautifulSoup(source, features="html.parser")

title = soup.find("meta", attrs={'property': 'og:title'})
image = soup.find("meta", attrs={'property': 'og:image'})

print(title["content"] if title is not None else "No meta title given")
print(image["content"] if title is not None else "No meta title given")


Source: stackoverflow