Skip to content
Advertisement

How to search multiple predefined string in a webpage using request and beautifulsoup in python

I wanted to search a page for multiple strings that contains a predefined pattern. Currently, my code seems problematic.

import requests, re
url = "https://bscscan.com/address/0x88c20beda907dbc60c56b71b102a133c1b29b053#code"

queries = ["t.me", "twitter", "www."]

r = requests.get(url)
for q in queries:
    print (q)
    if q.startswith(tuple(queries)):
        print(q, 'Found')
    else:
        print(q, 'Not Found')

Current Output:

t.me
t.me Found
twitter
twitter Found
www.
www. Found

Wanted Output:

www.shibuttinu.com - Found
https://t.me/Shibuttinu - Found
twitter - not found

Advertisement

Answer

Here a sample example on how use the modules, it could be even the solution. Sorry but no ideas of what do you really want… but I hope it helps you anyway

import requests
from bs4 import BeautifulSoup

res = requests.get(url)
soup = BeautifulSoup(res.text)

queries = ["t.me", "twitter", "www."]
results = []
for q in queries:
   # match condition here, in case change it
   results += soup.find_all(lambda t: q in str(t.string), string=True)

for r in results:
   print(r)

User contributions licensed under: CC BY-SA
3 People found this is helpful
Advertisement