I trust all is well with you and yours / thanks for taking a moment to read this. I am trying to figure out why this section of code does not respect 2 things.
- the count limit, it is supposed to only pull that many tweets, and it does whatever it wants to do.
- the code seems to loop indefinitely, and unless I put a hard stop, it doesn’t stop (hard stop is not currently included, though it would be something akin to adding a for in range loop that matches count. Though, I am not sure that would work either.
import tweepy import Auth_Codes import json import os def tweet_search(query=""): print(f"'Query = {query}'") twitter_auth_keys = { "consumer_key" : Auth_Codes.consumer_key, "consumer_secret" : Auth_Codes.consumer_secret, "access_token" : Auth_Codes.access_token, "access_token_secret" : Auth_Codes.access_token_secret } auth = tweepy.OAuthHandler( twitter_auth_keys["consumer_key"], twitter_auth_keys["consumer_secret"] ) auth.set_access_token( twitter_auth_keys["access_token"], twitter_auth_keys["access_token_secret"] ) api = tweepy.API(auth,wait_on_rate_limit=True) cursor = tweepy.Cursor(api.search_tweets,q = f'{query}',lang = 'en',result_type = 'recent',count = 1) print("Query Complete") tweet_payload = [] for tweet in cursor.items(): current_searches_remaining = api.rate_limit_status()['resources']['search']['/search/tweets']['remaining'] current_search_limit = api.rate_limit_status()['resources']['search']['/search/tweets']['limit'] max_searches = current_search_limit * .25 if current_searches_remaining >= (max_searches): print("Checking rate limit") print("Searches Remaining: ", current_searches_remaining, "/", current_search_limit) print("Converting payload to a useable format") tweet_payload.append(tweet._json) with open("Tweet_Payload.json", "w") as outfile: outfile.write(json.dumps(tweet_payload,indent=4, sort_keys=True)) tweet_filter() else: print("Rate Limit Hit, Moving On") tweet_filter() print("Search Complete") tweet_filter()
Advertisement
Answer
While I am still working out the finer details, after making the following change, it now loops nearly as expected.
cursor = tweepy.Cursor(api.search_tweets,q = f'{query}',lang = 'en',result_type = 'recent',count = 11) print("Query Complete") tweet_payload = [] for tweet in cursor.items(11):