I’m working on a Python library that interfaces with a web service API. Like many web services I’ve encountered, this one requests limiting the rate of requests. I would like to provide an optional parameter, limit, to the class instantiation that, if provided, will hold outgoing requests until the number of seconds specified passes.
I understand that the general scenario is the following: an instance of the class makes a request via a method. When it does, the method emits some signal that sets a lock variable somewhere, and begins a countdown timer for the number of seconds in limit. (In all likelihood, the lock is the countdown timer itself.) If another request is made within this time frame, it must be queued until the countdown timer reaches zero and the lock is disengaged; at this point, the oldest request on the queue is sent, and the countdown timer is reset and the lock is re-engaged.
Is this a case for threading? Is there another approach I’m not seeing?
Should the countdown timer and lock be instance variables, or should they belong to the class, such that all instances of the class hold requests?
Also, is this generally a bad idea to provide rate-limiting functionality within a library? I reason since, by default, the countdown is zero seconds, the library still allows developers to use the library and provide their own rate-limiting schemes. Given any developers using the service will need to rate-limit requests anyway, however, I figure that it would be a convenience for the library to provide a means of rate-limiting.
Regardless of placing a rate-limiting scheme in the library or not, I’ll want to write an application using the library, so suggested techniques will come in handy.
Advertisement
Answer
This works out better with a queue and a dispatcher.
You split your processing into two sides: source and dispatch. These can be separate threads (or separate processes if that’s easier).
The Source side creates and enqueues requests at whatever rate makes them happy.
The Dispatch side does this.
Get the request start time, s.
Dequeues a request, process the request through the remote service.
Get the current time, t. Sleep for rate – (t – s) seconds.
If you want to run the Source side connected directly to the remote service, you can do that, and bypass rate limiting. This is good for internal testing with a mock version of the remote service.
The hard part about this is creating some representation for each request that you can enqueue. Since the Python Queue will handle almost anything, you don’t have to do much.
If you’re using multi-processing, you’ll have to pickle your objects to put them into a pipe.