Skip to content
Advertisement

Tag: middleware

Unable to send requests in the right way after replacing redirected url with original one using middleware

I’ve created a script using scrapy to fetch some fields from a webpage. The url of the landing page and the urls of inner pages get redirected very often, so I created a middleware to handle that redirection. However, when I came across this post, I could understand that I need to return request in process_request() after replacing the redirected

Advertisement