I am making a request to get a download link through the following request:
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; rv:91.0) Gecko/20100101 Firefox/91.0',
'Accept': '*/*',
'Accept-Language': 'en-US,en;q=0.5',
'Accept-Encoding': 'gzip, deflate, br',
'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
'X-Requested-With': 'XMLHttpRequest',
'Origin': 'https://x2download.com',
'DNT': '1',
'Connection': 'keep-alive',
'Referer': 'https://x2download.com/fr54',
'Sec-Fetch-Dest': 'empty',
'Sec-Fetch-Mode': 'cors',
'Sec-Fetch-Site': 'same-origin',
'TE': 'trailers',
}
video= 'https://www.youtube.com/watch?v=kpz8lpoLvrA'
data = f'q={video}&vt=home'
response = requests.post('https://x2download.com/api/ajaxSearch', headers=headers, data= data)
print(response.content)
From my windows laptop and my ubuntu server I am getting the following content:
b'{“vid”:”kpz8lpoLvrA”,”title”:”Interstellar Main Theme – Hans Zimmer”,”fn”:”X2Download.com-Interstellar Main Theme – Hans Zimmer”,”a”:”Aura Music”,”t”:244,”links”:{“ogg”:{“1”:{“f”:”ogg”,”k”:”128″,”q”:”128kbps”,”size”:”4.02 MB”,”key”:”128kbps”,”selected”:null}},”mp3″:{“2”:{“f”:”mp3″,”k”:”128″,”q”:”128kbps”,”size”:”4.02 MB”,”key”:”128kbps”,”selected”:null}},”mp4″:{“3”:{“f”:”mp4″,”k”:”1080p”,”q”:”1080p”,”size”:”16.87 MB”,”key”:”1080″,”selected”:””},”4″:{“f”:”mp4″,”k”:”720p”,”q”:”720p”,”size”:”12.48 MB”,”key”:”720″,”selected”:”selected”},”5″:{“f”:”mp4″,”k”:”480p”,”q”:”480p”,”size”:”4.21 MB”,”key”:”480″,”selected”:””},”6″:{“f”:”mp4″,”k”:”360p”,”q”:”360p”,”size”:”7.39 MB”,”key”:”360″,”selected”:””},”7″:{“f”:”mp4″,”k”:”240p”,”q”:”240p”,”size”:”7.19 MB”,”key”:”240″,”selected”:””},”8″:{“f”:”mp4″,”k”:”144p”,”q”:”144p”,”size”:”817.20 KB”,”key”:”144″,”selected”:””}},”3gp”:{“9”:{“f”:”3gp”,”k”:”144p”,”q”:”144p”,”size”:”817.20 KB”,”key”:”144″,”selected”:null}}},”token”:”1cc3a03822a2582bcb47b70da2012cdf43fc66d899e6f0a5d14064c7dcec1154″,”timeExpires”:”1660554472″,”status”:”ok”,”p”:”convert”,”mess”:””}’
But when I try on a heroku app, AWS lambda or even a docker container, I am getting this:
b’x83%x02x00xc4/x9dxf9UxcbxbcZfx14x96xb4x9dxfdCxee~xeetxb0x17%Avxe4x7foxf9xb6xd2Yxc6x17x0ehxe4xffx00x0c,xe2xcbfxd1Ixf1xfdxbcx17
xa9Ex10qxc6ixbbLx13xc9obxaexcex9bxe3x15xdbxa5x03xe36xbcxd4axe8xbfoxd3=x14x96xcbx12x04x8c/ix91i^$x04?e?xfc%e?xcfx12%xbbxcb>xfb4gxff/1xcax04x85cx02xe3xaf}xdf?xa6xd0xfb.oxfbxxf7xe5xfexfdx1ex8cxfbuxf2xd9x8fuxbexb4PXxc0x96H!\xd2mx06xbfxa2?x9dxc0xaf0xe0Wx1cxc1{xc7n70x8cxadxa10xaa'xdcx0exc3x0cx85xf9xf2"Pxaemxf6xe3x01n7y>t]x82xb4x8btxe0xb4x86xb0xefnqpxe0Wxd8G>x15x07Wx04x85x9cOx99xaexf5xd0Qxe3x1ax9b2xafxabVxd7xb5x1ex13x80]x81x8cxd3xcax12%Mxcbxe6Txbb*Kxaex03xbbBxe1x9exa3x88x8bxc4x0exe5xd6xb0x92x98PjWxa2DSxabxca]creBxa5-!x18Qcxb2x94Px07x86x08x81xd8xadMx95[x9d}Kx92x93(xcexb3]xc1x9dx06xf0+x9a5xecNx83=xfcxaex9fxd9x15x96xfe&tx8cFWxe2xa5xb2xaexf6djxd3xd9xb6xf2x9axa8pxb5o\xa3xa5xf4Exebx9dxf1x9dx18{(x8co}xeb;[jrxf5xd1xdaxeex00x85xe5x12xe5xc3xd3px99dx06xc3x94xa5nxc8x14Mtx85yxf1xcb:x83xd1xdfxa00x80xd1xf6ix93ix81Bx94yx06x03xbbx01x03′
I tried:
- Modifying the parameters of the platform (LANG, C-LANG, etc)
- Decoding in any way I found
- Putting the same python + libraries to the same versions
- with urllib
- all other UnicodeDecodeError related solutions
The result stays the same. Any idea on how to change the received result or to decrypt it is welcome
Advertisement
Answer
Your headers have 'Accept-Encoding': 'gzip, deflate, br' which means you are telling the server that your request accepts compressed responses.
The reason this is working on your local but not on docker is that your local environment has brotli library installed but your docker container might not.
In this case, the server is sending a broti compressed response which your docker container cannot decode.
You can simply install brotli in your docker image by including this in your Dockerfile
RUN pip3 install brotli
or adding brotli in your requirements.txt. The same goes for your lambda.
Alternatively, you can request the server to not send brotli compressed responses by using 'Accept-Encoding': 'gzip, deflate'