Can’t convert curl to python request

I copied request as CURL with help of dev-tools and received:

curl 'https://ok.ru/web-api/v2/search/suggestCommunities' 
  -H 'authority: ok.ru' 
  -H 'accept: application/json, text/javascript, */*; q=0.01' 
  -H 'dnt: 1' 
  -H 'strd: false' 
  -H 'ok-screen: searchResult' 
  -H 'content-type: text/plain;charset=UTF-8' 
  -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36' 
  -H 'tkn: eezDYrXx08aiDcsqCCCheb1fGdGIvuJ9' 
  -H 'origin: https://ok.ru' 
  -H 'sec-fetch-site: same-origin' 
  -H 'sec-fetch-mode: cors' 
  -H 'sec-fetch-dest: empty' 
  -H 'referer: https://ok.ru/' 
  -H 'accept-language: ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7' 
  -H 'cookie: bci=5639450919803662892; _statid=72da67cb-c979-4442-948d-b76b70d16b59; tmr_lvid=2ac18e93052a4516697406103a6db5b3; tmr_lvidTS=1575669087459; msg_conf=2468555756792551; _flashVersion=0; cudr=0; klos=0; AUTHCODE=EjUzQb4TKlF4mTbJklDGt-dg-65th4TTwrbLdAPrK8mvgyWCFTIdNoeMcBkVrvZruEBK9frXso45DCrvoq8K9p-7ZUcDvASmGvV13LQ5gxgls-AFQjI7ZEPmb6u08TLFJ57ZCb4R00LhHto_3; JSESSIONID=a381dc641575389c384c9c7eacd747d14853670edfe4c807.f00f63fb; LASTSRV=ok.ru; viewport=762; TZ=6; CDN=; nbp=; tmr_detect=1%7C1605813256183; tmr_reqNum=495; TZD=6.683; TD=683' 
  --data-binary $'{n  "id": 38,n  "parameters": {n    "category": "UNIVERSITY",n    "cityId": "10399843062",n    "query": "Рос"n  }n}' 
  --compressed

Copied request to console (curl works fine). Received result

{"id":"38","success":true,"result":[{"id":"6100259920","category":"UNIVERSITY","name":"СПбГУКиТ, Санкт-Петербургский государственный университет кино и телевидения (бывш. ЛИКИ)","badgeName":"СПбГУКиТ"},{"id":"6098569210","category":"UNIVERSITY","name":"СПбГУЭФ (ФИНЭК), Санкт-Петербургский государственный университет экономики и финансов (бывш. ЛФЭИ им. Н.А. Вознесенского)","badgeName":"СПбГУЭФ (ФИНЭК)"},{"id":"6101852454","category":"UNIVERSITY","name":"СПбГУКИ, Санкт-Петербургский государственный университет культуры и искусств (бывш. ЛГИК им. Н.К. Крупской, СПбГАК)","badgeName":"СПбГУКИ"},{"id":"6096539930","category":"UNIVERSITY", ....

Converted curl to python requests with help of https://curl.trillworks.com/ Received:

import requests

headers = {
    'authority': 'ok.ru',
    'accept': 'application/json, text/javascript, */*; q=0.01',
    'dnt': '1',
    'strd': 'false',
    'ok-screen': 'searchResult',
    'content-type': 'text/plain;charset=UTF-8',
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36',
    'tkn': 'eezDYrXx08aiDcsqCCCheb1fGdGIvuJ9',
    'origin': 'https://ok.ru',
    'sec-fetch-site': 'same-origin',
    'sec-fetch-mode': 'cors',
    'sec-fetch-dest': 'empty',
    'referer': 'https://ok.ru/',
    'accept-language': 'ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7',
    'cookie': 'bci=5639450919803662892; _statid=72da67cb-c979-4442-948d-b76b70d16b59; tmr_lvid=2ac18e93052a4516697406103a6db5b3; tmr_lvidTS=1575669087459; msg_conf=2468555756792551; _flashVersion=0; cudr=0; klos=0; AUTHCODE=EjUzQb4TKlF4mTbJklDGt-dg-65th4TTwrbLdAPrK8mvgyWCFTIdNoeMcBkVrvZruEBK9frXso45DCrvoq8K9p-7ZUcDvASmGvV13LQ5gxgls-AFQjI7ZEPmb6u08TLFJ57ZCb4R00LhHto_3; JSESSIONID=a381dc641575389c384c9c7eacd747d14853670edfe4c807.f00f63fb; LASTSRV=ok.ru; viewport=762; TZ=6; CDN=; nbp=; tmr_detect=1%7C1605813256183; tmr_reqNum=495; TZD=6.683; TD=683',
}

data = '${\n "id": 38,\n "parameters": {\n "category": "UNIVERSITY",\n "cityId": "10399843062",\n "query": "u0420u043Eu0441"\n }\n}'

response = requests.post('https://ok.ru/web-api/v2/search/suggestCommunities', headers=headers, data=data)
print(response.content)

Tried this code and received error

Body is not valid Latin-1. Use body.encode(‘utf-8’) if you want to send it encoded in UTF-8.

Added .encode(‘utf-8’))

response = requests.post('https://ok.ru/web-api/v2/search/suggestCommunities', headers=headers, data=data.encode('utf-8'))

But received invalid result:

b'{“success”:false,”error”:{“type”:1,”typeName”:”INVALID_REQUEST”,”errorCode”:”api.invalid-format”,”errorMessage”:”Invalid request format: Unrecognized token ‘$’: was expecting (‘true’, ‘false’ or ‘null’)n at [Source: org.apache.catalina.connector.CoyoteInputStream@5399802b; line: 1, column: 3]”},”result”:null}’

How can I receive valid result?

{"id":"38","success":true,"result":[{"id":"6100259920","category":"UNIVERSITY","name":"СПбГУКиТ, Санкт-Петербургский государственный университет кино и телевидения (бывш. ЛИКИ)","badgeName":"СПбГУКиТ"},{"id":"6098569210","category":"UNIVERSITY","name":"СПбГУЭФ (ФИНЭК), Санкт-Петербургский государственный университет экономики и финансов (бывш. ЛФЭИ им. Н.А. Вознесенского)","badgeName":"СПбГУЭФ (ФИНЭК)"},

UPDATE

Tried to use data without $

data = ‘{n “id”: 38,n “parameters”: {n “category”: “UNIVERSITY”,n “cityId”: “10399843062”,n “query”: “u0420u043Eu0441″n }n}’

Received error

UnicodeEncodeError: ‘latin-1’ codec can’t encode characters in position 99-101: Body (‘Рос’) is not valid Latin-1. Use body.encode(‘utf-8’) if you want to send it encoded in UTF-8.

Tried

data = ‘{ “id”: 38, “parameters”: { “category”: “UNIVERSITY”, “cityId”: “10399843062”, “query”: “u0420u043Eu0441” }}’

Received error:

UnicodeEncodeError: ‘latin-1’ codec can’t encode characters in position 89-91: Body (‘Рос’) is not valid Latin-1. Use body.encode(‘utf-8’) if you want to send it encoded in UTF-8.

Answer

I think you accidentally copied the $ into your python code. Try changing to this line:

data = '{\n "id": 38,\n "parameters": {\n "category": "UNIVERSITY",\n "cityId": "10399843062",\n "query": "u0420u043Eu0441"\n }\n}'

You could even remove the n‘s:

data = '{ "id": 38, "parameters": { "category": "UNIVERSITY", "cityId": "10399843062", "query": "u0420u043Eu0441" }}'

Edit

As for the

UnicodeEncodeError: ‘latin-1’ codec can’t encode characters in position 89-91: Body (‘Рос’) is not valid Latin-1. Use body.encode(‘utf-8’) if you want to send it encoded in UTF-8.

check out the end of this forum

a = "u13E0u19E0u1320"
a.encode('latin1')  # Throws UnicodeEncodeError, proves that this can't be expressed in ISO-8859-1.
a.encode('utf-8')  # Totally fine.

So you could probably try something like:

data = '{ "id": 38, "parameters": { "category": "UNIVERSITY", "cityId": "10399843062", "query": "Poc" }}'

data = '{ "id": 38, "parameters": { "category": "UNIVERSITY", "cityId": "10399843062", "query": "u0420u043Eu0441" }}'
data = data.encode('utf-8')

Advertisement

Answer

Edit