Skip to content
Advertisement

UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xc4 in position 1: ordinal not in range(128)

So, as title of the questions says, I have a problem with encoding/decoding of strings.

I am using: python 2.7 | django 1.11 | jinja2 2.8

Basically, I am retrieving some data from data base, I serialize it, set cache on it, then get the cache, deserialize it and rendering it to the template.

Problem:

I have first names and last names of persons that have characters like “ă” in the names. I serialize using json.dumps.

A sample of serialized dictionary looks like (I have 10 like this):

JavaScript

Then, when I set the cache I do it like:

JavaScript

, where value is the list of dictionaries returned by .values() from the aforementioned code and key is region_agents_by_commission_last_month (like the variable from the previous code)

Now, I have to get the cache. So I am doing the same process, but reversed.

JavaScript

where _decode_dict for object_hook looks like:

The result from print: Cătălin Pintea , which is ok. But in the dictionary I render: 'full_name': 'Cxc4x83txc4x83lin Pintea',

JavaScript

Basically, I use this object hook function in order to encode() to utf-8 all keys and value when json.loads.

This is how I avoided this error to be thrown in views.py.

Error

Somewhere on template, I am using:

<td>{{ agent.full_name }}</td>

And agent.full_name comes from : 'full_name': 'Cxc4x83txc4x83lin Pintea',

Traceback

JavaScript

And this is from where the error comes. I tried other things, but I guess it is a limitation of python 2.7. I usually use python 3.9, but for this project I have to use 2.7. I tried other answers around here but nothing really helped.

Can anybody help me to serialize this dictionary properly and how can I avoid this mess?

I hope I made myself clear.

Have a nice day everyone !

Advertisement

Answer

So, I managed to solve my issue.

  1. I figured out that active_agents.values(...."first_name", "last_name").order_by('-total_paid_transaction_value_last_month') retrieved a dictionary where its key and values were already in unicode (bacause of the way it was configured in models.py, django 1.11 and python2.7. So, the process of serializing was just fine. It is indeed true that the final result that went to template was looking like ’Cxc4x83txc4x83lin'. The error came from /xc4/.
  2. In order to fix it on template, I just did this: {{ agent.full_name.decode(“utf-8”) }}, which gave me the right result: Cătălin Pintea

Thanks @BoarGules. It was true that d['last_name'] and d['first_name'] were in unicode. So when I did the concatenation, I had to add u" ".

User contributions licensed under: CC BY-SA
7 People found this is helpful
Advertisement