Can’t figure out error: “UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xdc in position 0: ordinal not in range(128)”

I have a program that is communicating with another machine that sends (or is supposed to send) ASCII characters, the code below is how I write and read code to the machine. def writeCode(send): address = ‘COM4’

ser = serial.Serial(
        port=address,
        baudrate=9600,
        parity=serial.PARITY_NONE,
        stopbits=serial.STOPBITS_ONE,
        bytesize=serial.EIGHTBITS
    )
ser.close()
ser.open()
message=send
message=message+str(checksum(message))+"r"
ser.write(bytes(message, "ascii"))

time.sleep(1)  # give com time to respond
out=''
while ser.inWaiting() > 0: 
    out+=ser.read(1).decode('ascii')

if out != '':
   ser.close()
return out

I get the error on the “out+=ser.read(1).decode(‘ascii’) line. I looked online but most of the advice seems to be based around if you aren’t using the proper decoding library. However, for this machine I know that the ASCII library should be correct as it says so in the manual (https://www.idealvac.com/files/manuals/PfeifferGauge_MPT200_1.pdf page 16). What am I missing here? Any help would be much appreciated.

Answer

Your byte string is clearly not “ASCII”, as the value 'xdc', as the error message states, fall outside of the ASCII range.

Unfortuantelly, there is no way to tell the encoding the other side is sending you text, but by checking documentation, or by trial and error.

“xdc” is a valid utf-8 prefix (for characters in the Syriac alphabet though), so you could try to decode your stream using utf-8. That would involve changing your code, though, becahse any byte > 128 in utf-8 would mean the character is composed of more than a single byte, and your code calls decode with one byte at a time.

However, your data source may be encoding the text using latin1 or other single byte encoding – in that case, just replacing your out+=ser.read(1).decode('ascii') line with out+=ser.read(1).decode('latin1') will work.

Using latin1 will work in the sense it will fix your error: but may produce garbage output if it is not the correct source encoding.