I am switched from Python 2.7 to Python 3.6. I have scripts that deal with some non-English content. I usually run scripts via Cron and also in Terminal. I had UnicodeDecodeError in my Python 2.7 scripts and I solved by this. Now in Python 3.6, it doesnt work. I have print statements like print(“Here %s…
Tag: unicode
Python – Difference Between Windows SystemParametersInfoW vs SystemParametersInfoA Function
I have a quick question that I cannot seem to clarify, despite my research on Stack Overflow and beyond. My questions involves the Windows SystemParametersInfo function with its variants SystemParametersInfoW (Unicode) and SystemParametersInfoA (ANSI) in relation to a Python 3.x script. In a Python script I a…
List of unicode character names
In Python I can print a unicode character by name (e.g. print(u’N{snowman}’)). Is there a way I get get a list of all valid names? Answer Every codepoint has a name, so you are effectively asking for the Unicode standard list of codepoint names (as well as the *list of name aliases, supported by P…
python urllib2 and unicode
I would like to collect information from the results given by a search engine. But I can only write text instead of unicode in the query part. give this error Answer Encode the Unicode data to UTF-8, then URL-encode: Demo: Using urllib.urlencode() to build the parameters is easier, but you can also just escap…
Use isinstance to test for Unicode string
How can I do something like: But I would like isinstance to return True for this Unicode encoded string. Is there a Unicode string object type? Answer Test for str: or, if you must handle bytestrings, test for bytes separately: The two types are deliberately not exchangible; use explicit encoding (for str -&g…
Saving UTF-8 texts with json.dumps as UTF-8, not as a u escape sequence
Sample code (in a REPL): Output: The problem: it’s not human readable. My (smart) users want to verify or even edit text files with JSON dumps (and I’d rather not use XML). Is there a way to serialize objects into UTF-8 JSON strings (instead of uXXXX)? Answer Use the ensure_ascii=False switch to json.du…
How to convert unicode accented characters to pure ascii without accents?
I’m trying to download some content from a dictionary site like http://dictionary.reference.com/browse/apple?s=t The problem I’m having is that the original paragraph has all those squiggly lines, and reverse letters, and such, so when I read the local files I end up with those funny escape charac…
Python Requests and Unicode
I am using the requests library to query the Diffbot API to get contents of an article from a web page url. When I visit a request URL that I create in my browser, it returns a JSON object with the text in Unicode (right?) for example (I shortended the text somewhat): {“icon”:”http://mexico.…
Convert ASCII chars to Unicode FULLWIDTH latin letters in Python?
Can you easily convert between ASCII characters and their Asian full-width Unicode wide characters? Like: to Answer Those “wide” characters are named FULLWIDTH LATIN LETTER: http://www.unicodemap.org/range/87/Halfwidth%20and%20Fullwidth%20Forms/ They have range 0xFF00 – -0xFFEF. You can make…
How do you find out what the “system default encoding” is?
The documentation for fileobject.encoding mentions that it can be None, and in that case, the “system default encoding” is used. How can I find out what this encoding is? Answer You should use sys.getdefaultencoding()