The error UnicodeEncodeError: 'ascii' codec can't encode character occurs when Python tries to convert a Unicode character into ASCII, but the character does not exist in the ASCII character set (0-127).
1. Common Causes and Fixes
Cause 1: Printing Non-ASCII Characters in an ASCII Environment
If your system’s default encoding is ASCII, and you try to print non-ASCII characters, this error may occur.
Incorrect Code:
print("Café") # Contains 'é' which is outside ASCII range
Problem: The é character is not in the ASCII table (0-127).
Solution: Explicitly set UTF-8 encoding:
import sys
sys.stdout.reconfigure(encoding="utf-8") # For Python 3.7+
print("Café")
For older versions:
import codecs
sys.stdout = codecs.getwriter("utf-8")(sys.stdout.buffer)
print("Café")
Cause 2: Writing Non-ASCII Characters to a File with ASCII Encoding
If you open a file without specifying encoding, Python may default to ASCII.
Incorrect Code:
with open("output.txt", "w") as f:
f.write("Café")
Problem: ASCII does not support é.
Solution: Use UTF-8 encoding when opening the file:
with open("output.txt", "w", encoding="utf-8") as f:
f.write("Café")
Cause 3: Encoding Strings with ASCII Instead of UTF-8
If you explicitly encode a string using ASCII, Python fails when non-ASCII characters exist.
Incorrect Code:
text = "Español"
encoded_text = text.encode("ascii") # ASCII does not support 'ñ'
Problem: ASCII does not include the ñ character.
Solution: Encode using UTF-8:
encoded_text = text.encode("utf-8")
Or ignore or replace unsupported characters:
encoded_text = text.encode("ascii", errors="ignore") # Removes unsupported characters
encoded_text = text.encode("ascii", errors="replace") # Replaces with '?'
Cause 4: Default System Encoding is ASCII (Python 2 or Old Systems)
In Python 2 (or some misconfigured environments), the default encoding may be ASCII.
Solution: Set the default encoding to UTF-8:
import sys
reload(sys)
sys.setdefaultencoding("utf-8")
Note: sys.setdefaultencoding() was removed in Python 3.
For Python 3, check system encoding:
import sys
print(sys.getdefaultencoding()) # Should be 'utf-8'
If it’s ASCII, fix it by setting environment variables:
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
Cause 5: Converting Unicode to Bytes Using ASCII Encoding
Using str() on Unicode strings in Python 2 or improper encoding in Python 3 can cause this issue.
Incorrect Code:
text = u"Café"
encoded_text = str(text) # Fails in Python 2
Problem: Python tries to encode using ASCII by default.
Solution: Use .encode("utf-8"):
encoded_text = text.encode("utf-8")
Cause 6: Using json.dumps() Without Specifying Encoding
The json.dumps() method may default to ASCII encoding.
Incorrect Code:
import json
data = {"name": "José"}
json_string = json.dumps(data) # Defaults to ASCII in older Python versions
Problem: ASCII does not support é.
Solution: Specify ensure_ascii=False:
json_string = json.dumps(data, ensure_ascii=False)
2. Summary of Fixes
| Issue | Fix |
|---|---|
| Printing non-ASCII characters | sys.stdout.reconfigure(encoding="utf-8") |
| Writing non-ASCII text to a file | Use open("file.txt", "w", encoding="utf-8") |
| Encoding a string with ASCII | Use .encode("utf-8") instead of .encode("ascii") |
| System default encoding is ASCII | Set UTF-8 via sys.setdefaultencoding("utf-8") (Python 2) |
Using json.dumps() | Set ensure_ascii=False |