normalize NFC and NFD
unicodedata.normalize(form, unistr)NFC and NFD implement canonical composition and decomposition. Equal appearance does not mean equal Python strings until they are normalized the same way.
| Parameter | Description |
|---|---|
form | One of "NFC", "NFD", "NFKC", or "NFKD". |
unistr | The Unicode string to normalize. |
Returns: A new normalized str.
Raises
ValueError— Raised when form is not one of the four supported names.
import unicodedata
composed = 'café'
decomposed = 'cafe\u0301'
print(composed == decomposed)
print(unicodedata.normalize('NFC', composed) == unicodedata.normalize('NFC', decomposed))
for form in ('NFC', 'NFD'):
for label, value in (('composed', composed), ('decomposed', decomposed)):
out = unicodedata.normalize(form, value)
print(form, label, len(out), [hex(ord(ch)) for ch in out])Expected output
False
True
NFC composed 4 ['0x63', '0x61', '0x66', '0xe9']
NFC decomposed 4 ['0x63', '0x61', '0x66', '0xe9']
NFD composed 5 ['0x63', '0x61', '0x66', '0x65', '0x301']
NFD decomposed 5 ['0x63', '0x61', '0x66', '0x65', '0x301']
Related: Unicode inspector tool, str methods