Unicode Normalizer — NFC, NFD, NFKC and NFKD Text Conversion

Unicode Normalizer At a Glance

Unicode often permits more than one sequence of code points to represent the same apparent text. For example, an accented letter may exist as one precomposed code point or as a base letter followed by a combining accent. A system comparing raw sequences can treat those forms as different even when they look identical. Unicode normalization selects a predictable representation so software can compare or process the text more consistently.

NFC composes canonical sequences where possible, while NFD separates them into their decomposed forms. NFKC and NFKD add compatibility transformations that may fold stylistic or compatibility distinctions, including some circled digits and typographic ligatures. That extra transformation can be valuable for analysis but inappropriate for names, branded typography, mathematical notation or identifiers whose visible distinction matters.

How to Normalize Text Step by Step

Paste the source into the editor, select NFC, NFD, NFKC or NFKD, and choose Normalize text. The tool shows the original and normalized output side by side, including whether the string changed. It compares Unicode code-point counts and UTF-8 byte sizes and reports the number of differing point positions. That report is particularly useful when a database value looks identical but occupies a different number of bytes after normalization.

Copy or download the result once you have checked it. Run a representative sample before applying the same form to a large collection of identifiers. Normalization does not correct spelling, convert languages, or decide whether a particular character is permitted in your software. It transforms Unicode sequences according to standardized equivalence rules and leaves those other decisions to the receiving application.

NFC and NFD: Canonical Equivalence

NFC is frequently useful for publishing text and storing consistent strings because it prefers composed forms when Unicode provides them. NFD uses canonically decomposed sequences and can be useful in text processing workflows that examine individual marks. Neither method means that an entire language is reduced to ASCII. An accented letter can remain accented after normalization; only its underlying representation changes.

Consider the word café. The final letter may be encoded as U+00E9 or as U+0065 followed by U+0301. NFC tends to produce the composed representation; NFD tends to produce the decomposed representation. A typical reader sees the same word in either case, but byte-level hashes can differ. If you are checking exact bytes, use the Text Hash Generator to see why normalization choice matters for reproducible digests.

NFKC and NFKD: Compatibility Changes

Compatibility normalization can map characters with special presentation forms to other sequences considered equivalent for certain processing tasks. A circled numeral can become an ordinary digit, and a ligature may expand into separate letters. Those operations can improve matching in search systems, but they may erase intentional distinctions. An organization using compatibility characters in product codes should never replace them blindly with normalized versions.

NFKC applies compatibility mapping and composition; NFKD applies compatibility mapping and canonical decomposition. Neither option is a general-purpose security sanitizer. Confusable letters from different alphabets may remain distinct, and normalization alone cannot establish whether a username is safe or unambiguous. When working with account identifiers or access policies, apply an explicit allowed-character policy in addition to whichever normalization form the system requires.

Byte Length, Code Points and Visual Characters

The number of Unicode code points does not always equal the number of visual symbols. A grapheme cluster may contain a base character plus combining marks, and an emoji sequence may include joiners and variation selectors. UTF-8 byte length adds another measure because different characters require different numbers of bytes. The output panel therefore reports code points and bytes rather than pretending those figures always equal the count a user sees.

If a form enforces a length restriction in visible characters or UTF-16 code units, normalization can change whether input meets that limit. Compare the exact measurement policy of the destination system with the reported output. The Character Limit Checker is useful when the practical question is how a field will count text rather than whether two Unicode representations are canonically equivalent.

Where Normalization Helps in Real Workflows

A publishing team may receive contributor names copied from different devices and discover that a search or deduplication operation treats visually identical accented names differently. Canonical normalization can make the stored values easier to compare while preserving their human-readable appearance. A developer may also normalize incoming content before producing a digest or comparing it with a documented reference value, provided both sides agree on the same normalization form.

A different use case involves imported spreadsheets with mixed Unicode forms. Instead of overwriting the complete spreadsheet immediately, normalize a sample column and compare before-and-after values. If a value represents a legally significant personal name or a code assigned by a third party, retain the original alongside the normalized search key. Distinguishing an authoritative value from a processing key reduces the risk of unintentional alteration.

Unicode Normalization Versus Invisible Character Removal

Normalization and invisible-character cleanup solve different problems. Unicode normalization addresses canonical and compatibility representations; invisible-character detection identifies formatting marks that may not appear on screen. NFC normally does not remove every zero-width space or directional control, nor should it, because those characters can have legitimate purposes. A text string can be normalized and still contain a confusing invisible character.

The Invisible Character Detector reports controls and joiners so you can review them separately. If a string appears unexpectedly reversed or contains zero-width marks, do not assume selecting NFKC will make it safe. Inspect each type of difference, then apply the minimum transformation needed. This is especially important when changing content in right-to-left languages or emoji-rich descriptions.

Reviewing Changes and Keeping an Original Copy

Check unusual punctuation, symbols, ligatures, product identifiers and names after applying a compatibility form. Compare the visual output and the reported byte and code-point counts. A change count of zero means the selected normalization operation produced the same string; it is not a guarantee that the content contains no spelling problems, bidirectional controls or visually confusable characters.

Where exact preservation is important, store the original before normalizing. A normalized search key may be a useful additional field, while the original should remain available for display, legal records or verification. Treat NFKC and NFKD as explicit transformations rather than silent cleaning steps. This allows the software to benefit from consistent comparisons without discarding distinctions the original author intended.

Normalization Policies for Search and Stored Records

A search interface and a stored record do not necessarily need the same string representation. It may be appropriate to keep a supplied author name unchanged for display while storing an NFC-normalized copy for matching. That approach also makes it possible to review whether two apparently identical names were originally provided with different code-point sequences. For transactional identifiers, the comparison policy should be documented and tested, because applying compatibility normalization to an external identifier could create a collision with another valid code.

Normalization should be applied consistently at defined entry points. If one subsystem compares NFC values and another hashes the original bytes, the two processes can disagree without either producing an obvious visual error. Decide whether normalization takes place before indexing, before comparison or only in a derived search field. Test values containing combining marks, ligatures, circled numbers, Arabic text, emoji sequences and symbols relevant to your application. Those examples reveal more than a sample consisting entirely of basic English letters.

Normalized text may also change future rendering in small but noticeable ways under different fonts and operating systems. Code-point equivalence does not guarantee every font produces identical pixels, and a compatibility mapping may intentionally replace one glyph with another. For editorial content, review headings, logos, mathematical expressions and brand names after transformation. The appropriate form is determined by how the destination uses the text, not by a blanket assumption that every normalization operation is harmless cleanup.

Frequently Asked Questions