Get Unicode Sequence from Text: How-To

To get the Unicode sequence from text, you convert each character into its numeric code point (like U+0041 for 'A') or its UTF-8 byte sequence (like 41 for 'A'). This article explains the different ways to do that, why you might need it, and how to avoid common pitfalls.

What Is a Unicode Sequence?

Every character you type—letters, digits, emoji, symbols—is assigned a unique number by the Unicode standard. That number is called a code point. For example, the letter 'A' is U+0041, the Euro sign '€' is U+20AC, and the emoji 😀 is U+1F600. The term “Unicode sequence” can mean a few things:

Code point sequence: the list of code points for each character in the text, e.g., “Hi” → U+0048 U+0069.
UTF-8 byte sequence: the actual bytes stored in a file when encoded as UTF-8, e.g., “€” → E2 82 AC.
UTF-16 code units: the 16-bit values used by Windows and JavaScript, e.g., “😀” → D83D DE00.

Which one you need depends on what you’re doing—debugging a font, writing a parser, or checking how a string is stored.

How to Get the Code Point of a Character

In most programming languages, there’s a built-in function to get the code point of a character. Here are a few common ones:

Python: ord('A') returns 65, hex(ord('A')) gives '0x41'.
JavaScript: 'A'.codePointAt(0) returns 65; 'A'.charCodeAt(0) also works for BMP characters.
Java: 'A' as a char is already the code unit; use Character.codePointAt("A", 0) for full Unicode.
C#: (int)'A' gives 65.
PHP: mb_ord('A') (with mbstring) returns 65.

For a whole string, loop over each character and collect the values. In Python, [hex(ord(c)) for c in "Hi"] gives ['0x48', '0x69']. In JavaScript, use for...of to iterate by code point, not by code unit, to handle emoji correctly.

Getting the UTF-8 Byte Sequence

UTF-8 is the most common encoding on the web. Each code point is encoded into 1 to 4 bytes. To see the bytes, encode the string and print the hexadecimal values.

Python: "€".encode("utf-8").hex() → e282ac.
JavaScript: new TextEncoder().encode("€") gives a Uint8Array; then convert each byte to hex.
Command line: echo -n "€" | xxd shows e2 82 ac.
Online tools: many sites let you paste text and see the UTF-8 bytes.

Remember that UTF-8 is variable-length: ASCII characters (U+0000 to U+007F) take 1 byte, up to U+07FF take 2 bytes, up to U+FFFF take 3 bytes, and beyond that take 4 bytes.

Using Online Tools vs. Programming

If you don’t code, online Unicode converters are the fastest. You paste your text, and they show code points, UTF-8, UTF-16, and HTML entities. Some also show the character names (e.g., “LATIN CAPITAL LETTER A”). These tools are handy for one-off checks, but they have downsides: you’re sending text to a third party (privacy concern), and they may not handle obscure scripts or combining characters perfectly. For batch processing or integration into a workflow, a few lines of code is better.

Common Pitfalls and Misconceptions

Here are the mistakes people often make when extracting Unicode sequences:

Confusing code points with UTF-16 code units: In JavaScript, charCodeAt returns a UTF-16 code unit, not the full code point. For emoji like 😀, charCodeAt(0) gives 55357 (0xD83D), which is a surrogate half. Use codePointAt(0) to get 128512.
Forgetting that combining characters are separate: “é” can be a single code point (U+00E9) or two (e + U+0301). The sequence you get depends on how the text was created.
Assuming UTF-8 bytes equal code points: They don’t. The code point for “€” is 0x20AC, but its UTF-8 bytes are E2 82 AC. Always know which representation you’re looking at.
Using ord() on a multi-byte character in Python 2: Python 2’s ord() only works on single-byte characters; Python 3 handles full code points. If you’re on Python 2, upgrade.

What About HTML Entities and \u Escapes?

Sometimes you need the Unicode sequence in a specific format for a document or code. HTML entities use decimal or hex: A or A for 'A'. In many programming languages, you can write a string with \u escapes: \u0041 for 'A'. In JSON, the same format applies. If you’re generating these from text, you can compute them from the code point. For example, in Python: f"&#{ord(c)};" gives the decimal HTML entity. In JavaScript: `&#${c.codePointAt(0)};`.

Frequently Asked Questions

How do I get the Unicode sequence for a whole string in Python?

Loop over the string and call ord() on each character. For example: [hex(ord(c)) for c in "Hello"] returns ['0x48', '0x65', '0x6c', '0x6c', '0x6f']. For multi-byte characters, ord() still gives the full code point.

Why does my emoji show as two codes in JavaScript?

Because charCodeAt() returns UTF-16 code units, and emoji outside the Basic Multilingual Plane are represented as a surrogate pair. Use codePointAt() to get the actual code point. For example, "😀".codePointAt(0) returns 128512.

What is the difference between code point and UTF-8 bytes?

A code point is the abstract number assigned to a character (like U+20AC for €). UTF-8 bytes are the specific byte sequence used to store that character in a file (E2 82 AC). The encoding rules determine how the code point becomes bytes.

Can I get the Unicode sequence from a file without writing code?

Yes. Many text editors (like Notepad++ or VS Code) have a “Hex Viewer” or “Show Unicode” feature. Online tools also work, but be mindful of privacy if the text is sensitive.

How do I convert a code point back to a character?

Use the inverse function: chr(65) in Python, String.fromCodePoint(65) in JavaScript, Character.toChars(65) in Java. This is useful for round-tripping.

Why does my text contain U+FFFD (replacement character)?

That means the text was decoded incorrectly—the original bytes weren’t valid in the encoding you assumed. It often happens when a file is UTF-8 but you opened it as Latin-1, or vice versa. Check the file’s actual encoding.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →