How is Shannon entropy calculated for text?

Shannon entropy H = −Σ(pᵢ × log₂(pᵢ)) for each unique character i, where pᵢ is the probability (frequency) of that character in the text. For example, in the string "aab", p(a)=2/3, p(b)=1/3, so H = −(2/3×log₂(2/3) + 1/3×log₂(1/3)) ≈ 0.918 bits/char. For password security, the entropy estimate is often reported as raw (total entropy) and per-character. A 10-character password with 6.5 bits/char entropy has 65 bits of total entropy — a strong password that would take billions of attempts to brute-force.

What is the difference between character count and byte count?

Character count is the number of Unicode code points (roughly, the number of visible symbols and spaces you see). Byte count depends on the encoding. In UTF-8, "a" = 1 byte, "é" = 2 bytes, "字" = 3 bytes, "😀" = 4 bytes. A string of 5 emoji is 5 characters but 20 bytes in UTF-8. This distinction matters for database column sizing (VARCHAR(255) may mean 255 characters or 255 bytes depending on the database and collation), network bandwidth, and API payload limits.

How can I use this tool for SEO meta tag optimization?

Toggle SEO Mode in the analyzer. This adds counters calibrated to Google's displayed character limits: title tag (50-60 characters before truncation), meta description (150-160 characters), and URL slug (ideally 50-60 characters, though technically unlimited). It also shows pixel-width estimates for title and description (Google uses proportional fonts, so "W" takes more pixels than "i"). Enter your draft meta text and the tool flags when you exceed the limit, helping you write titles and descriptions that display fully in search results.

What does the character frequency chart tell me?

The frequency chart shows every unique character in your text, ranked by occurrence count with percentage. This has several uses: (1) cryptography — substitution ciphers are cracked by comparing ciphertext frequency to known language frequency; (2) content quality — overuse of specific characters or words can indicate keyword stuffing; (3) encoding efficiency — texts dominated by ASCII characters are cheap in UTF-8; (4) language identification — different languages have distinct frequency signatures; (5) forensic linguistics — authorship analysis based on habitual character patterns.

What is the unique word ratio and why does it matter for writing quality?

The unique word ratio (Type-Token Ratio, TTR) measures vocabulary diversity. It's calculated as: unique words / total words. For a 500-word blog post, a TTR of 0.5 means 250 distinct word forms are used — indicative of varied, natural writing. A TTR of 0.2 means only 100 distinct words form the entire 500-word text — highly repetitive, characteristic of keyword-stuffed or poorly written content. However, TTR is sensitive to text length (longer texts naturally have lower TTR because function words like "the" keep repeating while new content words slow down), so compare TTRs only for texts of similar length.

Calculators2026-05-10

String Analyzer: Character Count, UTF-8 Byte Size, Entropy, and Text Diagnostics

Text looks simple — just a sequence of characters. But ask a computer "how long is this string?" and the answer depends on what you mean by "length." Is it 140 characters? 160 bytes? 10 words? And what about the invisible properties — the entropy that measures randomness, the frequency distribution of letters, or the fact that "e" appears three times more often than "k"? A string analyzer unpacks all of these dimensions at once. Our free online text analyzer gives you everything from basic counts (characters with and without spaces, words, lines, paragraphs) to advanced diagnostics (byte size in UTF-8/UTF-16/UTF-32, Shannon entropy, character frequency distribution, unique word ratio, longest word, and palindrome detection) — all computed instantly in your browser with zero data sent to any server.

manage_search

String Analyzer

Free · No registration

Try this tool for free →open_in_new

Step-by-Step Guide

Paste or Type Your Text

Enter your text in the input area — anything from a single word to a full document. The analyzer updates in real time as you type, showing live counts. A word counter tracks changes as you write; the character counter distinguishes between total characters and characters excluding whitespace. The text stays entirely in your browser — no data is ever sent over the network, making this safe for sensitive content, passwords under analysis, or confidential documents.

Review the Basic and Advanced Metrics

The results panel displays: character count (total and without spaces), word count (using Unicode-aware word boundary detection), line count, paragraph count, sentence count, average word length, longest word, unique word count and ratio (unique/total words, a measure of lexical diversity), and estimated reading time. Below the basics, advanced metrics show byte size in three encodings (UTF-8, UTF-16, UTF-32), Shannon entropy, and character frequency distribution.

Explore Byte Size, Entropy, and Frequency

The encoding section shows how many bytes your text consumes in UTF-8 (1-4 bytes per character, ASCII characters use 1 byte, CJK characters use 3), UTF-16 (2 or 4 bytes per character), and UTF-32 (always 4 bytes per character). The entropy score (0-8 bits per character) tells you how random and unpredictable the text is — useful for password strength estimation. The frequency table ranks every character by occurrence count and percentage, revealing patterns like the classic ETAOIN SHRDLU frequency order of English.

Tips & Best Practices

check_circle

UTF-8 is the dominant encoding on the web (used by 98%+ of websites). It's variable-width: ASCII characters (English letters, digits, common punctuation) take 1 byte; Latin Extended and Greek/Cyrillic take 2 bytes; CJK (Chinese, Japanese, Korean) characters take 3 bytes; emoji and rare scripts take 4 bytes. This means the string "Hello" is 5 bytes in UTF-8, while "こんにちは" is 15 bytes — three times larger for the same number of characters.

check_circle

UTF-16 is used internally by JavaScript, Java, and Windows APIs. It uses 2 bytes for characters in the Basic Multilingual Plane (BMP, U+0000 to U+FFFF, covering most living languages) and 4 bytes (surrogate pairs) for characters beyond — like emoji, historical scripts, and rare CJK characters. The string "Hello😀" is 5 characters but 10 bytes in UTF-16 (5×2) and 9 bytes in UTF-8 (5+4).

check_circle

Shannon entropy measures the average information content per character. English text typically scores 3.5-4.5 bits/char due to predictable letter frequencies and patterns. Fully random text (all 26 letters equally likely) approaches 4.7 bits/char. A random password mixing upper, lower, digits, and symbols can reach 6.5+ bits/char. Higher entropy = more random = stronger password. As a rule of thumb: below 2.5 bits/char is highly structured text; 3.5-4.5 is natural language; above 5.5 is random/generated.

check_circle

For SEO meta tags: Google typically displays title tags up to 600 pixels wide (roughly 50-60 characters) and meta descriptions up to 920 pixels (roughly 150-160 characters). Exceed these limits and your text gets truncated with an ellipsis in search results. Our string analyzer's "SEO Mode" adds counters specifically calibrated to these display limits, flagging when you exceed the visible threshold.

check_circle

SMS messages use a 160-character limit for standard GSM 7-bit encoding (Latin characters + basic symbols), but messages containing non-GSM characters (Cyrillic, CJK, emoji) use UCS-2 encoding, which drops the limit to 70 characters per segment. Messages longer than one segment are concatenated, reducing the effective length further. Our analyzer indicates whether your text fits in one SMS segment and which encoding applies.

check_circle

Character frequency analysis reveals patterns useful for cryptography and linguistics. In English prose, the letter "e" appears about 12.7% of the time, followed by "t" (9.1%), "a" (8.2%), and "o" (7.5%). The least common letters (q, z, j, x) each appear under 0.2% of the time. A substitution cipher can be cracked by aligning the ciphertext's frequency distribution with the known distribution of the plaintext language — the analyzer's frequency chart makes this visual.

check_circle

The unique word ratio (unique words ÷ total words) is the Type-Token Ratio (TTR), a measure of lexical diversity. For conversational English, TTR typically ranges 0.45-0.55. Higher values (0.6+) indicate varied vocabulary; lower values (under 0.3) indicate repetition. Children's books and instructional texts have low TTR; literary fiction has high TTR. Text with a TTR below 0.2 often reads as robotic or keyword-stuffed — a red flag for SEO content quality.

Frequently Asked Questions

UTF-8 uses 1-4 bytes per character based on Unicode code point. ASCII-range characters (U+0000-U+007F) use 1 byte, making UTF-8 very efficient for English text. UTF-16 uses 2 bytes for BMP characters (U+0000-U+FFFF) and 4 bytes for supplementary characters via surrogate pairs. For mostly-ASCII text, UTF-8 is about half the size of UTF-16. For mostly-CJK text, UTF-8 and UTF-16 are comparable (CJK needs 3 UTF-8 bytes vs 2 UTF-16 bytes). UTF-32 always uses 4 bytes per character, making it predictable but space-inefficient — rarely used for storage, mainly for internal processing where fixed-width access matters.

Text is the most common data type in computing, yet its hidden properties — byte size across encodings, entropy, frequency patterns, lexical diversity — are invisible without a string analyzer. Our tool reveals everything about your text in real time, from character counts for SEO to entropy scores for password strength. Paste your text and see what's really there.

Try this tool for free →open_in_new