Why I Choose My Blog Name To Be Called: WWW.ZQXJ.BLOGSPOT.COM =D Is The Rarest and Most Valuable Word On Google Search Engine :)

Letter frequency

From Wikipedia, the free encyclopedia

Jump to: navigation, search

The frequency of letters in text has often been studied for use in cryptography, and frequency analysis in particular. No exact letter frequency distribution underlies a given language, since all writers write slightly differently. Linotype machines assumed the letter order, from most to least common, to be etaoin shrdlu cmfwyp vbgkjq xz based on the experience and custom of manual compositors. Likewise, Modern International Morse code encodes the most frequent letters with the shortest symbols; arranging the Morse alphabet into groups of letters that require equal amounts of time to transmit, and then sorting these groups in increasing order, yields e it san hurdm wgvlfbk opjxcz yq. Similar ideas are used in modern data-compression techniques such as Huffman coding.

Contents 1 Introduction 2 Relative frequencies of letters in the English language 3 Relative frequencies of the first letters of a word in the English language 4 Relative frequencies of letters in other languages 5 See also 6 References 7 External links

Introduction

More recent^[when?] analyses show that letter frequencies, like word frequencies, tend to vary, both by writer and by subject. One cannot write an essay about x-rays without using frequent Xs, and the essay will have an especially strange letter frequency if the essay is about the frequent use of x-rays to treat zebras in Qatar. Different authors have habits which can be reflected in their use of letters. Hemingway's writing style, for example, is visibly different from Faulkner's. Letter, bigram, trigram, word frequencies, word length, and sentence length can be calculated for specific authors, and used to prove or disprove authorship of texts, even for authors whose styles aren't so divergent.
Accurate average letter frequencies can only be gleaned by analyzing a large amount of representative text. With the availability of modern computing and collections of large text corpora, such calculations are easily made. This Deafandblind link details examples from a variety of sources, (press reporting, religious text, scientific text and general fiction) and there are differences especially for general fiction with the position of 'h' and 'i'. The example differs from the linotype 'etaoin shrdlu' to come out as 'etaoHn Isrdlu'.
Herbert S. Zim, in his classic introductory cryptography text "Codes and Secret Writing", gives the English letter frequency sequence as "ETAON RISHD LFCMU GYPWB VKJXQ Z", the most common letter pairs as "TH HE AN RE ER IN ON AT ND ST ES EN OF TE ED OR TI HI AS TO", and the most common doubled letters as "LL EE SS OO TT FF RR NN PP CC".^[1]
The 'top twelve' letters comprise about 80% of the total usage. The 'top eight" letters comprise about 65% of the total usage. Letter frequency as a function of rank can be fitted well by several rank functions, with the two-parameter Cocho/Beta rank function being the best.^[2] Another rank function with no adjustable free parameter also fits the letter frequency distribution reasonably well^[3] (the same function has been used to fit the amino acid frequency in protein sequences.^[4]) A spy using the VIC cipher or some other cipher based on a straddling checkerboard typically uses a mnemonic such as "a sin to err" (dropping the second "r") to remember the top 8 characters.
The use of letter frequencies and frequency analysis plays a fundamental role in cryptograms and several word puzzle games, including Hangman, Scrabble, Bananagrams, and the television game show Wheel of Fortune. One of the earliest description in classical literature of applying the knowledge of English letter frequency to solving a cryptogram is found in E.A. Poe's famous story The Gold-Bug, where the method is successfully applied to decipher a message instructing on the whereabouts of a treasure hidden by Captain Kidd.^[5]
Letter frequencies had a strong effect on the design of some keyboard layouts. The most-frequent letters are on the bottom row of the Blickensderfer typewriter. The most-frequent letters are on the home row of the Dvorak Simplified Keyboard.

Relative frequencies of letters in the English language

Relative frequencies of letters in text.

Relative frequencies ordered by frequency.

The letter frequencies for English are listed below.^[6] However, this table differs slightly from others, such as Cornell University Math Explorer's Project, which produced this table after measuring 40,000 words. Analysis of entries in the Concise Oxford dictionary is published by the compilers.^[7] The table below is taken from this table^[8]

Letter	Relative frequency in the English language
a	8.167%
b	1.492%
c	2.782%
d	4.253%
e	12.702%
f	2.228%
g	2.015%
h	6.094%
i	6.966%
j	0.153%
k	0.772%
l	4.025%
m	2.406%
n	6.749%
o	7.507%
p	1.929%
q	0.095%
r	5.987%
s	6.327%
t	9.056%
u	2.758%
v	0.978%
w	2.360%
x	0.150%
y	1.974%
z	0.074%

In English, the space is slightly more frequent than the top letter (e) ^[9] and the non-alphabetic characters (digits, punctuation, etc.) occupy the fourth position, between t and a.^[10]

 ^{Source: https://en.wikipedia.org/wiki/Letter_frequency}