Thread View: sci.crypt
18 messages
18 total messages
Started by Leonard Erickson
Wed, 06 Jan 1988 05:36
how do you tell encrytped data from random data?
Author: Leonard Erickson
Date: Wed, 06 Jan 1988 05:36
Date: Wed, 06 Jan 1988 05:36
33 lines
2059 bytes
2059 bytes
An interesting question has crossed my mind. If someone presents you with an allegedly encrypted message, How can you tell if it really is encrypted as opposed to being a bunch of random characters? I know that transposition and *simple* substitution can be detected by letter frequency analysis. But is a "flat" distibution evidence of random data? For my purposes, both "one-time pad" ciphers and anything that operates on units other than characters can be considered random! If it is that complex, then I'm not likely to crack it! This is inspired by my fiinding some notes I made a year or so ago when I and some friends were exchanging encrypted msgs over a public BBS. A few of our friends cracked or first efforts and a typical "arms race" of better ciphers followed by better analytic techniques ensued. We finally came (after about 5 changes) came up with a scheme that only one outsiders could read part of. Some of the other users started using their own ciphers. The sysop didn't mind until someone started posting HUGE messages which he suspected were just garbage. But the challenge was, were they garbage or real messages? Under his rules, messages (even encrypted!) were ok, junk wasn't. I made a few attempts to analyze them, but while I was fairly certain, I could never be sure. (when an 8k msg uses the 26 letters so evenly that the spread better most used and least used is 12, you get *real* suspicious :-) -- Leonard Erickson ...!tektronix!reed!percival!bucket!leonard CIS: [70465,203] "I used to be a hacker. Now I'm a 'microcomputer specialist'. You know... I'd rather be a hacker."
Re: how do you tell encrytped data from random data?
Author: Doug Gwyn
Date: Fri, 08 Jan 1988 06:37
Date: Fri, 08 Jan 1988 06:37
29 lines
1864 bytes
1864 bytes
In article <660@bucket.UUCP> leonard@bucket.UUCP (Leonard Erickson) writes: >An interesting question has crossed my mind. If someone presents you with >an allegedly encrypted message, How can you tell if it really is encrypted >as opposed to being a bunch of random characters? The only foolproof way is to decrypt it. >I know that transposition and *simple* substitution can be detected by >letter frequency analysis. But is a "flat" distibution evidence of random >data? No, most really good encryption systems pretty much randomizes the cipher stream, insofar as simple statistical tests can determine. Conversely, just because some putative cipher stream appears nonrandom does not mean that it has meaningful underlying plaintext. It could be a hoax. There are those who think the two undeciphered Beale cryptograms are hoaxes. >For my purposes, both "one-time pad" ciphers and anything that operates on >units other than characters can be considered random! If it is that complex, >then I'm not likely to crack it! You can't crack a true one-time pad system by analysis anyway, no matter how competent a cryptanalyst you are. >(when an 8k msg uses the 26 letters so evenly that the >spread better most used and least used is 12, you get *real* suspicious :-) Flatter-than-random distributions are not necessarily hoaxes. They're actually pretty easy to obtain; Mil Cryp has an example.
Re: how do you tell encrytped data from
Author: leather
Date: Fri, 08 Jan 1988 19:15
Date: Fri, 08 Jan 1988 19:15
44 lines
3064 bytes
3064 bytes
> /* Written 11:36 pm Jan 5, 1988 by leonard@bucket.UUCP ... */
> /* ---------- "how do you tell encrytped data from" ---------- */
> An interesting question has crossed my mind. If someone presents you with
> an allegedly encrypted message, How can you tell if it really is encrypted
> as opposed to being a bunch of random characters?
>
> [...]
>
Although the _suspicious_ nature of an even distribution of letters
may be a give-away indication of *JUNK* messages for amateur and "semi-pro"
encryption schemes, it cannot be absolute proof.
I have dabbled some in the crypto-analysis field, myself. It is not
difficult to think of some methods for creating *SMOOTH* distributin messages:
1. massage the message with your existing encryption scheme,
2. perform a distribution analysis on the encoded result,
3. obtain the complementary distribution analysis - for *SMOOTHNESS*,
4. apply a padding of additional characters to the previously
encoded result, yielding the smoothed encrypted message:
a. padding could be in the form of "encasing" the previously
encoded result (either/both ends of message by line, block,
...),
b. or "enbedding" the characters in some simple/complex
pattern.
The real question is why bother, except to annoy or distract?
If you have a sufficiently effetive cypher to begin with, why add complexity
to its encoding/decoding process? If you are going to hide a *REAL* message
in a diversionary barrage of *JUNK* messages, you could just as well hide it
in *JUNK* that used ordinary English, French, Russian, Chinese, ... sentence
packages, in which case the _normal_ distribution patterns are going to lead
to equally wrong conclusions. The point is, there are ever so many ways for
going about this business of encryption that one analysis technique alone is
insufficient to determine what *IS* or *IS_NOT* encrypted data. Even if one
goes to the trouble of using a less complex technique for encryption of some
message, who is to say that the _message_ was not the *JUNK*. GIGO (garbage
in, garbage out)? It may be *JUNK*, but it _was_ encrypted! But then, ....
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ David A. Leatherman ~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (ARPA: leather@osiris.cso.uiuc.edu) ~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ leathrmn@uxe.cso.uiuc.edu ~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Re: how do you tell encrytped data from random data?
Author: Marcus J. Ranum
Date: Sat, 09 Jan 1988 19:55
Date: Sat, 09 Jan 1988 19:55
20 lines
1499 bytes
1499 bytes
Reminds me of a wonderful concept in a book by Stanitslaw Lem
(Memoirs Found in a Bathtub):
The underground of the Pentagon is all that remains after the end of
the world, but the spooks are sure they have been infiltrated. One of the best
tools they have is a super-dooper computer than can decrypt *ANYTHING* if it
tries hard enough. In fact, it can take plain text, and find the secret and
subversive messages hidden in it - even if there weren't any to begin with...
Another problem with Gywn's suggestion that the only way to tell if it
is encrypted (decrypt it) - how can you tell that it is really "meaningful" ?
Anyone who has listened to first year latin students trying to "decrypt"
latin will realize that garbage in does not always imply the same garbage out.
--mjr();
--
------------------------------------------------------------------------------
...ich bin in einem dusenjet ins jahr 53 vor chr...ich lande im antiken Rom...
einige gladiatoren spielen scrabble...ich rieche PIZZA...
Re: how do you tell encrytped data from random data?
Author: Henry Schaffer
Date: Sun, 10 Jan 1988 02:22
Date: Sun, 10 Jan 1988 02:22
14 lines
1100 bytes
1100 bytes
These days it is quite common to compress text (and other files too.) The various compressions schemes such as Huffman encoding and Lempel- Ziv-Welch tend to even out (smooth out) the character distribution (i.e., they hope to maximize the information transmitted per character, or per byte). If this file is then encrypted, the smoothness may end up in the encrypted data. Conversly, random data need not have a smooth character distribution. (It is common to confuse "random" with "equi-probable". However, even though most examples given in courses of "random" are "equi-probable there is no necessary connection.) --henry schaffer n c state univ
Re: how do you tell encrytped data from
Author: gordon
Date: Sun, 10 Jan 1988 07:24
Date: Sun, 10 Jan 1988 07:24
75 lines
4817 bytes
4817 bytes
> An interesting question has crossed my mind. If someone presents you with > an allegedly encrypted message, How can you tell if it really is encrypted > as opposed to being a bunch of random characters? You can't. And if I wanted to really screw things up, and had plenty of transmission bandwidth, I could use an encoding method that was mostly, but not completely noise. For example, each netnews article I post could have encoded in it one bit, which is either 1 or 0 depending on whether the number of lines in it plus the number of newsgroups it is cross-posted to is odd or even. I have to post about 200 netnews articles to transmit one sentence. I defy anyone to prove that there is really a message! For a less blatant waste of transmission bandwidth, how about using the "parity" bits only to contain the message, and the lower 7 bits contain random quotes from "talk.abortion" and "talk.bizarre"? > I know that transposition and *simple* substitution can be detected by > letter frequency analysis. But is a "flat" distibution evidence of random > data? No. Among other things, crypt(1) output gives a fairly flat distribution in its 256-character output alphabet. (the version of crypt I am thinking of is based on a WW-II rotor machine, and uses DES only to scramble the user's typed key. (You don't need to crack the DES to crack the encryption). This version was in System III, among others). In fact, it gives an absolutely flat distribution for any message consisting of one character repeated a multiple of 256 times and any key. (If not a multiple of 256, the distribution is as flat as you can get with that message length). You also get a flat distribution, more or less, by encoding techniques as "encode the i th character of the message by rotating it i positions in the alphabet", assuming the message was reasonably long. This doesn't even have an encryption key. A "real" encryption might use a keyword J characters long, and encode the i th character of the message by rotating it by its position in the message (i) plus an amount determined from the (i mod J)th character of the key. For messages that are long relative to the key length and the alphabet size, this will give a fairly flat distribution. You will be able to detect distribution patterns by taking every (J * alphabet size)th character, and looking at the frequency distribution for that. > For my purposes, both "one-time pad" ciphers and anything that operates on > units other than characters can be considered random! If it is that complex, > then I'm not likely to crack it! I'm not quite sure how to interpret this. Would this mean that running the message through uuencode(1) (which maps every 3 characters of 8 bits each into 4 characters out of a 6-bit 64-character alphabet) is too complex? It doesn't even have an encryption key! Running stuff through uuencode does tend to flatten out the distribution somewhat, but a message consisting mostly of the letter x will still have several high-frequency characters. To REALLY flatten out the distribution, there are several popular UNIX and DOS utilities to do this, under the classification "data compression programs". These include pack, compress, ARC, and compact. If you don't like the use of all 8 bits in the output, run the result through a filter to transform it to a smaller alphabet. These include things like uuencode/uudecode, btoa/atob (distributed with compress 4.0), encode/decode (distributed with B news 2.11), and, if you're really desparate, od, the octal dump program. (A few years back, someone posted a program to take an octal dump and re-create the file it was a dump of). These programs do not encrypt (that is, there is no encryption key, and just knowing which programs are used and in what order is sufficient to recover the plaintext. To get reasonable security, transform the text in several stages: plaintext -> compress -> your favorite encryption -> uuencode -> ciphertext Compress first, then encrypt. You save on message transmission costs and you mess up some clues used to attack the encryption - such as "all characters going into the encryption probably have the high-order bit turned off", because the compression removed much of the redundancy. Gordon Burditt ...!ihnp4!sys1!sneaky!gordon
Re: how do you tell encrytped data from random data?
Author: Doug Gwyn
Date: Mon, 11 Jan 1988 09:50
Date: Mon, 11 Jan 1988 09:50
44 lines
3137 bytes
3137 bytes
In article <1499@osiris.UUCP> mjr@osiris.UUCP (Marcus J. Ranum) writes: >Another problem with Gywn's suggestion that the only way to tell if it >is encrypted (decrypt it) - how can you tell that it is really "meaningful"? It's not that hard in practice. Witness the following, which have been uniquely cryptanalyzed into plaintext which all involved agreed was correct (and in cases where it could be checked with the encryptor, its correctness was verified): (a) the Chaucer transposition with spelling variations that was posted to this newsgroup not long ago; (b) "challenger" cryptograms Some of the latter are extremely short, and have plaintext that nobody in his right mind would really use. I've personally broken quite short messages, some 25 letters long, encrypted by some of the simpler methods. Usually, student cryppies would all get essentially the same result (sometimes off in one or two letters) for cryptanalysis of practice cryptograms, if they were able to crack them at all. It is extremely hard to make up a cryptogram of modest length that has two valid but different decryptions. In fact, I don't know of any of length more than a couple of dozen letters, although I can imagine some ways to try to do this for somewhat longer messages. This would make an interesting puzzle for someone. On the other hand, there are people such as those who find Baconian ciphers in Shakespeare's works, who put the message there themselves. It's pretty easy to tell when this happens; either the decipherment method is ambiguous, so the message could never have been realistically expected to be read even by its intended recipient, or so many "corrections" and otherwise unjustified special twists on the decipherment scheme have been introduced that the information content of the decipherment rivals that of the supposed message being extracted. It is interesting to note that William F. Friedman, perhaps the greatest cryptanalyst of all time, and his future wife got their start in the field by being employed at "Col." Fabyan's Riverbank Labs, which was trying to find just such hidden messages in Shakespeare. The Friedmans showed how the same methodology could be used to produce messages with precisely opposite meaning, i.e. the methodology was making up the whole content of the messages based on the analyst's preconceptions about what was to be found. On the other side of this issue, several rum-runner codes (generally much more ambiguous to crack than ciphers) were broken by the Friedmans, and they were able to demonstrate their methodology satisfactorily to the federal courts.
Re: how do you tell encrypted data from random data?
Author: Jim Gillogly
Date: Mon, 11 Jan 1988 17:25
Date: Mon, 11 Jan 1988 17:25
36 lines
2503 bytes
2503 bytes
In article <1499@osiris.UUCP> mjr@osiris.UUCP (Marcus J. Ranum) writes:
> Another problem with Gywn's suggestion that the only way to tell if it
>is encrypted (decrypt it) - how can you tell that it is really "meaningful" ?
>Anyone who has listened to first year latin students trying to "decrypt"
>latin will realize that garbage in does not always imply the same garbage out.
With the kinds of ciphers Doug Gwyn was referring to, you can tell. There
can be ambiguity if the message is short enough, but the usual case is that
you figure out the system and the keyword, use it to break part of the
message, and the rest of the message falls out nicely. For example, we
could apply Mr. Ranum's argument to DES with a key search: we've tried
our 7th New Collegiate on a 64-bit block, and discover that the key
"methionine" (suitably folded) results in legal 7-bit ascii plaintext
looking like "he purpo" for that 64-bit block. Proceeding further, we
find that "rhinencephalon" (also folded) results in "73ux&aUc", which is
also legal ascii and could easily be part of a uuencoded piece of source
code. In order to see whether either of these is the correct keyword, we
try them on the next 64-bit block...
There can be ambiguity in decrypting some kinds of cipher, such as a "book
code". A lot of people have "decrypted" the Beale Cyphers without being
able to prove it -- and unless they come up with a key text that supports
their own view (or they dig up the treasure, of course) nobody else will
believe them.
For interesting reading on times when people have had to prove (i.e.
demonstrate to a court) that they got the right reading, try "The
Codebreakers" (David Kahn), "The Zimmermann Telegram" (Barbara Tuchman),
and "The Shakespearean Ciphers Examined" (William and Elizebeth Friedman).
[Elizebeth Friedman specialized in helping convict drug runners by
breaking their ciphers; she would testify in court about the validity of
her decryption.]
--
Jim Gillogly
{hplabs, ihnp4}!sdcrdcf!randvax!jim
jim@rand-unix.arpa [HASA: U (Spam) division]
Re: how do you tell encrytped data from random data?
Author: Phil R. Karn
Date: Mon, 11 Jan 1988 23:30
Date: Mon, 11 Jan 1988 23:30
25 lines
1895 bytes
1895 bytes
Re random and encrypted messages: There is a strong connection between randomness and cryptography. "Randomness" is in the eye of the beholder. All it means for you to say that something is "random" is that *you* can't predict what the next element in the sequence is, not that it doesn't contain some underlying redundancy that someone else with additional information (or computing power) might discover. It therefore follows that any strong encryption algorithm would necessarily produce what appeared to be totally random ciphertext. Feed a counter into DES and you have an excellent random number generator; unfortunately it is usually too slow to be practical. > On the other hand, there are people such as those who find Baconian > ciphers in Shakespeare's works, who put the message there themselves. > It's pretty easy to tell when this happens; either the decipherment > method is ambiguous, so the message could never have been > realistically expected to be read even by its intended recipient, > or so many "corrections" and otherwise unjustified special twists > on the decipherment scheme have been introduced that the > information content of the decipherment rivals that of the supposed > message being extracted. Quite true. I know I'm going to get flamed for this, but this is *precisely* what people do all the time with the Bible. Phil
Re: how do you tell encrytped data from
Author: truett
Date: Tue, 12 Jan 1988 10:45
Date: Tue, 12 Jan 1988 10:45
29 lines
2049 bytes
2049 bytes
Randomness or flatness of the probability distribution of the result is neither necessary nor sufficient for a good encryptor. First, it is not necessary. Consider the one time pad. In a binary stream, a one time pad will convert "random" streams to "nonrandom" ones just as often as it converts "nonrandom" (presumably plaintext) streams to apparently "random" ones. The only way to camouflage this effect is to increase the size of the apparent alphabet or state space of the message channel (somebody correct me if I'm wrong here.). It isn't sufficient either. Consider the linear congruential pseudorandum number generator: y = (ax + b) mod (n). For suitable a,b,n, you can use the successive characters of the message as values of x and the resulting stream of y values will be quite random. Another way is to use a data compression scheme like the ARC system used on many electronic bulletin board systems. The reduction of redundancy in a data stream makes it look more like noise than the original. Actually, good random number generators make quite good encryptors. So do ECC generators. Remember that Shannon demonstrated that the closer a noisy channel got to zero error, the more the data stream has to look like noise. Ergo: apply a really good ECC to the data stream (thus adding redundant data, see my comment above) and "BINGO" you have a good encryptor. Again, the answer is NO, you can't tell noise and encrypted data apart without a known encryption system. In the latter case, you tell by decrypting. Truett Lee Smith, Sunnyvale, CA UUCP: truett@cup.portal.com
Re: how do you tell encrytped data from
Author: Leonard Erickson
Date: Fri, 15 Jan 1988 05:07
Date: Fri, 15 Jan 1988 05:07
72 lines
4745 bytes
4745 bytes
In article <-63293657@sneaky> gordon@sneaky.UUCP writes: <> I know that transposition and *simple* substitution can be detected by <> letter frequency analysis. But is a "flat" distibution evidence of random <> data? < <No. Among other things, crypt(1) output gives a fairly flat distribution <in its 256-character output alphabet. (the version of crypt I am thinking of <is based on a WW-II rotor machine, and uses DES only to scramble the user's <typed key. (You don't need to crack the DES to crack the encryption). This <version was in System III, among others). In fact, it gives an absolutely <flat distribution for any message consisting of one character repeated a <multiple of 256 times and any key. (If not a multiple of 256, the <distribution is as flat as you can get with that message length). < <You also get a flat distribution, more or less, by encoding techniques <as "encode the i th character of the message by rotating it i positions in <the alphabet", assuming the message was reasonably long. This doesn't even <have an encryption key. A "real" encryption might use a keyword J characters <long, and encode the i th character of the message by rotating it by its <position in the message (i) plus an amount determined from the (i mod J)th <character of the key. For messages that are long relative to the key length <and the alphabet size, this will give a fairly flat distribution. You will <be able to detect distribution patterns by taking every (J * alphabet size)th <character, and looking at the frequency distribution for that. All the above examples, including the crypt that simulates a rotor machine are polyalphabetic substitutions. I have program that will do analysis of a text by calculating the variance for all the key lengths up to a user specified limit. It outputs the length of with the "best fit". On both random text and the suspect text, it always output the limit length as the best fit. And the fit was none too good. <I'm not quite sure how to interpret this. Would this mean that running the <message through uuencode(1) (which maps every 3 characters of 8 bits each <into 4 characters out of a 6-bit 64-character alphabet) is too complex? It <doesn't even have an encryption key! Running stuff through uuencode does tend <to flatten out the distribution somewhat, but a message consisting mostly of <the letter x will still have several high-frequency characters. < <To REALLY flatten out the distribution, there are several popular UNIX and <DOS utilities to do this, under the classification "data compression programs". <These include pack, compress, ARC, and compact. If you don't like the <use of all 8 bits in the output, run the result through a filter to transform <it to a smaller alphabet. These include things like uuencode/uudecode, <btoa/atob (distributed with compress 4.0), encode/decode (distributed with B <news 2.11), and, if you're really desparate, od, the octal dump program. <(A few years back, someone posted a program to take an octal dump and <re-create the file it was a dump of). Well, I've never even seen any _hints_ on how to attack a cipher where the plaintext characters can't be "localized". Since you can't do statistical analysis on the characters (at least not and get results that _mean_ anything!! :-) it would appear that you are reduced to trial and error. Uuencode and atob are at the extreme edge of the ability range expected of the probable originators of those old messages. Compress and ARC were out of the question! (we're talking c-64s and BASIC here!). A favorite that has a fair amount of popularity, is XORing the plaintext with a key on a byte-by-byte basis. It is simple, and you decode by running the ciphertext thru the encryptor again. Of course it is only as secure as key. You would also be surprised at how long you can stump even a skilled amateur with something silly like switching character sets. (eg using EBCDIC where ASCII is expected) -- Leonard Erickson ...!tektronix!reed!percival!bucket!leonard CIS: [70465,203] "I used to be a hacker. Now I'm a 'microcomputer specialist'. You know... I'd rather be a hacker."
Re: how do you tell encrytped data from
Author: Doug Gwyn
Date: Fri, 15 Jan 1988 15:10
Date: Fri, 15 Jan 1988 15:10
19 lines
1543 bytes
1543 bytes
In article <2415@cup.portal.com> truett@cup.portal.com writes: >In a binary stream, a one time pad will convert "random" streams to "nonrandom" >ones just as often as it converts "nonrandom" (presumably plaintext) streams to >apparently "random" ones. The only way to camouflage this effect ... This is quite misleading. There is no effect to "camouflage", since the so-called effect being described is the purely theoretical one where the entire space of possible messages is considered. Useful messages are a tiny fraction of this space. In practice, the cipher stream does not really exhibit "nonrandomness" (on the average) when the one-time pad is applied to random plaintext. >Actually, good random number generators make quite good encryptors. Actually, assuming you're still talking about such pseudo-random sequences as are produced by linear congruential feedback shift registers or other such simple schemes, they make TERRIBLE encryptors. They have far too much structure with far too small a key. Therefore they're quite easy to break, using the proper tools.
Re: how do you tell encrytped data from
Author: truett
Date: Mon, 18 Jan 1988 09:14
Date: Mon, 18 Jan 1988 09:14
17 lines
1385 bytes
1385 bytes
The idea that the set of possible plaintext messages is small relative to the set of possible messages is not always valid. How about the encryption of binary object codes or the vectorized files of the plans of a classified mechanical device? A good cryptographer must assume that any possible message has a chance of eventually appearing for encryption. So long as the encryption process is one-to-one between the spaces of possible plaintext messages and cyphertext messages, then "nonrandom" (whatever that means!) messages are encrypted into "rnadom" messages just as often as the "random" ones are encrypted into "nonrandom" ones. What *is* rare, so long as the nonrandom messages are a smaller space than the space of all messages, is the event in which a "nonrandom" message is encrypted into a "nonrandom" message. There are exceptions: remember what the Germans thought of Navajo? Truett Lee Smith, Sunnyvale, CA UUCP: truett@cup.portal.com
Re: how do you tell encrytped data from random data?
Author: Brian Chapman Mx321
Date: Mon, 18 Jan 1988 23:34
Date: Mon, 18 Jan 1988 23:34
38 lines
2091 bytes
2091 bytes
In article <660@bucket.UUCP> leonard@bucket.UUCP (Leonard Erickson) writes: < An interesting question has crossed my mind. If someone presents you with < an allegedly encrypted message, How can you tell if it really is encrypted < as opposed to being a bunch of random characters? It is the goal of any good encryption technique, in this class of techniques, to make the data look random. Any sort of pattern in the encrypted data is a crack you can drive a decryption wedge into. < I know that transposition and *simple* substitution can be detected by < letter frequency analysis. But is a "flat" distibution evidence of random < data? < For my purposes, both "one-time pad" ciphers and anything that operates on < units other than characters can be considered random! If it is that complex, < then I'm not likely to crack it! Yet UNIX crypt is a simple substitution code with the extra twists that the offset in the file is added to the character (mod 256) and you change substution tables every 256 characters. This tends to flatten out a naively taken distrubution. < (when an 8k msg uses the 26 letters so evenly that the spread < better most used and least used is 12, you get *real* suspicious :-) ^^^^^^ you mean "between"? You mean the character counts vary between 309 and 321. Sounds about right to me for random looking data. Artificialy flat distributions mean that the data is either a fraud or the encryption method pads with extra characters. -- uunet!-\ Brian Chapman microsof!-->sco!chapman ihnp4!-/ Pay no attention to the man behind the curtain! -- The Great & Powerfull Oz
Re: how do you tell encrytped data from random data?
Author: Henry Schaffer
Date: Wed, 20 Jan 1988 04:53
Date: Wed, 20 Jan 1988 04:53
26 lines
1680 bytes
1680 bytes
In article <275@sysco>, chapman@sco.COM (Brian Chapman Mx321) writes: > In article <660@bucket.UUCP> leonard@bucket.UUCP (Leonard Erickson) writes: > < An interesting question has crossed my mind. If someone presents you with > < an allegedly encrypted message, How can you tell if it really is encrypted > < as opposed to being a bunch of random characters? > ... > You mean the character counts vary between 309 and 321. > Sounds about right to me for random looking data. I'm taking "random" as meaning multinomial with equi-probability. In this case - looking at any one character should be found about 8k/26=307 times, with a standard deviation of about 17. That makes a spread of 12 between the most and least frequent of the 26 look quite small, indeed. > Artificialy flat distributions mean that the data is > either a fraud or the encryption method pads with extra > characters. Hmm, is this correct? I don't know how flat the output of something like Huffman encoding can be. I know it isn't encryption - but perhaps something like this in conjunction with encryption would give a flatter-than-random character distribution without character padding. > -- > Brian Chapman microsof!-->sco!chapman --henry schaffer n c state univ
Re: how do you tell encrytped data from random data?
Author: duncs home
Date: Wed, 20 Jan 1988 15:54
Date: Wed, 20 Jan 1988 15:54
18 lines
1206 bytes
1206 bytes
In article <660@bucket.UUCP> leonard@bucket.UUCP (Leonard Erickson) writes:
> ...
>I know that transposition and *simple* substitution can be detected by
>letter frequency analysis. But is a "flat" distibution evidence of random
>data?
No. Consider this simple substitution cipher:
'a' = 'abcdefghijklmnopqrstuvwxyz'
'b' = 'bcdefghijklmnopqrstuvwxyza'
'c' = 'cdefghijklmnopqrstuvwxyzab'
...
'z' = 'zabcdefghijklmnopqrstuvwxy'
The frequency distribution of the letters will be absolutely flat, yet it
does contain a message. This shows that a flat distribution does not mean
absence of a message (and also that flat distribution, by itself, does not
guarantee a good cipher!)
Re: how do you tell encrytped data from random data?
Author: Michael Mauldin
Date: Wed, 20 Jan 1988 22:52
Date: Wed, 20 Jan 1988 22:52
22 lines
1586 bytes
1586 bytes
In article <275@sysco>, chapman@sco.COM (Brian Chapman Mx321) writes: > Artificialy flat distributions mean that the data is > either a fraud or the encryption method pads with extra > characters. The encryption algorithm might very easily generate purposefully "flat" distributions by a variety of methods. You could switch tables after every block to shuffle "overused" bytes to less commonly used meanings. You can even guarantee perfectly flat counts by not altering the "meaning" of each byte. This turns out to require extra bytes, since once you make a decision that "I've used \035 enough, now", the remaining bytes encode fewer bits. Still, the result would be neither fraudulent nor padded with extra characters (the extra bytes would be "necessary"). If you got a really good encryption algorithm, this would be a pointless add-on feature, but that doesn't mean that seomone else isn't doing it. Michael L. Mauldin (Fuzzy) Department of Computer Science ARPA: Michael.Mauldin@NL.CS.CMU.EDU Carnegie-Mellon University Phone: (412) 268-3065 Pittsburgh, PA 15213-3890
Re: how do you tell encrytped data from random data?
Author: Jack Morrison
Date: Fri, 22 Jan 1988 16:16
Date: Fri, 22 Jan 1988 16:16
44 lines
1955 bytes
1955 bytes
In article <39394@sun.uucp> dunc@sun.UUCP (duncs home) writes: >In article <660@bucket.UUCP> leonard@bucket.UUCP (Leonard Erickson) writes: >> ... >>I know that transposition and *simple* substitution can be detected by >>letter frequency analysis. But is a "flat" distibution evidence of random >>data? > >No. Consider this simple substitution cipher: > >'a' = 'abcdefghijklmnopqrstuvwxyz' >'b' = 'bcdefghijklmnopqrstuvwxyza' >'c' = 'cdefghijklmnopqrstuvwxyzab' > ... >'z' = 'zabcdefghijklmnopqrstuvwxy' > >The frequency distribution of the letters will be absolutely flat, yet it >does contain a message. This shows that a flat distribution does not mean >absence of a message (and also that flat distribution, by itself, does not >guarantee a good cipher!) Cute, but keep in mind that a "distribution" can be more than just a histogram of letter frequencies; the example cipher would have a very lopsided distibution in other measurements, for example digram frequencies. (I realize the proposal was not meant as a good cipher, and I'm just adding to the discussion; this is NOT a flame, I just don't see where to stick the :-) !) [What? [The [followup [has [to [be [longer [than [the [original? [Than [eat [this!]-- Jack C. Morrison Jet Propulsion Laboratory (818)354-1463 jackm@jpl-devvax.jpl.nasa.gov "The paycheck is part government property, but the opinions are all mine."
Thread Navigation
This is a paginated view of messages in the thread with full content displayed inline.
Messages are displayed in chronological order, with the original post highlighted in green.
Use pagination controls to navigate through all messages in large threads.
Back to All Threads