CrunchBang - Code, Design & GNU/Linux

5 Or More Consecutive Consonants

"A consonant please Carol, and another, and another, and another, and another." — actually, this post is not about Carol Vorderman or Countdown, it is about some interesting[?] script output I came across when attempting to write a new spam filter. I will explain…

Just lately my website has been receiving some rather odd junk comments. The comments make no sense and they have quite obviously been sent by some automated junk flinging robot. The reason the comments make no sense is because they seem to be constructed from random characters. Apart from making no sense, these comments were also becoming a nuisance as they were easily slipping past my existing keyword filters.

So, the other night I decided to sit down and write a new filter to try and catch these random character junk comments. I started by analysing some previously submitted comments to try and find any common patterns. One such pattern I found was multiple strings containing 5 or more consecutive consonants. Thinking this to be unusual, I ran some tests against a flat file containing 21110 common English words. I thought the results were interesting. Here is what I found:

• 85 unique strings containing 5 or more consecutive consonants.
• 113 words containing 5 or more consecutive consonants.
• 9 words containing 5 or more consecutive consonants and no vowels: crypt, lymph, lynch, myrrh, nymph, pygmy, rhythm, sylph, tryst
• 10 words containing 6 or more consecutive consonants: latchstring, metempsychosis, polysyllabic, polysyllable, porphyry, rhythm, skyscraper, strychnine, synchronize, synchronous
• 1 word containing 6 consecutive consonants and no vowels: rhythm
• 1 word containing 7 consecutive consonants: strychnine

I should state that the above results are in no way definitive. I know this because I also ran the same test against another file containing 311141 words found in the Merriam-Webster dictionary. Still, by using the results of the initial test I was able to construct a list of safe words to use with my new spam filter.

Finally, yes, I did consider not writing this post; however, I am sure my publishing of these results will not change anything. Besides, Arthur, my 80 year old neighbour, is the biggest Countdown fan on the planet, he is also quite Internet savvy and definitely thinks Carol Vorderman is hot — he may find these results quite useful in increasing his daily Countdown score!