Tuesday, August 03, 2004

Why do some emails reach slower than others

Why is it that I can send a message to a group and have it appear literally within seconds, and then send another, which will take more than 24 hours to appear?
-Puzzled friend from Belgaum

Ah. I'm happy to report that you have come to the right place for the answer to this deep question, for my vast (nay, encyclopedic) knowledge commands such respect.
Before I reveal the cause of the common phenomenon you're wondering about, though, I'd like to point out some other quirky behaviors that you may have noticed.
Some days your bike starts on the first kick. Some days it doesn't start at all.
Some recent evenings have been brilliantly lighted by the full moon. Yesterday I got drenched before I got out of my office (the office faucet is broken, but still).
These have nothing to do with why one message is transmitted immediately while another takes 24 hours.
The reason is complex, and technologists don't often expect even to hear such sophisticated questions from those outside the inner circle (HAHAHAHAHAH THEY THINK THAT THEY KNOW EVERYTHING), and many of them are loathe revealing the hidden cause.
But you seem trustworthy, so...
Look at your keyboard. Notice how the keys are all out of order? You'd think they'd be in alphabetical order, wouldn't you? But no, they are arranged in an odd pattern called QWERTY, originally devised by a typewriter manufacturer to slow typists down to the point where his machines wouldn't jam. Imagine, now, when you send a message down the wires, how differently the many routers and interfaces that the message goes through are affected by different juxtapositions of letters in your message. Just as a modest change in the original position of a chess pawn has a dramatic effect on the time required to solve it, the tiniest change in the arrangement of letters in your message - often not even noticeable to any but the expert eye, and even then only with complex measurement equipment - can wreak havoc with every interface the message must pass through. Imagine you had a vehicle wider than the normal highway. Going through interchanges would be a particular trial; how quickly you could pass through would depend on the amount of other traffic, the number of odd-shaped oncoming vehicles, and many other factors -- much too complex to summarize quickly. But I'm sure you get the idea. And just as if you drove through many interchanges in your odd-shaped vehicle you could be delayed dramatically, changes in the letter composition of your message slow it down every time it goes through a router, the internet's interchange.
The letters W and M are particularly noxious in this way. If they happen to fall within the same word, as in women, or if multiples of them fall within a word, as in mammal, or wow, their retardant effect is in fact squared; Puttanna Kanagal proved this in a film in 1944, although Hema Malini in ‘Sholay’ during the 70’s indicated that she, too, had the basic idea by competing with Dharmendra’s bike on a tonga.
The vowels, on the other hand, particularly O, and I are quite slippery and can speed up the trip of your message through a router; in fact, an I almost cancels an M, and words with many Is and Os, such as oil, lion, noise, and onion, can have a remarkable accelerating effect.
These are only the extreme cases. Each letter, and in fact each key, has its own lexical friction coefficient (LFC), which often depends on the relationship of the letter to other letters in the word and to other words in the message. LFC tables were originally compiled by Amithabh Bachan in 1981, for use in his famous Amar akbar anthony song, but were not made available to the general public until Microsoft brought out the Microsoft word processor, and published a full set of lexical friction data in the documentation that was issued with the Software. Much before, in 1962, when IBM first produced the electric typewriter, new LFC tables had to be constructed; these were made available only in technical libraries.
Thus high-LFC messages not only move more slowly through the Internet, but actually raise the Internet's traffic load while they do so.
This issue has been studied in great detail by my erstwhile colleagues, whose interest in fiber plaque, LFC aggregation, and the resulting internet congestion is so high that it has formed a special task force to study the matter and recommend solutions within a year. I fully expect, however, that since the matter is dependent on keyboard design originally, these studies will probably result in little improvement, and once again we will be left anxiously awaiting the next-faster generation of optics, routers, and computers, meanwhile helplessly floundering in a stew of such technical complexity that only the few can comprehend it and Amitabh bachan can get back to films than lexicology.
I would suggest that to improve your transmission times you should begin by tabulating the letter counts in your messages, and correlate them with message delivery delays. This technique is crude, but should give you a rough idea of what to expect but alas all the counting can actually slow the email further as it takes a lot of time, but if you want to get so technical.
I'm glad to have been of service in this matter, and will make myself available for further questions as they occur to you.

Rishi