Information Theory. The prehistory of Unix part seven

Information theory

Are we the result of the decisions we make under our free will or is there a superior force that guides our steps? The bibliography on Claude Shannon consulted for this series of articles would seem to give arguments to the supporters of Destiny. Shannon was a person who was interested in many things; juggling, jazz, aviation, crossword puzzles or building game machines. Part of his professional life was dedicated to researching mathematics applied to genetic research or the arms industry. However, time and time again circumstances led him to build the basis of his top work: The Theory of Information

A little notice on the wall at the University of Minnesota led him to work with MIT's differential analyzer. There he related Boolean Algebra to the construction of circuits. While doing an internship at Bell Laboratories, he had access to an article that gave him the idea that a single theory could explain the transmission of information independent of the medium. Permanently incorporated into Bell, where he basically entered to avoid being called up, he became interested in cryptography and discovered the redundancy of language and the need not to have to transmit complete sentences for the message to make sense.

Information Theory

Shannon was part of a team working on a paradigm shift in telecommunications, the so-called PCM technology or Pulse Code Modulation. Instead of the transmission of the voice by means of electric waves, as had been done since Alexander Graham Bell invented the telephone, the aim was to transmit information about the behavior of electric waves so that the receiver is able to reconstruct them. Here we must take into account two important points.

  • The information about the behavior of the waves is not transmitted during the whole conversation, but a sample is made every certain amount of time, and the receiver fills in the spaces. Recall Shannon's remarks on redundancy and not having to send the full message.
  • Information about waves can be transmitted by encoding them with zeros and ones. Here they apply Shannon's ideas about the use of Boolean algebra for the transmission of information through a circuit.

However, this method does not apply only to the voice. The same technology can be applied to the remote transmission of any content that can be converted into zeros and ones; still and moving images, texts, graphics, music, etc.

Ensuring the fidelity of the message

Probably in every country there is a variant of the game that in my childhood we called "broken phone." One person whispers a message to the partner next door who in turn does the same to the next in line. Thus, until the last one you must repeat the message out loud. It hardly ever matches what the first one said.

The challenge for Bell Labs is to prevent this from happening in communications. And, this is where Information Theory comes in.

Shannon proposed a general communications model in which the sender emits through a transmitter a signal that, traveling through a channel, reaches the receiver. This is in charge of decoding the message previously encoded by the transmitter and delivering it to the recipient. In every channel there is what is called "noise" which are distortions that affect the reception of the message.

His proposal includes the following statements:

  • All communications, regardless of format, can be thought of in terms of information.
  • All information can be measured in bits. One bit (Short for binary digit) indicates the choice between two possible alternatives, the dot or dash of the telegraph, heads or tails when flipping a coin, or pulses on or off in PCM technology
  • The most complex information can be transmitted by means of a string of bits in a predefined format. For example, the base 2 representation of a number code assigned to a letter.

In her work on cryptography, Shannon had shown that message size could be reduced by eliminating redundancy. Here he proposed to go the opposite way; combat noise by adding extra bits so that the receiver is able to correct errors that occurred during transmission.

Although in many cases the theoretical formulation would take decades to become practical applications, engineers already had a way of measuring the ability of different channels to transmit information. Everything was ready for new materials to replace the traditional copper cable, exponentially increasing the number of circulating messages. And of course you would need a way to handle all that amount of information. We will see that in the next articles.

The prehistory of Unix
Related article:
The prehistory of Unix and the role of Bell Labs
Bringing scientists and engineers together
Related article:
Bringing scientists and engineers together. The prehistory of Unix. Part 2
Vacuum tubes
Related article:
Vacuum tubes. The prehistory of Unix part 3
The arrival of the transistor
Related article:
The arrival of the transistor. The Prehistory of Unix Part Four
Related article:
Mr. Claude Shannon. The Prehistory of Unix Part Five
Shannon's work
Related article:
Shannon's work. The prehistory of Unix part 6

2 comments, leave yours

Leave a Comment

Your email address will not be published. Required fields are marked with *

*

*

  1. Responsible for the data: AB Internet Networks 2008 SL
  2. Purpose of the data: Control SPAM, comment management.
  3. Legitimation: Your consent
  4. Communication of the data: The data will not be communicated to third parties except by legal obligation.
  5. Data storage: Database hosted by Occentus Networks (EU)
  6. Rights: At any time you can limit, recover and delete your information.

  1.   symbi said

    Please sources! the sixth part had a chunk that made me think it was a translation, although I'm not sure. Where does all this come from?

    1.    Diego German Gonzalez said

      I promise by Friday that a detailed list of the sources and what I got from each one comes. What probably sounds like plagiarism to you is the first sentence. I stole it from Isaac Asimov from a book that compiles his answers on scientific questions.