- Email: [email protected]
The entropic and symbolic components of information
BioSystems 182 (2019) 17–20
Contents lists available at ScienceDirect
BioSystems journal homepage: www.elsevier.com/locate/biosystems
The entropic and symbolic components of information
T
Wanderley Dantas dos Santos State University of Maringá, Av. Colombo 5790, Maringá, PR, 87020-900, Brazil
A R T I C LE I N FO
A B S T R A C T
Keywords: Code Sign Signal Meaning Cognition Semiotics
At the turn of the 19th and 20th centuries, Boltzmann and Plank described entropy S as a logarithm function of the probability distribution of microstates w of a system (S = k ln w), where k is the Boltzmann constant equalling the gas constant per Avogadro’s number (R NA−1). A few decades later, Shannon established that information, I, could be measured as the log of the number of stable microstates n of a system. Considering a system formed by binary information units, bit, I = log2 bit From this, Brillouin deduced that information is inversely proportional to the number of microstates of a system, and equivalent to entropy taken with a negative signal −S or ‘negentropy’ (I = k ln (1/w) = −S). In contrast with these quantitative treatments, more recently, Barbieri approached the ‘nominal’ feature of information. In computing, semantics or molecular biology, information is transported in specific sequences (of bits, letters or monomers). As these sequences are not determined by the intrinsic properties of the components, they cannot be described by a physical law: information derives necessarily from a copying/coding process. Therefore, a piece of information, although an objective physical entity, is irreducible and immeasurable: it can only be described by naming their components in the exact order. Here, I review the mathematical rationale of Brillouin’s identitification between information and negentropy to demonstrate that although a gain in information implies a necessary gain in negentropy, a gain in negentropy does not necessarily imply a gain in information.
1. The entropic component of information Claude Shannon (1949) proposed that information could be understood as a decision between alternatives. Thus, the smallest piece of information is a state defined between two possible states and the amount of information could be measured in binary unities of information (bit). Each binary unity can be represented by a variable that can assume at least two values (such as 0 or 1) resulting in a code. To express larger amounts of information in binary units, one can align a series of bits, for example, 0110. A group of bits (b) can assume 2b states. Words of 8 bits (byte) can take 28 = 256 distinct values, and so on. In computer science, the bits are stored and processed in electronic keys (flip-flops) that oscillate between two states. As each bit can assume two states, the number of potential states grows exponentially with the number of bits. Thus, a byte can transport 28 = 256 different messages, but any one needs 8 bits to be transmitted. The amount of information I in bits can be measured as the logarithm of the number of unities: I = log2 b
(1)
At first, Shannon was reluctant to call a sequence of bits of ‘information’. On the contrary, he considered using the term 'uncertainty',
because a byte is the amount of uncertainty you need to transmit an 8bit message. For this reason, von Neumann, suggested he use the term 'entropy', since entropy denotes the degrees of freedom of the components in a physical system. The degree of freedom of each bit is two (1 or 2) and of a byte is (256), while the degree of freedom (entropy) of a specific 8-bits message (containing information, like 11001001) is indeed zero. In thermodynamics, entropy S reflects the amount of heat Q a system can absorb without changing its temperature (or still the amount of heat necessary to change the temperature T of a system): δS = δQ/T
(2)
The amount of heat per degree varies non-linearly with temperature because the number and kinds of chemical bonds among the components varies at different temperatures. Each kind of chemical bond (van der Waals, H bonds, ionic etc.) presents a different binding energy. When a system is heated, its internal energy increases and, eventually, the kinetic energy of its components surpasses the binding energy of some bond, increasing the entropy (degree of freedom). Below 0 °C, the kinetic energy of water molecules is not enough to overcome the binding energy of the hydrogen bonds, which restricts their degree of freedom (entropy). Above 0 °C, the water molecules’ kinetic energy
E-mail address: [email protected]. https://doi.org/10.1016/j.biosystems.2019.05.003 Received 16 March 2019; Received in revised form 3 May 2019; Accepted 6 May 2019 Available online 07 May 2019 0303-2647/ © 2019 Elsevier B.V. All rights reserved.
BioSystems 182 (2019) 17–20
W.D. dos Santos
The precise definition of information as a decision among a set of possibilities became useful for molecular biologists. In living beings, a specific sequence of DNA (a gene) can be transcribed into a specific RNA molecule, which in turn can be translated into a peptide with a specific sequence of amino acids. The ability to conserve the monomer sequences during DNA replication and to determine a precise sequence of RNA nucleotides during transcription and of amino acid sequences during protein translation is essential for the existence of the living beings. For convenience, most authors started to refer to these sequences of monomers in biopolymers as information. However, by lack of a precise definition, biologists use the concept rather as a useful metaphor than as a real property of biopolymers, until Marcello Barbieri (2016) finally decided to grasp the nettle. The monomeric sequence of a biological polymer is ordered by the monomeric sequence of another polymer: a template sequence. For instance, DNA and RNA strands are synthesised by specific base pairing with a DNA template strand. In turn, a sequence of amino acid residues in a polypeptide is determined from a template of mRNA. Therefore, Barbieri proposes that information is a sequence based on another sequence. Although to define information from a biological model in order to extend the concept to biology may seem to be tautological, the fact is that this definitory property of information seems to apply to all informational systems. In computing, telecommunications or cryptography information is never random. Only when a sequence of signs is generated from a conversion process (e.g. an analogic/digital converser), does it gain the status of information. Barbieri splits information out of the determinism that comprises the inorganic world. Monomeric sequences from biological copolymers are not determined by physical laws. Although the chemical linkages that produce a protein necessarily obey the laws of thermodynamics and chemistry, its specific monomeric sequence is not determined by these laws. Therefore, such a sequence cannot be described by, or reduced to a mathematical formula, neither deterministic, nor statistical. The monomeric sequence of a biopolymer is determined by a process governed externally (by a template), not by the properties of the monomers themselves. Hence, information cannot be described by a natural law. To describe a piece of information is necessary to perform the sequential denomination of its components. In other words, a sequence is irreducible. In opposition, the synthesis of inorganic compounds is deterministic. Their chemical formula is the necessary result of a dynamic determined by the intrinsic properties of their components. In the same way, the synthesis of a racemic mixture of an asymmetric organic compound is also deterministic. Therefore, Barbieri defines information as a sequence set from outside by a template sequence. Only this type of sequence must be considered as information. Sequences not produced from a guideline are random and have no meaning. Thus, Barbieri concludes that the status of information applies only to sequences generated by a codemaker. For Barbieri a code is a set of rules that establishes a correspondence between objects in two different words. In this context, the meaning of an object in a set A is its correspondent object in the set B. For instance, rouge in French, means red in English (Brassier, 2016). If the byte 01101011 in binary language corresponds to letter k in the alphabet, then 01101011 means k. Likewise, if the AAA codon in an mRNA leads to the insertion of a lysine into a peptide, AAA in the genetic code means lysine in protein code. In this way, Barbieri suggests that codes and meanings are inseparable: to give meaning to something is to establish a code. In the above discussion, information was defined as a sequence generated from another sequence. However, considering that the smallest piece of information can be defined as one state defined between two possible states, we should not consider that information is restricted to sequences, but instead, should be a property of the smallest component of a given sequence in a code a: bit, codon, monomer and so on. Electrical engineers refer to wanted information as the ‘signal’, bequeathing to the unwanted information the term ‘noise’. Signal is
surpasses the H-bound binding energy and they become able to assume a higher number of states, i.e. higher entropy. Therefore, more heat per degree is necessary to increase the temperature. Boltzmann was the first to define entropy in terms of the number of possible microstates the components of a system can assume. Like information (Eq. (1)) the entropy S increases with the log of the number of microstates w. S = kB ln w
(3)
where, kB is the Boltzmann constant expressing the amount of kinetic energy of each particle in a given temperature. The similarity between Shannon’s information and Boltzmann’s entropy was explored in a series of articles published by Léon Brillouin. Considering a system with P possible structures, Brillouin (1953) demonstrated that for an a priori equal distribution of probabilities, the amount of information in a physical system can be measured as the logarithm of the ratio between the initial number of possible structures P0, and the number of possible structures P1 after a gain in information (I1 > 0). I = K ln P0/P1
(3b)
where K is a proportionality constant that can be adjusted according to the desired unities: for K = log2e, then I= log2 P0/P1 gives the information in bits. On the other hand, for K = kB then, I = kb ln P0/P1
(3c) −1
gives I in Joules K , i.e. the unities of entropy. Considering the initial entropy S0 = k ln P0 and the final entropy (after a gain in information) S1 = k ln P1, it is possible to deduce that S1 = S0 − I1
(4)
In other words, the analysis proves that information = negative entropy (negentropy). The idea behind the correlation between negentropy and information is that a chemical bound is a decision (definite state) among other possibilities (other microstates), the very definition of information provided by Shannon. The amount of bonds is inversely proportional to the entropy (1/w); therefore, information (order) can be expressed in reciprocal terms to those of Boltzmann entropy: −S = k ln (1/w)
(5)
While entropy measures the degree of freedom, negentropy is a measure of the ‘degree of order’ in a system. Negentropy is related to high-grade energy, since it implies an asymmetry in the system, which, once released, can produce work. Therefore, Brillouin (1962) concludes that negentropy can be converted into information and vice-versa. 2. The symbolic component of information A message is a set of specific states (e.g. 01100011, exactly), while Shannon information expresses only that which is necessary for 8 bits to express that message – or any other 255 alternative messages. Shannon information is therefore a physical quantity and it must be clearly denoted when the term is used in sensu stricto to avoid misunderstandings. For example, information theory affirms that ‘information is not a property of a given message, but of all sets of possible messages’. Let’s understand this claim. Consider a 4-letter message in binary code. The amount of information is 24 = 16 possible messages. On the other hand, in a 4-letter message using an alphabetic code, the amount of information is 264 = 456,976 different messages. Thus, the statement information is not a property of the message, but of the whole set of messages. Shannon did not define what information is, just how it can be measured. Besides reinforcing the concept of information as a magnitude, the digression above highlights another fundamental feature of information: it is a property of codes. 18
BioSystems 182 (2019) 17–20
W.D. dos Santos
‘what you want to convey’, while noise can both be a random interference in the signal or a signal out of context (such as a crossed line in a call). In this sense, we could dub signal the smallest piece of information in a given code and define information as a signal or sequence of signals produced in correspondence with another signal or sequence. The meaning of a signal in one set (a) is the correspondent signal in set (b). As this correspondence is determined by the code rules, the trinity ‘code’, ‘information’ and ‘meaning’ emerge altogether and are inextricable from each other. However, in linguistics and semiotics, the term used to designate an object is ‘sign’. Objects, qualities and happenings in the world can be represented and communicated through sounds, gestures, drawings or even by other objects (mock-ups, trophies, statues and so on). Charles Pierce distinguished three kinds of signs: indexes, icons and symbols, which differ by the kind of relationship with the object they represent. 1) Indexes are effects that indicate something else through a causal relationship (e.g. smoke indicating fire, apple scent indicating apples or footprints indicating the steps of an animal). 2) Icons are signs resembling the object they refer to, which include sacred images, pictograms (used in graphical interfaces, traffic signs and bathroom doors), hieroglyphs or onomatopoeic sounds. Finally, 3) symbols are completely arbitrary signs, related with an object by mere convention. It is the case of a Christian or family name given to a person, a written word using alphabetic letters or the Latin alphabet, itself. The meaning of sign here bears some resemblance with the sense in which the term signal is used by engineers, in the sense that both signal and sign are ‘distinguishable marks’ representing something else. As information requires a decision between at least two alternatives, a code must present at least two signs. The binary code has two signs, 0 and 1, a decimal code demands 10 signs and an alphabetic code presents 26 signs. Physical informational systems also require at least two different signs: a compact disc uses pits and lands and CPUs use flip-flop circuits. In turn, genetic information is stored in nucleic acids that contain 4 signs (A, C, G, T), and so on. The components of a thermodynamic system are frequently identical to each other (e.g. a homogeneous phase). In this case, the many possible discrete negentropic states are also indistinguishable from each other (Fig. 1).
negentropy, resulting from the reduction in the degrees of freedom of water molecules (Eq. (5)). For Brillouin, this gain in negentropy corresponds to a gain in information (Eq. (4)). However, one water molecule cannot be distinguished from the others and their bounds also cannot exist in distinct states. As we saw, entropy can be calculated in the function of the logarithm of the number of microstates the system’s components can assume. Each microstate is the combinatory distribution of the kinetic energy of each component, what Boltzmann called a complexion. The number of complexions is proportional to the probability of finding the components in such a state (Eq. (3)). Conversely, the negentropy can be calculated in the function of the inverse of such a probability distribution (Eq. (5)). If w is the number of accessible microscopic states, or components’ degrees of freedom, 1/w is the degree of order or bonds that restrain the freedom of components. Shannon’s equation measures the amount of information in a system as the number of unities that can assume one of two states (Eq. (1)). In contrast, Brillouin’s equation (Eq. (5)) measures negentropy as proportional to the inverse of the number of accessible microscopic states, which is proportional to the number of bonds. The more important difference is that Shannon notoriously approaches the amount of information in terms of the number of elements of a ‘minimum code’ with two signs. In turn, Brillouin refers to information as the number of bounds in a system. However, the bounds and components might be indistinguishable from one another. If each bound or bounded component could assume one of two distinct states, negentropy would be a measure of the system’s capacity to store information. Using Eq. (3b) it would be possible to obtain the information in bits. But, in general, this is not the case, since each bound or component exists in one sole distinguishible state. To call this ordered state of information is like creating a new ‘code’ with only one sign. Such a ‘code’ is more precisely a non-code since the amount of information (in bits) it can represent is 1n = log2 20 = 0 bits, no matter how big the number of elements of the non-code (n). For instance, while a DNA strand 20 nucleotides length can carry log2 420 = 40 bits of information, a DNA homopolymer strand (e.g. a poly A tail) with same length, can store only log2 120 = 0 bit, although both oligomers present very similar negentropies.
3. Negentropy is not information
4. Conclusion
When cooled, water molecules pass, roughly speaking, from an undefined state (liquid) to a defined state (ice) with a gain of
Although entropy can reversibly change in many systems, only coding systems can produce codes. Coding systems contain components Fig. 1. Sketch of a nanodevice entrapping six molecules of butene in six different niches capable of reading/altering the configurations their special configurations. There are 2 positional isoforms of butene: 1-butene, that has no stereoisomers, and 2-butene, presenting cis and trans stereoisomers. Let’s make a device trapping six molecules of 1-butene (Fig. 1A) and another one, trapping six 2-butene molecules (Fig. 1B). While device A presents only one state, device B can exist in 26 different states. Each state can be used to code a different message, making device B useful for storing and communicating information, which it is not possible with device B. At any given state, B presents about the same entropy, since the number of microstates is roughly the same despite the number and position of each cis/ trans isomer. By the same reason, any state of B presents an entropy similar to A. Such a property of isomeric monomers has been investigated to produce solid-state photoswitches for ultra-high capacity data storage media (e.g. Steiner et al., 1978; Hatcher and Raithby, 2013). 19
BioSystems 182 (2019) 17–20
W.D. dos Santos
that can be arranged in two (binary) or more (alphabetic) stable discrete states or signs. Information is the arbitrary pattern in which these signs are arranged, when, and only when this arrange is determined from outside by a template, or guideline i.e. a codemaker. Thus, a gain in information requires at least an equivalent gain in negentropy, but a gain in negentropy does not necessarily imply a gain in information.
0: Castalia, The Game of Ends and Means. http://www.glass-bead.org/article/ transcendental-logic-and-true-representings/?lang=enview. Brillouin, L., 1962. Science and Information Theory. Academic Press Inc, New York 347 pp. Hatcher, L.E., Raithby, P.R., 2013. Solid-state photochemistry of molecular photoswitchable species: the role of photocrystallographic techniques. Acta Cryst. C. 69 (12), 1448–1456. Shannon, C., 1949. A mathematical theory of communication. Bell Sys. Tech. J. 27 (379–423), 623–656. Steiner, U., Abdel-Kader, M.H., Fischer, P., Kramer, H.E.A., 1978. Photochemical cis/ trans isomerization of a stilbazolium betaine. A protolytic/photochemical reaction cycle. J. Am. Chem. Soc. 100 (10), 3190–3197.
References Barbieri, M., 2016. What is information? Philos. Trans. A R. Soc. 374, 20150060. Brassier, R., 2016. Transcendental logic and true representings. Glass Bead Journal, Site
20
Contents lists available at ScienceDirect
BioSystems journal homepage: www.elsevier.com/locate/biosystems
The entropic and symbolic components of information
T
Wanderley Dantas dos Santos State University of Maringá, Av. Colombo 5790, Maringá, PR, 87020-900, Brazil
A R T I C LE I N FO
A B S T R A C T
Keywords: Code Sign Signal Meaning Cognition Semiotics
At the turn of the 19th and 20th centuries, Boltzmann and Plank described entropy S as a logarithm function of the probability distribution of microstates w of a system (S = k ln w), where k is the Boltzmann constant equalling the gas constant per Avogadro’s number (R NA−1). A few decades later, Shannon established that information, I, could be measured as the log of the number of stable microstates n of a system. Considering a system formed by binary information units, bit, I = log2 bit From this, Brillouin deduced that information is inversely proportional to the number of microstates of a system, and equivalent to entropy taken with a negative signal −S or ‘negentropy’ (I = k ln (1/w) = −S). In contrast with these quantitative treatments, more recently, Barbieri approached the ‘nominal’ feature of information. In computing, semantics or molecular biology, information is transported in specific sequences (of bits, letters or monomers). As these sequences are not determined by the intrinsic properties of the components, they cannot be described by a physical law: information derives necessarily from a copying/coding process. Therefore, a piece of information, although an objective physical entity, is irreducible and immeasurable: it can only be described by naming their components in the exact order. Here, I review the mathematical rationale of Brillouin’s identitification between information and negentropy to demonstrate that although a gain in information implies a necessary gain in negentropy, a gain in negentropy does not necessarily imply a gain in information.
1. The entropic component of information Claude Shannon (1949) proposed that information could be understood as a decision between alternatives. Thus, the smallest piece of information is a state defined between two possible states and the amount of information could be measured in binary unities of information (bit). Each binary unity can be represented by a variable that can assume at least two values (such as 0 or 1) resulting in a code. To express larger amounts of information in binary units, one can align a series of bits, for example, 0110. A group of bits (b) can assume 2b states. Words of 8 bits (byte) can take 28 = 256 distinct values, and so on. In computer science, the bits are stored and processed in electronic keys (flip-flops) that oscillate between two states. As each bit can assume two states, the number of potential states grows exponentially with the number of bits. Thus, a byte can transport 28 = 256 different messages, but any one needs 8 bits to be transmitted. The amount of information I in bits can be measured as the logarithm of the number of unities: I = log2 b
(1)
At first, Shannon was reluctant to call a sequence of bits of ‘information’. On the contrary, he considered using the term 'uncertainty',
because a byte is the amount of uncertainty you need to transmit an 8bit message. For this reason, von Neumann, suggested he use the term 'entropy', since entropy denotes the degrees of freedom of the components in a physical system. The degree of freedom of each bit is two (1 or 2) and of a byte is (256), while the degree of freedom (entropy) of a specific 8-bits message (containing information, like 11001001) is indeed zero. In thermodynamics, entropy S reflects the amount of heat Q a system can absorb without changing its temperature (or still the amount of heat necessary to change the temperature T of a system): δS = δQ/T
(2)
The amount of heat per degree varies non-linearly with temperature because the number and kinds of chemical bonds among the components varies at different temperatures. Each kind of chemical bond (van der Waals, H bonds, ionic etc.) presents a different binding energy. When a system is heated, its internal energy increases and, eventually, the kinetic energy of its components surpasses the binding energy of some bond, increasing the entropy (degree of freedom). Below 0 °C, the kinetic energy of water molecules is not enough to overcome the binding energy of the hydrogen bonds, which restricts their degree of freedom (entropy). Above 0 °C, the water molecules’ kinetic energy
E-mail address: [email protected]. https://doi.org/10.1016/j.biosystems.2019.05.003 Received 16 March 2019; Received in revised form 3 May 2019; Accepted 6 May 2019 Available online 07 May 2019 0303-2647/ © 2019 Elsevier B.V. All rights reserved.
BioSystems 182 (2019) 17–20
W.D. dos Santos
The precise definition of information as a decision among a set of possibilities became useful for molecular biologists. In living beings, a specific sequence of DNA (a gene) can be transcribed into a specific RNA molecule, which in turn can be translated into a peptide with a specific sequence of amino acids. The ability to conserve the monomer sequences during DNA replication and to determine a precise sequence of RNA nucleotides during transcription and of amino acid sequences during protein translation is essential for the existence of the living beings. For convenience, most authors started to refer to these sequences of monomers in biopolymers as information. However, by lack of a precise definition, biologists use the concept rather as a useful metaphor than as a real property of biopolymers, until Marcello Barbieri (2016) finally decided to grasp the nettle. The monomeric sequence of a biological polymer is ordered by the monomeric sequence of another polymer: a template sequence. For instance, DNA and RNA strands are synthesised by specific base pairing with a DNA template strand. In turn, a sequence of amino acid residues in a polypeptide is determined from a template of mRNA. Therefore, Barbieri proposes that information is a sequence based on another sequence. Although to define information from a biological model in order to extend the concept to biology may seem to be tautological, the fact is that this definitory property of information seems to apply to all informational systems. In computing, telecommunications or cryptography information is never random. Only when a sequence of signs is generated from a conversion process (e.g. an analogic/digital converser), does it gain the status of information. Barbieri splits information out of the determinism that comprises the inorganic world. Monomeric sequences from biological copolymers are not determined by physical laws. Although the chemical linkages that produce a protein necessarily obey the laws of thermodynamics and chemistry, its specific monomeric sequence is not determined by these laws. Therefore, such a sequence cannot be described by, or reduced to a mathematical formula, neither deterministic, nor statistical. The monomeric sequence of a biopolymer is determined by a process governed externally (by a template), not by the properties of the monomers themselves. Hence, information cannot be described by a natural law. To describe a piece of information is necessary to perform the sequential denomination of its components. In other words, a sequence is irreducible. In opposition, the synthesis of inorganic compounds is deterministic. Their chemical formula is the necessary result of a dynamic determined by the intrinsic properties of their components. In the same way, the synthesis of a racemic mixture of an asymmetric organic compound is also deterministic. Therefore, Barbieri defines information as a sequence set from outside by a template sequence. Only this type of sequence must be considered as information. Sequences not produced from a guideline are random and have no meaning. Thus, Barbieri concludes that the status of information applies only to sequences generated by a codemaker. For Barbieri a code is a set of rules that establishes a correspondence between objects in two different words. In this context, the meaning of an object in a set A is its correspondent object in the set B. For instance, rouge in French, means red in English (Brassier, 2016). If the byte 01101011 in binary language corresponds to letter k in the alphabet, then 01101011 means k. Likewise, if the AAA codon in an mRNA leads to the insertion of a lysine into a peptide, AAA in the genetic code means lysine in protein code. In this way, Barbieri suggests that codes and meanings are inseparable: to give meaning to something is to establish a code. In the above discussion, information was defined as a sequence generated from another sequence. However, considering that the smallest piece of information can be defined as one state defined between two possible states, we should not consider that information is restricted to sequences, but instead, should be a property of the smallest component of a given sequence in a code a: bit, codon, monomer and so on. Electrical engineers refer to wanted information as the ‘signal’, bequeathing to the unwanted information the term ‘noise’. Signal is
surpasses the H-bound binding energy and they become able to assume a higher number of states, i.e. higher entropy. Therefore, more heat per degree is necessary to increase the temperature. Boltzmann was the first to define entropy in terms of the number of possible microstates the components of a system can assume. Like information (Eq. (1)) the entropy S increases with the log of the number of microstates w. S = kB ln w
(3)
where, kB is the Boltzmann constant expressing the amount of kinetic energy of each particle in a given temperature. The similarity between Shannon’s information and Boltzmann’s entropy was explored in a series of articles published by Léon Brillouin. Considering a system with P possible structures, Brillouin (1953) demonstrated that for an a priori equal distribution of probabilities, the amount of information in a physical system can be measured as the logarithm of the ratio between the initial number of possible structures P0, and the number of possible structures P1 after a gain in information (I1 > 0). I = K ln P0/P1
(3b)
where K is a proportionality constant that can be adjusted according to the desired unities: for K = log2e, then I= log2 P0/P1 gives the information in bits. On the other hand, for K = kB then, I = kb ln P0/P1
(3c) −1
gives I in Joules K , i.e. the unities of entropy. Considering the initial entropy S0 = k ln P0 and the final entropy (after a gain in information) S1 = k ln P1, it is possible to deduce that S1 = S0 − I1
(4)
In other words, the analysis proves that information = negative entropy (negentropy). The idea behind the correlation between negentropy and information is that a chemical bound is a decision (definite state) among other possibilities (other microstates), the very definition of information provided by Shannon. The amount of bonds is inversely proportional to the entropy (1/w); therefore, information (order) can be expressed in reciprocal terms to those of Boltzmann entropy: −S = k ln (1/w)
(5)
While entropy measures the degree of freedom, negentropy is a measure of the ‘degree of order’ in a system. Negentropy is related to high-grade energy, since it implies an asymmetry in the system, which, once released, can produce work. Therefore, Brillouin (1962) concludes that negentropy can be converted into information and vice-versa. 2. The symbolic component of information A message is a set of specific states (e.g. 01100011, exactly), while Shannon information expresses only that which is necessary for 8 bits to express that message – or any other 255 alternative messages. Shannon information is therefore a physical quantity and it must be clearly denoted when the term is used in sensu stricto to avoid misunderstandings. For example, information theory affirms that ‘information is not a property of a given message, but of all sets of possible messages’. Let’s understand this claim. Consider a 4-letter message in binary code. The amount of information is 24 = 16 possible messages. On the other hand, in a 4-letter message using an alphabetic code, the amount of information is 264 = 456,976 different messages. Thus, the statement information is not a property of the message, but of the whole set of messages. Shannon did not define what information is, just how it can be measured. Besides reinforcing the concept of information as a magnitude, the digression above highlights another fundamental feature of information: it is a property of codes. 18
BioSystems 182 (2019) 17–20
W.D. dos Santos
‘what you want to convey’, while noise can both be a random interference in the signal or a signal out of context (such as a crossed line in a call). In this sense, we could dub signal the smallest piece of information in a given code and define information as a signal or sequence of signals produced in correspondence with another signal or sequence. The meaning of a signal in one set (a) is the correspondent signal in set (b). As this correspondence is determined by the code rules, the trinity ‘code’, ‘information’ and ‘meaning’ emerge altogether and are inextricable from each other. However, in linguistics and semiotics, the term used to designate an object is ‘sign’. Objects, qualities and happenings in the world can be represented and communicated through sounds, gestures, drawings or even by other objects (mock-ups, trophies, statues and so on). Charles Pierce distinguished three kinds of signs: indexes, icons and symbols, which differ by the kind of relationship with the object they represent. 1) Indexes are effects that indicate something else through a causal relationship (e.g. smoke indicating fire, apple scent indicating apples or footprints indicating the steps of an animal). 2) Icons are signs resembling the object they refer to, which include sacred images, pictograms (used in graphical interfaces, traffic signs and bathroom doors), hieroglyphs or onomatopoeic sounds. Finally, 3) symbols are completely arbitrary signs, related with an object by mere convention. It is the case of a Christian or family name given to a person, a written word using alphabetic letters or the Latin alphabet, itself. The meaning of sign here bears some resemblance with the sense in which the term signal is used by engineers, in the sense that both signal and sign are ‘distinguishable marks’ representing something else. As information requires a decision between at least two alternatives, a code must present at least two signs. The binary code has two signs, 0 and 1, a decimal code demands 10 signs and an alphabetic code presents 26 signs. Physical informational systems also require at least two different signs: a compact disc uses pits and lands and CPUs use flip-flop circuits. In turn, genetic information is stored in nucleic acids that contain 4 signs (A, C, G, T), and so on. The components of a thermodynamic system are frequently identical to each other (e.g. a homogeneous phase). In this case, the many possible discrete negentropic states are also indistinguishable from each other (Fig. 1).
negentropy, resulting from the reduction in the degrees of freedom of water molecules (Eq. (5)). For Brillouin, this gain in negentropy corresponds to a gain in information (Eq. (4)). However, one water molecule cannot be distinguished from the others and their bounds also cannot exist in distinct states. As we saw, entropy can be calculated in the function of the logarithm of the number of microstates the system’s components can assume. Each microstate is the combinatory distribution of the kinetic energy of each component, what Boltzmann called a complexion. The number of complexions is proportional to the probability of finding the components in such a state (Eq. (3)). Conversely, the negentropy can be calculated in the function of the inverse of such a probability distribution (Eq. (5)). If w is the number of accessible microscopic states, or components’ degrees of freedom, 1/w is the degree of order or bonds that restrain the freedom of components. Shannon’s equation measures the amount of information in a system as the number of unities that can assume one of two states (Eq. (1)). In contrast, Brillouin’s equation (Eq. (5)) measures negentropy as proportional to the inverse of the number of accessible microscopic states, which is proportional to the number of bonds. The more important difference is that Shannon notoriously approaches the amount of information in terms of the number of elements of a ‘minimum code’ with two signs. In turn, Brillouin refers to information as the number of bounds in a system. However, the bounds and components might be indistinguishable from one another. If each bound or bounded component could assume one of two distinct states, negentropy would be a measure of the system’s capacity to store information. Using Eq. (3b) it would be possible to obtain the information in bits. But, in general, this is not the case, since each bound or component exists in one sole distinguishible state. To call this ordered state of information is like creating a new ‘code’ with only one sign. Such a ‘code’ is more precisely a non-code since the amount of information (in bits) it can represent is 1n = log2 20 = 0 bits, no matter how big the number of elements of the non-code (n). For instance, while a DNA strand 20 nucleotides length can carry log2 420 = 40 bits of information, a DNA homopolymer strand (e.g. a poly A tail) with same length, can store only log2 120 = 0 bit, although both oligomers present very similar negentropies.
3. Negentropy is not information
4. Conclusion
When cooled, water molecules pass, roughly speaking, from an undefined state (liquid) to a defined state (ice) with a gain of
Although entropy can reversibly change in many systems, only coding systems can produce codes. Coding systems contain components Fig. 1. Sketch of a nanodevice entrapping six molecules of butene in six different niches capable of reading/altering the configurations their special configurations. There are 2 positional isoforms of butene: 1-butene, that has no stereoisomers, and 2-butene, presenting cis and trans stereoisomers. Let’s make a device trapping six molecules of 1-butene (Fig. 1A) and another one, trapping six 2-butene molecules (Fig. 1B). While device A presents only one state, device B can exist in 26 different states. Each state can be used to code a different message, making device B useful for storing and communicating information, which it is not possible with device B. At any given state, B presents about the same entropy, since the number of microstates is roughly the same despite the number and position of each cis/ trans isomer. By the same reason, any state of B presents an entropy similar to A. Such a property of isomeric monomers has been investigated to produce solid-state photoswitches for ultra-high capacity data storage media (e.g. Steiner et al., 1978; Hatcher and Raithby, 2013). 19
BioSystems 182 (2019) 17–20
W.D. dos Santos
that can be arranged in two (binary) or more (alphabetic) stable discrete states or signs. Information is the arbitrary pattern in which these signs are arranged, when, and only when this arrange is determined from outside by a template, or guideline i.e. a codemaker. Thus, a gain in information requires at least an equivalent gain in negentropy, but a gain in negentropy does not necessarily imply a gain in information.
0: Castalia, The Game of Ends and Means. http://www.glass-bead.org/article/ transcendental-logic-and-true-representings/?lang=enview. Brillouin, L., 1962. Science and Information Theory. Academic Press Inc, New York 347 pp. Hatcher, L.E., Raithby, P.R., 2013. Solid-state photochemistry of molecular photoswitchable species: the role of photocrystallographic techniques. Acta Cryst. C. 69 (12), 1448–1456. Shannon, C., 1949. A mathematical theory of communication. Bell Sys. Tech. J. 27 (379–423), 623–656. Steiner, U., Abdel-Kader, M.H., Fischer, P., Kramer, H.E.A., 1978. Photochemical cis/ trans isomerization of a stilbazolium betaine. A protolytic/photochemical reaction cycle. J. Am. Chem. Soc. 100 (10), 3190–3197.
References Barbieri, M., 2016. What is information? Philos. Trans. A R. Soc. 374, 20150060. Brassier, R., 2016. Transcendental logic and true representings. Glass Bead Journal, Site
20