- Email: [email protected]
Elements of a more comprehensive theory of computing
BioSystems 52 (1999) 111 – 122 www.elsevier.com/locate/biosystems
Elements of a more comprehensive theory of computing Toni Kazic * Institute for Biomedical Computing, Washington Uni6ersity, St. Louis, MO 63110, USA
Abstract Problems implementing DNA computers stem from the physical nature of molecules and their reactions. The present theory of computation requires assumptions that, at best, are extremely crude approximations of the physical chemistry. Here, I consider the hypothesis that discarding those assumptions in favor of more physically realistic descriptions would produce a more comprehensive theory of computing, yielding both theoretical insights and help in designing better molecular computers. I describe the discordances between the theories of physical biochemistry and computation, indicate some elements of a more comprehensive theory, and discuss some of the challenges the construction of a unified theory faces. © 1999 Elsevier Science Ireland Ltd. All rights reserved. Keywords: Molecular computing; DNA computing; Physical biochemistry; Theory of computing
1. Hypothesis Molecular computing is justifiably exciting, not least for the alluring prospect of biologically-inspired machines nicely handling NP-complete problems. However, current molecular computers are limited in three ways. First, it is technically hard to compute by biochemical means, and matters worsen with problem size, reagent inhomogeneities, and the number of manipulations. Second, each computer is a special-purpose machine tuned for particular problems, data structures, algorithms, and problem sizes, often fusing them together; not a general-purpose device. Third, algorithms for combinatoric and number – theoretic problems only have been attempted so
Prepared for the DNA4 Proceedings, January 26, 1999. * Tel.: +1-314-362-3121; fax: + 1-314-362-0234. E-mail address: [email protected] (T. Kazic)
far, not just because they dominate the NP-complete problems (Garey and Johnson, 1979), but because they fit the architectures chosen so far. Thus, many important numerical problems are excluded. Do these limitations have any significance apart from posing technological challenges for clever engineering to surmount? The answer depends on how well the theory of computation can predict the behavior of molecular algorithms and molecular computers. Here experience is less than encouraging. The outcome of iterating a molecular algorithm ten times, let alone 10 000, is not predicted by the theory, and the properties of iteration vary depending on whether the computer is electronic or biochemical. The computed output is a mixed population of ‘answer’ and ‘others’, not the single unambiguous result the theory presumes. Since it assumes the execution of each step
0303-2647/99/$ - see front matter © 1999 Elsevier Science Ireland Ltd. All rights reserved. PII: S 0 3 0 3 - 2 6 4 7 ( 9 9 ) 0 0 0 3 8 - 6
112
T. Kazic / BioSystems 52 (1999) 111–122
is error-free, the theory cannot help us estimate how good our engineering of the biochemical machine must be or the theoretical consequences of error rates of 1 – 50%, reasonable values for many reactions in vitro. A theory failing such empirical tests would be judged inadequate by a natural scientist. Empirical adequacy properly has little weight in pure mathematics, and one may object the theory was intended only to elucidate the nature of and requirements for computation. The extent to which electronic machines emulate the discrete premises and operations of the theory of computation is remarkable, but perhaps this is incidental to the theory’s predictive success. The claim would be that as long as the abstract properties of the molecular computer match those of the theory, the theory is adequate. If true, then changing the computing machinery should have no effect on the theory’s ability to predict the outcome of computations implemented on the new machine. Since this does not seem to be the case in any but the most limited instances, the logical conclusion is that the abstract properties of the molecular system do not match those of the theory of computation. This result is hardly surprising to those with any acquaintance with physical biochemistry. As I sketch below, the intrinsic physical nature of molecules and their reactions are inconsistent with the fundamental premises of the present theory of computation; and physics governs the behavior of machines. One must therefore consider the hypothesis that the present theory is simply a special case of a more comprehensi6e theory of computing, one which would unify aspects of the theories of computation and physical biochemistry to account equally well for the properties of algorithms when implemented on abstract, electronic, and biochemical computers. This paper considers some of the elements of a unified theory and what issues it would need to address to represent an improvement over the present situation. It is not a pre´cis of that theory: at best it is only a crude sketch of the territory. Space constrains many truncations; for a fuller exposition see Kazic (1999). Are new theories of computation worth the effort of their creation? There are four reasons to
think the effort would be well spent. First, it is important to determine the conditions under which the present theory of computation holds. If the premises or methods of present proofs are restricted, then their results, including the properties of computations, will be less general than previously thought and correspondingly modified. Second, a more physically grounded theory would likely stimulate substantial changes in our understanding of computing and the design of computers. Third, our present notion of effective computability may be unnecessarily limited if physical systems indeed subsume the class of general recursive functions (Smith, 1993) and if an expanded definition is as useful as the current one for abstract problems. Finally, a theory that helped design better algorithms and molecular computers would distinctly benefit the current situation.
2. The present theories To understand why the present theory of computation is insufficient for biochemically based systems, I briefly compare the fundamental axioms and basic results of the theories of physical biochemistry and computation. There are a number of excellent references which describe the physics of systems of molecules and reactions, and any good text can serve as a starting point (Blinder, 1969; Jost, 1971; Astarita, 1989). Similarly, the references (Post, 1936; Turing, 1936; Minsky, 1967; Moret, 1998) cover the foundations of modern computational theory.
2.1. Physical biochemistry An informal summary of the major intrinsic physical properties of reactions and molecules may be helpful. All chemical reactions, including those involving molecules of biological derivation, arise from populations of different types of molecules interacting in physical space and time. These populations are large: at standard temperature and pressure 1 ml of water contains : 3× 1019 molecules; if it was also one femtoMolar in an oligonucleotide the total population would rise
T. Kazic / BioSystems 52 (1999) 111–122
to : 3× 1028 molecules (neglecting the oligonucleotide’s counterions). The thermodynamic extensional variables which describe the system—for example, concentration (an adequate approximation to chemical potential for our purposes), free energy, enthalpy, entropy, and other state variables—are averages both over the entire population of molecules and the infinite number of possible quantum states each molecule can assume, and their values range over the real or positive real numbers. The kinetic properties of a system—variables such as reaction rate and flux —are determined by the statistical mechanics of the molecular populations. Some states of molecules are discretely approximated, such as covalent structure, but the quantum mechanical description of those structures is a set of continuous functions; and each molecular species has characteristic probabilities for different types of structural changes, including covalent modifications, under different environmental conditions. Even though molecules are discrete, the number of molecules is a finite subset of the natural numbers, and many properties of molecular species are discrete (e.g. their quantum states), the variables and equations which describe how a molecular computer and its components behave are as continuous as those for any other system of reactions, independent of any idealization of a molecular specie or reaction. This does not mean that approximations are impossible (see Section 4), but it does mean they are exactly that and their quality must be evaluated. I now present a brief rationale for the main points in the preceding summary, either by indicating a derivation or summarizing the physical chemistry. Three axioms are assumed: that matter and energy are conserved; that space and time are continuous variables; and that mass is a discrete variable. The first is simply the fundamental assumption of thermodynamics. The second accords with the way these observables are treated in the physical theory at these size scales, and with our accumulated experience that space and time can assume any real value (molecular reactions operate at length and time scales 23 and 29 orders of magnitude greater than the scales at which space and time may be better thought discrete (Misner
113
et al., 1973). The third axiom expresses a fundamental observation of the last few hundred years.
2.1.1. E6ery reaction is continuous To illustrate the continuous nature of reactions, I sketch the derivation of the key equations. Let the set R be a system of reactions ri, 15 i5 N among a set of molecular species X= {xj }, 1 5 j5 M (called reactants); i, j, N, and M are any positive integer. Each reactant xj may participate in more than one reaction, and every reaction has at least two reactants. Each reaction ri is described by a formal reaction equation of the form: Xs,i
ki
Xd,i
j=1
k−i
j=1
% ni,s, j xj X % ni,d, j xj
(1)
where the sets of reactants written sinistralaterally (s) and dextralaterally (d) in the formal reaction equation are Xs,i and Xd,i, respectively, the cardi= nality of any set S is denoted by S , so that Xs,i and Xd,i are the number of reactants in each set; ni,{s d},j is the stoichiometry of reactant xj in Xs,i or Xd,i ( is logical or); and ki and k − i are the forward and reverse reaction rate constants, respectively. A species appearing catalytically in a reaction equation is included on both sides. R is then described by the set of formal reaction equations, one for each ri R. Call the set of reactions in which any particular reactant participates Rj, Rj ¤ R. To characterize the behavior of reactions chemists measure reactions’ kinetics and thermodynamics. If the formal reaction equations generated by Eq. (1) describe elementary reactions and certain chemical conventions are applied, then one can describe the kinetics of R in terms of the changes in concentration (xj ) for each species. These are simply the derivative of concentration with respect to time (x; j ) for each molecular species in R. The system of ordinary differential equations describing the kinetics of R is generated by: N
x; j = % i= −N i"0
M
a(ri, xj )bkbi 5 x nl l, − bi l=1
where the additional variables are: a(ri, xj )= − 1 if xj Xs,i
(2)
T. Kazic / BioSystems 52 (1999) 111–122
114
a(ri, xj )= 1 if xj Xd,i a(ri, xj )= 0 if xj QXs,i, xj Q Xd,i
(3)
define the state of system R. For each molecular species xj call the set of quantum states it can assume Gq,j. Applying a common notation (e.g. Casti, 1992), the state of the system at any time t, SR:
(4)
SR = {X, Sq, X, X: , GR,t, G, t}
b = − 1 if considering Xs,i b = 1 if considering Xd,i l= the index of reactants in each reaction ri Rj
(5)
nl, − bi =stoichiometry of reactant xl on the b side of reaction ri, ri Rj,
(6)
n]0
− bi B0 if xl Xs,i for ri Rj − bi \ 0 if xl Xd,i for ri Rj
(7)
The Gibbs free energy of the system, dGR, summarizes its collective thermodynamics. Similar expressions can be written for the dG components enthalpy and entropy. Strictly speaking all these variables are only defined for equilibrium, but here, and in Section 2.1.2, I will follow the common biochemical practice of evaluating the same expressions at arbitrary times to estimate these variables for ‘reasonable’ displacements from equilibrium. The differential change in the Gibbs free energy for a reaction ri, dGi, is: 5x nl,dl , − bi dGi = DG 0% i + RT ln
(8) 5x nl,sl , − bi
where both products are multiplied over l, 15l 5 M, DG 0% i is the free energy change for reaction ri under standard biochemical conditions, R the gas constant, and T the temperature. The total dG for the system R is simply the sum of those for each reaction: N
dGR = % dGi
(9)
i=1
2.1.2. E6ery reaction has an infinite number of intrinsic states Even reactions involving only two species can assume an infinite number of states. To see this,
(10)
where GR,t, is the summed free energy of the system at t and the other variables are: Sq = {Sq,1, …, Sq, j, …, Sq,M }
(11)
X= {x1, …, xj, …, xM }
(12)
X: = (x; 1, …, x; j, …, x; M )
(13)
G= (dG1, …, dGi, …, dGN )
(14)
the variables in the set X and vectors X: and G each evaluated at time t. The set of all states the system assumes over time is SR. The range of Sq is the natural numbers. The ranges for GR,t and the component variables of each of X, G, and X: are the reals, so the range of GR is the reals.
2.1.3. Intrinsic states do not persist in a system of reactions Sq, X, X: , GR,t, and G determine a multidimensional phase space through which the system passes over t. From the quasi-ergodic theorem of Birkhoff and von Neumann, we know that the state of the system will eventually approach arbitrarily near every accessible point in the phase space (Blinder, 1969). Thus, over time the system will evolve, ‘discarding’ previous states and entering new ones. 2.1.4. E6ery reaction is stochastic The steps involved in any instance of a reaction are stochastic processes, and chemists have identified many relevant ones. Some of these are sequential, but many are not. Further, there are many instances of a given reaction (how many being determined by the reaction’s kinetics and thermodynamics), forming a random population of events. For any transition state intermediate there are usually more than two outcomes: the individual frequency of these ‘side reactions’ is by definition less than that of the reaction of interest, but often significant. In toto, the probability of
T. Kazic / BioSystems 52 (1999) 111–122
any particular outcome is less than unity (often much less so).
2.1.5. E6ery reaction is re6ersible When the concentrations of the dextralateral reactants exceeds that of the sinistralateral, the reverse direction will be favored and the sinistralateral set produced. Which direction is favored changes moment to infinitesimal moment, as the ensemble concentrations shift and as each molecule randomly teeters on the brink of several stochastic processes simultaneously. Equilibrium for the population of molecules — the concentrations of sinistralateral and dextralateral reactant sets of reaction ri — is strictly defined only at infinite time under thermodynamic standard conditions and is given by the equilibrium constant for that reaction, Keq,i : l, − bi 5x nl,d,
(15)
Keq,i = l, − bi 5x nl,s,
= ki /k − i
(16)
where xl,{s d}, denotes the concentration of a given xl in Xd,i or Xs,i at infinite time, the products are indexed over l as in Eq. (8), and the remaining variables are the same as in Eq. (2).
2.1.6. No reaction halts This is a consequence of the foregoing facts. Reactions slowly proceed at extremely cold temperatures, and would only halt completely at absolute zero (which cannot be achieved for thermodynamic reasons).
115
Modern electronic machines go to considerable lengths to emulate this discreteness. The number of intrinsic states the machine may assume is finite. Despite the inverse relationship between number of states and size of the alphabet, to the best of my knowledge no one has formulated a model of computation whose processing unit is allowed an infinite number of intrinsic states. An intrinsic state persists indefinitely. The current machine state persists until explicitly changed, and certainly long enough for the computation to complete and the output to be detected. Indeed, this persistence of state may account for the tendency to call states ‘memory’. Execution of the computations is nonstochastic. Each step in an algorithm may accept any of a set of inputs, in turn producing a set of one or more outputs; but the symbols representing input and output exist absolutely, without loss or change. With effective error rates in the order of 10 − 15 per operation, electronic computers give a very good approximation to this condition. Execution can halt if the computation completes. A computation has a detectable output; this output is unique to the state of completion; the output’s correctness can be determined; and the algorithm’s halting is absolute, stopping further change in output. Input, output, and program share an abstract, very general language. Problems are formally represented in an abstract language the machine interprets; the language’s operations are sufficiently abstract to be at least very general; input, output, and program are in the same abstract language; and their encoding and decoding are uniquely determined in any given abstract language.
2.2. Computation By way of example I summarize the central premises of the main computational model of the present theory, the universal Turing machine. A compact definition is given in Illingworth et al. (1990) and reproduced in Kazic (1999). The data, the processes which operate on the data, and time are all discrete. Models of computation remain formulated within their original context of number theory and mathematical logic.
3. Toward a more comprehensive model of computation When one compares the assumptions of the theory of computing to the physical nature of reactions, the discordances are fundamental. The theory of computing describes machines which are general, discrete, finite, nonstochastic, haltable, and usually memorizing: reactions are none of
116
T. Kazic / BioSystems 52 (1999) 111–122
these. Instead, they are at the very minimum continuous, infinite, stochastic, and interminable. A theory of computing which assumed such properties would surely produce very different models of computation. What might such a theory be? It might reasonably be expected to hybridize descriptions of physical biochemistry and analogue computers (Karplus and Soroka, 1959). The ‘theory of computation’ for analogue computers was the theory of differential equations; they were programmed by connecting particular components; and they suffered from high noise levels (Fred Rosenberger and Gerald Johns, personal communication). Landweber and Lipton have pointed out that a potentially significant application of DNA computers might be their use to infer properties of naturally occurring DNA in an analogue process (Landweber and Lipton, 1997). Erik Winfree’s experiments indicate the value of using even simple kinetic models in designing molecular computers (personal communication). The anticipated theory would define the abstract properties of computations implemented with molecules and reactions without arbitrarily idealizing their physics. Below I suggest some key notions from the point of view of a unified, more comprehensive theory as examples. Additional applications may be found elsewhere (Kazic, 1999). The principles basic to a more comprehensive theory may be informally stated: the computing machine is a continuous, nonhalting, stochastic system; it may assume an infinite number of transient intrinsic states; a computation operates over populations of symbols having probability distributions over time and composition; these distributions can assume any value over the interval [0, 1] of the real numbers; because of this diversity in symbol sets, the computation must map among subpopulations of symbols; and those mappings can be deterministic or nondeterministic according to the properties of the computation’s mapping function.
3.1. Basic notions Imagine an experimenter who wishes to build an adder using terminal deoxynucleotidyl trans-
ferase (Brutlag et al., 1977; Kornberg, 1980). To a solution containing a single-stranded tetramer of sequence 5%CATC (hereafter CATC) and deoxyguanosine triphosphate (dGTP) at one-tenth the concentration of the tetramer, an excess of the enzyme is added. This enzyme adds one or more nucleotide triphosphates to the 3%OH of the oligonucleotide, at each addition eliminating a pyrophosphate. Each addition leaves a free 3%OH, and to a first approximation the enzyme cannot distinguish between an oligonucleotide having no added bases and one which has. Each oligonucleotide is continuously subject to spontaneous hydrolytic reactions including cytosine deamination (which changes cytosines to uracils) which can change its sequence and length independently, and the rates and rate constants of these reactions differ. Similarly, dGTP molecules are also hydrolyzing, usually to the di- or monophosphates. Each reaction in this description, and many other processes not mentioned, operates randomly with characteristic rate constants. I percolate this example into the following, but it should be understood the ideas are not limited to computers constructed with molecules. Variables and indices repeated from Section 2.1 have the same meanings here. Let S (all the molecules) be the set of symbols in a language L understood by a computational device (the enzyme responds appropriately to each), which may itself be considered a member of S if this is useful. Each symbol sj , 15 j5 M, sS, is represented by a population of identical physical objects (the molecules of each molecular species). Structurally changing an object removes it from one population and places it in another population representing a different symbol. SI, SO are the sets of all symbols that could be initially presented for or eventually produced by computation. Here the subscripts simply designate the relevant subset of S. In the example SI is the set {dGTP, O1, O2, …, Oj }, where Oj is an oligonucleotide of length three or greater. SO is {dGMP, PPi, O1+G, O2+G, …, Oj+G}, where Oj+G is Oj with a G added 3% and PPi is pyrophosphate (the biochemical abbreviation for pyrophosphate, not the index i ). The set of symbols internal to the computation is designated SC
T. Kazic / BioSystems 52 (1999) 111–122
(enzyme); this set need not be disjoint with either SI or SO. From the populations of molecules composing SI and SO, only some molecules will actually collide with the enzyme and suffer reaction to the transition state intermediate; of those, only some will actually complete the reaction to yield product. SI %, SO%, represent these sets of symbols actually presented for or eventually produced by a computation, and are the inputs and outputs of the computation in the classical sense. For each computational step si, 1 5 i5N, Si,I and Si,O are the sets of symbols which could be presented to that step as input and output respectively (only C-containing oligonucleotides are subject to cytosine deamination; uracil-containing oligonucleotides are produced; but both sets include many possible oligonucleotides present in S). Si,I % and Si,O% are the sets of symbols actually chosen for a computational step si as input and output, subject to the constraint that each member of both sets is required for the other members of that set to be mapped; i.e. that all the members of Si,I % and Si,O% are obligatorily comapped. For a single cytosine deamination event where Si,I = {CATC, UATC, CATU, CAT, ATC, H2O}, Si,I% = {CATC, H2O}; then Si,O% is one of {CATU, NH3} and {UATC, NH3}, and Si,O is their union. The most important relationships among the sets are: Si,{I O}% ¤ Si,{I O}; S{I O}% ¤S{I O}; Si,{I O} ¤ S{I O} S= SI SO SC.
3.2. Some definitions With this taxonomy one may now define several terms. Definition 1. A computational system is a set of objects and a set of processes acting on those objects, such that each member of both sets and the results of their interactions can be mapped to an abstract formulation of a computational problem. In the present context the obvious example is a reaction system R and its included molecules and reactions; but the definition would apply equally well to mechanical or electronic computing devices. No claim is made as to the power of the computational system or the class of its computations; equally, it is not an assertion that extant
117
natural systems perform computations. It is reasonable to identify R as the computing device and the computational steps si as ri, but other partitions of roles among objects are possible. Definition 2. The concentration of a symbol, [sj], is defined as the concentration of the physical objects representing that symbol in the computational system. The notion that a symbol has the property of concentration certainly does not derive from formal logic, but is appropriate in the present context. Definition 3. A symbol sj is said to exist if its concentration is sufficient to be detected by some de6ice. The probability that this condition obtains is called the probability of the symbol’s existence, Pe(sj), and ranges o6er [0, 1] of the real numbers. For a set S, Pe(S) is the 6ector of Pe(sj), sj S. Clearly a computation cannot proceed if the objects instantiating that computation vanish during it or before the output object is detected. A probability density function for Pe (S) may be defined with respect to time and the composition of S for a given detection scheme. Definition 4. A computational step si consists of three components: (1) recognizing a set of obligatorily co-input species, Si,I %, from Si,I; (2) recognizing a set of obligatorily co-output species, Si,O %, from Si,O; (3) and mapping Si,I % Si,O %. A computational step includes distinguishing which symbols will actually be used (Si,I %, Si,O%) from the sets of symbols which could be used (Si,I, Si,O), then mapping among the used sets. The cardinality of each S set may be one or greater. Each symbol is represented by a population of homogeneous physical objects. The recognition and mapping are applied to sets of such populations. If the computational step happens to be a reaction ri, then Si,I % is one set of coreacting species (say Xs ) and Si,O% the other (say Xd for an arbitrarily written reaction equation). In the example terminal deoxynucleotidyl transferase distinguishes CAT from CA, both members of SI and both derived from CATC by hydrolysis, because it reacts only with oligonucleotides of length
118
T. Kazic / BioSystems 52 (1999) 111–122
three or greater. The enzyme also distinguishes CATG and CATG300, both members of SO. In adding a single deoxyguanosine triphosphate to CAT, the enzyme maps {CAT, dGTP} {CATG, dGMP, PPi }. Definition 5. A state of a system S is a set of distinct, obser6able, intrinsic system 6ariables e6aluated at computational step Si. S may be a finite or infinite set. The obvious example is SR defined in Section 2.1.2, but the definition applies equally to states in classical Turing and von Neumann machines. This definition and that for memory distinguish the two terms; by them Post production systems have neither. Definition 6. A half-life t1/2 is the time required for one half of a population of objects to randomly change from some obser6able property s to (a not necessarily obser6able) property of not-s ( ¬s). Of the initial population of CATC oligonucleotides, some will be substrates in the enzymatic reaction and some will be substrates of the various hydrolytic reactions. The population will decrease as members change from the state of ‘being CATC’ to ‘being something else’, and one can measure the half-life of the population as a function of reaction conditions. Thus over time [sj ] can change just as RAM fails in electronic computers, destroying the stored data. Definition 7. A memory is a storage de6ice of measurable half-life for data and programs that permits any combination of operations which add information to or extract information from it, nondestructi6ely or otherwise. Suppose that CATCG is assigned to be a memory device; in the forward direction the reaction CATCG+ dGTP =CATCGG + PPi destroys CATCG. Definition 8. A deterministic computation is the set of computational steps Cd which collecti6ely map the set of presented inputs SI %, to the set of produced outputs SO %, or Cd: SI % SO% It has the following properties. SI % =SI and SO% =SO. For each computational step, ci Cd, the mapping is
one-to-one. For all symbols in each of SI %, SO%, SI, and SO, the probabilities that each exists are unity. Definition 9. A non-deterministic computation is the set of computational steps Cn which collecti6ely map the set of presented inputs SI % to the set of produced outputs SO% or Cn: SI % SO%. It has the following properties. Either SI % ¦ SI; or SO% ¦ SO; or both. At least one mapping ci Cn is either one-to-many, many-to-one, or many-to-many: that is, there is more than one set of obligatorily coprocessed inputs, Si,I %, in Si,I; or more than one set of obligatorily coprocessed outputs, Si,O%, in Si,O, or both. For all symbols in each of SI %, SO%, SI and SO, the probabilities that each exists are unity. Definition 10. A computation C is executed stochastically if for at least one sj S, Pe (sj)B 1; or at least one ci C is a stochastic function. In Definitions 8–10 I have distinguished determinism from stochasticity. Determinism is a property of a computation (and its specifying algorithm, if there is one) and stochasticity is a property of the algorithm’s execution. In an enzymatic context, three types of nondeterminism are relevant. The first is when a particular set of presented substrates can give more than one set of products (there is more than one Si,O% set and the mapping is one-to-many; e.g. DNA polymerase I of E. coli, which catalyzes five different reactions among DNA and deoxynucleotide triphosphates). One may separately itemize each enzymatic activity as a reaction, but a multifunctional enzyme will perform all catalyses at their characteristic rates unless conditions are carefully arranged or the other activities are ablated from the enzyme (for example by protein cleavage or gene mutation). The second is when an enzyme can accept more than one set of substrates and yields a set of products unique to each substrate set (there are more than one of both Si,I % and Si,O%, but the mapping between given substrate and product sets is one-to-one; e.g. hexokinase, which specifically adds a phosphoryl to carbon six of a wide variety of hexoses, yielding the appropriate 6-phosphohexose). Such an enzyme is usually considered to be ‘broadly specific’ in the biochemical literature.
T. Kazic / BioSystems 52 (1999) 111–122
The third is when an enzyme can accept more than one set of substrates and yield more than one set of products (there are more than one of both Si,I % and Si,O%, and the mapping between them is many-to-many; e.g. terminal deoxynucleotidyl transferase, which will produce polymers of different chain length in our example, but each addition is deterministic). Each of these reactions executes stochastically, subject to the processes described in Section 2.1. Under these definitions one could reasonably speak of measurably different degrees of determinism and stochasticity, depending on the designs of R and C and the conditions under which R performs C. When characterizing a computation C determinism and stochasticity would be separately evaluated. For evaluating whether C was deterministic or not one would momentarily assume Pe (sj )= 1 for each sj, determine the subset relationships of the various S sets, and check to see if any ci C was other than a one-to-one mapping. If more than one of either Si,I % or Si,O%, exists for any ci C, or a non-one-to-one mapping was present the computation would be nondeterministic overall (though clearly it could also have deterministic steps). To evaluate stochasticity one would then determine if the actual value of Pe (sj ) B 1 for at least one sj or if the execution of any step ci is most accurately represented by a stochastic function. If either condition obtains then C is executed stochastically.
4. Issues facing a unified theory Any attempt at unification will need to reconcile the discordances between the two contributing theories. There are two basic approaches to doing this. The first is to redesign the biochemical system so that it more closely approximates the premises of the theory of computing and then see how well the theory predicts its properties. This requires both developing metrics to quantitatively evaluate the quality of the approximations and of the results and understanding how to map the quantitative phenomena to the qualitative, logical premises of the theory. The second basic approach is to replace the theory of computation
119
with one that explicitly and quantitatively includes the physical biochemistry, roughly at the level described above. While this also requires the development and use of quantitative metrics for algorithm and computer performance, more importantly it mandates rethinking fundamental ideas. In the main this more exciting approach awaits the future. What are the issues a more comprehensive theory must confront, and how can one tell how well it performs?
4.1. Theoretical issues 4.1.1. Generality An interesting phenomenon of molecular computing so far is that data structure and algorithm often merge. The ‘machine’—just now a human in a laboratory—is not readily programmable in the sense von Neumann machines are. Thus the generality of machines is more a multivalued function than a binary property. What are appropriate metrics for generality? A crude measure might count the number of algorithms a system permits without structural modification, including some measure of their mutual differences, rather than simply the machine’s parts. 4.1.2. Partition of information and functions Since a general machine would clearly be more useful and possibly more economical, a rule of thumb would be to logically partition SI %, SO% and C from the computer, the data structures, and each other to the maximum possible extent, along the lines of the von Neumann machine; recall the option to include the enzyme in the symbol set S in Section 3.1. Doing so provides the opportunity to evaluate alternative partitioning schemes and trade-offs. 4.1.3. Stochasticity Much of the discordance between the theories lies in the stochastic nature of reactions. While the emphasis has been on forcing molecular systems to be digital, the obvious theoretical alternative is to define the meaning of computation and computability in terms of stochastic processes, explicitly treating the kinetics and thermodynamics of
120
T. Kazic / BioSystems 52 (1999) 111–122
the molecular computer. Substituting statistics for logic has some interesting theoretical consequences, apart from the practical issue of output evaluation functions (see Section 4.2). First, what is the appropriate definition of correctness if it varies with algorithm or machine? This is not unlike the mathematician’s concern for the nature of proof (Fallis, 1997). Second, it embraces the notion of analogue computing. What was previously an erroneous computation may now become a meaningful one, precisely because it directly samples a large solution space. Third, it requires a clear distinction between stochasticity and nondeterminism. Fourth, since it implies explicit consideration of probability density functions, it may rescue parts of the present theory of computation by allowing a well-reasoned neglect of minor species, effectively unpopulated states, and rare events. Finally, it fuses the abstract and physical properties of the system, since the former directly depend on the latter. For the resulting theory to be general it will need to consider the varieties of reaction systems and not content itself with one or two exemplars.
4.1.4. Encoding Current molecular computers encode problems by relying on the representation of molecular sequence as strings and reactions of oligonucleotides as string operations, limiting the problems that can be treated to combinatorial ones. Abandoning this restriction could convolve logical, analogue, and numerical operations, perhaps for greater computational power. What are the theoretical implications of nonrecursively enumerable output constructs? 4.1.5. Shared language Logical/analogue/numerical encodings may need to relax the assumption of the mutual intelligibility of SI %, SO%, and C. The rationale may be as simple as shifting from determining the existence of tokens to counting them (e.g. concentration), or as canny as sequestering SO% in a different language unintelligible to the computing device to mimic halting. What the theoretical consequences of this molecular tower of Babel and what the metrics for describing similarities and differences among the languages might be are not yet clear.
4.1.6. Discretization The obvious way to discretize equations is to bin the values the variables can assume. Discretization of time and steps introduces problems of defining those notions in systems of reversible, continuous, and nonhalting reactions. Many discretization schemes for reactions are possible; in practice consideration of optimal schemes would be essential. Standard statistical techniques applied to actual and simulated data provide the appropriate metrics for evaluating possible schemes. 4.1.7. Finiteness What are the theoretical consequences of the computer having an infinite number of intrinsic states? Does the answer depend on the probability distribution of the states? 4.1.8. Haltability Absent 0 K freezers, completeness of halting will be a variable. Incorporating the frequency, kinetics, and consequences of these demi-halts into the theory should provide additional insight into computational failures. 4.2. Pragmatic issues 4.2.1. Machine complexity A likely cost of increasing the generality of a machine would be greater size and complexity in its design, execution, and the detection of SO%. On the other hand, the benefits would likely be an increase of and qualitative change in the nature of the parallelism and a reduction in the required number of manipulations. 4.2.2. Loose and tight couplings among reactions Designing systems that ‘self-bin’, for example because their characteristic functions (and corresponding variables) behave more like sigmoid, d, or step functions under the system’s conditions introduces coupling issues. Biochemistries exhibiting such behaviors are known: reactions with very large –dGs, enzymes with unusually fast kinetics, and cooperative transitions in the presence of a regulatory ligand. Consider installing an enzymatic detector for SO%: difficulties can arise if sI %,
T. Kazic / BioSystems 52 (1999) 111–122
121
for the detector is not equal to SO% for the computation and the two sets must be connected by one or more intermediary reactions. Depending on the properties of those intermediary reactions, the coupling between the sets of symbols may be loosened too much.
but since the monomer is added many times its stoichiometric requirement is quite high. Reducing the use of polymers cuts the requirement for materials considerably, but may introduce problems of interchangeability.
4.2.3. Alternati6e output e6aluation functions The practical effect of stochasticity is a decrease in the yield of SO%, producing detection problems. Most current DNA computing schemes rely on detecting the ‘right’ answer from a large population of very many competing species, a very stringent condition. However, there are other possible evaluation functions, such as the concentrations of all molecular species at some arbitrary time in the computation, that could turn molecular heterogeneity to account.
5. Epilogue
4.2.4. Memory Universal Turing and von Neumann machines require a memory, usually a string; oligonucleotides have so functioned in some molecular computers. A molecular computer which emulated a Post system would at least be free of the requirements for memory and persistence of states. 4.2.5. Interchangeability A benefit of oligonucleotides is that their polymeric structure makes it relatively easy to define a small set of reactions which will operate on any instantiation of a molecule (a particular sequence) approximately equally well. However, DNA is less stable than many other molecules, especially small monomers (Dawson et al., 1986); hybridization is a low-yield reaction with varying yields; and separations based on Stokes radius (crudely, molecular size and shape) are quantitative only over relatively narrow ranges, usually with significant contamination. Designing with monomeric species offers a much larger range of choices, perhaps at the cost of machine complexity. 4.2.6. Stoichiometry An economic disadvantage of polymers is the stoichiometry of their production. Conditions that drive polymerization to completion include substantially raising the concentration of monomers,
While it is premature to speculate on the details of a more comprehensive theory, I suggest one could be developed along the lines of Section 3. But perhaps the best test will be what will be required for us to believe the result of a molecular computation for which we do not have, or cannot determine, the result of a corresponding electronically implemented algorithm. A more comprehensive theory which allows us to ask how correct a probabilistic algorithm and how reliable a stochastic machine are will be important in determining the credibility of that computation.
Acknowledgements The author thanks Denise Beusen, Michael Conrad, Mike Foster, Max Garzon, Al Globus, Tom Head, Gerald Johns, Deborah Joseph, Laura Landweber, Dana Latch, Andrew McDonald, Webb Miller, Fred Rosenberger, Gregorz Rozenberg, Harvey Rubin, Mike Steuerwalt, WaiMo Suen, Keith Tipton, Bill Wise, David Wood, and Zeke Zalcstein for provocative questions and error-trapping. I am infinitely grateful to Tom Blackwell for help with derivations (he is not responsible for my stubborn insistence on mathematically less elegant but chemically more meaningful forms) and to he and Will Gillett for help in translating across disciplines. This work stems from work supported by NSF (IRI-9117005), NIH (GM-56529-01), and the Institute for Biomedical Computing.
References Astarita, G., 1989. Thermodynamics. Plenum Press, New York.
122
T. Kazic / BioSystems 52 (1999) 111–122
Blinder, S.M., 1969. Advanced Physical Chemistry. MacMillan, London. Brutlag, D., Fry, K., Nelson, T., Hung, P., 1977. Synthesis of hybrid bacterial plasmids containing highly repeated satellite DNA. Cell 10, 509–519. Casti, J.L., 1992. Reality Rules, vol. 1. Wiley, New York. Dawson, R.M.C., Elliott, D.C., Elliott, W.H., Jones, K.M., 1986. Data for Biochemical Research, 3rd ed. Clarendon Press, Oxford. Fallis, D., 1997. The epistemic status of probabilistic proof. J. Phil. 94, 165 – 186. Garey, M., Johnson, D., 1979. Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman, San Francisco. Illingworth, V., Glaser, E.L., Pyle, I.C., (Eds.), 1990. Dictionary of Computing, 3rd ed., Oxford University Press, Oxford. Jost, W., (Ed.), 1971. Thermodynamics, vol. 1 of Physical Chemistry. An Advanced Treatise, Academic Press, New York. Karplus, W.J., Soroka, W.W., 1959. Analog Methods. Computation and Simulation. McGraw-Hill, New York. Kazic, T. 1999. After the Turing Machine. Technical report, Institute for Biomedical Computing, Washington Univer-
sity, St. Louis, MO. http://www.ibc.wustl.edu/klotho/papers/techl.ps. Kornberg, A., 1980. DNA Replication. W.H. Freeman, San Francisco. Landweber, L.F., Lipton, R.J., 1997. DNA2DNA computations: a potential ‘killer app’? In: Degano, P., Gorrieri, R., Marchetti-Spaccamela, A. (Eds.), Automata, Languages, and Programming: 24th International Colloquium, ICALP97, number 1256 in Lecture Notes in Computer Science. Springer Verlag, Berlin, pp. 56 – 64. Minsky, M.L., 1967. Computation: Finite and Infinite Machines. Prentice-Hall, Englewood Cliffs, NJ. Misner, C.W., Thorne, K.S., Wheeler, J.A., 1973. Gravitation. W.H. Freeman, New York. Moret, B.M., 1998. The Theory of Computation. AddisonWesley, Reading, MA. Post, E.L., 1936. Finite combinatory processes. Formulation I. J. Symb. Log. 1, 103 – 105. Smith, W.D. 1993. Church’s thesis meets the N-body problem. Technical report, NEC Research Institute, http:// www.neci.nj.nec.com/homepages/wds/church.ps. Turing, A.M., 1936. On computable numbers with an application to the Entscheidungs problem. Proc. Lond. Math. Soc. 42, 230 – 265.
. .
Elements of a more comprehensive theory of computing Toni Kazic * Institute for Biomedical Computing, Washington Uni6ersity, St. Louis, MO 63110, USA
Abstract Problems implementing DNA computers stem from the physical nature of molecules and their reactions. The present theory of computation requires assumptions that, at best, are extremely crude approximations of the physical chemistry. Here, I consider the hypothesis that discarding those assumptions in favor of more physically realistic descriptions would produce a more comprehensive theory of computing, yielding both theoretical insights and help in designing better molecular computers. I describe the discordances between the theories of physical biochemistry and computation, indicate some elements of a more comprehensive theory, and discuss some of the challenges the construction of a unified theory faces. © 1999 Elsevier Science Ireland Ltd. All rights reserved. Keywords: Molecular computing; DNA computing; Physical biochemistry; Theory of computing
1. Hypothesis Molecular computing is justifiably exciting, not least for the alluring prospect of biologically-inspired machines nicely handling NP-complete problems. However, current molecular computers are limited in three ways. First, it is technically hard to compute by biochemical means, and matters worsen with problem size, reagent inhomogeneities, and the number of manipulations. Second, each computer is a special-purpose machine tuned for particular problems, data structures, algorithms, and problem sizes, often fusing them together; not a general-purpose device. Third, algorithms for combinatoric and number – theoretic problems only have been attempted so
Prepared for the DNA4 Proceedings, January 26, 1999. * Tel.: +1-314-362-3121; fax: + 1-314-362-0234. E-mail address: [email protected] (T. Kazic)
far, not just because they dominate the NP-complete problems (Garey and Johnson, 1979), but because they fit the architectures chosen so far. Thus, many important numerical problems are excluded. Do these limitations have any significance apart from posing technological challenges for clever engineering to surmount? The answer depends on how well the theory of computation can predict the behavior of molecular algorithms and molecular computers. Here experience is less than encouraging. The outcome of iterating a molecular algorithm ten times, let alone 10 000, is not predicted by the theory, and the properties of iteration vary depending on whether the computer is electronic or biochemical. The computed output is a mixed population of ‘answer’ and ‘others’, not the single unambiguous result the theory presumes. Since it assumes the execution of each step
0303-2647/99/$ - see front matter © 1999 Elsevier Science Ireland Ltd. All rights reserved. PII: S 0 3 0 3 - 2 6 4 7 ( 9 9 ) 0 0 0 3 8 - 6
112
T. Kazic / BioSystems 52 (1999) 111–122
is error-free, the theory cannot help us estimate how good our engineering of the biochemical machine must be or the theoretical consequences of error rates of 1 – 50%, reasonable values for many reactions in vitro. A theory failing such empirical tests would be judged inadequate by a natural scientist. Empirical adequacy properly has little weight in pure mathematics, and one may object the theory was intended only to elucidate the nature of and requirements for computation. The extent to which electronic machines emulate the discrete premises and operations of the theory of computation is remarkable, but perhaps this is incidental to the theory’s predictive success. The claim would be that as long as the abstract properties of the molecular computer match those of the theory, the theory is adequate. If true, then changing the computing machinery should have no effect on the theory’s ability to predict the outcome of computations implemented on the new machine. Since this does not seem to be the case in any but the most limited instances, the logical conclusion is that the abstract properties of the molecular system do not match those of the theory of computation. This result is hardly surprising to those with any acquaintance with physical biochemistry. As I sketch below, the intrinsic physical nature of molecules and their reactions are inconsistent with the fundamental premises of the present theory of computation; and physics governs the behavior of machines. One must therefore consider the hypothesis that the present theory is simply a special case of a more comprehensi6e theory of computing, one which would unify aspects of the theories of computation and physical biochemistry to account equally well for the properties of algorithms when implemented on abstract, electronic, and biochemical computers. This paper considers some of the elements of a unified theory and what issues it would need to address to represent an improvement over the present situation. It is not a pre´cis of that theory: at best it is only a crude sketch of the territory. Space constrains many truncations; for a fuller exposition see Kazic (1999). Are new theories of computation worth the effort of their creation? There are four reasons to
think the effort would be well spent. First, it is important to determine the conditions under which the present theory of computation holds. If the premises or methods of present proofs are restricted, then their results, including the properties of computations, will be less general than previously thought and correspondingly modified. Second, a more physically grounded theory would likely stimulate substantial changes in our understanding of computing and the design of computers. Third, our present notion of effective computability may be unnecessarily limited if physical systems indeed subsume the class of general recursive functions (Smith, 1993) and if an expanded definition is as useful as the current one for abstract problems. Finally, a theory that helped design better algorithms and molecular computers would distinctly benefit the current situation.
2. The present theories To understand why the present theory of computation is insufficient for biochemically based systems, I briefly compare the fundamental axioms and basic results of the theories of physical biochemistry and computation. There are a number of excellent references which describe the physics of systems of molecules and reactions, and any good text can serve as a starting point (Blinder, 1969; Jost, 1971; Astarita, 1989). Similarly, the references (Post, 1936; Turing, 1936; Minsky, 1967; Moret, 1998) cover the foundations of modern computational theory.
2.1. Physical biochemistry An informal summary of the major intrinsic physical properties of reactions and molecules may be helpful. All chemical reactions, including those involving molecules of biological derivation, arise from populations of different types of molecules interacting in physical space and time. These populations are large: at standard temperature and pressure 1 ml of water contains : 3× 1019 molecules; if it was also one femtoMolar in an oligonucleotide the total population would rise
T. Kazic / BioSystems 52 (1999) 111–122
to : 3× 1028 molecules (neglecting the oligonucleotide’s counterions). The thermodynamic extensional variables which describe the system—for example, concentration (an adequate approximation to chemical potential for our purposes), free energy, enthalpy, entropy, and other state variables—are averages both over the entire population of molecules and the infinite number of possible quantum states each molecule can assume, and their values range over the real or positive real numbers. The kinetic properties of a system—variables such as reaction rate and flux —are determined by the statistical mechanics of the molecular populations. Some states of molecules are discretely approximated, such as covalent structure, but the quantum mechanical description of those structures is a set of continuous functions; and each molecular species has characteristic probabilities for different types of structural changes, including covalent modifications, under different environmental conditions. Even though molecules are discrete, the number of molecules is a finite subset of the natural numbers, and many properties of molecular species are discrete (e.g. their quantum states), the variables and equations which describe how a molecular computer and its components behave are as continuous as those for any other system of reactions, independent of any idealization of a molecular specie or reaction. This does not mean that approximations are impossible (see Section 4), but it does mean they are exactly that and their quality must be evaluated. I now present a brief rationale for the main points in the preceding summary, either by indicating a derivation or summarizing the physical chemistry. Three axioms are assumed: that matter and energy are conserved; that space and time are continuous variables; and that mass is a discrete variable. The first is simply the fundamental assumption of thermodynamics. The second accords with the way these observables are treated in the physical theory at these size scales, and with our accumulated experience that space and time can assume any real value (molecular reactions operate at length and time scales 23 and 29 orders of magnitude greater than the scales at which space and time may be better thought discrete (Misner
113
et al., 1973). The third axiom expresses a fundamental observation of the last few hundred years.
2.1.1. E6ery reaction is continuous To illustrate the continuous nature of reactions, I sketch the derivation of the key equations. Let the set R be a system of reactions ri, 15 i5 N among a set of molecular species X= {xj }, 1 5 j5 M (called reactants); i, j, N, and M are any positive integer. Each reactant xj may participate in more than one reaction, and every reaction has at least two reactants. Each reaction ri is described by a formal reaction equation of the form: Xs,i
ki
Xd,i
j=1
k−i
j=1
% ni,s, j xj X % ni,d, j xj
(1)
where the sets of reactants written sinistralaterally (s) and dextralaterally (d) in the formal reaction equation are Xs,i and Xd,i, respectively, the cardi= nality of any set S is denoted by S , so that Xs,i and Xd,i are the number of reactants in each set; ni,{s d},j is the stoichiometry of reactant xj in Xs,i or Xd,i ( is logical or); and ki and k − i are the forward and reverse reaction rate constants, respectively. A species appearing catalytically in a reaction equation is included on both sides. R is then described by the set of formal reaction equations, one for each ri R. Call the set of reactions in which any particular reactant participates Rj, Rj ¤ R. To characterize the behavior of reactions chemists measure reactions’ kinetics and thermodynamics. If the formal reaction equations generated by Eq. (1) describe elementary reactions and certain chemical conventions are applied, then one can describe the kinetics of R in terms of the changes in concentration (xj ) for each species. These are simply the derivative of concentration with respect to time (x; j ) for each molecular species in R. The system of ordinary differential equations describing the kinetics of R is generated by: N
x; j = % i= −N i"0
M
a(ri, xj )bkbi 5 x nl l, − bi l=1
where the additional variables are: a(ri, xj )= − 1 if xj Xs,i
(2)
T. Kazic / BioSystems 52 (1999) 111–122
114
a(ri, xj )= 1 if xj Xd,i a(ri, xj )= 0 if xj QXs,i, xj Q Xd,i
(3)
define the state of system R. For each molecular species xj call the set of quantum states it can assume Gq,j. Applying a common notation (e.g. Casti, 1992), the state of the system at any time t, SR:
(4)
SR = {X, Sq, X, X: , GR,t, G, t}
b = − 1 if considering Xs,i b = 1 if considering Xd,i l= the index of reactants in each reaction ri Rj
(5)
nl, − bi =stoichiometry of reactant xl on the b side of reaction ri, ri Rj,
(6)
n]0
− bi B0 if xl Xs,i for ri Rj − bi \ 0 if xl Xd,i for ri Rj
(7)
The Gibbs free energy of the system, dGR, summarizes its collective thermodynamics. Similar expressions can be written for the dG components enthalpy and entropy. Strictly speaking all these variables are only defined for equilibrium, but here, and in Section 2.1.2, I will follow the common biochemical practice of evaluating the same expressions at arbitrary times to estimate these variables for ‘reasonable’ displacements from equilibrium. The differential change in the Gibbs free energy for a reaction ri, dGi, is: 5x nl,dl , − bi dGi = DG 0% i + RT ln
(8) 5x nl,sl , − bi
where both products are multiplied over l, 15l 5 M, DG 0% i is the free energy change for reaction ri under standard biochemical conditions, R the gas constant, and T the temperature. The total dG for the system R is simply the sum of those for each reaction: N
dGR = % dGi
(9)
i=1
2.1.2. E6ery reaction has an infinite number of intrinsic states Even reactions involving only two species can assume an infinite number of states. To see this,
(10)
where GR,t, is the summed free energy of the system at t and the other variables are: Sq = {Sq,1, …, Sq, j, …, Sq,M }
(11)
X= {x1, …, xj, …, xM }
(12)
X: = (x; 1, …, x; j, …, x; M )
(13)
G= (dG1, …, dGi, …, dGN )
(14)
the variables in the set X and vectors X: and G each evaluated at time t. The set of all states the system assumes over time is SR. The range of Sq is the natural numbers. The ranges for GR,t and the component variables of each of X, G, and X: are the reals, so the range of GR is the reals.
2.1.3. Intrinsic states do not persist in a system of reactions Sq, X, X: , GR,t, and G determine a multidimensional phase space through which the system passes over t. From the quasi-ergodic theorem of Birkhoff and von Neumann, we know that the state of the system will eventually approach arbitrarily near every accessible point in the phase space (Blinder, 1969). Thus, over time the system will evolve, ‘discarding’ previous states and entering new ones. 2.1.4. E6ery reaction is stochastic The steps involved in any instance of a reaction are stochastic processes, and chemists have identified many relevant ones. Some of these are sequential, but many are not. Further, there are many instances of a given reaction (how many being determined by the reaction’s kinetics and thermodynamics), forming a random population of events. For any transition state intermediate there are usually more than two outcomes: the individual frequency of these ‘side reactions’ is by definition less than that of the reaction of interest, but often significant. In toto, the probability of
T. Kazic / BioSystems 52 (1999) 111–122
any particular outcome is less than unity (often much less so).
2.1.5. E6ery reaction is re6ersible When the concentrations of the dextralateral reactants exceeds that of the sinistralateral, the reverse direction will be favored and the sinistralateral set produced. Which direction is favored changes moment to infinitesimal moment, as the ensemble concentrations shift and as each molecule randomly teeters on the brink of several stochastic processes simultaneously. Equilibrium for the population of molecules — the concentrations of sinistralateral and dextralateral reactant sets of reaction ri — is strictly defined only at infinite time under thermodynamic standard conditions and is given by the equilibrium constant for that reaction, Keq,i : l, − bi 5x nl,d,
(15)
Keq,i = l, − bi 5x nl,s,
= ki /k − i
(16)
where xl,{s d}, denotes the concentration of a given xl in Xd,i or Xs,i at infinite time, the products are indexed over l as in Eq. (8), and the remaining variables are the same as in Eq. (2).
2.1.6. No reaction halts This is a consequence of the foregoing facts. Reactions slowly proceed at extremely cold temperatures, and would only halt completely at absolute zero (which cannot be achieved for thermodynamic reasons).
115
Modern electronic machines go to considerable lengths to emulate this discreteness. The number of intrinsic states the machine may assume is finite. Despite the inverse relationship between number of states and size of the alphabet, to the best of my knowledge no one has formulated a model of computation whose processing unit is allowed an infinite number of intrinsic states. An intrinsic state persists indefinitely. The current machine state persists until explicitly changed, and certainly long enough for the computation to complete and the output to be detected. Indeed, this persistence of state may account for the tendency to call states ‘memory’. Execution of the computations is nonstochastic. Each step in an algorithm may accept any of a set of inputs, in turn producing a set of one or more outputs; but the symbols representing input and output exist absolutely, without loss or change. With effective error rates in the order of 10 − 15 per operation, electronic computers give a very good approximation to this condition. Execution can halt if the computation completes. A computation has a detectable output; this output is unique to the state of completion; the output’s correctness can be determined; and the algorithm’s halting is absolute, stopping further change in output. Input, output, and program share an abstract, very general language. Problems are formally represented in an abstract language the machine interprets; the language’s operations are sufficiently abstract to be at least very general; input, output, and program are in the same abstract language; and their encoding and decoding are uniquely determined in any given abstract language.
2.2. Computation By way of example I summarize the central premises of the main computational model of the present theory, the universal Turing machine. A compact definition is given in Illingworth et al. (1990) and reproduced in Kazic (1999). The data, the processes which operate on the data, and time are all discrete. Models of computation remain formulated within their original context of number theory and mathematical logic.
3. Toward a more comprehensive model of computation When one compares the assumptions of the theory of computing to the physical nature of reactions, the discordances are fundamental. The theory of computing describes machines which are general, discrete, finite, nonstochastic, haltable, and usually memorizing: reactions are none of
116
T. Kazic / BioSystems 52 (1999) 111–122
these. Instead, they are at the very minimum continuous, infinite, stochastic, and interminable. A theory of computing which assumed such properties would surely produce very different models of computation. What might such a theory be? It might reasonably be expected to hybridize descriptions of physical biochemistry and analogue computers (Karplus and Soroka, 1959). The ‘theory of computation’ for analogue computers was the theory of differential equations; they were programmed by connecting particular components; and they suffered from high noise levels (Fred Rosenberger and Gerald Johns, personal communication). Landweber and Lipton have pointed out that a potentially significant application of DNA computers might be their use to infer properties of naturally occurring DNA in an analogue process (Landweber and Lipton, 1997). Erik Winfree’s experiments indicate the value of using even simple kinetic models in designing molecular computers (personal communication). The anticipated theory would define the abstract properties of computations implemented with molecules and reactions without arbitrarily idealizing their physics. Below I suggest some key notions from the point of view of a unified, more comprehensive theory as examples. Additional applications may be found elsewhere (Kazic, 1999). The principles basic to a more comprehensive theory may be informally stated: the computing machine is a continuous, nonhalting, stochastic system; it may assume an infinite number of transient intrinsic states; a computation operates over populations of symbols having probability distributions over time and composition; these distributions can assume any value over the interval [0, 1] of the real numbers; because of this diversity in symbol sets, the computation must map among subpopulations of symbols; and those mappings can be deterministic or nondeterministic according to the properties of the computation’s mapping function.
3.1. Basic notions Imagine an experimenter who wishes to build an adder using terminal deoxynucleotidyl trans-
ferase (Brutlag et al., 1977; Kornberg, 1980). To a solution containing a single-stranded tetramer of sequence 5%CATC (hereafter CATC) and deoxyguanosine triphosphate (dGTP) at one-tenth the concentration of the tetramer, an excess of the enzyme is added. This enzyme adds one or more nucleotide triphosphates to the 3%OH of the oligonucleotide, at each addition eliminating a pyrophosphate. Each addition leaves a free 3%OH, and to a first approximation the enzyme cannot distinguish between an oligonucleotide having no added bases and one which has. Each oligonucleotide is continuously subject to spontaneous hydrolytic reactions including cytosine deamination (which changes cytosines to uracils) which can change its sequence and length independently, and the rates and rate constants of these reactions differ. Similarly, dGTP molecules are also hydrolyzing, usually to the di- or monophosphates. Each reaction in this description, and many other processes not mentioned, operates randomly with characteristic rate constants. I percolate this example into the following, but it should be understood the ideas are not limited to computers constructed with molecules. Variables and indices repeated from Section 2.1 have the same meanings here. Let S (all the molecules) be the set of symbols in a language L understood by a computational device (the enzyme responds appropriately to each), which may itself be considered a member of S if this is useful. Each symbol sj , 15 j5 M, sS, is represented by a population of identical physical objects (the molecules of each molecular species). Structurally changing an object removes it from one population and places it in another population representing a different symbol. SI, SO are the sets of all symbols that could be initially presented for or eventually produced by computation. Here the subscripts simply designate the relevant subset of S. In the example SI is the set {dGTP, O1, O2, …, Oj }, where Oj is an oligonucleotide of length three or greater. SO is {dGMP, PPi, O1+G, O2+G, …, Oj+G}, where Oj+G is Oj with a G added 3% and PPi is pyrophosphate (the biochemical abbreviation for pyrophosphate, not the index i ). The set of symbols internal to the computation is designated SC
T. Kazic / BioSystems 52 (1999) 111–122
(enzyme); this set need not be disjoint with either SI or SO. From the populations of molecules composing SI and SO, only some molecules will actually collide with the enzyme and suffer reaction to the transition state intermediate; of those, only some will actually complete the reaction to yield product. SI %, SO%, represent these sets of symbols actually presented for or eventually produced by a computation, and are the inputs and outputs of the computation in the classical sense. For each computational step si, 1 5 i5N, Si,I and Si,O are the sets of symbols which could be presented to that step as input and output respectively (only C-containing oligonucleotides are subject to cytosine deamination; uracil-containing oligonucleotides are produced; but both sets include many possible oligonucleotides present in S). Si,I % and Si,O% are the sets of symbols actually chosen for a computational step si as input and output, subject to the constraint that each member of both sets is required for the other members of that set to be mapped; i.e. that all the members of Si,I % and Si,O% are obligatorily comapped. For a single cytosine deamination event where Si,I = {CATC, UATC, CATU, CAT, ATC, H2O}, Si,I% = {CATC, H2O}; then Si,O% is one of {CATU, NH3} and {UATC, NH3}, and Si,O is their union. The most important relationships among the sets are: Si,{I O}% ¤ Si,{I O}; S{I O}% ¤S{I O}; Si,{I O} ¤ S{I O} S= SI SO SC.
3.2. Some definitions With this taxonomy one may now define several terms. Definition 1. A computational system is a set of objects and a set of processes acting on those objects, such that each member of both sets and the results of their interactions can be mapped to an abstract formulation of a computational problem. In the present context the obvious example is a reaction system R and its included molecules and reactions; but the definition would apply equally well to mechanical or electronic computing devices. No claim is made as to the power of the computational system or the class of its computations; equally, it is not an assertion that extant
117
natural systems perform computations. It is reasonable to identify R as the computing device and the computational steps si as ri, but other partitions of roles among objects are possible. Definition 2. The concentration of a symbol, [sj], is defined as the concentration of the physical objects representing that symbol in the computational system. The notion that a symbol has the property of concentration certainly does not derive from formal logic, but is appropriate in the present context. Definition 3. A symbol sj is said to exist if its concentration is sufficient to be detected by some de6ice. The probability that this condition obtains is called the probability of the symbol’s existence, Pe(sj), and ranges o6er [0, 1] of the real numbers. For a set S, Pe(S) is the 6ector of Pe(sj), sj S. Clearly a computation cannot proceed if the objects instantiating that computation vanish during it or before the output object is detected. A probability density function for Pe (S) may be defined with respect to time and the composition of S for a given detection scheme. Definition 4. A computational step si consists of three components: (1) recognizing a set of obligatorily co-input species, Si,I %, from Si,I; (2) recognizing a set of obligatorily co-output species, Si,O %, from Si,O; (3) and mapping Si,I % Si,O %. A computational step includes distinguishing which symbols will actually be used (Si,I %, Si,O%) from the sets of symbols which could be used (Si,I, Si,O), then mapping among the used sets. The cardinality of each S set may be one or greater. Each symbol is represented by a population of homogeneous physical objects. The recognition and mapping are applied to sets of such populations. If the computational step happens to be a reaction ri, then Si,I % is one set of coreacting species (say Xs ) and Si,O% the other (say Xd for an arbitrarily written reaction equation). In the example terminal deoxynucleotidyl transferase distinguishes CAT from CA, both members of SI and both derived from CATC by hydrolysis, because it reacts only with oligonucleotides of length
118
T. Kazic / BioSystems 52 (1999) 111–122
three or greater. The enzyme also distinguishes CATG and CATG300, both members of SO. In adding a single deoxyguanosine triphosphate to CAT, the enzyme maps {CAT, dGTP} {CATG, dGMP, PPi }. Definition 5. A state of a system S is a set of distinct, obser6able, intrinsic system 6ariables e6aluated at computational step Si. S may be a finite or infinite set. The obvious example is SR defined in Section 2.1.2, but the definition applies equally to states in classical Turing and von Neumann machines. This definition and that for memory distinguish the two terms; by them Post production systems have neither. Definition 6. A half-life t1/2 is the time required for one half of a population of objects to randomly change from some obser6able property s to (a not necessarily obser6able) property of not-s ( ¬s). Of the initial population of CATC oligonucleotides, some will be substrates in the enzymatic reaction and some will be substrates of the various hydrolytic reactions. The population will decrease as members change from the state of ‘being CATC’ to ‘being something else’, and one can measure the half-life of the population as a function of reaction conditions. Thus over time [sj ] can change just as RAM fails in electronic computers, destroying the stored data. Definition 7. A memory is a storage de6ice of measurable half-life for data and programs that permits any combination of operations which add information to or extract information from it, nondestructi6ely or otherwise. Suppose that CATCG is assigned to be a memory device; in the forward direction the reaction CATCG+ dGTP =CATCGG + PPi destroys CATCG. Definition 8. A deterministic computation is the set of computational steps Cd which collecti6ely map the set of presented inputs SI %, to the set of produced outputs SO %, or Cd: SI % SO% It has the following properties. SI % =SI and SO% =SO. For each computational step, ci Cd, the mapping is
one-to-one. For all symbols in each of SI %, SO%, SI, and SO, the probabilities that each exists are unity. Definition 9. A non-deterministic computation is the set of computational steps Cn which collecti6ely map the set of presented inputs SI % to the set of produced outputs SO% or Cn: SI % SO%. It has the following properties. Either SI % ¦ SI; or SO% ¦ SO; or both. At least one mapping ci Cn is either one-to-many, many-to-one, or many-to-many: that is, there is more than one set of obligatorily coprocessed inputs, Si,I %, in Si,I; or more than one set of obligatorily coprocessed outputs, Si,O%, in Si,O, or both. For all symbols in each of SI %, SO%, SI and SO, the probabilities that each exists are unity. Definition 10. A computation C is executed stochastically if for at least one sj S, Pe (sj)B 1; or at least one ci C is a stochastic function. In Definitions 8–10 I have distinguished determinism from stochasticity. Determinism is a property of a computation (and its specifying algorithm, if there is one) and stochasticity is a property of the algorithm’s execution. In an enzymatic context, three types of nondeterminism are relevant. The first is when a particular set of presented substrates can give more than one set of products (there is more than one Si,O% set and the mapping is one-to-many; e.g. DNA polymerase I of E. coli, which catalyzes five different reactions among DNA and deoxynucleotide triphosphates). One may separately itemize each enzymatic activity as a reaction, but a multifunctional enzyme will perform all catalyses at their characteristic rates unless conditions are carefully arranged or the other activities are ablated from the enzyme (for example by protein cleavage or gene mutation). The second is when an enzyme can accept more than one set of substrates and yields a set of products unique to each substrate set (there are more than one of both Si,I % and Si,O%, but the mapping between given substrate and product sets is one-to-one; e.g. hexokinase, which specifically adds a phosphoryl to carbon six of a wide variety of hexoses, yielding the appropriate 6-phosphohexose). Such an enzyme is usually considered to be ‘broadly specific’ in the biochemical literature.
T. Kazic / BioSystems 52 (1999) 111–122
The third is when an enzyme can accept more than one set of substrates and yield more than one set of products (there are more than one of both Si,I % and Si,O%, and the mapping between them is many-to-many; e.g. terminal deoxynucleotidyl transferase, which will produce polymers of different chain length in our example, but each addition is deterministic). Each of these reactions executes stochastically, subject to the processes described in Section 2.1. Under these definitions one could reasonably speak of measurably different degrees of determinism and stochasticity, depending on the designs of R and C and the conditions under which R performs C. When characterizing a computation C determinism and stochasticity would be separately evaluated. For evaluating whether C was deterministic or not one would momentarily assume Pe (sj )= 1 for each sj, determine the subset relationships of the various S sets, and check to see if any ci C was other than a one-to-one mapping. If more than one of either Si,I % or Si,O%, exists for any ci C, or a non-one-to-one mapping was present the computation would be nondeterministic overall (though clearly it could also have deterministic steps). To evaluate stochasticity one would then determine if the actual value of Pe (sj ) B 1 for at least one sj or if the execution of any step ci is most accurately represented by a stochastic function. If either condition obtains then C is executed stochastically.
4. Issues facing a unified theory Any attempt at unification will need to reconcile the discordances between the two contributing theories. There are two basic approaches to doing this. The first is to redesign the biochemical system so that it more closely approximates the premises of the theory of computing and then see how well the theory predicts its properties. This requires both developing metrics to quantitatively evaluate the quality of the approximations and of the results and understanding how to map the quantitative phenomena to the qualitative, logical premises of the theory. The second basic approach is to replace the theory of computation
119
with one that explicitly and quantitatively includes the physical biochemistry, roughly at the level described above. While this also requires the development and use of quantitative metrics for algorithm and computer performance, more importantly it mandates rethinking fundamental ideas. In the main this more exciting approach awaits the future. What are the issues a more comprehensive theory must confront, and how can one tell how well it performs?
4.1. Theoretical issues 4.1.1. Generality An interesting phenomenon of molecular computing so far is that data structure and algorithm often merge. The ‘machine’—just now a human in a laboratory—is not readily programmable in the sense von Neumann machines are. Thus the generality of machines is more a multivalued function than a binary property. What are appropriate metrics for generality? A crude measure might count the number of algorithms a system permits without structural modification, including some measure of their mutual differences, rather than simply the machine’s parts. 4.1.2. Partition of information and functions Since a general machine would clearly be more useful and possibly more economical, a rule of thumb would be to logically partition SI %, SO% and C from the computer, the data structures, and each other to the maximum possible extent, along the lines of the von Neumann machine; recall the option to include the enzyme in the symbol set S in Section 3.1. Doing so provides the opportunity to evaluate alternative partitioning schemes and trade-offs. 4.1.3. Stochasticity Much of the discordance between the theories lies in the stochastic nature of reactions. While the emphasis has been on forcing molecular systems to be digital, the obvious theoretical alternative is to define the meaning of computation and computability in terms of stochastic processes, explicitly treating the kinetics and thermodynamics of
120
T. Kazic / BioSystems 52 (1999) 111–122
the molecular computer. Substituting statistics for logic has some interesting theoretical consequences, apart from the practical issue of output evaluation functions (see Section 4.2). First, what is the appropriate definition of correctness if it varies with algorithm or machine? This is not unlike the mathematician’s concern for the nature of proof (Fallis, 1997). Second, it embraces the notion of analogue computing. What was previously an erroneous computation may now become a meaningful one, precisely because it directly samples a large solution space. Third, it requires a clear distinction between stochasticity and nondeterminism. Fourth, since it implies explicit consideration of probability density functions, it may rescue parts of the present theory of computation by allowing a well-reasoned neglect of minor species, effectively unpopulated states, and rare events. Finally, it fuses the abstract and physical properties of the system, since the former directly depend on the latter. For the resulting theory to be general it will need to consider the varieties of reaction systems and not content itself with one or two exemplars.
4.1.4. Encoding Current molecular computers encode problems by relying on the representation of molecular sequence as strings and reactions of oligonucleotides as string operations, limiting the problems that can be treated to combinatorial ones. Abandoning this restriction could convolve logical, analogue, and numerical operations, perhaps for greater computational power. What are the theoretical implications of nonrecursively enumerable output constructs? 4.1.5. Shared language Logical/analogue/numerical encodings may need to relax the assumption of the mutual intelligibility of SI %, SO%, and C. The rationale may be as simple as shifting from determining the existence of tokens to counting them (e.g. concentration), or as canny as sequestering SO% in a different language unintelligible to the computing device to mimic halting. What the theoretical consequences of this molecular tower of Babel and what the metrics for describing similarities and differences among the languages might be are not yet clear.
4.1.6. Discretization The obvious way to discretize equations is to bin the values the variables can assume. Discretization of time and steps introduces problems of defining those notions in systems of reversible, continuous, and nonhalting reactions. Many discretization schemes for reactions are possible; in practice consideration of optimal schemes would be essential. Standard statistical techniques applied to actual and simulated data provide the appropriate metrics for evaluating possible schemes. 4.1.7. Finiteness What are the theoretical consequences of the computer having an infinite number of intrinsic states? Does the answer depend on the probability distribution of the states? 4.1.8. Haltability Absent 0 K freezers, completeness of halting will be a variable. Incorporating the frequency, kinetics, and consequences of these demi-halts into the theory should provide additional insight into computational failures. 4.2. Pragmatic issues 4.2.1. Machine complexity A likely cost of increasing the generality of a machine would be greater size and complexity in its design, execution, and the detection of SO%. On the other hand, the benefits would likely be an increase of and qualitative change in the nature of the parallelism and a reduction in the required number of manipulations. 4.2.2. Loose and tight couplings among reactions Designing systems that ‘self-bin’, for example because their characteristic functions (and corresponding variables) behave more like sigmoid, d, or step functions under the system’s conditions introduces coupling issues. Biochemistries exhibiting such behaviors are known: reactions with very large –dGs, enzymes with unusually fast kinetics, and cooperative transitions in the presence of a regulatory ligand. Consider installing an enzymatic detector for SO%: difficulties can arise if sI %,
T. Kazic / BioSystems 52 (1999) 111–122
121
for the detector is not equal to SO% for the computation and the two sets must be connected by one or more intermediary reactions. Depending on the properties of those intermediary reactions, the coupling between the sets of symbols may be loosened too much.
but since the monomer is added many times its stoichiometric requirement is quite high. Reducing the use of polymers cuts the requirement for materials considerably, but may introduce problems of interchangeability.
4.2.3. Alternati6e output e6aluation functions The practical effect of stochasticity is a decrease in the yield of SO%, producing detection problems. Most current DNA computing schemes rely on detecting the ‘right’ answer from a large population of very many competing species, a very stringent condition. However, there are other possible evaluation functions, such as the concentrations of all molecular species at some arbitrary time in the computation, that could turn molecular heterogeneity to account.
5. Epilogue
4.2.4. Memory Universal Turing and von Neumann machines require a memory, usually a string; oligonucleotides have so functioned in some molecular computers. A molecular computer which emulated a Post system would at least be free of the requirements for memory and persistence of states. 4.2.5. Interchangeability A benefit of oligonucleotides is that their polymeric structure makes it relatively easy to define a small set of reactions which will operate on any instantiation of a molecule (a particular sequence) approximately equally well. However, DNA is less stable than many other molecules, especially small monomers (Dawson et al., 1986); hybridization is a low-yield reaction with varying yields; and separations based on Stokes radius (crudely, molecular size and shape) are quantitative only over relatively narrow ranges, usually with significant contamination. Designing with monomeric species offers a much larger range of choices, perhaps at the cost of machine complexity. 4.2.6. Stoichiometry An economic disadvantage of polymers is the stoichiometry of their production. Conditions that drive polymerization to completion include substantially raising the concentration of monomers,
While it is premature to speculate on the details of a more comprehensive theory, I suggest one could be developed along the lines of Section 3. But perhaps the best test will be what will be required for us to believe the result of a molecular computation for which we do not have, or cannot determine, the result of a corresponding electronically implemented algorithm. A more comprehensive theory which allows us to ask how correct a probabilistic algorithm and how reliable a stochastic machine are will be important in determining the credibility of that computation.
Acknowledgements The author thanks Denise Beusen, Michael Conrad, Mike Foster, Max Garzon, Al Globus, Tom Head, Gerald Johns, Deborah Joseph, Laura Landweber, Dana Latch, Andrew McDonald, Webb Miller, Fred Rosenberger, Gregorz Rozenberg, Harvey Rubin, Mike Steuerwalt, WaiMo Suen, Keith Tipton, Bill Wise, David Wood, and Zeke Zalcstein for provocative questions and error-trapping. I am infinitely grateful to Tom Blackwell for help with derivations (he is not responsible for my stubborn insistence on mathematically less elegant but chemically more meaningful forms) and to he and Will Gillett for help in translating across disciplines. This work stems from work supported by NSF (IRI-9117005), NIH (GM-56529-01), and the Institute for Biomedical Computing.
References Astarita, G., 1989. Thermodynamics. Plenum Press, New York.
122
T. Kazic / BioSystems 52 (1999) 111–122
Blinder, S.M., 1969. Advanced Physical Chemistry. MacMillan, London. Brutlag, D., Fry, K., Nelson, T., Hung, P., 1977. Synthesis of hybrid bacterial plasmids containing highly repeated satellite DNA. Cell 10, 509–519. Casti, J.L., 1992. Reality Rules, vol. 1. Wiley, New York. Dawson, R.M.C., Elliott, D.C., Elliott, W.H., Jones, K.M., 1986. Data for Biochemical Research, 3rd ed. Clarendon Press, Oxford. Fallis, D., 1997. The epistemic status of probabilistic proof. J. Phil. 94, 165 – 186. Garey, M., Johnson, D., 1979. Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman, San Francisco. Illingworth, V., Glaser, E.L., Pyle, I.C., (Eds.), 1990. Dictionary of Computing, 3rd ed., Oxford University Press, Oxford. Jost, W., (Ed.), 1971. Thermodynamics, vol. 1 of Physical Chemistry. An Advanced Treatise, Academic Press, New York. Karplus, W.J., Soroka, W.W., 1959. Analog Methods. Computation and Simulation. McGraw-Hill, New York. Kazic, T. 1999. After the Turing Machine. Technical report, Institute for Biomedical Computing, Washington Univer-
sity, St. Louis, MO. http://www.ibc.wustl.edu/klotho/papers/techl.ps. Kornberg, A., 1980. DNA Replication. W.H. Freeman, San Francisco. Landweber, L.F., Lipton, R.J., 1997. DNA2DNA computations: a potential ‘killer app’? In: Degano, P., Gorrieri, R., Marchetti-Spaccamela, A. (Eds.), Automata, Languages, and Programming: 24th International Colloquium, ICALP97, number 1256 in Lecture Notes in Computer Science. Springer Verlag, Berlin, pp. 56 – 64. Minsky, M.L., 1967. Computation: Finite and Infinite Machines. Prentice-Hall, Englewood Cliffs, NJ. Misner, C.W., Thorne, K.S., Wheeler, J.A., 1973. Gravitation. W.H. Freeman, New York. Moret, B.M., 1998. The Theory of Computation. AddisonWesley, Reading, MA. Post, E.L., 1936. Finite combinatory processes. Formulation I. J. Symb. Log. 1, 103 – 105. Smith, W.D. 1993. Church’s thesis meets the N-body problem. Technical report, NEC Research Institute, http:// www.neci.nj.nec.com/homepages/wds/church.ps. Turing, A.M., 1936. On computable numbers with an application to the Entscheidungs problem. Proc. Lond. Math. Soc. 42, 230 – 265.
. .