Chapter 4 Kolmogorov and Martin-Lof's Complexity Theory

297

CHAPTER 4

4. KOLMOGOROV AND MARTIN-LOF’S COMPLEXITY THEORY

This chapter is devoted to the complexity analysis of “algorithmic descriptions”. Due to the fact that in what follows the input/output processes play the decisive r&, we are dealing with string-functions (over a non-binary alphabet). This study hae led to a satisfactory definition of random strings, which turn out to have many interesting applicationa.

4.1. EXAMPLES

In this section we present two problem which motivate the complexity-theoretic approach to random atrings, namely the opthisation of program length and the probabilistic tests of primality (with an application to cryptosystems). Paradoxes often turn out to be an important source of fertile mathematical ideas. That is why, we begin the analysis of the optimisstion problem below by presenting the famous BERRY paradox (see, for example, VAN HEIJENOORT [1967]). Consider the smallest natural number that cannot be defined by an English phrase whose length is less than lo3 characters. Let us suppose that n, is this smallest natural number and that the defmition of n, hm at lead lo3 characters. It is easy to see that, in fact, nocan be defmed by the following E n g h h phrase having much less than los characters: “the smallest natural number that cannot be defmed by an English phrase with less than lo3 characters”. Consequently, the natural number no cannot eziut!

Someone may argue against the linguistic and mathematical correctness of our former defmition (for an interesting discussion see BOREL

298

Cduda

[1946]). To avoid criticism, one can employ a formal axiomatic system having precise specifications. The problem of checking if a proof written in this system is correct is completely mechanical (for i=1,2, generate all strings of length i in the finite alphabet of the system and check for every such string whether is a correct proof; print all theorems of the system, i.e. our defmitiona, whose proofs were found), and consequently, it can be simulated by a computer (CHAITIN [1974)). In what follows we shall describe thin problem in detail (see aleo, DAVIS [1978], MACHTEY and YOUNG [1978]). We shall use a sufficiently strong programming language PL in order to give precise mathematical defmitions of natural numbers. A definition of a natural number n is a pair D = (P,m), where P is a (correct) program in PL and m is a natural number subject to the following condition: the program P accepts m as input and after running on m it prints n and halts. Clearly, every natural number n has at least a definition. Take, for example, D, = (P,,n), where P, is the program

...

BEGIN READ n PRINT n STOP T h e length of a defmition D = (P,rn) is the sum of the number of characters of program P and the characters number of the input data. For example, the length of D, is 20+[log,n]+l, where p is the base in which n is written. Consider now the following optimization problem: Find the shortest definition for every natural n. Every natural n has at least one defmition and there exist natural numbers VL which have significantly shorter defmitions than D,. For example, let us take n = p', for some k 2 35, and consider the program R, BEGIN READ k i = l

n = p

WHILE i

PRINT n

# k

DO: n = n . p , i = i + l

STOP

Clearly, (R,,,k) is a defmition of n w h i h is shorter than D,, since A < p'-=, for every k 2 35. Let us SUPPOS~ that PL satisfies the following three conditions: PL is aufficiently strong in order to:

Chapter 4

299

a) contain a program which, for every given program P in PL, computes the number of characters of P and halts, b) accept subroutines, and c) perform some basic algorithmic operations

(including WHILE constructions), PL uses a fmite alphabet (consequently,

(1.1)

the number of correct programs in PL having a fued length is finite),

(1.2)

PL works with natural numbers written in a base p

2. 2

.

(1.3)

Each natural number has at least one shorter defmition, though it may well have several ones. So, the optimisation problem has at least one solution which, by (1.2) should be searched for in the fmite set of all defmitions of shorter length than the length of 0,. It seems quite natural to ask whether the optimal solutions (or, a t least, one such solution for every natural n) could be algorithmically obtained, i.e. by using a program in PL. We shall prove that under the hypotheses ( l J F ( l . 3 ) no such program exists. Suppose, for the sake of a contradiction, that there exists a program P in PL such that for every natural n, 4.) gives as output a shortest defmition of n and halts. Let us denote by (fln),rn(n)) the defmition f l n )*

For every natural t

2 1 we construct the program Qr as follows:

BEGIN READ t y=o z=o

WHILE z < t DO: CALL fly),

PRINT y STOP

z = length ( f l y ) ) + length (m(g)), y = y+l

In view of (1.1) and (1.2) it follows that Q, ia a correct PL program. For distinct naturala t , and t,, the corresponding prograxm Qt, and Q,, diifer with respect to the input data and the condition tested in the WHILE atatement. Consequently, there exists a constant c > 16 (which

300

Cdude

depends upon the programs in (l.l),a) and b)) such that for every natural t 2 2-e we have: length ( Q t ) = length ( t ) + e = [log,t+l]+c

< e+log,t+2

,

and, consequently, defmition ( Q r , t )is shorter than t , length

(8,)< t

.

In view of (1.2), the program Q, prints the smallest natural number requiring a defmition of length greater than t . This is a contradiction, becauae there exists already a defmition of it shorter than t . The above example suggests the possibility of measuring the complexity of fmite objects (natural numbers, strings etc.) by means of their shortest defmitions induced by a fmed computing device. The reader will a h fmd this idea in the next section. Recent results in approaching the complexity of concrete classes of problems have led to an important distinction between problems having a potynomial time algorithm and those requiring exponential running time. It is largely agreed that problems requiring exponential running time are intractable. Yet, problems for which no polynomial algorithm is known can sometime8 be solved by means of non-deterministic polynomial algorithms. The non-deterministic algorithms are very useful for theoretical purposes, but they are rather incongruous in practice. Furthermore, the problem of transforming non-deterministic polynomial time algorithms into the equivalent deterministic ones is open. In contrast, a new class of algor i t h m running in polynomial time has appeared: the clam of probabilistic algorithms. The probabilistic methods have been used for several decades (Bee

PAZ [1971], and the bibliography JANKO [1982]). The interest in these

new algorithm has increased when difficult problems turned out to be solvable by probabilistic algorithms that are working faster than the known deterministic ones. A typical example is the factoring problem. The complexity of factoring natural numbers is unknown (the best of the steps, for all large n a t u r b present algorithms work in (log2m)c'-*m m, where c ia a constant, NILEMAN, POMERANCE and RUMELY !1983]; thia means that within fdteen seconds a etrong computer can check the primality of a fifty-digit natural number, LANDAU (19831). MILLER [1976], using the Extended R I E M A " Hypothesis (ERH), has d e v k d a polynomial algorithm to test primality. More exactly, let n and b be natural numbers, such that 1 5 6 < n. Denote by M(6,n) the

Chapter 4

301

b”-’ fl(mod n), or

(1.4

following condition: rn = (n-1)2+ is natural and 1 < gcd(bm-l,n)

< n, for some natural i .

(1.5)

If M(l,n) holds, for some b , then n must be composite. On the bagis of ERH, there exists a constant c such that if n is composite, then we can for which fmd a natural 6 satisfying the inequalities 1 5 6 5 c M(b,n) holds. The number b is called the witncer of the compositeners of n. RABIN 119761 has shown, without ERH that, when n is compoaite at lesst half of the natural numbers 1 5 b < n are witnesses of the compositenem of n. SOLOVAY and STRASSEN [1977] constructed a probabilistic algorithm for testing primality by means o,f the JACOB1 symbol j(b,n). For every natural b , 1 <_ 6 < n, set 6 = J(l,n) (see RIVEST, S H A M I R and ADLEMAN [1978] for an efficient algorithm that computes j(b,n)). Assume that b is relatively prime to n. If n is prime, then

6 =- b(”-’)D ( mod n)

.

(1.6)

Since, for a composite n, the set of all naturals b, 1 2 6 < n, sstisfying (1.6) is a proper subgroup of unite of Z,,, it follows that at leaat half of the naturals b smaller than n and relatively prime to n do not satisfy (1.6). The SOLOVAY and STRASSEN probabilistic algorithm runs k independent trials. If for some trial the answer is “composite”, then the number n ia declared composite (and in this caae the output i s correct). In the opposite situation, the natural n is declared prime (and the probability that the output is correct is greater than 1-2-’). The general form of the MILLER and W I N , reepectively, SOLOVAY and STRASSEN probabilistic algorithm is the following. Take k naturals uniformly distributed between 1 and n-1, inclusively, and for every trial b, check the validity of some fmed predicate W(b,n). If for some b, W(b,n) holds then n is composite. If not, n is prime with p r o b e The predicates W(b,n), which are ddferent for bility greater than 1-2-‘. the MILLER and W I N , respectively, SOLOVAY and STRASSEN probabilistic algorithms, can be evaluated by polynomial algorithms. These probabilistic algorithms act deterministically, but they use a “random device” a coin flipping, for example in order to obtain, a t mme moments, “an auxiliary entry”. The given output is only probably correct; neverthekss, the correctness probability is sufficiently large. The study of the correctnetw of thew probabilistic algorithm requires an adequate notion of

-

-

302

Cdudc

randomness; some related results will be presented in Section 4.8. We end this example with an application in cryptography which will show the practical significance of the above probabilistic algorithms (see RIVEST, SHAMIR and ADLEMAN [1978], LANDAU (19831). In short, the idea is that on the basis of the diificulty of factoring large naturals a cryptosystem can be constructed, in which the encryption method is “public”, but for possible interceptor, decryption is very dBicult and it could take years to get. Such a mechanism ia called a f i b l i e - K e y Syetem.

RIVEST-SHAMIR-ADLEMAN’smethod use8 EULER’s phi-function

<

(#(m) = card{n E PV 11 5 n m,ged(n,m) = 1)) and the fact that calculating d(m) ia polynomial time equivalent to factoring m. Each partici-

4

pant in the cryptosystem fmds two large primes (each having about fdty digits) p and q, and a natural b, 1 5 6 < pg, which is relatively prime to d(pq) = (p-l)(q-1). Such a 6 does exist since most natural numbers smailer than pq are relatively prime to #(PQ). Every participant in the cryptosystem appears in the Public-Key Book with the pair (b,n), where n = pq. The encryption of a message is given by the following algorithm: 1. Translate the message into natural numbers.

2. Break the resulting message into blocks of convenient length. 3. Transform each block into (block)’( mod n).

The decryption of a message can be done by the following algorithm:

I. Compute the integers z and y , such that bz++(n)y = 1. 2. Break the message up into blocks. 3. For each block, compute (block)'( mod n). 4. Glue the block8 back together.

5. Decode the resulting message.

The decryption algorithm is correct because from the relation

ged(b,+(n)) = 1 it follows that we can (eaaily) fmd the integers z and y, such that bz+g)(n)y = 1 (for example, by EUCLID’s modified algorithm in

MARCUS [1981]). Consequently, ((block)’)’

3

(bl~ek)’~

= (block)’*”)’ E

(block) (mod n)

,

by EULER-FERMAT’s Theorem (For every integer m and every natural u which is relatively prime to rn we have: ad”’) E 1 (mod m).). The decryption is “feasible”, for it takes polynomial time to compute

Chapkr 4

309

z if one disposes of +(n). The ddference between the addrewee and a possible interceptor lies in the former knowing #(n), while the latter knows only n (which is the product of two fdty-digit primes); factoring such a large natural number is a t present an intractable problem.

4.2. KOLMOGOROV’S COMPLEXITY

In this section we develop a complexity-theoretic method for defming “random strings”. This approach waa initiated independently by SOLOMONOFF [1964], KOLMOGOROV [1965], and CHAITIN [1966]. The adequacy of this method ia mainly due to MARTIN-LOF[1966a], [1966b]. We begin with an illuminating example. (2.1) Example.

(MARTIN-LOF [1966a]) Consider the following four binary strings of length 32: z = z y = 10011001100110011001100110011001 u = 00001001100000010100000000100000 u = 01001110100111101000001100101101.

Suppose that a random device produces reros and ones with probability 1/2. According to classical probability theory the strings z , y, u and u are equally probable (i.e., the probability of each is 2”*). We can compare the above strings from another, totally different, point of view, namely “regularity”. The string z is of maximum regularity, hence its information content is lower. It in easy to see the regularity involved in the string y: eight substrings of the form 1001. As concerns u and u , we can hardly specify a defrnite regularity. Nevertheless, a deeper analysis would point out a drastic difference in the structures of u and u. To see this let us return to the strings z and y and kt us defme them in a shorter way: z can be described by the English statement “only scrod”, y can be defined by “eight strings 1001”. In order to distinguish the strings u and v , we order the binary strings of a given kngth according to the increasing frequency of the ones and within c b s of equal frequency, in the lexicographical order induced by the relation 0 < 1. We define a string by its number in the above enumeration. The number of a string of length n having a small frequency of ones (i.e., m/n 5 1F,where m is

304

Cdade

the number of ones) requires approximately n*H(m/n) binary digits, H is the binary entropy function: H ( 0 ) = 0, where H ( t ) = -t.loglt-(l-t)*log2(l-t), 0
,

binary digits, which is smaller that n for small values of the quotient m/n. Note that u is such a string (8/32 < lb);consequently, u haa a shorter defmition. In contrast, the string v appears to have no shorter defmition. It is only one of the four strings which would be regarded as being the output of a random experiment which gives preference to neither ieros or ones. These examples point out a closer relation between complexity and randomness. Moreover, the length is not a decisive factor in this analysis. For example, the string u (or even, the string u ) is more complex than the string a consisting of 40 s e r a . The last remark suggests that the complexity basically refers to cognition processes (see also LOFGREN [1977]). KOLMOGOROV [1965] has defmed the complexity of a given string z with respect to the algorithm 4 as the length of the ahortest program which computes it. More formally, let us flx a fmite alphabet

x= with p

{Q1lQ2,...,QI)

1

2 2 elements, and consider a p.r. function

g:x*xm-Qbx* .

(2.2) Definition. The KOLMOGOROV complczity induced by

function

K + : X *x N

-+

JV U

{m}

4 ia a

,

defmed by the formula

(X.

m w r ) Iv EX1,9(y,m) = 4 , in ca&e 2

KdZ

Im) =

for every z in X * and m in

=

d(v,m),

for some y in X * otherwise,

N.

,

306

Chapter 4

(2.8) Example. Let 4:X*x R'V + X* be the recursive function defmed by #(z,rn)= z, for every z in X* and rn in N. A simple computation shows that

K+(2 Im) = t ( z ) for every z in

X* and

m

in

,

RV. 0

(2.4) Example. Let f:Hv a X * be a p.r. function. Asaociate the p.r. x RV *X* given by t$,(z,rn) = f(rn), for every z in X* function +,:X* and rn in N. (Clearly, if f(m) = q for some rn in RV, then #,(z,rn) = q for every z in X*.)If z = f(rn), for Bome rn in R'V, then t$,(A,rn) = 2. Consequently,

i

0 , if2 = f(rn),

K+,(z Irn)

=

00,

otherwise.

(2.6) Example. Consider the p.r. function

in case t ( z ) = rn ot herwiee.

,

[(z), i f t ( 2 ) = rn 00 , otherwise.

,

z d(z,m) =

Then

K + ( z Irn)

=

+:X*x Hv A X *given by

i

(2.6) Example. (MARTIN-LOF [1966a]) Take p = 2, a1 = 0 and We defme a p.r. function #:X* x R'V *X* in this way: #(z,n) is the kth element in the enumeration of the binary strings of length n intrduced in Example (2.1), where k ie the natural number whose binary expansion is 2. Let m(z) be the number of ones in 2. Then, for every z a2 = 1.

in x*,

K,(Z

when m(z)/t

(2)

IW)

W.H(m(z)/W)9

5 lfi. 0

308

Cdude

(2.7) Lemma. Let

+:X*x IV a x ' be a p.r. function. Then:

For all natural numbers n and r we have: card{z

EX*l t ( 2 ) = A , K,(z

For all n and m in card{z

N ,if

n

In) = r }

5 p'

.

(2.1)

2 m, then

EX*l t ( z ) = n,K+(z In)

< n-m} 5 (p""-l)/(p-1)
t

(both equalities can occur).

Proof. (2.1) Let A = {Z E X * l t ( z ) = n, K,(Z In) = r). Assume A (the case A = 0 is obvious).

z0

We defme the auxiliary function D:A-,~x* by D ( 4 = {ar E X * It (u) = r , db,t (2)) = 2). For every z in A we have D ( z ) # 0; moreover, if z1 # z2, then D(zl) n D ( z , ) = 0, so D ( z J # D ( z q ) . Since the function D is injective it follows that

cardA = cardD(A)

5carW

.

= pc

(2.2) The case n = m being obvious, assume n

(2.1), we have:

card{z =

> m.

In view of

EX*lt(z) = n,.K4(z In) < n-m}

n-m-1

card{z E X ' It (z)= n&,(z

In) = i}

id

5

c Pi

n-m-1 i4

= (p""-l)/(p-l)

. 0

For

all

c(n,m) = card{z

natural

numbers

n

EX* l t ( z )= n, K,(Z In) 2 n-m}.

and

m,

set

307

Chapter 4

(2.8) Corollary. Let 4:X' x RV ax*be a p.r. function. Then for all natural numbers n and m such that n 2 m, we have:

> P"(l-P-AP-1))

+,m)

20

(the last equality holds for p = 2 and m = 0). In particular,

> P"(P-2)/(P-1)

+PO)

20

(2.3)

9

-

(2.4)

Remarh. 1) Let 4:X' x RV S , X * be a p.r. function,'and let n and m be in RV such that n 2 m. I f p > 2 or m > 0, then limc(n,m) =

n-ca

00

.

The aesertion above fails to be true in caae p = 2 and m = 0. Actually, we shall present an example of a p.r. function 4:X' x RV having the property that e(n,O) = 1, for every n in RV. Let p = 2, a1 = 0, a2 = 1. Let < be the lexicographical order on X* = {0,1}* induced by 0 < l : X < 0 < 1 < 00 < 01 < 10 < 11 < 000 <... Recall that y(n) is the n t h string in the lexicographical order; it is seen that y(2"+'-1) = l", and y(2"+') = On+1. (Recall that for every a EX, a" = a...a, (n copies) for n 2 1, and ao = X.)

ax*

Let E(0) = {(A,O)}.

For every natural number R

E ( n ) = {(g(i),n) li=1,2,3 ,...,2"-1,2"+'-1} ina ally, put E

uE ( n ) c X*x N.

2 1 set

.

00

=

n 4

Now we defme by cases the announced p.r. function +:E + X*: a) For n = 0,

4(X,O)

=

A.

b) For n 2 1, d(y(i),n) = y(i+2"-1), 4(y(2"+'-l),n) = y(2"+'-1).

for i=1,2,3 ,...,2"-1,

and

A rapid examination of the above defmition tells us that for.every natural n 2 1, one hae

{z EX* It(%) = n&+(z In) 2 n } = {z EX*l t ( z ) = n&,(z

In) = n }

308

Cdudc

.

= {y(2"+'-1)}

Hence c(n,O) = 1. This fact shows the existence of a drastic distinction between the binary ease and the non-binary e a ~ e s .

2) From the inequality (2.4) we deduce that for every p.r. function

d:X*x N A X * and for every n in N ,there exists a t least one string z in X*such that l ( z ) = n and K,(z In) 2 n. (KOLMOGOROV [1965]) There exists a per. function A X * with the following universality property: For every p.r. function # : X * x RV a x * , we can effectively fmd a natural e (which depends upon w and 4) such that (2.9) Theorem.

w

:X*x lN

K,(z for every z in Proof.

Let

I4 I K,(z

Im)+e

(2.5)

9

X*and m in RV. us

+i:x*x RV ax*.

consider

an

acceptable

giideljsation

u*,

(#i)i

For further purposes we defme the following auxiliary (primitive) recursive functions: 1) T : X * -+ X * , T(X)= a1a2, and for every z EX*, z = z1z2...2,, with n = l ( z )2 1, T(z)= 21212222...2,z,a1a2, 2)

f:X'+x*,

f ( z )=

and 3)

g:x*+x*, !7(4 =

1 1

z

, if z = T(z)y, for some

X

,

y

, if z = T(z)y,for some

X

,

in X * otherwise, y and z

in otherwise.

y and z

,

X* ,

Notice that the last two recursive functions are well-defined because the representation z = T(z)y is unique (when it holds). Let & : X * x (X*x N )A X * be an universal p.r. function, i.e. for all i,z in X*,and every m in N ,we have: #i2iv(i,(z,m)) = di(z,m). The p.r. function w :X*x N a x x will . be defined by the formula

309

Chapter 4

w (2 ,m =

4Eie ( f ( ~ ) , ( g ( ~ )1), m *

Now let 4:X' x PJ ax' be a p.r. function and let i be an index for 4, i.e. d = bi. Aseume that K& Im) = t < cq i.e. there elrista a string y in X ' with [(y) = t and #(y,m) = 2. Take z = T(i)y and compute: w(z,m) = w(T(i)y,m) =

4Eie(f ( ~ ( i ) y ) , ( g ( ~ ( i ) y ) , m ) )

= 4%e(it(y,m)) =

6i(r,m) = 2

and

e (2) = e ( T ( i ) y )= e (y)+2([

(i)+l)= t+2(! (i)+l)

Set c = 2(e (i)+l). We have proved that

K,(z Im) 5 e ( 2 )

= tfC

= K,(z Im)+e

.

.

Remark. The constant e appearing in the asymptotic relation (2.5) depends, in fact, upon the gcdelisation (di)=* and #.

(2.10) Definition. A p.r. function w :X' x PJ 4X' which satisfies the asymptotic relation (2.5) will be called an universal KOLMOGOROV algorithm. I n the following we shall choose a fized universal KOLMOGOROV algorithm w and we ehall write K ( z Im) inetead ofK,(z Im). (2.11) Corollary. There exists a natural number e (which depends upon the fixed universal KOLMOGOROV algorithm w ) such that for every z in

x*.

K(. It (4)I e(4+C

(2.6)

9

Proof. Take 4:X' X PJ -+ X' to be the recursive function in Example (2.3), i.e. t#(z,rn)= 2, for every z in X ' and m in PV. We have already noticed that K 4 ( z lt(z)) = e(z), for every z in X'. In view of KOLMOGOROV's Theorem (2.9) we have:

K ( z I+))

5 K,(z

(!(z))+e

=

for iome natural number e and every z in X'.

!(Z)+C

,

310

Cdudc

Remark. Corollary (2.11) enables us to claim that in case p EH of natural numbers, exists a eequence

>2

there

,

n
such that

Indeed, we use the equality (Z

EX*

= ~ , K ( zIn)

U{ZE X * I!(%)

2 n)

C

=

,

= n,K(z In) = n + i )

id

and Corollaries (2.8) and (2.11) to fmd, for every n in {n,n+l, ...,n+c} such that number m, card{z

EX' le(z)

RV,

= n&(z In) c- mn}2 pn(p-2)/(p-l)(c+l)

a natural

.

(2.12) Corollary. There exists a natural number q (which depends upon the fmed universal KOLMOGOROV algorithm) such that for every z in X' there exists a natural number m such that

KbIm)Gl

-

(2-7)

Proof. Let f :RV + X * be a recursive bijection and let qj,:X' x A' + X* be the recursive function defined by (p,(z,rn) = f(rn). In view of Example (2.4) we have: Kd,(z ~ m =)

k,

if

f(m) =

otherwise.

2

,

By KOLMOGOROVL Theorem (2.9) the existence of a natural number g follows, such that for every z in X * there exists an m in IV (i.e. the unique m such that z = f (m)), such that

K(.

I 4 I K,,(z

Im)+cl = Q 0

311

Chapter 4

(2.18) Deflnitlon. (KOLMOGOROV [1966]) A string z in X*is called a (KOLMGOROV) random string provided

(2.14) Theorem. For every natural number n there exists a random string of length n.

Proof. In view of Corollary (2.8), with card{z

w

instead of 4, we have:

EX*I t ( %= ) n , K ( z In) 2 n }

> P”(P-2)/(P-l) 2 0

-

Remarks. a) The defmition of random strings depends upon the chosen universal KOLMOGOROV algorithm w

.

b) The random strings are those elements of X* of maximal complexity with respect to w . c) In Sections 4.3 and 4.4 we shall prove the adequacy of KOLMOGOROVL defmition of randomness, both from the statistical and the recursion-theoretic points of view. Notation. Denote by R A N D ( w ) , (or, shortly, dom strings.

RAND)the set of

all ran-

(2.16) .Theorem. The number of random strings of a fmed length increases “proportionally” to the number of elements in the alphabet X.

Proof. By Corollary (2.7), with w instead of 4, we have: card{z

EX*l t ( z ) = n,K(z

Comment. If p are random.

In) < n}*p”

< l/(p-l)

.

> 2, then more than half of the strings of a given length

(2.16) Deflnitlon.

a) Let m be a fmed natural. We say that the string z (KOLMOGOROV) m-random provided

EX* is

312

Cdudc

.

K ( Z l e ( 2 ) ) 2 +)-m

(2.9)

b) A string in X* M said to be (KOLMOGOROV)ueymptotic random provided it is rn-random, for some natural number m. Random strings are precisely the &random atrings. Notation. Denote by RAND,(w), (or, shortly, RAND,) the set of all m-random atrings. (2.17) Theorem. For all natural numbers n and rn auch that n there exists an m-random string of length n.

Proof. In view of Corollary (2.8), with

w

> m,

instead of $, we have:

card{z E X * lt'(z) = n&(z In) 2 n-m}

> pn(l-p-/(p-l))

20 . 0

Comment. The proofs of Theorems (2.14) and (2.17), which rely on the proof of Corollary (2.8), are non-constructive. In Section 4.4 we shall prove that there are no (uniform) constructive proofs for these existential

theorems. Actually, a bit stronger version of Theorem (2.15) can be stated: (2.18) Theorem. For every natural number rn m-random.

2 1, almost all strings are

Proof. Again, by Corollary (2.8) we have: card(z

EX*lt(z) = n&(z

In) >_ n-m}/p"

> 1-p-/(p-l)

.

Remark. (DAVIS [1978]) Take p = 2, a 1 = 0, a2 = 1, and m = 10. Then, more than 99.8% of all strings of length greater than n-10 are m-random.

313

Chapter 4

4.8.

MABTIN-LOF TESTS

In this section we study the statbtieal propertier of KOLMOGOROV’s (asymptotic) random strings. We prove that KOLMOGOROV’s (ssymptotic) random strings poaaew, in a sense, all poi+ sible properties of stochasticity. (8.1) Example. (MARTIN-LOF [1966a], [1966b]) Consider the binary case, i.e. p = 2, a1 = 0, a2 = 1. h u m e that we hare a test of randomness for binary strings which rejects relative frequencies of ones differing too much from the expected value 1/2. In other worda, if z = z122...z2,is a binary string and an =

czi,then our test of randomness rejects the n

ill

hypothesis when

bta/n-lFI

9

is too large. Since we are interested in the magnitude order of the significance level, we may reatrict our attention to a discrete set of critical values (Sicherheitawahrscheinlichkeit (MARTIN-LOF [1966a], p.3.9)) c = 2-’,2-2

,..’,2- ,....

Consequently, the above MARTIN-LOF test rejects the hypothesis of randomness on the level E = 2- provided l8Jn-lF

where f :RV x RV tione:

+

I 2 f(m,n)

(34

9

N hi a function satisfying the following two condk

The number of strings of length n for which the the inequality (3.1) holda is less than 2”-’

.

One cannot diminish f without violating condition (3.2).

(3.2)

(3.3) 0

The above example euggesta a general notion of randomnetm teat. In order to get this defmition we establish a useful notation: For every set V c X*x (RV;{O}) and for every natural number m 2 1, denote by V, the set {z EX I(z,m) E V), that is the inverse image of the m t h projection.

314

Caludc

(8.2) Deitdtlon.

(MARTIN-LOF [1966b],CALUDE and CHITESCU [lSSZa])A non-empty r.e. set V C X* X ( N - ( 0 ) )is called a MARTINLOF test (M-L t e s t ) if it possesees the following two properties: For all n and m in dv, m 2 1, we have:

v,,, c v m card{z E X * l!(z) = n , EV,} ~

9

< p”“/(p-1)

.

(3.4) (3.5)

We agree upon the fact that the empty set in a M-L test.

Comments. (Motivation of Defmition (3.2)) a) The choice of a strict inequality in condition (3.5) is motivated by Example (3.3), which in turn relies on the strict inequality in Lemma (2.7). b) Each test of randomness must be capable of being effectively specifled before the running experiment; it follows that such a teat must be a r.e. set (this is the weakest constructive restriction). c) Condition (3.4) can be regarded M an “efficiency” requirement.

Remark. For every M-L test V and for every t(z)>m?l (8.8)

Example.

d:X*x Bv * X * be

(2,“)

.

in V one has

(3.6)

and CHITESCU [1982a]) Let function. We claim that the set V(4) defmed

(CALUDE

a p.r.

~ ( d =) {(z,m) Iz EX*,^ E N , m 2 1, and K,(z (!

(2))

< 1 (2)-m} ,

(3.7)

is a M-L test.

First of ail we see that V(+)is a r.e. set. Let UE consider a (primirecursive enumeration of X*x RV given by the bijection e. :N X* x N. At etep i let e ( i ) = (2,m). Check if the finite set

tive)

A = (1 E X * l W < W - m , d ( u , W ) = 21 , is empty. Thin can be done by dovetailing. Let the algorithm which computes the p.r. function 4 run one step for all inputs (g,t(z)), with t ( y ) < !(z)-m. If for some such y we have #(y,P(z)) = 2, then the set A is non-empty. If not, let the algorithm run the second etep for all inputs M before. Again, if for uome y we have d(y,t(z)) = 2 , then the set A is non-empty. If not, p m to the next step, a.a.0. If A in non-empty, accept the pair (z,m). The set V(4)C O M M ~ U of all accepted pairs (z,m). For z in (V(+)),+,,we have K,(z lt(z))< 1(z)-m-l < l(z)-rn,

315

Chapter 4

thus proving that z is in (V($)),,,. Fmally, by Lemma (2.7), we have:

E X *le(z) = n

(8.4) Example. Let z be in t ( z ) > m 2 1. Then the set

, E~ (V(4)),,,)

X* and

m

in N. h u m e that

H(z,m) = {(Z,l),(Z,2),'..,( z t m ) )

,

(3.8)

is a M-L test. The only condition in Defmition (3.2) which must be checked ia (3.5). We have:

card{y E X * l l ( y ) = n , ( z , q ) . E H ( z , m ) )

=i

1 , i f n = t ( z ) , and 1 0 , otherwise,

< P"'/(P-1) because l ( z )> m 2 q (i.e.

q

5m,

9

p"'/(p-l)

> 1). 0

(8.6) Example. Let z be in X* and m in N. Assume that [(z)> m 2 1. Then the set i ( ( z , m )=

{(w) lo EX*,%E N , l I 12 I m,y 3 z} ,

(3.9)

is a M-L teat. (Recall that the relation y 2 z means that y = zz, for some z in Clearly, F(z,m) ia a r.e. set (in fact,a primitive recursive set). TO prove condition (3.4), let (y,n+l) be in H(z,m), i.e. 2 5 n + l 5 m and y> z;it follows that 1 5 n 5 m, and y 2 z,which shows that (g,n) is in H ( z ,m 1. Finally, let n and q be in RV such that q 2 1. We have:

x*.)

card{y E X * l q Y ) = n,(u,q) E@z,m))

318

Cdudc

Remarks. Let (z,m) be in are equivalent: a)

i) [ ( z )

X* x ( N - ( 0 , ) .

Then the following statements

> m 2 1,

a M-L test, iii) H(z,m) is a M-L test.

ii) H_(z,m)is

From Exampleel3.4) and (3.5) we know that i) implies both ii) and iii). If H ( z , m ) (or, H ( z , m ) )is a M-L test, then 1 < p'(')-/(p-l), for every 1 5 q <_ rn, i.e. P(z) > rn 2 1. b) For all z in

X*and

m in

Ehm)=

N-{O} we have:

u

U

H(y,m)

3 '

We finish the presentation of the examples of M-L tests with a nonrecursive M-L test. (8.6) Example. Take A c {a,}* = {X,al,a~,...,a; ,...} a r.e. but not recursive set (see Example (2.5.16)). Then V = (A-{X,al}) X {I} is a nonrecursive M-L test. (The condition (3.5) is fulfilled because the left member of the inequality is always less than or equal to 1.) 0

In order to obtain the main enumeration theorem we shall give the following lemma.

317

Chapter 4

(8.7) Lemma. There exiets a p.r. function

f:(N-{o})2 ax*x

nv ,

with the following two properties: For all natural numbers i and j such that j(i,j)# oq we have f ( i , k ) # oq for all k

Aset A

c X'

.

5j

(3.10)

x N is r.e.

iff

A = { j ( i , j )lj=1,2 ,...}-{.0},for some i

21

.

(3.11)

Proof. Let (fp))im-j0) be an acceptable gEidelisation, where + / ' ) : I V - { O } a X x N. We defrne the p.r. function

f:(RV-{o})2 by the formula:

I,

ax' x nv ,

&!io(i,j) 1 if 4E!ie(i,k) for all k 5 j f( i d=

where

4gi0is an universal p.r.

otherwise,

function for

+ 04 ,

(+/'))i E~-{o).

Let us notice that among the p.r. functions h i : N - { O } A X *X RV, i EN-{O}, hi(j) = f ( i , j ) , for all j in RV-{0}, we can fmd all recursive functions as well as the empty function, thus ending the proof. 0 (MARTIN-LOF [1966b], CALUDE and CHITESCU [1982a]) The set of all M-L testa is r.e. More precisely, there elrists a r.e. set T C Bv x X*X N auch that for every V C X*x Bv the following equivalence holds: V is a M-L test iff V = {( z, k) ] ( i , z , m )E T), for some i in N. (8.8) Theorem.

Proof. Throughout the proof we shall constantly use a fmed pa. function given by Lemma (3.7). Thia allows us to write 4 instead of {f (i ,j)li=1,2,...}-{4. We display a procedure of ge:eration, baaed upon Lemma (3.7), which constructs all r.e. subsets of X x IN and modfier only those sets which are not M-L tests (more exactly, makes empty all r.e. eta which are not M-L tests). Roeedute of constructing the eeetion of T, i.e.,

318

Cdudc

z = {(z,m)I(i,m,z) E T), for some f i z e d natural number i 2 1: 1. Put z = 0. 2. P u t j = 1.

If f(i,j) = + then continue indefmitely. 4. Compute f(i,j)= (zj,mj). 5. If l(zj) <_ mi, then put Ti = 0 and STOP. 6. P u t Ti = Ti U H(~j,mj). = 0 and STOP. 7. If Ti is not a M-L test, then put 3.

8. P u t j = j+l and go to step 3.

Comments. a) The test in step 3 is not recursive. But, in case f(i,j) = oq then the algorithm computing f will never atop, i.e. it will continue indefmitely, remains unchanged. If f ( i , j ) # cq then the algorithm computing and f will eventually give aa output the pair (zj,mj). b) In view of the Remark a) following Example (3.5), the step 5 can be considered a preliminary rejection, because if H(z,m)is included in aome M-L test, then H(z,rn) is a M-L teat itself. c ) An concerns step 7, it is obvious that the problem of deciding if for all natural numbers j , m1,m2,...,mi, and for all strings z1,z2,...,2,’ the fmite set

i-1

is recursive. Indeed, condition (3.4) is obviously fulfilled for and condition (3.5) must be checked only for 1 5 rn 5 max(m1,m2 mi) and z in the set {z1,z2,...,2 , ) . The test in step 7 ia essential, because otherwise we can generate sets Ti which are not M-L tests. This fact relies on the possibility that a finite union of M-L tests of the form H(z,m) need not be a M-L test. See, in this respect, Example (3.9) and the following Remark (CALUDE and CHITESCU [1982a]).

is a

H,

M-L test,

,...,

d) The procedure either stops after a fmite number of steps or conthUe6 indefmitely. It stops only when the reetion G is empty; but, it may ia fmite. well continue indefmitely even when All that remains t o be proved can be divided into two steps: 1) For every natural number i , the set Ti is a M-L test, and 2) For every natural number i , if 4. is a M-L teat, then Ti = A,. (Recall that A, = {f(i,j) I j=1,2 }-{a}.)

,...

319

Chapter 4

1) The interesting case is that one when the procedure continues indefinitely (see the procedure and Comment d) above). Obviously, q is r.e. If (z,m+l) is in then ( z , m + l ) is in some H(zj,mj), therefore (2,m) is in H(Zj,rnj) c

z, z.

For the sake of a contradiction suppose the existence of the natural numbera m and n, such that

.

card{z EX* l t ( z ) = A , ( z , ~ E) q} 2 p""/(p-l) But,

q

t

5 cq

= UH(zj,mj), for some 1 5 t

where f(i,j)= (zjymj).

j-1

Hence there exist distinct strings ~,~,z,~,...,zof j~ length n, such that jl < j2 <...< j,, r 2 p""/(p-l), and m 5 min(mjl,mja mjr). In view of step 7 the set

,...,

jr

H

= IJH(zj,mj)

v

j-1

is a M-L test, consequently,

.

= n,(z,m) EH)< #"'/(p-l) ciud{z EX* It(%)

Since the set {z EX* le(z) = n,(z,m) EH} contains at least r elements it follows a contradiction. (i.e. zjl,zjs, zjr)and r 2 p""/(p-l),

...,

2) Notice that 4. = 0 implies stops). Aasume 4. z 0.

We prove that

f(i,j)= (zj,mj)*

4. c Ti, i.e.

q

=

0 (though the procedure

(zj,mj) E q,for every j

never

2 1 such that

i) For k=1,2 ,...,j , we have

(because the test in step 5 ie passed). Indeed, it is Been that zk

EB

It (2) = t (zh),(Z,mh)E Ai)

=

{z E X *

1

< card3 < p

therefore

because A, is a M-Ltest. ii) For every natural number t

(sk 1-k

AP-1)

9

9

2 j , the set

t

H = IJH(zi,mi)

9

i-1

is a

M-L test (step

7 is paesed). Indeed, it is Been that all the fmite sets

...a( zt,m,) are included in

H(zl,m1),H(z2,m2),

the

M-L t e n t 4.. Their

320

Cdudc

union H is also included in the M-L test tional propert7: (z,m+l) E H for every m

4. and

haa the following addi-

* (2,m)EH ,

2 1. Consequently, H is also a M-L test.

iii) Following i) and ii) we conclude that H(zj,mj) c Ti and no # 0.The reader will realire that we “rerorisation” may occur, i.e. have apparently proved more in order to avoid a possible “seroriation” given by steps 5 or 7. Finally we prove that Ti c A,. Let (2,”) be in $; there exists j 2 1 such that z = z,,m 5 mi, and f ( i , j ) = (zj,mj). Since A, is a M-L test and (zj,mj) belongs t o 4. it follows that (z,m) belongs to A, too. 0

(8.9) Example. A fmite union of M-L tests of type H(z,m) need not be a M-L test.

For instance, take p = 2, a 1 = 0, o2 = 1, z1 = 00, z2 = 01, and 2 3 = 10. Then H = H ( z , , l ) u H(z2,1) u H(z,,l) ia not a M-L test (it violates condition (3.5)). 0

Remark. Assume, for the sake of contradiction, that the procedure in the proof of Theorem (3.8) would contain only the steps 1,2,3,4,5,6,!, and p = 2, a, = 0, a2 = 1. Then take the p.r.. function h:{1,2,3} + X x RV defmed by h ( j ) = z j , j = 1,2,3, where the 2’s are those of Example (3.9). For a fmed f M in Lemma (3.7) there exists some natural number i such that h ( j ) = f ( i , j ) ,j = 1,2,3. The procedure (without step 7) will produce the non M-L test which is exactly the set H in Example (3.9). The next theorem is analoguous to KOLMOGOROV’s Theorem (2.9) expressed in terms of M-L tests. (8.10) Theorem. (MARTIN-LOF [1966b]) There exists a M-L test U with the following universality property: For every M-L test V we can effectively fmd e in PV (depending upon U and V) such that

vm, c u m for every natural number m 2 1.

9

(3.12)

321

Chapter 4

Proof. Let T be the r.e. set constructed in Theorem (3.8). Defme

u = {(Z,rn)

12

d , m EN-{0},

(i,z,rn+i)

E T,for some i > 0)

.

Clearly, V is r.e. We prove that it ie in fact a M-2, test. Firstly, take some (z,rn+l)in U and prove that (z,m) is a h in U. Our hypothesis is therefore that (z,m+i+l) ie in Ti. Hence (z,m+i) is in the M-L test q,i.e. (z,m) is in U. Secondly, for all natural numbers n and m with n 2 1 we have: card{z EX4It(=)=

E Urn}

A,Z

= card{z

EX*lt(z) = n, there exiata an i such that (z,m+i) E K }

< gp"++)/(p-l) it1

= p""/(p-1)2

SP*-/(P--l)

*

Finally, let V be an arbitrary M-L test. In view of Theorem (3.8) we find some natural number i such that V = {(z,m) 1% EX*,m E N-{0}, (i,z,m) E 2"). Consequently,

v,,

=

{z E X *I(i,z,n+i) E TI

c {z EX*I(z,m) E U } =

u,

.

The constant e is therefore equal to i,and this ends the proof. 0

The previous theorem enables us to give:

(S.11) Definition. A M-L test U satisfying condition (3.12) in Theorem (3.10) will be called an uniuereal M-L tcet. Comment. An universal M-L test possesses the following intuitive prperty: if a string is random with respect to that test, then it is random with respect to every conceivable test, neglecting a change in the level of significance.

322

Caludc

(8.12) Lemma. A M-Ltest V is finite iff the section V, is fmite.

Proof. We must prove that in case Vl is fmite, V is itself fmite. Assume V,# 0 (in the opposite case V itself is empty) and put

n = max(t(z) 12 E Vl). Take a natural number m 2 n. Then V , must be empty, because otherwise every z in V, would have e(z) > m (notice that V, c VJ On the other hand

V1 IJ Vz

>see>

Vn-1

and

(8.1s) Theorem. (CALUDE and CHITESCU [1982a)) Every universal M-L test U has all its sections U, infmite.

Proof. We shall exhibit an example of a W, infinite. Defme

w = {(%,ma)

12

EX.,m

M-L test W having all sections

2 1,z 3 uY+l} .

a) The set W is a M-L test. Indeed, it is obvious that W is recurm 2 1, Wm+l c W, sive, hence r.e. For every natural = {z EX*Iz 2 a;""}. To check the last property, we fu the natural numbers 1~ > m and we compute: card{z

EX*It (z)= n,z E W,}

= card{y E X * l t ( y ) = n-m-1)

- pn--l

< Pn-/(P-l)

*

b) Obviously, for every natural m

2

1, the section W, is infmite.

c) According to Theorem (3.10), there exists a natural number e such that W,,, C U,, for every m 2 1. Since W,+c is infmite, it follows that Urnis infmite. 0

We introduce another measure of complexity: the critical level induced by a M-Ltest.

323

Chapter 4

(8.14) Definition. (MARTIN-LOF [1966b]) ZRe critical level induced by a M-L teet V is the function

rnV:X*4v

,

given by max(m

2 1 Iz EV,), i f z E V , , otherwise.

Remarks. a) As in statistical practice, the critical level is the smallest level of significance (i.e., p") on which the randomness hypothesis is rejected. b) We have 0 5 mv(z) 5 t (z), for every z in X '. If z # X, then m v ( 4 < e (4. (8.16) Theorem. Let U be a M-L test. Then U ia an universal M-L test iff for every M-L test V there exists a natural number c (depending upon U and V) such that: (3.13)

Proof. Let U be 'an universal M-L test. We prove that the constant e in (3.13) can be taken to be the same as that one in Theorem (3.10), relation (3.12).

The result ia obvious in caae mdz) = 0. Consequently, aaaume mdz) #O, i.e. z E V,,,+). We must check the inequality (3.13) only in caae mv(z)-e

> 0.

In view of Theorem (3.10) we have: Vmy(r)= Vm~(s)-e+c C u m ~ ( r ) - e

Consequently, z ia in Um,,+]-c,i.e. mdz)-e

5 mu(.).

Suppose now that U is a M-L test satisfying condition (3.13). We prove that V,, C U,, for all rn 2 1. h u m e that for some natural rn 2 1, Vm+c# 0. Let z be in Vm+c. In view of Defmition (3.14), rnV(z)2 m+e, or equivalently, my(z)-e 2 m. According to (3.13) we have: mu(%)2 my(+

which proves that mv(z) 2 rn

2m

9

2 1, i.e. z E U,,,dsiC U,. a

In the follow'ng

we

shall ehoore a fized universal M-L test U and

324

Cdudc

rhell write m inatcad of my. The following result establishes an aaymptotic relation between KOLMOGOROV’a compkxity K (induced by a fued universal KOLMOGOROV algorithm) and the critical level m (associated to an univehal MARTIN-LOF test). we

(MARTIN-LOF[1966b], CALUDE and [1982a]) There exists a natural number q such that

(8.16) Theorem.

I5

le(Z)-K(z l+))--m(z) for all z in x*.

CHITESCU (3.14)

9

Proof. Firstly we prove the existence of a natural number c that

e (45 K(.

It (Z))+m(z)+c

2

1 such

(3.15)

1

for all z in x*. For this inequality we shall use the M-L test (3.3), with w instead of 4.

V(4)given in Example

Let ua notice that my(,,(.)

Assume

that

K ( 2 l e ( t ) ) + m(z)+l.

.

= 0 iff t ( z ) - K ( 2 le(z)) 5 1

rnq,,)(z)= 0.

Therefore

((2)

(3.16)

5 K ( z lt(z))+l 5

Now suppose that rnqU1(z)# 0, i.e. t ( z ) - K ( z It! (2)) > 1. According to defmition, r n ~ ( , ) ( ~=) max(rn =

2 1 ( K ( z (t(2))< e(z)-m)

P(z)-K(z ( e ( Z ) ) - l

,

(3.17)

(see a h the Remark at the end of the proof). Theorem (3.15) furnishes a natural c’ such that rn for all z in X I . We take e = e’+l, because in case mq,$))’

.

e ( z ) - K ( z le (2))-1 5 m(z)+e’

(2)

5 rn(z)+c’,

# 0 one

has

Secondly we prove the existence of another natural number, aay d , such that

e (42 K(.

It (z))+m(z)-d

7

(3.18)

for all z in x*. The universal M-L k r t U,being infmite by Theorem (3.13), can be written U = g(aV-{O}), where

Chapter 4

g:N-(0)

325

-+x*x Hv ,

ie an injective recursive function (see Theorem (2.5.18)). Using the recursive function g we shall construct a p.r. function d:X*X N S , X * such that (3.19)

for all 2 in

x*.

We construct the p.r. function 4 by cases. First we consider the set A = { ( ~ , t ( g ) )IV EX*}and we put d(g,t(y)) = y, for every y in X*. Clearly, the “graph” { ( t , d ( t ) )It E A} ie recursive. Now we construct the second “part” of 4. Consider the range of g and partition it according to the following equivalence relation: The equivalence class of g ( i ) = (zi,mi) contains a t most h elements, where because U is a M-L test. Moreover, We have therefore a sequence (Ei)i20of equivalence classes. Let a fued Ei contain r elements. We can order lexicographically the fust r strings of length .f (zi)-mi, obtaining the ordered set Ci. Here Ei is the cl&s of (zi,mi) and the lexicographical order on X* is induced by a1

< a1 <...< a,.

We construct

the sets Bi = {(g,t(zi)) Ig ECi},

and we put

B = U B i . Notice that for distinct i and j we have Bi

n Bj= (3, and

00

ill

AnB=@. The domain of 4 will be A U B. The p.r. function 4 will be completely determined by the following procedure (defmition of 4 on B). Let g ( l ) = (zl,ml). Then put and notice that t(zl)-ml 2 1 because (zl,ml) is in the M-L test U. Clearly, (a~(’l)-l,t (ZJ) is the fust element in B1,if El is the equivalence ClaM of (zl,ml). Let g(2) = (z2,m2). There are two possibilities: either (t(z2),mz)# (t(zl),ml), and in this case put

326

Calude

or, (t (z2),m2)=

( t (zl),ml),and in this case put d ( ay Z ) - x - 1

4 ( z 2 ) )=

22

In the rust case ( a 1'(sx)-',t(z2)) is the fust element in B2, if E2 is the '(aJ-+a2,e (z2)) is the equivalence class of (z2,mZ). In the second case, (aI second element in El. Notice that the last elements do not exist since L'(z2)-m2 2 1. The procedure continues: g(3) = (zs,m3). If we have two subcases:

(e (zs),m3)= (e (zz),ml)

i) If (t'(z2),m2)= (t(zl),ml),then one aet f(.))-ms-l

in case p

in

aS,e(ZS))

= 2s

9

2 3, or w-1-1

case

p = 2. The elements (a1 a d (2s)) and (a;(=a)-a-2 a 2 a , e (z3))are respectively, the third element of E l , or the fvst element in B2, according to p > 2 or p = 2. We must check that we have

"enough" elements to carry out our construction. Indeed, if p 2 3, the above computation.showa that t'(z3)-m3 2 1. If p = 2 we use the condition (3.5) and we have: 3 5 card{z EX*It (2) = e (zs),(z,m3)E V ) < 2 W-I 9 hence e(z3)-m3 2 2.

ii) If (t(z2),rn2)# (t(zJ,mJ, then one set W-ma-1

a2re ( 2 3 ) ) =

zs

*

Notice that ( a 1'('""'-'a2,e (zs))is the second element in B2. In case ( l (z3),m3)# ( l (z2),m2),three subcases appear: a)

If (P(z2),m2)= (!(z1),ml), then one set

+(ayl)-? ,e(z3)) =

23

where (al'(as)-tns,t(zs))is the fist element in

9

B2,if E2 is the clase of

(239m3).

set

b) If (f(z,),m,) #

(f(Zl),ml)

and (e(zl),ml) =

(e ( 2 s ) m s ) ,

then one

327

Chapter 4

4(a:(~t)-m,--l

where (a1W - t - 1 set

c)

If

a2,t (23))

= 2s

.I,t(ZS))

I

is the second element in B,.

(e(z2),m2) f (e(Zl),ml) and ( t ( Z l ) , r n l )

d(=;(=t)** ,%))

= 23

f (t(zs),rns), then one

9

where (a1“‘””8,t (zs))is the fist element in Bj. Notice that the set B has “enough” elements, hence the procedure is well-defmed. Clearly A and B are r.e. sets. Therefore, d is a p.r. function. We c l a i i that the above constructed p.r. function 4 satisfies the equality (3.19). Indeed, for a given z in X*two possibilities may occur: A) m(z) = 0. In thia case, for all natural numbers i 2 1 and rn 2 1, we have g ( i ) Z (z,rn), so +(z,t(z)) = z and no other possibility for obtaining z via 4 occurs. Consequently, K,(z It (2)) = t (2).

B) m(z) f 0. In thia case there exist the natural numbers i 2 1 and rn 2 1 such that g ( i ) = (zi,rni) = (z,rn). The string z = zi may be obtained in two ways via +: or

d ( ~ (.if) 8 = zi

9

with y in X*,t ( y ) = t(zi)-rni, and g ( i ) = (z,rni). Hence:

K,(z It (2)) = &([

(Z)-mi

lo(;)

=

= min (t(z)-m I(z,rn)

= t (2)-max(rn = t(z)-m(z)

(Zrmi))

E V)

_> 1 I(z,rn) E U)

.

According to Theorem (2.9) there exists a natural d such that:

K ( z It (2))

< K+(z It (z))+d

= t (z)-m(z)+d

.

From the inequalities (3.15) and (3.18) we conclude:

- d 5 ! ( z ) - K ( z lt(z))-rn(z)

5e

.

Taking q = max(e,d) we obtain the required asymptotic evaluation (3.14). 0

328

Cdudc

Remarkm. a) The constant g appearing in the inequality (3.14) depends upon w and U. b) Notice that the equality (3.17) used in the proof of Theorem (3.16) is valid only in c u e mqw)(z)> 0. We are in a position to specify the statistical properties of the KOLMOGOROV random strings. We begin by proving that the M-L test BBM)ciated to an universal KOLMOGOROV algorithm ie universal.

(8.17) Theorem. (CALUDE and CHITESCU [1984]) Let w : X x N A X * be an universal KOLMOGOROV algorithm. Then V ( w )i an universal M-Ltest.

Proof. Recall that putting w instead of # in (3.7) we get the M-L test ~ ( w=) {(z,m) Iz E X*,m E N , m and K , ( z [!(Z))

2 1,

< !(z)-m}

.

In view of Theorem (3.15) it is sufficient to prove that V ( w ) satisfies the relation (3.13). To this aim we use an universal M-L test U provided by Theorem (3.10): For every M-L test V there elcists a natural e such that

mvk) tor every z in

I mu(z)+e

(3.20)

t

x*.

Using Theorem (3.16) for the pair (w ,U)we get a natural number q such that muk)

for every z in

IW - K &

lW)+q

7

(3.21)

x*.

Now let V be an arbitrary M-L test and set t = q+c+l, where c is obtained from (3.20) and g is the fmed constant satisfying (3.21). We analyse two cases, according to mqw)(z)= 0 or m+)(z) > 0. If my,)(z) = 0,then by (3.16) we have:

e (.)-&

(2

It (4) I 1

In view of (3.20), (3.21)and (3.22) we have:

md.1

5 mu(z)+e 5W-K&

lW)+P+C

(3.22)

329

Chapter 4

< q+c+l = t = mV(&)+f

If m+,(z)

*

> 0, then by (3.17) we have: mV(w,(z) =

W - K & lW)--l

*

(3.23)

Using again (3.20), (3.21) and (3.23) we derive: mv(4

I mv(z)+c IW - K , ( z

I W ) + Q + C

= my,)(z)+1+q+c = mV(w)(z)+t

9

which concludes the proof. 0

(8.18) w

Corollary.

Fix an universal KOLMOGOROV algorithm

:X x IV 4X*.Then every random string (associated to

stands the universal M-L test V ( w).

w ) with-

Proof. Since z is random it follows that Kw(z

I+))

2 “4 > w - 1

f

which proves that (z,l)!$ V(w). Comment. Corollary (3.18) shows that the KOLMOGOROV random strings possess almost all conceivable statistical properties of randomness (i.e. they withstand an universal M-L test).

(8.19) Corollary. Every asymptotic random string withstands the universal M-L test V ( w ) .

Proof. For some natural number m 4 [(z) we have

w. I W ) L w - - m

consequently (z,m) 4 V(u).

I

330

Cdudc

(8.20) Corollary. To every asymptotic random string z and every M-L teat V we can associate a natural number q such that

(ztm+q) 4

for all naturals m

v

9

2 1.

Proof. In view of Theorem (3.17), there exists a natural c (depending for every natural m 2 1. On upon w and V) such that Vn+c C (V(W))=, the other hand, ( z , d ) 4 V(W), for some natural d. Hence z ( V ( w ) ) d ,i.e. z Vm+(d+e), for every natural m 2 1. The constant q = d + c works. 0

Remark. When z is random, q can be taken to be c . We

close

this section

by

stressing

that

KOLMOGOROV’s

complezity-theoretic dc finition of (finite) randomness is compatible with almost all statietieal requirements.

4.4. UNDECIDABILITY THEOREMS

We pursue our analysis of the relevance of KOLMOGOROV’s complexity-theoretic definition of (fmite) randomness by proving that the (asymptotic) random strings are, in a strong sense, non-conatructable. This result reinforces the adequacy of KOLMOGOROV’s approach. We begin with aome preliminary results. (4.1) Theorem. Let w :X* x IV A X * be an universal KOLMOGOROV algorithm and let a:RV -+ nV be a (non-neceesarily recursive) function

such that lim a(.) n -3c

=

03.

Then every partial function f K , ( f ( n ) In)

:IV * X * satisfying the condition

2 a(*) and n E dom(f)

7

(4.1 1

for an infmity of naturals n, hae no p.r. extension. In particular f itself ie not a p.r. function.

331

Chapter 4

Proof. For the sake of a contradiction, w u m e the existence of a p.r. function F:BV S,X* which extends f. We construct the audliary p.r. function 4F:X* x

Ax* in X* and

given by 4F(z,n) = F(n), for all z n in dom(F). Notice that from condition (4.1) it follows that the domain of f is infmite, hence, the domain of F is abo infmite. In view of Example (2.4) (with F instead of f) we have:

K,p(.)

).I

=0

(4.2)

9

for all n in dom(F). Furthermore, if n is in dom(f), then because F(n) = f(n).

KOLMOGOROV’s Theorem (2.9) furnishes a natural number e (depending upon w and F)such that for all n in dom(f).

( . ) > e 0.c. From the divergence of function a we conclude that a We use condition (4.1): ultimately we can fmd a natural number n in dom(f) such that

K u ( f ( 411. 2 4.)

>e

-

(4.5)

The relations (4.4) and (4.5) are contradictory, thus ending the proof. 0

The recursive functions a(.) = n, a(.) = n%, and = Pog2n] satisfy the condition of divergence used in Theorem (4.1).

Remark. a ( . )

(4.2) Lemma. Let A C X* be an infmite r.e. set. Then there exists a p.r. function F:N S, X * having the following properties:

’

then:

The domain of F is infmite.

(44

The range of F is contained in A.

(4.7)

For every n in dom(F), t (F(n)) = n.

(4.8)

Furthermore, if for all z and y in A, z # y impliss t ( z ) # t ( p ) ,

332

Cdude

The function F can be taken to be injective.

(4.9)

The range of F is exactly A.

(4.10)

Proof. From the recursive enumerability of A it follows that A is the range of some injective recursive function f :N+ X*.The p.r. function F:RV %X* will be given by the following procedure: 1. P u t

a

= 0.

2. Put F(!(f(i))) = f ( i ) .

3. Put i

=

i+l.

4. If t ( f ( i ) ) = t(f(j)),for some j

< i , then go to step 3.

5. G o t o s t e p 2.

Property (4.6) follows from the infinity of the set A and the fact that step 2 in the procedure which computes F is reached infinitely many times. From the construction of F we easily conclude that (4.7) and (4.8) hold. Under the assumption that for every natural number n there is a t most one string of length n, the test in step 4 is always overpaaeed. Consequently, for all naturals n and m in dom(F) such that F ( n ) = F(rn), there exist two other naturals i and j satisfying the equalities

flil F(m) = F ( e ( f ( j ) ) )= f ( i ) F(n) = F ( W i ) ) ) =

I I

(see the procedure defming F). Hence n =

e(f(;)) = e(f(j))= m

which proves (4.9). We fmish the proof with the equality F(t(f(i)))= f ( i )which shows that range(F) = range(f) = A. 0

ax*

(4.S) Theorem. Let w : X * x RV be an universal KOLMOGOROV algorithm and let a:BV --+ BV be a (non-necesearily recursive) function such that lim a(.) = oa n

-00

For a set A C

X*we consider the following properties: The eet A is infmite.

(4.11)

333

Chapter 4

For almost all z in A , K , ( z It(.))

2 a(l(z)) .

For all z and y in A, if [(z) = l ( y ) , then z = 9

For infdtely many z in A,

K, (z it (2)) 2 a(t (2))

.

.

(4.12) (4.13) (4.14)

If A has the propertiee ((4.11) and (4.12)) or ((4.13) and (4.14)), then

A is not r.e.

Proof. Suppose that A f u l f i i (4.11) and (4.12). For the sake of a contradiction, assume that A is r.e. In view of Lemma (4.2) there exists a p.r. having an infmite domain and range(F) C A. For function F:RV ax*, almost all n E dom(F) we have: t (F(n)) = n and K,(F(n) ( t(F(n)))= K,(F(n) In) 2 a ( n ) , because F(n) E A. This contradicts Theorem (4.1). Suppose now that fulzills A (4.13) and (4.14) and asoume by rcduetio ad obeurdum, that A is r.e. By Lemma (4.2) there exists an injective p.r. function F:N a x *such that range(F) = A and for every n in dom(F),

t ( F ( n ) )= n. Then there exists an infmite subset Y of dom(F) such that for every n in Y we have: F(n) € A , t'(F(n)) = n and K, (F(n) It (F(n))) = K, (F(n) in) 2 a(n), thus contradicting Theorem (4.1).

0

Remark. Theorem (4.3) is consistent! In fact, conditions (4.11) and (4.12)

are satisfied by the identity function a(.) = n and the set of all raidom strings (see Theorem (2.14)). Again let a(.) = n and let A be a set of strings satisfying (4.13) and containing an infrnite set of random strings; we have a model for (4.13) and (4.14). For the rest of this section fm an universal KOLMOGOROV algo-

rithm w ; K = K,.

(4.4) Corollary. For every natural number m

2 0, RAND,

is immune.

( . ) = n A m . Let A be an Proof. Let a : N + N be the function a infmite set of m-random strings. Obviously, lim a(.) = 00; since A fuln-m

f i conditions (4.11) and (4.12) in Theorem (4.3) we conclude that RAND, is immune. R

334

Cdudc

(4.6) Corollary. (ZVONKIN and LEVIN [1970]; CALUDE and CHITESCU [1982b]) The set RAND is immune.

Proof. Set m

= 0 in Corollary (4.4). 0

(4.6) Corollary. The set X*-RAND, ia not recursive, for every natural number m . In particular, XI-RAND is not recursive.

Proof. Directly, from Corollary (4.4) and Theorem (2.5.7). 0

-

(4.7) Corollary. Let m be a natural number. Then there ia no recursive function f :N X* such that for every natural n we have t ( f ( n ) ) = n and K(f ( n ) In) 2 n A m . = range( f ). Since A satisfies conditions (4.13) and (4.14) in Theorem (4.3), it follows that A is not r.e. We have arrived at a cont r adic t ion.

Proof. Take A

0

Remsrks. a) Corollary (4.7) shows that the “m-random strings are not constructable”, in fact, strongly non-constructable, by Corollary (4.4). Tireee reeults point out that, in a strong .sense, there i s n o “algorithmic rule” for generating m-random strings. In other worde, there are no afgorithmie t o o h for uniformly recognizing m-random stringe, thus reinforcing the adequacy of KOLMOGOROV’s defmition of fmite randomness. b) Corollary (4.7) stresses that the existence of m-random strings cannot be constructively proved. Hence, Theorems (2.14) and (2.17) cannot be proved in any constructive way. (4.8) Corollary.

(MARTIN-LOF(1966bl) The function

K:x*+w , defmed by

I?(=)

=

K ( z It(=))ia not recursive.

Proof. If I? were recursive, then one_ should fmd a recursive function which produces aiI 2 in X* such that K ( z ) _> t(z), i.e. the set of all random strings, thus contradicting Corollary (4.5). 0

Consider the set

335

Chapter 4

(4.9) Corollary. If for an infmity of natural numbers n one has

K(zlz2...z2,In) >_

n nrn

,

for some fmed m, then A is not r.e.

Proof. The set A satisfies conditions (4.13) and (4.14) in Theorem (4.3), with m(n) = A A m . Consequently, A is not r.e. 0

Comment. The non-constructable global property is not a decisive argument for a good defmition of randomness. For example, the set 1 A = {z E X * IK(z It (4)2 [logp(t (.) I> is clearly not r.e. (see Theorem (4.3)), but it must be clearly rejected as L candidate for the random strings set. We continue the analysis of the undecidable properties of (asymp totic) random strings with a characteriration of recursive M-Ltesta.

(4.10) Lemma. A M-L test V is recursive iff the critical level induced by V, my, is a recursive function.

Proo;.

Assume that my is recursive. The characteristic function of V, -+ {0,1} can be defmed by

xv:X x IV

1, ifrnv(z)2m>O, xv(z7m)= (0

, otherwise,

which proves that V is recursive. Conversely, suppose that xv is recursive. Then the critical level induced by Vcan be expressed by the formula

E WT Ixv(z,m) = 1) , if xv(zJ) otherwise. Hence, my is recursive.

=1

,

338

Cdudc

(4.11) Theorem. The universal M-L test V ( w )is not recursive.

Proof. Suppose, for the sake of a contradiction, that V ( w ) is recursive. In view of the equuality

X'-(V(w))1

= (2

E X * IK&

lt(Z))

2 q+1>

9

we deduce that the set of 1-random strings is recursive (for V(w) recursive implies the recursiveness of section (V(w))l). Thie contradicts Corollary (4.4).

(4.12) Corollary. a)

The critical level induced by a M-L test V ( w )ie not recursive.

b) Each universal KOLMOGOROV algorithm w has non-recursive graph; in particular, w is not recursive.

Proof. a) Directly from Lemma (4.10) and Theorem (4.11).

-

b) Assuming w has a recursive graph, the equivalence (2,m)

for some y in

E V(w)

w(ar,W)=z

,

X* with e ( ~<) e(z)-m, contradicta Theorem (4.11). 0

Following CHAITIN [1974], we consider a formal system to be a program for listing the set of theorem and the time at which a theorem is written out to be the length of its proof. We try to formaliie these ideas. Let P,(X*) be the set of all fmite subsets of X*. Consider a recursive function

F:X* x lN

-.,

P,(X*),

such that F ( z , n ) C F(z,n+l)

for all z in

X*,and

n

in

,

(4.15)

I?.

The recursive function F defines the rules of i n f e r e n c e of a class of formal systems. The value F(u,n) is the fmite (pomibly empty) set of the theorems that can be proved from the axioms coded by the string E in X * , by means of the proofs of length shorter than n. A recursive function F defming the inference rules of a formal eye tern can be described by the composition of a recursive function

337

Chapter 4

+:x*xBV-+x*, and a (primitive) recursive bijection

a:x*+ P,(X*).

More precisely,

(4.12) Definition. (CHAITIN [1974]) The ordered pair where

F:X.x Hv

+

P,(X*),

ia a recursive function satisfying (4.15), and 8 is a string in X* (the codification of the set of axiom) is a f o r m a l eyetern.

Notice that a formal system implies both the choice of inference rules and the codification of axioms. (4.14) Definition. The set of all theorems deducible in the formal system is (4.16)

Recall that in the proof of Theorem (2.9) we have constructed an universal KOLMOGOROV algorithm w :X*X RV a x *by means of an universal p.r. function $mie:X* x (X*x nV) S,X* and three recursive functions T,f , and g. In what follows we shall make use of these functions. Fix a formal system having the following two properties: The propositions of the form “ K ( z In)

> m”

can be coded as a recursive subset of X*

.

(4.17)

The formal system ia mund with respect to all propositions of the form “ K ( z In) > m”, i.e. if a proposition “K(zIn) > m” belongs to m ( F , 8 ) , then K ( z In) > m

.

(4.18)

338

Cduds

(4.16) Theorem. (CHAITIN [1974]) For every formal system having properties (4.17) and (4.18), there exists a constant e (depending upon the system), such that for every proposition of the form “ K ( z In) > m”, that belongs to Z%(F,s),we have m < t?(a)+e.

Proof. We begin with the construction of a p.r. function 4:X* x RV ax*by the following procedure: 1. Read (z,n) in X* x hV. 2. If z # T ( z ) s ,for all z in X*, then 4(z,n) = OQ 3. P u t h = 0. 4. Generate F(8,h). 5. If F ( s , h ) contains the code of a proposition of the form “K(y In) > m”, with rn > t(s)+2t(T(z)), then +,n) = y, where y comes from the fmet generated proposition “ K ( vIn) > m”. STOP. 6. Put h = h + l , and go to step 4. It was already noticed that the representation z = T(z)a is unique, when it happens. Moreover, the test in step 2 is obviously recursive. In view of (4.17) and (4.18) it follows that the above procedure defmes a p.r. function. According to Theorem (2.9), we can fmd a constant q such that K(ar I.) for

5Q

Y In)+q

v

(u,n) in X* x N. Now let (z,n) be in X* x hV such that 2

= T(z)a

,

for some z in X* with t(T(z)) = 2(q+l). If for some etring y in X* we have d(z,n) = 4(T(z)8,n) = Y

,

then K(Y/ I

4 I K,(Y In)+q I e (W4+Q = e ( 8 ) i - t (T(Z))+q = q8)+2(q+i)+q

= e(4+3q+2

.

In view of (4.8) and step 5 in the procedure which computes 4, we

339

Chapter 4

deduce that there exists a natural m such that

We conclude that no such y exists, so

W(+,n)

=

03

,

for all n in RV. Take c = 3q+2. We have proved that every proposition of the form “K(y In) > m ” is a theorem of the formal system only when m


0

(4.16) Corollary. (GODEL) For every formal system having the properties (4.17) and (4.18), there exista a true proposition of the form “ K ( z 11 (z))> t (2)” which is independent of (i.e., neither the proposition nor its negation is in Th(F,u)).

Proof. According to Theorem (2.14) we can fiid a random string z with t? ( 2 ) > l (s)+e, where c is the constant furnished by Theorem (4.15). The proposition “ K ( z lt?(z))> e(z)” is not a theorem of the formal system , for it violates the condition C (z)< e ( 5 ) + e . The negation of the proposition “ K ( z It? ( 2 ) ) > t (2)” cannot be a theorem of the formal system < F , 5 >, for it violates the soundness assumption (4.18). 0

Remsr ks. a) To realire the importance of Theorem (4.16) let us notice that there is no limitation to the form of axioms or rules of proof (presumably sound) for proving statements of the form “ K ( z In) > m”. We may include here all (constructive or non-constructive) methods of proof avsilable in usual mathematics. Theorem (4.15) puts down a severe limitation to the power of mathematics, since it indicates the existence of a constant, t ( e ) + e (depending upon the system) such that it is impossible to prove within the system that a string is more complex than 1 ( u ) + e . b) P ut p = 2, u1 = 0, a2 = 1. Theorem (4.15) asserts that we can find a constant e such that if a theorem constitutes more than t ( e ) + e bits of information, then it ia impossible to deduce it from the system (here

340

Cdadc

C ( 8 ) denotes the information contained in the axioms of the system).

c) GODEL [1931] pointed out that there are statements about natural numbers which can be neither proved nor disproved in the logical system Principia Mathernatica or in a i m i i systems. Corollary (4.17) allows a deeper look into the matter (see also Theorem (2.6.15); DAVIS (19781, CALUDE (1982bj).

4.6. REPRESENTABILITY THEOREMS

The critical levele induced by M-L testa constitute themselves aa an alternative to the KOLMOGOROV theory of complexity. The results of Sections 4.3 and 4.4 suggest that these complexity theories are "nearly equivalent". In the present section we shall prove that these theories are not equivalent and we shall investigate the possibility of expressing the M-L tests in terma of KOLMOGOROV's complexity. We shall show that this is possible by adding an element to the primary alphabet. The starting point ia Example (3.3), which we shall briefly recall. To every p.r. function 4:X* x IV a x * we aesociate the M-L test .V(#) defmed by

V(4) =

.I d , m E J V - { O ) , I+)) < W - m )

{(z,m)

K,(z

Notice that (z,m) cV(4) iff there exiats a string g in ( ( y ) < t'(z)--m and 4(p,t'(z)) = 2 . This example suggests the following defmition.

(5.1)

X* with

(6.1) Definition. (CALUDE and CHITESCU [1983a]) Let V C X*x IV be a M-L test. We say that V ia (KOLMOGOROV) representcable if there X N S , X * such that V = V(#). exists a p.r. function #:X*

(6.2) Example. The M-L test H ( z , m ) in Example (3.4) is representable.

Take for instance the p.r. q5(a:(*)*-',t'(z))

=

2.

Since

# : X * x N A X * , given by K,(z lC(z)) = .t?(z)--m-l, we have

function

H(.,m) = V(0). 0

341

Cbrpkr 4

(6.8) Example. The M-L test g((z,m) in Example (3.5) is representable. We shall construct a p.r. function d:X*X h' ax*,such that for all (y,n) in X* x h' satisfying the conditions y 3 2 and 1 5 n 5 m , we have

q a r It ( 9 ) ) = t ( d - m - 1 thus proving that H(z,m) = V(4). Set d(apl(s)*-1

For every natural t and

=

.

2

> !(z), we consider the sets 4 = {Y EX*lY 3 z , w = t }

9

. EX*l t ( 2 ) = t-rn-1} > t(z)-rn-l 2 0; so, Bt # 0. Moreover, Bt

Clearly, t-m-1

,+))

(54

9

=

(2

c a r w = pt-'(s)

< - pt--l

=

CardBt

.

(5.3)

We order the sets 4 and Bt in the lexicographical order induced by up, thus obtaining 4 = {gl,v2 g,} and Bt = {zl,iq,...,z,}. In view of (5.3), 8 5 r . Now defme u1

,...,

< u2 <...<

+(Zi,t) = gi

9

for every i=1,2 ,...,8 . Clearly is a p.r. function satisfying the relation (5.2). 0

(6.4) Example. There exist universal and representable M-L tests. Take, x h' ax*, for example, an universal KOLMOGOROV algorithm w :X* and consider the M-L test V ( w )(see Theorem (3.17)). 0

Unfortunately, not all M-L testa are representable. (6.6) Example. Take p = 2, u 1 = 0, a2 = 1. The set

v=~

~

~

,

~9

~

~

~

b a non-representable M-L test.

Obviously, V is a (fmite) M-L test. We shall prove that V is not representable. Suppose, by absurd, that there exists a p.r. function d:X* x hT a x * such that V = V(4).We can infer the existence of three 1, such that strings go,yI,gl in X*, l?(ui)

~

~

~

342

Cdude

+(y0,3) = +t111,3) = 010

9

+ ( ~ 2 , 3= ) 111

It follows that {y0,yl,y2} = {A,O,l}. For instance, we choose @,3) = OOO

(and 4(0,3) = 010, +(1,3) = 111). For this t# we must have (000,2) E V(4), because t?(A) = 0 < t? (000)-2 = 3-2 = 1. This proves that (000,2) E V(4)= V, which is a contradiction. 0

In order to avoid this situation we “enlarge” the primary alphabet X = {a1,a2,...,ap}, p 2 2, by adding a single new elment ap+l (distinct from ai, 1 5 i 5 p). We obtain the new alphabet Y = {a1,a2,...,ap,ap+1}. In this case, every M-L test V c X* x IV c a n be viewed as a M-L test V c r‘ x N. We shall prove that all such M-L teats are representable and, in fact, the p.r. function 4 : f x lN af which represents V (i.e. V = V ( + ) )can be taken to have its range included in X*. Theorem. (CALUDE and CHITESCU [1983a]) Let {al,a2~..,ap}, p 2 2, and Y = X U aa before. For every M-L test V C X x lN there exists a p.r. function

(6.6)

X

=

4:y‘xN&f, such that V = V(+),and range(+) C X*.

Proof. Only the non-trivial case V # 0 will be considered. Let UB consider the lexicographical order on f induced by a 1 < a2 <...< ap < ap+l. We shall construct a p.r. function 4 : f x N &f having the property

K&

I[ (4)= e ( z ) - w ( z ) - l

?

(5.4)

for every z in X* for which (z,1) E V. (For the motivation see the p.r. function 4 exhibited in Example (5.3).) We distinguish two cases: a) The M-L test V is infinite, and in‘this cane there exists an injective recurshe function g : N - ( 0 ) -+ X* x IV,such that range(g) = V.

b) The M-L test V is finite, and in this caee there exists a (p.r.) injective function g:(1,2 ,...,q } + X* x hr, such that g({1,2 ,...,q } ) = V (we have assumed that c a r d V = q). In both canes we shall write g(i) = (zi,m,), for every i in dom(g). We shall describe the action of 4, on the basis of an uniform p r e cedure (which stops after a fmite number of steps in case V is fmite). We

343

Chapter 4

proceed similarly to the construction of the p.r. function Theorem (3.16).

4 in the proof of

Let g ( l ) = (zl,ml) and set

In the latter case set The construction is possible because t (za)-m2 > 0 ((z2,m2)belongs to the M-L test V). If the additional equality (t (z2),m2)= (t (zl),mr)holds, then we have (in view of (3.5)): 2

5 card{z EX.

(2)

=

t ( z z ) ,(%,ma)E v)

which proves that t (2,)-m2 2 2. In general, at step i > 1, let g(i) = (zipti). (t(zi),mi) (t(zj),rnj),for'all j=l,2,...,i-1, put

+

In

case

In the opposite case, let 1 <_

8

= card{j E IV

l i 5 j < 1, and (t(zj),mj) = (t(zi),mi))

(the k t inequality is a consequence of condition (3.5) for the M-L test V). The elements 8 in f,having t ( y ) = t(zi)-mi-l, are (in lexicographical order): Y 19Y21.*-9Yr

where r = (p+l)

Put The construction is possible because

e (s;)-mi

I

-1

344

Cdudc t(+6;-1

r = (p+l)

> ((pw+ -l)/(p-1))-1

28

.

It is obvious that 4 sets as a function. Moreover, the above procedure stops after the analysis of g(q), when V is fmite and hss exactly q elements; otherwise, the procedure continues indefmitely. To be more precise, we shall describe the domain of 4. To this aim, we partition the range of g according to the following equivalence relation: g(i)

g( i )

* (k'(zi),mi) = (k'(Zj),mj)

*

The equivalence class of an element (zi,mi) contains a t most h elements, where h = (p'(si)*iW(P-1) * So, the range

V of g is the union

00

UE j , of equivalence classes E j (m

j-I

case V

8

is infmite) or is a fmite union U E , (in case V is fmite). For every j-1

-

-

equivalence clasa E j which contains t elements we consider the set Cj consisting of the lexicographically last t strings of length k' (zj)-mj-l;here Ej is the CIME of (zj,rnj). P u t then Bj = {(y,t (zj)) Iy E Ci},for the above pair (zj,mj). The domain of s

B = UE j , (in case V ia fmhe).

4

m

ia B = UEj, (in case V is infmite), or j-1

j-1

Take z in X* such that (z,l) . . E V, so mv(z) - . > 0. There exists an unique natural i > 0 such that g(i) = (z,mv(z)).According t o the procedure, there exists a string y in f with l ( y ) = k'(z)-rnV(z)-l, and 4 ( y , l ( z ) ) = z. This shows that ~

K,(z le (2)) 5 e (z)-mV(z)-l

.

(5.5)

On the other hand, the equality #(y',k'(z')) = 2 implies z' = 2 , and [(y') = l(z)--mj-l, in case g ( j ) = (zj,mj) = (z,mj). This can be done for some mi 5 mv(z). We conclude that l(p') 2 k' [z)-mv(z)-l, i.e.

K , ( ~le(2)) 2 e(~)--my(~)-i

.

(54

From (5.5) and (5.6) property (5.4) follows, thus proving the inclusion

v c V(4).

To prove the converse inclusion, V(4) C V, we should notice fust that, in view of the construction of 4, (z,m) E V(4) implies (z,l) EV. Now we take (2,m) in V(#), and we prove that rn 5 my(z), (i.e., (z,m)E V). For the sake of a contradiction we suppose that m > mV(z). According t o the defmition of the critical level we have

345

Chapter 4

(z,mv(z)+l) EV(#), l (y) < t (z)-m,,(z)-l,

which and

yields

an

in

y

Y'

such

that

d(v,.f ( 2 ) ) = z. This contradicts property

(5.4).

0

Remark. If V is a representable M-L test, then the equality V = V(4) holds for many p.r. functions 4. The p.r. function 4 furnished by the construction in the proof of Theorem (5.3) is injective. Actually, Example (5.5) can be generahed: (6.1) Propomition. For every alphabet X,having p 2 2 elements there exist a frnite M-L test V, and an infmite M-L test W, which are both nonrepresen table.

Proof. a) Let p strings

2 2 and put

8 =

We consider

(pp-l)/(p-l).

Y1,12,"',J,

8

difrerent

9

in X * , with length t(yi) = p + l . We claim that the frnite M-L test,

v = ((ri,i) li=1,2 ,...,4 , is non-representable. Indeed, if V were representable, then we could fmd the (mutually distinct) strings Zl,Z2,".9Z,

9

in X* each having the length l ( z i ) < p + l - 1 = p, and such that

#(s,P+~) = yi

9

for i = l , 2 , ..,a. Since p'-' < 8 , a t least one of the strings zi, say zt,should have the length shorter than p-2. So, b(z,,p+l) = yt, and

w IP-2

< %/,,)-2

*

This shows that (yt,2) E V(O), contradicting the construction of V. b) Put W = V u {(ai,l) )i=p+2,p+3, ...}, where V is the M-L t e s t constructed at a). 0

346

Cdude

(6.8) Propodtion. For every alphabet X, having p 2 2 elements, and alphabet Y 3 X with p + l elernenk, there exists a p.r. function

T?xBv

A X * such that the M-L test test over X * x BV.

V(4) over

f x RV

is not

a

M-L

Proof. Let X = {a1,a2,...,4p}and Y = X U {ap+,}. We order X* lexicographically according to a 1 < a2 <...< ap, and r‘ according t o a 1 < a2 <...< ap < ap+l. Let A = {y E f IL(y) < p ) = {~~~...~y~,...,g~}~ in 1exic:graphical order. It is seen t h a t t = ((p+l)P-l)/p. Let B = {z EX lt(z)= p + l } = {z1,z2,...,z,}, in lexicographical order. We have 8 = pp+’ > 1 .

-

...,

The domain of 4 is the set D = {(ui,p+l) li=1,2, t } . We defme X*,by 4(yi,p+1) = zi. The set V(4) is a M-L test over r‘ X dv. On the other hand V(4)C X* x RV; a simple computation showa that

4:D

card{% EX*le(z) = p + l , ( z , l ) E V(4)} = t asserting that

> (pp-l)/(p-l)

,

V(4)is not a M-Ltest over X* x N. U

Remarks. 1) We can interpret the result stated in Theorem (5.6) aa follows: a) The complexity theories of KOLMOGOROV (based on the complexity function K,) and MARTIN-LOF (bssed on the critical level rnv) are not equivalent, according to Example (5.5). b) Considering the MARTIN-LOF theory over an “enriched” alphabet (in fact, an alphabet containing one more element) we can exactly express the M-L tests ae objects in the KOLMOGOROV theory. c) For every natural p 2 2, and for every alphabet X with p elements, there elcists a M-L test over X*x RV which in non-representable So, every non-repreoentable M-L test (see Proposition (5.7)). V c X* x Bv become representable in Y‘ x N,by adding a single new element to X. But in r‘ X RV there exist abo other non-representable M-L tests! And the “enlargement” may continue indefmitely. 2) Proposition (5.8) goes in a “converse direction”. Here, there are “too many” representable M-L tests over the enriched alphabet. Hence, the KOLMOGOROV theory over an alphabet with p elements is not equivalent to the MARTIN-LOF theory over an alphabet with p + l elements.

347

Chapter 4

Comments. In Remark 1) following Corollary (2.8) we have pointed out the distinction between the binary and the non-binary cases in the KOLMOGOROV theory of complexity. The representability analysis stresses this distinction; moreover, the same remark can be done for the MARTIN-LOF theory of complexity. The following result shows that, in a sense, the representable M-L test are “economical”. (6.9) Propoeitton. For every representable M-L test V the following inequality holds:

card{z

,

(5-7)

#:X* x IV

ax*,

E X * It (2) = n,mV(z) = m } 5 pn-cn-l

for all natural numbers n and m, n

> m > 0.

Proof. From hypothesis there exists a p.r. function such that V = V(4).

Fix the naturals n > m > 0. For every z in X*satisfying the conditions t ( z ) = n, and mv(z) = rn, there exista a string y in X* with t ( y ) < l(z)-m, and #(g,t‘(z)) = 2. We have t ( y ) 5 n-m-1. Actually, we shall prove that t ( y ) = n-m-1. Suppose; by absurd, Let [(y ) = n - m - 1 4 , with h > 0. This leads to that t ( y ) 5 n-m-2. the false relation (z,m+h) EV. Indeed, t ( y ) = n-,-h-1 < n-m-h and 4(y,t (2)) = z,thus showing that (z,m+h) E V(4)= V. The just proved equality t ( y ) = n-m-1 shows that card{z

EX*l t(z ) = n,mv(z) = m } 5 card{y EX’ It (y) = n-m-1) =

pn-m-l

0

We continue this section with a result establishing a precise relation between the KOLMOGOROV complexity K, and the critical level induced by the M-L test V(#).

(6.10) Theorem. Let V = V(4)be a representable M-L teat. The following aseertions hold for all z in X.: my(.)

If my(.)

=0

iffK,(z I+))

2 t(z)-l

*

> 0, then K + ( z It (2))= t (z)-mv(z)-l

.

(54 (5.9)

In the particular case when range(#) = Vl, the equivalence (5.8) can be stated more precisely, namely:

348

Cdudc

Proof. Assume mv(z) = 0. Therefore (z,1)# V = V(+),i.e., for every string y in X’ with t ( y ) < t(z)-1, we have +(y,t(z)) # 2 . Then, either +(y,t(z)) # 2 , for all y in X * (which shows that K,(z 14(z))= co),or their exints a string y in X* with +(y,e(z)) = z, but this y must have e(y) 2 e(Z)-i. so, K,(Z l e ( Z ) ) 2 e(z)-i.

2 t (z)-l. There are two cases: then +(y,e(z)) # 2, for all y in X*,and then

Suppose that K,(z It (2)) i) if K,(z le(z)) = q

v,

(ZJ)B

v(4)

(ZJ)6!

V(+)= v.

=

ii) if K,(z le(z)) < cq then there exists at least one y in X* with d(y,t(z)) = 2, and one must have [(y) 2 e(z)-l. This shows that Hence (5.8)was proved. According to the hypothesis, there exists a string y in 2. We have:

X* such that

4(y,t(z)) =

card{z

EX*le(z) = p+l,(z,l)E V(+)}= t > (p’-l)/(p-1)

,

fde ch4.5, 1. 587 (ms 347):

m v k ) = mq,,(4 = max(m

E N Im 2 l,+(y,t(z))= 2 ,

for some y in X*with t (y) < t (z)-rn) = max(m

E N Irn 2 l,+(y,e(z))=

for some y in

X*with

m

2,

< t(z)-e(y)) .

The last maximum is attained for those y in X * which are of minimum length, i.e. for those y in X* with t (y) = K,(z It (2)). So, mv(z) = t ( z ) - K + ( z lt(z))-l

.

In the particular case when range(+) = V , we have: if mv(z) = 0 , then ( z , l ) # V, i.e. z range(+). Hence K,(z “(2)) = oa 0

C range(+). So, the condition V , = range(+) Remark. We have (V(+)), in Theorem (5.10)can be equally stated as range(+) C V,.

349

Chapter 4

X N S,X* be an universal KOLMO(6.11) Corollary. Let w :X* GOROV algorithm. Then, range(w) # (V(W))~.

Proof. Use Theorem (5.10) and the relation Ky(zle(z)) # cq for every z hX*. 0

Recall that the enumeration of X* in the lexicographical order induced by a y < az <...< a,, is given by {y(n) In 2 11, where y(1) = A, y(2) = a1?... y,( p + l ) = ap, y(p+2) = alal It follows that

,... .

ln

ap" = y(#(m)), where s ( m ) = z p ' = (pm+'-1)/(p-l). id

(6.1%)Theorem. (STAIGER [1984) (added in proof)) Every M-L test V C X*x IV satisfying the condition:

card{z EX*l l ( z )= n,(z,m) €V) 5 p"-"'-' for all naturala n,m

,

(5.11)

2 1, is representable.

Proof. As in the proof of Theorem (5.6), fu an injective recursive (or, fmite) enumeration g for V, i.e. range(g) = V.. P u t g ( i ) 7 (q,mi), for

I

each i E dom(g). We define the p.r. function

4 :X x IV A X

, if n 2 2, and t

= pr[t(z,) = n,

zt

bod

=

m, = n-11

00,

and I

zt

aa follows:

EN,

otherwise,

, if n 2 t (z)+2,

z = y ( 8 ( t (z)-l)+i),

and t = pr[e(z,) = n,m, = n-t(z)-l,

d.9.)

and card{j E h v 11 5 j

=

5 r,e(zi) = n,

mj = n-e (Z)-l} = i] E hv, 00,

otherwise,

\

for z # 1. We shall prove that V = V(4).

First, assume that (2,rn) EV, i.e. (z,m) = (zt,rnt), for mme t 2 1.

Since t ( 2 ) > m , two caaee may occur: a) If ! ( z ) = m+l, then put z = X and notice that +(X,m+l) = z . Indeed, in view of (5.11), there ie an unique string y of length m + l euch E V , namely y = 2. that (y,m)

350

Cdudc

b) If e(z)

> m+l,

then we compute the number

i = card{i EN11 5

J'

2

t , t ( z , ) = t(z),m, = m }

,

and we notice that, by hypothesis and (5.11), 1 5 i 5 pe(')--'. In this and it is easily seen that c w we put E = y(.g(t'(z)-rn-2)+i)

d(z,t (4)= 2 * In both cases we have found a string z of length e(z)-m-l such that ~(z,!(z)) = 2, thus proving that (z,m)E V(6). Conversely, if (z,m)E V(t#), then we can fmd a string z in X* with ! ( z ) 5 t ( z ) - m - l such that # ( E , ~ ( Z ) )= 2. In view of the construction of 4, we fmd a t E RV, with # ( E , ~ ( z ) ) = zt and mt = e(z)-e(z)-l 2 t ( z ) - e ( z ) + m + i - i = m. Consequently, z = zt, (zt,mt) E V , and m 5 mt; 80, (z,m)E V, thus ending the proof. 0

Remark. The condition (5.11) is not necessary.

Example (6.7).

See in this respect

(6.18) Corollary. (STAIGER [1964]) Let V C X* x RV be a M-L test and let u E X*-{A}. Then the set tbv

is a representable

= {(=,m) I(z,m) E v)

!

M-Ltest.

Proof. For all naturab n,m 2 1, we have: card{y

EX* (y) = n,(y,m) E uV) = card{z EX* (2)= n-e (u),(z,rn) E V)

5 (p"-e(.)5 pn--1

-1)/(P -1) ,

because t (u) 2 1. Hence uV satisfies the condition (5.11), being represent able by Theorem (5.12). 0

(6.14) Theorem. (STAIGER [1984]) There exieta a M-L test V c X* x Bv such that the relation V V(4)faib to hold for every p.r. function 4 xN 4

:x*

x*.

c

Chapter 4

351

.

Proof. Firstly, we prove the following combinstorial result: Intermediate etep. Let W C X* X PV be a M-L test which satisfies the condition card{% EX*I!(z)

= n,(~,mE ) W ) = (p""-l)/(p-1)

,

(5.12)

for some naturals m,n 2 1. If W C V(+),for some p.r. function +:X*x JV then 4 maps the set {(z,n) Iz EX', !(z) 5 n-m-1) in an one-to-one manner onto the set {y E X * It (y) = n, (y,m) E W). Indeed, in view of the relation W C V(+), we have mw(z) 5 myc,,(s) = e(z)- K,(z lt(z))-l, for all z in W,(see Theorem (5.10)). Hence, for every E X * with !(y) = n and (y,m) E W, (i.e. mw(y) 2 m), there is a string z, in X with t ( z , ) 5 n-rn-1 and +(z,,n) = y. Since there are a t most (p"--l)/(p-l) strings of length less than n-m-1, the assertion of the Intermediate step follows from

ax*,

(5.12).

We continue the proof by picking two r.e. sets A,B C JV which cannot be recursively separated (see Theorem (2.9.8)). Furthermore, assume that 0,1,2 A U B . We defme the M-L test V C X*x EV an follows:

I

0, (y(s(n)+l), ...,y( e(n)+p+l)},

= {y(e(n)+W,

b (8 (a )+2)1, ,0 9

if n 5 2,m 2 0, if n 2 3,l I m I n-2, if n EA,m = n-1, if n EB,m = n-1, otherwise.

(Recall that { ~ ( n In ) 2 1) is the enumeration of X* in lexicographical order.) Clearly, V is really a M-L test. Suppose, by contradiction, that v c ~ ( 4 1 , for some p.r. function #:x*x PV AX*. Since for every n 2 3, card{z EX*It (2) = n, (z,n-2) E V) = p + l , we can use the Intermediate step to derive that the restriction of the p.r. function # to the set X U {A} acts in an one-to-one manner onto the set (y(a(n)+l), ...,y(8( n)+p+l)}. In particular, $(A,.) # 00, for every n 2 3. According to the defmition of V, for each n 2 3, we have:

352

Cdudc

{YMn)+qh if

{4(X,n)} = { V ( 8 ( 4 + 2 ) } ,

if

€4 n

EB.

The recursive set C = {n E RV In 2 3, #(A,.) = Y(a(n)+l)} separates the r.e. sets A and 8,thus contradicting our working hypothesis. 0

4.6. RECURSIVE MARTIN-LOFTESTS

In this section we focus our attention to recursive M-L tests. In this context we are able to express some previous facts in a more precise setting. (6.1) Theorem. (CALUDE and CHITESCU [1983c]) Let V C X* X RV be s recursive M-L test. Then we can effectively find a p.r. function d:X*x N a X* such that V C V(t$). Furthermore, 4 can be taken to poeseea the following properties:

4 is injective. The graph of 4 is recursive.

(6.1)

For every z in X*,(z,l) E V iff (z,1) E V(t$)

(6.2)

.

(6.3)

Proof. The set A = {(zrmV(z))Iz E V,}is recursive in view of Lemma (4.10). We distinguish two caaes: i) V is infinite and in this case there exists an injective recursive function g:RV-{0} + X*X w’ such that range(g) = A; ii) V ie finite and A hss q elements, and in this case there X*X nV such that elrists an injective (p.r.) function g:{1,2, ...,q} g({1,2 ,...,g}) = A. In all caaes, if i is in the domain of 8, we put g(i) = (2; d z i 1). Due to the recursiveness of V, we may suppoee that g haa the following “lexicographical” property: for all natural numbers 1 i <

-

<

(zi) I-t(zj),

if e(zi)

=

e (zj),then mv(zi) 2 mv(zi)

+

(6.4) (6-5)

We are ready to defme the p.r. function 4. For i = 1, g(l) = (z1,my(z1)).Put z1 = ~(s(t(zl)-mV(zl)-l)) and

353

Chapter 4

4 ( ~ 1 , ~ ( Z l )= ) 21

Next, let i = 2, 80 g(2) = (z2,mv(z2)).

In c w e (tl)z t (4, we put

=

v ( ~ ( (22)-m~(z2)-1)). e

In case t ( z l ) = t(z2), we consider the greatest element (according to the lexicographical order) of the set {p(l),g(2),...,y( 8 (e (22)-mv(z2)-l))){zl}, and we call thia element z2. In both cases, put =

+2,+2))

Continuing

the

procedure

22

we

-

reach

the

step

i > 1,

g(i) = (zi,mv(zi)). There are two cases. In the former case e(zi) # e(z,),

for all 1 5 j < i;set zi = y(8(e(zi)-mv(zi)-1)). In the latter (opposite) case let 1 j(1) < j(2) <...< j ( b ) < i be all indices 1 5 j < i such that t ( q ) = t ( z j ) . In fact j(2) = J(l)+l,j(3) = d2)+i, due to properties (6.4) and (6.5). We defie zi to be the greatest element (m lexicographical ) , 8 (e (zi)-mV(zi))}-{zj(l),zj(2),...tzj(k)~. order) of the set { ~ ( 1 ) , ~ ( 2...,y( In both c u e s put

...,

d(zi,e (zi)) = zi

.

Notice that 4 acts ae a function, because if t(zi) = e ( z j ) , then is possible and here follows the motivation. Put =...- P ( ~ ~ ( =~ 1t ). We have:

zi # zj. The construction qz,)= e(Zj(q) = e(zj(a))

-

mi = mV(zi) <_ mk = mV(zj(t)) = mV(zj(k-1))

<...<

...,

ml

=

5 mk-l *

mV(zj(l))

For every natural t E {1,2, k} U {i}, let

4 = b(l)lY(2),...,Y(4 --mt--l))}

'

Notice that

B1 C 8 and for every 1 5 u

2

C-.C Bt C Bi

9


=

Bo iff

m, =

.

m,

We shall describe in detail the action of 4. Clearly, Zj(1)

J

~(8(e-m,-1))

*

In order to obtain ~ ~ ( we ~ 1differentiate , two possible cases:

354

Cdude

(6-7)

m1> m2

or

.

ml=m2

(6.8)

If (6.7) holds, then zj(?)= ~(s(t-rn,-l)); in the case of (6.8) we have It is obvious that in case (6.8) one has s(t-m2-l)-l 2 1, since

8,= B2,80 zj(2) = Y(d(t-m2-1)-1). 2

<_ card{z EX*l t ( z ) = e,(z,m2)E V)

I (P--l)/(P-l) = e(t-m2-1)

.

The case when the strict inclusion occurs in (6.6) being clearly favourable, we focus our attention on the “bad” situation, i.e.,

=...= m,

mh = mh+l = mh+2

for 1 5 h have

Here, in caae h

> 1,

z j ( , ) = y(s(t-m-l)-(r-h))

.

5 r 2 i.

=m

,

we conaider mh-l

> mh.

We

It remains to show that s(t-m-1)-(r-h) 2 1, i.e. r - - h + l 5 (p‘--l)/(p-l). This relation follows from the inequalitiee r-h+l

halts.

5 card{z EX*It(%)= t , ( z , m ) E V) L (Pl-4AP-1) *

It is worth adding that in caae V is fmite the procedure eventually

Property (6.1) is a coneequence of the injectivity of g:(zi,mV(zi)) # (mj,mdzj)) iff zi # z j or mv(zi) # mv(zj). This implies that for distinct i and j one must obtain dirrerent value8 4(zi,t?(zi)) = zi and +(zi,t(zj)) = zj. We prove now the inclueion: V C V(t$). Indeed, in case (z,m) is in

365

Chapter 4

V, let (z,rnv(t)) = (zi,rnv(zi)), in the enumeration given by g. So, rn 5 rny(zi), and ti = #(zi,t (ti)),where the length of zi is less than t(zi)-mv(ti)-l, i.e., K+(z (t(z))5 t (zi)-rnv(zi)-I < t (z)-mv(z) 5 t (z)-rn, showing that (zp)E V(4). It is seen that for every z in X* for which (z,1) is in V(d),there exists a natural number i 2 1 such that 2 = zi, and (zi,mv(zi)) EV. It follows that (z,l) .EV; hence (6.3) was proved. All it remains to show is the recursivenees of the graph.of 4. T h i ia proved by taking arbitrarily ((z,t),z) 2 (z,t,z) in X*x N X X*,and checking if ( z , t , z ) belongs to the graph of 4, according to the following algorithm (recall that r n V is a recursive function in view of Lemma (4.10)): 1. If mv(z) = 0, no. STOP. 2. If t (z)# t , no. STOP. 3. Chooee i such that g(i) = (zi,rnv(zi)) and z = zi. 4. Run enough steps in the procedure defming 4 in order to find zi. 5. If t = zi, gC8. STOP. 6. No.STOP. 0

Remark. For a given recursive M-L test V there are many p.r. functions

4 satisfying Theorem (6.1), e.g. our construction depends on the enumeration function g. The converse implication in Proposition (5.9) also holds for recursive M-L tests. (6.2) Theorem. Let V be a recursive M-L test. Then the following conditions are equivalent:

The M-L test V is representable, card{z EX*l l ( z ) = n , r n v ( ~ )= m } 5 p”--’ for all naturals n

>m >0 .

(6.9) 3

(6.10)

Proof. The implication “(6.9) (6.10)’’is in fact a weakened form of Proposition (5.9). We deal with the converse implication. We shall prove that V = V(4),where 4 is the p.r. function constructed in Theorem (6.1). All it remains to prove is the inclusion V(4) C V. Take (z,rn) in V(4). In any caae (z,l) E V (see Theorem (6.1)). We shall prove that (z,rn) is in V by showing that my(.) 2 rny+)(z). For the sake of a contradiction, assume that mv(t) < mq&). It followa that (z,rnv(z)+l) is in V(d),hence there exists a atring a in X’

356

Cdudr

with ! ( z ) < !(z)-rnv(z)-l and

b(z,P (2))

=

2.

Let g(i) = (zi,mv(zi)),where z = zi, in the enumeration given by g (see the construction of I# in the proof of Theorem (6.1)). We let the procedure giving 4 run enough steps and we obtain the string zi such that I#(zi,l(zi)) = zi. We shall show that e(zi) = e(zi)-mV(zi)-l = t! (z)-mV(z)-l, thus deriving a contradiction (in view of the injectivity of #; see (6.1)).

Remember the action of 4. In case e(z,) # [(zj),for all 1 2 j < i, we have !. ( z j ) = !(zi)-mv(zi)-l, and the proof is finbhed in this case. In CBBe

e (2j(2)) =...= e ( Z j ( & ) ) = e (zi) , 1 2 j(1)< j ( 2 ) <...< j(k) < i,we have analysed several

e (.

)=

. 1(1)

possibilities, according to the existence of some equalities in the sequence of inequalities

for

mv(zj(1))L mv(zj(2))

>...> m ~ ( z j ( i t L) ) 4 . i )

*

<

In the case of the strict inequality rnv(zi) mv(zj(k)), we saw that P(zi) = !(zi)-rnV(zi)-l, and again the proof b finished. The most complicated cme is when

mV(zi) = mv(zj(i))=

mv(zj(k-1)) =.**- -

md2j(k-))

9

where 0 5 r < k. In this case we must put zi = y(8(t(zi)-mv(zi)-l)-(r+1)). In any case we have r + 2 elements z such that P(z)= n and mv(z) = m (we put e(zi) = A , and rnv(zi) = rn) and the hypothesis gives r+2

2 pn--1=

card{z E X * l l ( z ) = n-rn-1)

.

But y(.~(n--m-l)) is the last element (in the lexicographical order) of the set x"--l=

{z

EX* lqz) = n-m-~)

.

It follows that zi EX"-"-', which shows that t ( z i ) = A-m-1. ie finished in this case too.

The proof 0

Using Theorem (5.10)we get

V be a recursive representable M-L test and let X* be a p.r. function such that V = V(#)and range(#) = Vl . (8.11)

(6.8) Corollary. Let

#:X* x N

S,

Then the partial function U,:X*A N Lr+(z)= K,(z {e(z)) is a p.r. function with recursive graph.

given

by

357

Chapkr 4

Proof. In view of hypothesis (6.11) we can apply Theorem (5.10). From (5.9) and (5.10) it followe that U, is a p.r. function. Moreover, ( ~ , mE) Graph(UJ w mV(z) # 0 and m =

e (z)-mV(z)-l

Here we have used the recursivenew of the function

my

.

(see Lemma

(4.10)). 0

Remark. The p.r. function q5 given by the proof of Theorem (6.1) satisfies the additional property: range(#) = Vl. (6.4) Theorem. Let q5:X’ x N 44’ be a p.r. function such that range(q5) = (V(#))l. Then the following assertions are equivalent:

The partial function

U,:X’ *Hv given

by V , ( z ) = K4(z le(z)) is a p.r. function with a recursive graph.

(6.12)

The M-L test V(#)is recureive.

(6.13)

Proof. In order to prove the implication “(6.12) + (6.13)” we establish a chain of equivalences: For all (z,m) in X’ X IV we have: (2,m)

E V(4)

--

U,(Z)

< e (2)-

U,(Z)

E {0,1,2,...,[(2)-m-1}

U d Z )= 0 v U&) = 1 v...v V&)

=

!(z)-m-l

* ( ~ $ 0E) Graph (Vb) V (z,l)E Graph (V,) v

...V (z,e (2)-,-1) By convention, when .t (2)

E Graph (V,)

.

< m+l,the set {O,l, ...,l(z)-m-l)

For the converse implication, i.e., “(6.13) V = V(+)and we apply Corollary (6.3) to V and 4.

is void.

(6.12)”, we put 0

The following theorem will furnish a c h of recursive representable M-L tests Recall that for every set V C X’ x ( N- { 0 } )and for every natural number m 2 1 we have written V, = {z E X * 1(z,m) E V). We defme the critical level induced by V to be the function

.

358

Caludc

mV:X* -+NU

{m}

,

sup {m E N Im 2 1,z E V,},

in case such rn exists, otherwise.

Clearly, thia defmition ia compatible with the defmition of the critical level induced by a M-L test. (6.6) Theorem. Let V

every natural number m

C X* x A’be a r.e. set such that V,,, C V,, for 2 1.

a) The following assertions are equivalent: For all natural numbers n

> m 2 1, we have:

card{z EX* le (2) = n,(z,m) E V) = (p”“-l)/(p-1) For all natural numbers n

.(6.14)

> rn 2 1, we have:

card(z EX* lt?(z)= n,my(z) = m } = pn-m-1

(6.15)

b) If one of the above conditions (6.14) or (6.15) iS fulfiied for a set V subject to the general hypothesis, then V is a recursive representable M-L test.

Proof. a) Firstly, we deal with the implication “(6.14) (6.15)”. The hypothesis of the theorem and condition (6.14) eneure that V ia a M-L test, hence my takes only fmite values. Fix a natural number j and let n 2 j+l. In view of the inclusion V,,, c V,, we have:

{z E X * lt?(z)= n,rnV(z)= n--(j+l)} = {z E X *l t ( z ) = n,(z,n-j-l)

EV)

- {z EX* lt(z) = n,(z,n-j) EV)

.

Consequently, according to (6.14), we have:

-(.

) n-(j+l)} card{z EX*( P ( z )= n , r n ~ ( z= =

((pn

-pJ

.

+-I)

- 1 )/( p - 1)) - ((pn -in 4- 1)/(p -1))

Taking m = n - ( j + l ) , we obtain (6.15). Secondly, we prove the implication “(6.15) =+ (6.14)”. natural numbers n > m 2 1. For every natural g 2 1, put

Fix

the

Chapter 4

359

In view of (6.15), cardA,,-l = 1. Since A,, C A,,-l, it follows that car* 5 1. The equality car- = 1 would imply A,, = A,,-1, a contradiction. Consequently, A,, = (3. Moreover, A, = 0, for all u 2 n. We have proved the equality: {z EX* le(z) = n,(z,m)

E v)

From (6.16) we infer the equalities: card{z E X *l t ( z ) = n,(z,m) E V) =

=

n -1

card{z EX*le(z) = n , m d z ) = j )

C pn-i-l

n -1

.

- (Fn--l)/(p-l)

b) All it remains to prove is that under the general hypothesis, condition (6.14) implies the recursivenew of V (because in this case V will be a recursive M-L test satisfying condition (6.10) in Theorem (6.2)). Clearly, in view of (6.14), V is infinite. Let g :N-(0)+ X ' x IV be an injective recursive function such that range(g) = V. Put g ( i ) = (zi,mi), for every natural number i 2 1. We take an arbitrary (z,m) in X*x N and we describe an algorithm for testing if (z,rn) is in V. Put t ( z ) = n. There exiets a natural q 2 1 such that the set

G

= {9(1),0(2),...,9(~))

9

contains all the elements (y,m) in V with t ( y ) = n. Moreover, q can be effectively found. For instance, q can be taken to be the l e d natural number h such that the set

1)

( 9 (1),!?(2),...,9(h

9

contains exactly (pn--1)/(p-l) pairs (g,m) with t ( y ) = n. If (z,m)is in G , then (z,m) EV;if (z,m) G ,then (z,rn) B V. 0

360

Cdudc

(6.6) Definition. A M-L test V satisfying condition (6.14) (or, equivalently, condition (6.15)) will be called full.

Remark. Every full M-L test is recursive and representable by Theorem (6.5).

(6.7) Example. We shall give an example of a full M-L test V and we shall construct its associate p.r. function 9 such that V = V(#),M in Theorem (6.1).

a) Denote, for all naturals n > m 2 1, by A(n,m) the set {(z,m) E V l e ( z ) = n}. It is clear that V d be completely determined by the sets A(n,m). Recalling the lexicographical enumeration of X * given by { y ( n ) In > O} we set

A(n,rn) = {(y(s(n-l)+i),m) li=1,2, ...,s(n-m-l)} For every natural rn

(6.17)

2 1 one has

.

a,

V, =

U

A(n,m)

n==m+l

haa:

.

(6.18)

Clearly, V ia a full M-L test. Moreover, for every natural n rny(y(s(n-l)+l)) = n-1

,

2 2, one (6.19)

and rnv(y(s(n-l)+i)) = n-k-1

,

for every 1 <_ k 5 n-2, and i E {s(k-l)+l, s(k-l)+2, Also, rnv(z) = 0, for the other z in X * . An inspection of A ( n , l ) shows that for n

...,e ( k ) } .

2 2 we have:

card{z E X * l!(z) = n,(z,rn) EV, for some rn

2 1) = e(n-2)

(6.20)

.

b) In order to carry out the construction indicated in the proof of Theorem (6.1), we choose an enumeration function p for the set A = {(z,rnV(z))Iz E Vl}. This g will satisfy conditions (6.4) and (6.5). Furthermore, p has the supplementary property (which completely determines 9):

If for some i


one haa [(z,)= e(zj) and

mv(zi) = rnv(zj),then zi is greater than zj

361

Chapter 4

in the lexicographical order.

(6.21)

> m 2 1, the set l t ( z ) = n,my(z) = m) ,

Property (6.21) says that for all n

{z EX*

is ordered by the “inverse” lexicographical order. The p.r. function 4:X*x PJ A X * produced by the proof of Theorem (6.1) is given by d(v(i),n) = y(s(n-l)+i), for every n 2 2 and i E {1,2,...,~ ( n - 2 ) ) . 0

4.7. INFINITE OSCILLATIONS

In thie section we deal with the set of all initial segments of a fixed .. It is shown that €or every recursive

infinite sequence x = z1z2. z,...

function f haa

an

00

:N--.* JV such that Cp-’(”)

infmite

number

of

n 4

initial

=

CQ

every infmite sequence x

segments

zl...zz,

such

that

K,(z l...z, in) 5 n-f(n); here w :X* x PJ A X * is an arbitrary universal KOLMOGOROV algorithm.

The motivation of the following results comes from probability theory. Recalling Example (3.1), it is known that the deviation of s,/n from 1/2 is of aeymptotical order of magnitude G / n (see also Exercise (10.7)). Furthermore, the law of the iterated logarithm tells us that, for each fixed infmite binary sequence x = z 1z2...z,..., there exist infmitely many more moments n when (sn/n)-(l/2) is bigger than G / n , i.e. conthere exist infmitely many moments n when the binary string zl...z,, sidered as an element of the population of all binary strings of length n, is “non-random”. Our aim is to prove that the phenomenon just described also occurs when the randomness is measured by KOLMOGOROVL complexity associated to an universal KOLMOGOROV algorithm.

362

Cdudc

Notation. By X" we denote the set of all infinite x = 2122...2... n of elements in X. m

If 'I E X", then a) x(n) = zl...zn,for every n EN-{O},

> 0, and

sequences

b) for all n,

We begin with an useful combinatorial reault.

(7.1) Lemma. (KATSEFF [1978]) For each (nl1...,nk)E nVk, k 2 1, satisfying the inequality k

cP-'rl

1

i=1

one can effectively fmd the strings e(8;)

= ni, for each 1

for every x i, 1

such that:

81,...,8k

5 i 5 k , and

EX", there exists an

5 i 5 k, such that 8 i

= x(ni)

.

Proof. Assume, without loss of generality, that n l <_ n2 I...< nt. The strings a,,..., at will be produced by the following algorithm: 1. Put e l = a "1 l

.

2. For every 1 < i 5 k, let 8i be the smallest string (according to the lexicographical order induced by a1 < a 2 <...< a,,)of the set

A,,; = {z E X * l l ( z ) = ni, card{j a j C 2 } isminimum}

E IN 11 < j 5 i ,

.

Condition (7.2) is obviously fulfiied. To prove (7.3) we proceed by h u m e , for the sake of a contradiction, the existence of an infmite sequence x EX", auch that for every 1 5 i 5 k , .si Z x(ni). In other words, we can fmd an infinite sequence x EX" such that si 6 x(nk),for each 1 _< i 2 k. For every 1 5 i <_ k, there exists a string z E X * such that c a r d ( j E RV 11 5 j < i , s l c z) = 0. Indeed, we take z = x(ni). Consequently, for all 1 5 n < rn 5 k , 6 8 , . reduetio ad absurdum.

To each string di, 1

5 i 5 k, we associate

the aet

Chapter 4

363

-

Ex' (e( V ) = n k , 8i c $f}

Si = {V

It is seen that card Si = pncei, and k

c

Usi + {V

i4

EX*

I ~ ( v =) n t }

9

.(tit) 4 Si, for each choice of i, 1 5 i 5 k. Furthermore, n Sj = 0, for distinct indices i and j , we can write:

because Si

h

h

k

i =I

ir l

i=1

c a r d ( U S i ) = x c a r d S i = x p n k l c i < pnk k

Hence, z p - " '

since

.

< 1, thus contradicting (7.1).

i-1

(7.2) Lemma. Let such that

7

:JV

S,

nV

be a p.r. function with a recursive graph, (7.4)

Then we can effectively fmd a p.r. function r ' : N following five properties: n 4 '

Graph(7') C Graph(7)

<

~ ' ( n ) n, for each n

card{n € R V In-.'(.) 7'

=

E dom(7')

.

k} 5 1, for each natural k

4 dom(T),

M

.

(7.7) (7.8) (7-9)

then r(m) =

p*(m) = 0 does not contribute to zp'(").

&PI

.

haa a recursive graph.

Proof. Recall that if m r':N

a N having the

so,

the term

Defme the p.r. function

n 4

by

~ ( n ) ,if r(n) 5 n, and for every m

oq

< R,

otherwise.

The properties (7.6)-(7.9) follow from construction. Hence, we focus our attention on (7.5). We define the sets

364

Cdudc

A = {a E N b ( n ) I

At

EN

= {n

In-r(n) = k}

,

.

From (7.4) it follows that

nEA

because

cI+)>.

p-+)
M

A=UAi t 4

.

For every natural k for which At # 0, we denote by nt the smallest element of A t . It is obvious that dom(r’) = (at Ik E N , Ak # @). We have:

In case At #

0, EAk

n fn,

Since

n =nk+l a h

n =nk+l ni(n)==t

365

Chapter 4

we deduce that 00

(p/(p-l)).cp-+'(")

=

c

p+)

=

00

.

nEA

m=u

which proves (7.5). 0

(7.8) Lemma. Let 7 :RV a N be a p.r. function with a recursive graph satisfying (7.4). Then we can effectively find a recuraive function g:N---c X*satisfying the condition: For every x EX"", the set

4 x 1 = {t

E

Idt) = x(e ( g ( t ) ) ) and e

=

9

(7.10)

is infinite.

Proof. Replace 7 by 7 ' , the p.r. function furnished by Lemma (7.2). It is seen that for every natural n, .'(TI) # 00 iff ~ ' ( n= ) m, for some m 5 n. It follows that the domain of T' is actually rccureivc. The recursive function g will be defmed by stages a8 follows: Stage 0. I. Compute no= p n [

C n

2 11.

p+'(j)

1 4

r'(j)+m Extract from the vector (7'(o),...,7'(nO)) all fmite components, thus obtaining the vector (r'(io), ~'(i~,)). 2.

...,

3. Use the al5orithm provided by Lemma (7.1) in order to fmd the strings 8,,...98k EX having e ( 8 , ) = r f ( j j ) ,0 j 5 k,, and such that for each x E xm, there exists a j E {(),I,...,k,} satisfying 8, = x(e (6,)). 4. Defme g(ij) = a j , for all j E {0,1,...,ko}, and g(m)= A, for all m E {o,l,...,~o}-{~o,...,jk~}. Stage ( q + i ) .

I. Compute nq+l = w[n,

< n,

i: p+'(j) 2

11.

jm,+l

r'(j)+m 2. Extract from the vector (~ '(n , +l), T ' ( A , + ~ ) ) the fmite components, thus obtaining the vector (r'(ik +l),...,Tf(ik,+l)).

...,

f

3. Use the algorithm provided by Lemma (7.1) in order to fmd the

366

Cdude

strings 8k,+l, ..., E X * , having [ ( a , ) = r ’ ( i j ) , J’ E {k,+l, ...,k,+l}, and such that for each x EX”,we can find a j E {k,+l,...,k,+l}, satisfying 8j

=

X(! (8j)).

4. Defme g ( i j ) = sir for all

E {kq+l,...,kq+l} and g(m) = A , for all

J’

m E (nq+l,..’,n,+l}-{iL,+~,...,i~,-~}.

It is easy to notice that the above procedure really defines a recursive function g:N -+ X*. Condition (7.10) follows from the construction of g and the infinity of the domain of 7 ‘ . 0

Remark. In case the p.r. function 7 itself comes from Lemma (7.2), i.e. 7 satisfies conditions (7.5), (7.7)-(7.9), then t ‘ ( g ( t ) ) = 7 ( t ) , for each t

E dom(7)

.

(7.11)

(7.4) Proposltlon. Let r :N3 N be a p.r. function with a recursive graph satisfying (7.4). Then for each universal KOLMOGOROV algrithm w :X*x N we can find a constant e such that for every x E X” there exist infmitely many naturals n E dom(.r), for which the following inequality holds:

ax*

.

K,(x(n) In) 5 n--7(n)+c

(7.12)

Proof. Construct, for the given 7 , the p.r. function 7’:N3 N furnished by Lemma (7.2). By Lemma (7.3) we can construct the recursive function g : N -c x*satisfying (7.10). Defme the p.r. function #:X* x N J+X*by

[

g(n)y,

4(ar,n) =

OG,

if n - 7 ’ ( 4

otherwise,

=

w,

for all y E X * , n EN. The above definition works because in case n--7’(n) = [(y), then this n must be unique by (7.8). Take now x EX”. For each t E A(x) = {t E IV I g ( t ) = x(t ( g ( t ) ) ) and P ( g ( t ) ) = 7 ‘ ( t ) } , we construct the string Y(t)

=x

e(u(t))+l.t

.

Clearly, P ( ~ ( ~ 1=) t - 7 ’ ( t ) and d(?+),t) = 9 ( l ) Y ( t ) = x u )

Consequently,

-

367

Chapter 4

K o ( x ( t )It) 5 % ( t ) ) = f - r ' ( t )

'

KOLMOGOROV's Theorem furnishes a constant

e

such that

.

5 t-r'(t)+e

K,(x(t) I t ) <_ K,(x(t) I t ) + c

Finally, t - ? ' ( t ) _> 0, so r ( t ) = r l ( t ) . Consequently, for an infiity of c dom(.r), (7.12) holds.

t E dom(r')

0

(7.6) Lemma. (MARTIN-LOF [1971)) Let f:RV function such that

+

RV be

a

recursive

C p - J ( n ) = 30 . 00

(7.13)

n d

that

Then we can effectively fid a recursive function w

p-1

.

=

f':m

4

PI such

co ,and

(7.14)

niO

for each c E N,there exists a natural that f'(n)

2 f(n)+c, for all

2 N,

N, such

.

(7.15)

Proof. We defme, by primitive recursion, the recursive function F:N + RV: F(0) = 0

,

F(m+1) = pn[n > F ( m ) , and

Finally, defme f * by: f(n) = j(n)+m

2

P-'(~)

i=F(m)+l

> P"]

-

if ~ ( m<) n 2 F(rn+~) .

Clearly f * hae the requited properties. 0

(7.6)

f

Theorem.

:RV -+ N

(MARTIN-LOF

[1971],

be a recursive function such that

every universal KOLMOGOROV algorithm w each I EX",

KATSEFF 00

[1978])

Cp-'(")= OQ

" 4 .

Let

Then for

:X X IV 44. and for

368

Cdude

K,(x(n) In) 5 n - f ( n )

(7.16)

i.0.

Proof. Let f *:N4 N be the recursive function coming from Lemma

(7.5). Clearly, T = f satisfies the hypothesis of Proposition (7.4). Hence, we can fmd a constant e such that for every x EX"",the set

I

= {n

EN IK,(x(n) In) 5 n-f.(n)+c)

,

is infmite. Lemma (7.5) guarantees the existence of the natural N, satisfying (7.15). Consequently, the set

I' = I n {n E N

If*(n)

2

f(n)+c)

is still infmite. For each n E I' we have:

K,(W

in) 5 n - f f . ( 4 + e I n -

w

9

thus completing the proof. 0

Remark. In contrast with the law of the iterated logarithm from probability theory, the inequality (7.16) holds for each x EXrn,and not only with probability one. Theorem (7.6) holds for f (a)= [logpa].

4.8. PROBABILISTIC ALGORITHMS

The probabilistic tests of primality diecussed in the rust section of this chapter are only probably correct. This ie a common feature for all probabilistic algorithms. In the present aection we shall describe the clam of all probabilistic algorithms and we shall prove that the ability t o make random decisions does not increase the global computational power. We shall analyse a complexity-theoretic condition under which a probabilistic algorithm is error-free. This result is only of theoretical interest, since it involves the use of an infmite set of random strings, which is not r.e., by Corollary (4.4). Roughly speaking, there is only a theoretical (but not practical) poeeibility to convert a large class of probabilistic algorithm into equivalent deterministic onee.

First of all we formabe the general notion of probabilistic algorithm.

369

Chapter 4

(8.1) Definition. (ZIMAND [1983a]) A pair (f ,e), where

f:nvxX*s,N

,

is a p.r. function and e E [0,2-'] is a recursive real, is called a probabilistic algorithm that c-computes the partial function g:nvanv

,

provided the following two conditions hold: If g(n) f

00

and f (n,z) = g(n), for some n in

N

and z in X*, then f (n,zy) = g(n), for every y in

X*

.

(8.1)

For every n in dom(g), there exists a natural number t , , (which depends upon e and n) such that:

card{z

EX*14(z)= t,,,, f (n,z) = g(n)} > (l-c)pf""

.

(8.2)

Remarks. a) The computation of a probabilistic algorithm is influenced by a "random" factor, for instance the tosses of an unbiased coin (in the binary case). When writing f ( n , z ) we denote by n the input value and by the string z the encoding of the "random" behaviour. Condition (8.1) says that if the probabilietic algorithm reaches an acceptable state, then further random experiments are superfluous. According to condition (8.2), the probability that f computes g is greater than 1-e (i the encoding of the "random" factor is sufficiently long). Finally, choosing e in the interval [0,2-'] will ensure the uniqueness of the function evaluated by f (see Theorem (8.5)).

b) A model of probabilistic algorithm is the probabilistic TURING is a TURING machine with distinguished states (the cointossing states, in the binary case). For each such dietinguished state, the fmite control unit specifies p possible next states (in case X = {a1,a3, tap}, p 2 2). The computation is deterministic except that in the distinguished states the machine uses the output of a random experL ment (having exactly p results) to decide among the p possible next states. See for example, SANTOS [1971),MA" [1973], GILL [1976]. machine which

...,

(8.2) Example. We shall prove that the MILLER and W I N , and SOLOVAY and STRASSEN probabilistic algorithms satisfy Definition (8.1). We shall take c = 2-' and g:N -* (0,l) the (primitive recursive) characteristic function of the set of primes: g = PRIME.

370

CJudc

To be more precise we recall the common construction of these probabilistic algorithm. For every natural number n (which can be tested) we take j natural numbers b uniformly distributed in the set {1,2,...,ta- l}. For each such 6 we check whether some rued predicate W(6,n) holds. If so, n is composite; if not, n is prime (with a probability greater than 1-23.

The encoding of the “random” experiment (which consists of the selection of 6’s in the set {1,2,...,n-1}) ia binary. Put p = 2, u1 = 0, a2 = 1. For every subset Zc {1,2,...,n-1}, consider the binary string z of length %-I, defmed by z = Z~Z~...Z,-~, 1, i f i c l ,

Condition (8.1) ia obviously fulfied. (8.2) holds too, because in case n ie prime,

Take t,,n = n-1.

Condition

card{z EX’ l!(z) = n-l,f(n,z) = g(n)}

- 2”-1 > (1-2-1)2m-l

,

and in case n is composite, at least half of 6’s between 1 and n-1 satisfy the predicate W(b,n), i.e. card{z EX* It (2) = n-l,f(n,z) = g(n)} = card{z E X *

lt(z) = n - 1 , ~=~ 1 and W(6,n) holds

...,

for some b in {1,2, n-1}}

2 2n-I

n-1

C(

n-1

-

)2-k

kI0

- 2”-1-(3/2)”-1 -

= 2”-’( 1-(3/4)”-’)

> 2n-l(l-2-1), for n

2 5. 0

(8.8) Example. We consider the p.r. function by f(0,z) = 0 and for n > 0,

f :NX X*

N ,defmed

371

Chapter 4

n

,

if z contains at least one in the fvst n positions, otherwise.

01

Take c = l -l /p and t , , = n. The probabilistic algorithm f c-computes the identity function g : N -+ N,g(n) = n. 0

(8.4) Lemma. Let /:Nx X* 4 N be a probabilistic algorithm that r-computes the partial function g :PI3 N.

For every natural n in dom(g), and for every natural number t 2 1 for which there exists a recursive real p E [0,2-'] satisfying the inequality card{z

E X * lt?(z)= t , f ( n , z ) = g(n)} > (1-p)p'

,

(8.4)

card{z

EX*l!(z)

> (l-p)pr

,

(8.5)

we have: for every r

= r,f(n,z) = g(n)}

2 t.

Proof. We proceed by induction upon s = r-t. fact (8.4). Consider the sets:

If 8

= 0 , then (8.5) is in

A = {z EX* l[(z) = r,f(n,z) = g(n)) and

B

= {y

EX*lg

= zui, for some

Suppose that cardA

z in A and ai in X}

(8.6)

.

(8.7)

> ( 1 - p ) ~ ~.

Clearly, cardB = p cardA

> (1-p)pr+' .

(8.8)

Furthermore, for every y in B (notice that t? (y) = r + l ) we have, by hypothesis and condition (8.1), f(n,g) = g(n). In view of (8.8) it follows that card{g E X * "(y) = r f l , f(n,g) = g(n)} 2 cardB > ( 1 - p ) ~ ' ~ ' . 0

372

Cdude

> is replaced

.

by 2 Moreover, the recursiveness of p can be dropped here; this condition will be used in Theorem (8.5).

Remark. Lemma (8.4) holds a h when

(DE LEEUW, MOORE and SHANNON [1956], MA" [1973]) The class of all partial functions computed by probabilistic algorithms coincides with the clasa of p.r. functions. (8.6) Theorem.

Proof. If g:N 4 N is a p.r. function, then g is c-computed by the probabilistic algorithm f :Nx X* 4 N defined by f(.,~) = g(n), for all z in X*and E = 2-2. Conversely, we shall prove that the partial function c-computed by the probabilistic algorithm f as in Defmition (8.1) ia a p.r. function. Actually we shall present a procedure which computes the value of g for an arbitrary input n in EV: 1. Run the fvst step in the computation of f(n,z), for all z in with P(z) = 1.

X*

2. If for no z in X*with P(z) = 1, the computation halts within one step, then go to step 5. 3. In the opposite case, there exists a string z in X* with e(z) = 1 and f(n,z) halts in one step. Denote by n, the output thus obtained.

If

4.

f(n,z) = nl}

card{z EX*le(z) = 1, f ( n , z ) halts in one step and > (1-c)p, then g(n) = nl. STOP.

5. Run the fvst two steps in the computation of f ( n , z ) for all z in

X*with e (2) E {1,2}.

8. If for no z in X*with [(z)= 1, the computation halts within two steps, then go to step 9.

7. In the opposite case, there exists a string z in X* with t ( z ) = 1 and f(n,z) halts within two steps. Denote by n l the output thus obtained

.

8. If card{z E X * le(z) = 1, f(n,z) halts within two steps, and f(n,z) = nl)> (1-c)p, then g(n) = n,. STOP.

9. If for no z in X*with P ( z ) = 2, the computation halts within two steps, then go to step 12.

10. In the opposite case, there exists a string z in X* with P(z) = 2 and f ( n , z ) halts within two steps. Denote by n2 the output thus obtained. 11. If card{z EX* lP(z) = 2, f ( n , z ) halts within two steps, and f(n,z) = n2}> (l-e)p2, then g(n) = n2. STOP.

12. Run the fvst three steps in the computation of f(n,z), for all

2

373

Chapter 4

in x' with t ( z ) E {1,2,3}. As.0. It is seen that the above procedure acts algorithmically (since c is a recursive real!). Two possibilities may occur. If for all natural numbers m and t , f(n,z) does not halt within m steps or card{z EX*le(z) = t , f(n,z) halts within m steps, and f(n,z) = k} 2 (l-c)pt, for every k in RV, then g(n) = a In the opposite case, there exist the natural numbers m, t , and k such that card{z E X * It (2) = t,f(n,z) halts within m steps, (8.9 1 and f(n,z) = k} > ( 1 - e ) ~ ' , and this inequality holds fnst for the given m and t . The procedure asserts that g(n) = k. Indeed, condition (8.9) cannot hold for a fmed t and different k's according to the fact that c E [0,2-']. According to (8.2), the inequality (8.9) holds at leaat for some m, k and t = te,n. Moreover, in view of relation (8.1) and Lemma (8.4), (used for p = Q, t = t,,,) for d m' 2 m and t' 2 t the inequality card{z EX* lt(z) = t', f(n,z) halts witbin m' steps, and f(n,z) = k} > (l-c)p*', holds too. 0

Fix a probabilistic algorithm given by a recursive function 4 nV which c-computes the recursive function g:RV -+ RV.

f :pV x X*

For every recursive function h :RV + RV we consider the set:

~ ( h=){(z,m) Iz EX*,m E N - { o } ,

r(Wz)),z)f g ( h ( W ) ) , card{v

EX*l l ( v ) = W , f ( h ( W ) ,

> (1-P-AP-1NPV

Y ) = s(h(C(v))N

*

(8.10)

(8.6) Lemma. The set W ( h )is a recursive M-L test.

Proof. Clearly, W ( h ) is a recursive set. If (z,m) is in W(h)L+', for some natural k 2 1, then f(h(t(z)),z) # g ( h ( t ( z ) ) ) and , c a r d b EX*lW = W , f ( h ( W ) , v = ) o(h(W))}

> (1-p ++')/(p -l))p[ > (l-p-k/(p-l))p~(*)

(=)

.

Consequently, (z,m) is in W(h)b. Finally,

374

Calude

EX*It (2) = j,(z,rn) E W ( h ) }

card{z

<_ C-W

EX*

It ( 2 ) = j , f W ( j ) , Z )

< PJ+-P-/(P-WPj = p'"/(p-l)

=

e(k(i)),

.

Fix an universal KOLMOGOROV algorithm w and an universal M-L teat U (see Defmitiona (2.10) and (3.11)). Recall that K = K, is the KOLMOGOROV complexity induced by w , and rn = m u is the critical level induced by U.

(8.7) Theorem. (CALUDE and ZIMAM) [1984]) Let f : N x

g,h :hT + N ,be three recursive functions.

X*

3

N,

Assume that: A) f is a probabilistic algorithm that ecomputea g. B) For every natural n there exist a natural tn and a recursive real p,, E [0,2-'] such that:

lim)c,=o

n -cm

,

(8.11)

and card{z E X * lt(z) = t , , , f ( n , t )= g(n)} 2 (l-pn)pt"

.

(8.12)

2 no

Then there elcists a natural number n, such that for every n satisfying n =

h(t(y)), and 4(y) 2 t,, for some y in X*

,

(8.13)

we have

f ( n 4= o ( 4 for every random string z in X*such that n

7

=

h(4 (2)).

Proof. In view of Lemma (8.6) and Theorem (3.10) one i 2 1 such that:

rnW(h)(a)

5 m(z)+i

7

geta a natural

(8.14)

for every a in x*. Let q be the constant furnished by Theorem (3.16) for the pair (w ,U), and put: kn = P ~ ~ ( l / ~ n ( p - l ) ) ] - ( g + ~ + l *)

(8.15)

In view of (8.11) there exists a natural number nosuch that kn

> 0,

375

Chapter 4

for every n 2 no. Let k = knk. We shall prove that for each n 2 no, if there exists a string y with h ( P ( y ) ) = n and t ( p ) 2 t,, then f(n,z) = g(n), for all random strings z such that h ( P ( z ) )= n. We proceed by rcduetio ad absurdum. Suppose z to be random, n = h ( t ( 2 ) )2 ti,, a d

f ( n , z ) # o(n)

(8.16)

In view of Lemma (8.4) (see also the Remark which follows) and the hypothesis (8.12) we have: card{z

EX' lP (2)

= t! (z),f(n,z) = g(n)}

.

2 (1-pn)P

(8.17)

From the construction of the critical level we conclude that (z,rnw(k)(z)+l) !$ W(h). Hence: card{z EX' l t ( z ) = !(z),f(n,z) = o(n))

s (1-P

-by*)(=)+1) /(P -1))P l ( = )

(8.18)

Combining the inequalities (8.17) and (8.18) we obtain the relation

Pn

2P

-c"'y*fi)f',

AP-1)

s

or, equivalently,

~ w ( A ) (2z )[logp(l/~n(~-1))J-1

-

(8.19)

*

(8.20)

From (8.14) and (8.19) it follows: m(z)

2 [logp(l/~n(~-1))]-(i+1)

-

Finally, we use Theorem (3.16) which has furnished the constant q and the relation (8.20):

K ( z It (4)I e (z)-+)+q

I

(z)+(q +i+ 1)-Pogp(1/pn (~-1))l = P(z)-k,

<+) since k,

> 0.

9

We contradict the randomness of z.

-

376

Cdudt

Remarks. a) The set RAND is not r.e. (Corollary (4.4)). Consequently, we cannot practically convert each probabilistic algorithm satisfying the hypotheses of Theorem (8.7) into an equivalent deterministic one. b) In view of Theorem (2.11), card{z E X*(!

(2)

= t&(z It (z))2 t}.p-'

< l/(p-1)

,

i.e. the probability that astring z is random is greater than (p-2)/(p-l). c) The consistency of Theorem (8.7) follows from Theorem (8.8) which shows that the MILLER and W I N , and SOLOVAY and STRASSEN pr*maIity tests satiafy hypothesis B) in Theorem (8.7). (8.8) Theorem. For almost all inputa n, the probabilistic algorithm of MILLER and W I N , and SOLOVAY and STRASSEN are error-free in case the encoding of the coin tosses is a random binary string.

Proof. We consider the recursive function h : N + N ,h(n) = n + l , and we set, for every natural number n, pn = 2-['"!4, tn = n n l . For every natural number n 2 5, the condition (8.12) folIows from (8.3). Since limp,, = 0, the condition B) in Theorem (8.7) holds. Conen

-00

quently, for almost all n, and every random string z in X* with P ( z ) = i - 1 , the primality tests of MILLER and W I N , and SOLOVAY and STRASSEN are error-free.

Remark. A slightly diierent version of Theorem (8.8) was fvst proved by CHAITIN and SCHWARTZ 119781.

4.9.BISTORY The concept of program sire structure waa fust studied by SOLOMONOFF (19641, KOLMOGOROV [1965], and CHAITIN [1966]. CHAITIN [1977] credited MINSKY [1962] for the fvst publication of these ideas. Since that time, a number of different measures of program sise complexity have been introduced and studied, for instance BLUM [1967b],

Chapter 4

377

LOWLAND [1969], CHAITM [1975]. AU these measures, though differing in the programming language used and in the the additional information helping the computation, and in spite of varioue tradeoffs (see DALEY [1980]), have the same asymptotic properties. A comparative analysis was done in KATSEFF and SIPSER [1981]. The basic link between the KOLMOGOROV notion of randomness and the statistical tests was discovered by MARTIN-LOF [1966a] and [1966b]. He extended this study for arbitrary sequences. There some pathological phenomena occur (see CALUDE and CHITESCU [1983b]). See also FINE [1973] and NALlMOV [1981] for a critical discussion. The non-binary approach was investigated by CALUDE and CHITESCU [1982a], [1982b], [1983a]. The infmite oscillations of the complexity have been first announced by MARTIN-LOF [1965]. Detailed proofs appear in MARTIN-LOF [1971] and KATSEFF [1978] (see ale0 KATSEFF and SIPSER [1981]). Useful overviews can be found in MARTIN-LOF [1966a], ZVONKIN and LEVIN (19701, CHATTIN (19771, SCHNORR [1977], USPENSKY and SEMENOV [1981]; MANIN [1977] gives a hint of some results concerning KOLMOGOROV’s complexity in a slight different context. Related works are: KNUTH [1969], KAMAE [1973]. See ae0 KOLMOGOROV [1968], [19831. MANIN [1981] contains an interesting informal discussion of the basic resuits overviewed in Section 4.2. Many applications in biology, mathematical logic, cryptography and algorithmic information theory were developed: for related work see L. BLUM and BLUM [1975], CHAITIN [1977], COOK [1983].

4.10.

EXERCISES AND PROBLEMS

Section 4.2

ax*

(10.1) Let b:X* x nV be a p.r. function. For every z in X* and 0 < i 5 4(z), denote by z(i) the i t h prefm of 2, i.e. z ( i ) C z and e(z(i))= i. Pu t

378

Cdudr

l+(u,i)

D ( ~ , z= ) {y EX*

= z(i), for every

o < i 5 t (2))

.

Following LOVELAND [1969], defme the uniform eomplezity to be the partial function

induced by

+

K,( ; ) :x* x pv A N ,

K,(z;t(z)) =

i-, &(!

(Y)

Iv

E W , z ) ) ,if W , z )# otherwise.

0,

a) Check the validity of Lemma (2.7) and Corollary (2.8) using the uniform complexity inatead of KOLMOGOROV'a complexity. b) Show the existence of a p.r. function q:X* x JV 4 X *such that for every p.r. function d:X* x RV 4 X * there exists a constant e (depending upon 9 and +) such that

K & ; W ) 5 K,(z;+))+e for every z in

(10.1)

9

x*.

(10.2) Evaluate the difference I K,(z l-!?(z))-K,,(z;t(z)) I, for a fixed p.r. function satisfying condition (10.1) in Exercise (10.1). (10.3) (LOWLAND [lSSS]) Let f :N4 X * he a function. Show that f is recuraive iff K , . ( f ( n ) In) < e , for some conatant e in N ,and all n E IV;w :X*x JV 4 X is an univeraal KOLMOGOROV algorithm. (10.4) (LOVELAND [1969]) Recall the function rev:X* defmed in Exercise (1.10.23) by rev(1) = 1, rev(ail...a. ' k ) = ai k ...ail. Let w :X*x Bv A X *be a universal KOLMOGOROV algorithm. a) Show the existence of a constant q m EV such that

-x*

lK,(z for all z in

lW)-Kw(rev(z)

I++)))

I
?

(10.2)

x*.

b) Show that the uniform complexity induced by a p.r. function q satisfying (10.1) lacks the property (10.2). Section 4.8

(10.5) Construct a function f :mX IV .-+ PV eatk~fyingthe conditions (3.1)-(3.3) in Example (3.1). (10.8) (MARTIN-LOF [1966a]) Consider the M-L test in Example (3.1). Prove the existence of a constant c such that for every natural number n,

I2*s,-n

I<

f ( n - K ( z l...z, In)+c,n)

.

(10.3)

(10.7) (MARTIN-LOF [1966a]) Use the M O W and LAPLACE

Chapter 4

379

Theorem to prove that in the context of Exercise (l0.6),

where

Conclude that

1 2-en-n I is of the order of magnitude & provided

K ( zl...zn In) approximately equals ta.

(10.8) (LOVELAND [1969]) Let V be a M-L test, n > m 2 1 two natural numbers and e = n-m. A string z in V, with t (2) = n is said to be terminal for claes c at m provided there does not exiet a string y in Vm+lwith !(y) = n + l , and J 3 2. Denote by r ( V ) the set of all terminal strings for claw c at r, for 1 5 r < m. A M-L test V is said to be an uniform M-L teat provided for d natural numbers n > m 2 1 we have:

card{z E X * It ( 2 ) = n , E~ Vm}+ cardc+(V)
*

(10.4)

a) Show that the set of aIl uniform M-L teats is r.e. b) Prove the existence of an universal uniform M-L test (i.e. an uniform M-L test satisfying condition (3.12) for the clam of d uniform M-L tests). c) Restate Theorem (3.16) for the uniform complexity and uniform M-L tests. (10.9) Does there exist a universal M-L test which is uniform?

(10.10) Exhibit a proof of Theorem (3.17) making no use of Theorem (3.16).

Section 4.4 (10.11) Is every universal M-L test non-recursive? (10.12) Is the set

X*-RAND, r.e. for some natural m?

(10.13) (Open) Is Corollary (4.16) a consequence of Theorem (2.6.15)? Check also the converse implication.

380

Cdudc

Sectton 4.6

(10.14) Is the universal M-L teat Theorem (3.10)representable?

U constructed in the proof of

(10.15)Does there exist a non-representable universal M-L teat? (10.16) (STAIGER (19843) Use Theorem (5.12) t o obtain a direct proof of Theorem (5.6). (10.17) (ZVONKIN and LEVIN [1970)) Let p.r. function satisfying the inequality

$:X*x N AX' be

a

I+))

F K,(z l W ) + C r (10.5) for every p.r. function 4:X' x nV a x *and each z in X'; here the conthere exist a stant e depends upon + and b. Show that for every u in X*, K,(z

natural 8 (depending upon $ and u) such that

K,(z for each z in

I+)) L K,(uz l w 4 ) + 8

?

X*.

(10.18) (CALUDE, CHITESCU and STAIGER [1985] (added in x Bv be a p.r. function. Then the following proof)) Let $:X* statements are equivalent:

ax'

a) The p.r. function $ satiafiea (10.5). b) For every M-L test V, there exists a natural number q (depending upon V and 4) such that

mv(4 for all z in

x*.

L e (z)-&(z

lW)+Q

?

c) The M-L teat V($) is universal and there is a natural number d such that K,(z I t ( = ) )5 ! ( z ) + d , for each z in X'.

way.

(10.19)Use the above exercise to derive Theorem (3.16) in a direct

Section 4.6

(10.20) (STAIGER (19841) A M-L test W ia called weakly recursive in case the set <(z,mw(z)) Iz E W,}ia r.e. Show that every recursive M-L teat is weakly recursive but the converae implication is falae. (10.2l) (CALUDE, CHITESCU and STAIGER [1985]) Show that every weakly recuraive M-L test is uniformly embeddable into a weakly recursive representable M-L test. (10.22) Show that Theorem (6.2)in valid for weakly recursive M-L tests. (10.23) Exhibit an a a m p l e of a M-L test which is not weakly

381

Chapter 4

recursive. (10.24) Show that the M-L test W is recursive and satisfies condition (6.10) in Theorem (6.2) iff W is representable by a (total) recursive func-

tion.

(10.25) Show that the M-L test W is weakly recursive and sathies condition (6.10) in Theorem (6.2) iff W is representable by an injective p.r. function.

SectSon 4.7 (10.26) (CALUDE and CHITESCU [1983b]) For each z in X*, put zX" = {y EX" I y ( t (2)) = z},in case z # X, and AX" = X".

a) Sow that the set of all fmite mutually disjoint unions of sets of the form z x " generates a o-algebra C.

b) Prove that the computable function p:{zXo31zE X'} given by

p(2X.q = p+)

,

induces a probability p on C (i.e. the LEBESGUE probability). (10.27) (MARTIN-LOF [1971]) Let f : N + RV be such that iro

Show that with LEBESGUE probability one, for each x in Xo3, K,(x(n) In)

Here w

2 n-f(n)

a.e.

:x'x PJ a x *is an universal KOLMOGOROV algorithm.

-+

[0,1]