A Constructive Description of SAGBI Bases for Polynomial Invariants of Permutation Groups

J. Symbolic Computation (1998) 26, 261–272 Article No. sy980210

A Constructive Description of SAGBI Bases for Polynomial Invariants of Permutation Groups † ¨ MANFRED GOBEL

International Computer Science Institute, 1947 Center Street (Suite 600), Berkeley, California 94704-1198, U.S.A.

Let R be a commutative ring with 1, let R[X1 , . . . , Xn ] be the polynomial ring in X1 , . . . , Xn over R, and let G be an arbitrary group of permutations of {X1 , . . . , Xn }. This note presents a detailed analysis and a constructive combinatorial description of SAGBI bases for the R-algebra of G-invariant polynomials. Our main result is a ground ring independent characterization of all rings of polynomial invariants of permutation groups G having a finite SAGBI basis. c 1998 Academic Press

1. Introduction In G¨ obel (1995), the computation of bases for rings R[X1 , . . . , Xn ]G of polynomial invariants of permutation groups G was investigated. The aim of this note is to study the properties of canonical or SAGBI bases B of R[X1 , . . . , Xn ]G . These are such that — w.r.t. a given term order — every head term in R[X1 , . . . , Xn ]G can be expressed as a product of head terms in B. We investigate the general reduction technique for SAGBI bases and two variants thereof. We also present degree bounds to decide finiteness and a constructive description of SAGBI bases of R[X1 , . . . , Xn ]G by a combinatorial approach. Our main result will be to show that only permutation groups G generated by direct products of symmetric groups lead to corresponding invariant rings R[X1 , . . . , Xn ]G , which have finite SAGBI bases, independent of the given ground ring R. Moreover, any finite SAGBI basis consists only of multilinear polynomials. The concept of SAGBI bases was introduced by Kapur and Madlener (1989) and Robbiano and Sweedler (1990); it is a method to compute subalgebra bases in a similar way to computing Gr¨ obner bases for ideals (Buchberger, 1985) and — from the algorithmic point of view — to perform a Knuth-Bendix completion for term rewriting systems (Knuth and Bendix, 1970). SAGBI bases and Gr¨ obner bases have analogous reduction properties. The main difference is that SAGBI bases need not be finitely generated. Recent applications of SAGBI bases in commutative algebra are, e.g. Conca et al. (1996) and Huber et al. (1997). An analysis of formal modular seminvariants using SAGBI bases can be found in Shanks (1997), where a generating set for the ring of invariants for the four- and five-dimensional indecomposable modular representations of a cyclic group of prime order was constructed. Other well-known occurrences of SAGBI bases † E-mail:

[email protected]

0747–7171/98/090261 + 12

$30.00/0

c 1998 Academic Press

262

M. G¨ obel

in invariant theory are the elementary symmetric polynomials for the ring of symmetric functions and the invariant ring of an additive group, which is finitely generated (cf. Fogarty, 1969), and which has a finite SAGBI basis (cf. Vasconcelos, 1998). We now proceed as follows: Section 2 contains our notation and the basic definitions. Section 3 deals with some general properties and examples of SAGBI bases. Section 4 presents a first non-trivial description of SAGBI bases, which is, in general, not minimal but representation preserving. Section 5 describes SAGBI bases satisfying an additional reduction property, and finally, Section 6 deals with the most general reduction situation. 2. Basics N denotes the natural numbers. Let R be a commutative ring with 1, let R[X1 , . . . , Xn ] be the commutative polynomial ring over R in the indeterminates Xi , let T be the set of terms (= power-products of the Xi ) in R[X1 , . . . , Xn ], and let T (f ) be the set of terms Pn occurring in f ∈ R[X1 , . . . , Xn ] with non-zero coefficients. Lete1 max{een1 , . . . , en } of t = X1 . . . Xn and let ( i=1 ei ) be the maximal variable degree (total degree) Pn max{max{e1 , . . . , en } | X1e1 . . . Xnen ∈ T (f )} (max{ Pi=1 ei | X1e1 . . . Xnen ∈ T (f )}) be the maximal variable degree (total degree) of f = t∈T (f ) at t ∈ R[X1 , . . . , Xn ]. The variable set of t is defined as V ar(t) = {Xi |ei 6= 0, 1 ≤ i ≤ n}. In the following we fix the lexicographical order lex . . . >lex Xn . HT (f ) and HC(f ) denote the highest term t of f w.r.t.
SAGBI Bases for Polynomial Invariants of Permutation Groups

263

of B1 ∩ B2 , or it can be reduced with a product of some G-invariant orbits in B1 ∩ B2 . Let orbitG (t) be a G-invariant orbit with a total degree d, which is not an element of B1 ∩ B2 , w.l.o.g. say orbitG (t) ∈ / B2 . Then orbitG (t) can be reduced with a product of some G-invariant orbits in B2 having a total degree less than d, and so orbitG (t) can be reduced with a product of some G-invariant orbits in B1 ∩B2 by induction assumption. 2 Definition 3.3. Let B1 , B2 , . . . beT an enumeration of all simple SAGBI bases of R R[X1 , . . . , Xn ]G . Then SAGBIG = i=1,2,... Bi is the minimal simple SAGBI basis of G R[X1 , . . . , Xn ] . R R R is finite, if |SAGBIG | < ∞; SAGBIG is multilinear, if all orbitG (t) ∈ SAGBIG R SAGBIG are multilinear. Note that the elements of a multilinear SAGBI basis have a total degree of at most n. Theorem 3.4. Let f ∈ R[X1 , . . . , Xn ]G . Then f has a computable representation as a R . polynomial in the G-invariant orbits of SAGBIG Proof. See Kapur and Madlener (1989); Robbiano and Sweedler (1990) or Sturmfels (1995, Section 11). The representation can be computed by the following algorithm: R ; lex. term order; INPUT f ∈ R[X1 , . . . , Xn ]G ; {ψ1 , ψ2 , . . . , } = SAGBIG ˆ f := f ; p := 0; WHILE fˆ 6= 0 DO t := X1e1 . . . Xnen = HT (fˆ); a := HC(fˆ); R select ψi1 , . . . , ψil ∈ SAGBIG such that t = HT (ψi1 ) . . . HT (ψil ); p := p + a · Xi1 . . . Xil ; fˆ := fˆ − a · ψi1 . . . ψil ; ENDWHILE; OUTPUT f = p(ψ1 , ψ2 , . . .) with p ∈ R[X1 , X2 , . . .]. 2

The (non-)existence of a finite SAGBI basis implies the (non-)existence of a finite simR for 1 ≤ i ≤ n, and orbitG (X1e1 . . . Xnen ) ple SAGBI basis. Note that orbitG (Xi ) ∈ SAGBIG R ∈ SAGBIG implies either e1 = . . . = en = 1, or ei = 0 for some 1 ≤ i ≤ n. The following lemma presents a first combinatorial criterion for G-invariant orbits, which are definitely R . not elements of SAGBIG Lemma 3.5. Let t = X1e1 . . . Xnen ∈ HSG , and let {a1 , . . . ak } = {e1 , . . . , en } such that R / SAGBIG . 1 6= a1 and ai−1 + 1 < ai for 2 ≤ i ≤ k. Then orbitG (t) ∈ Proof. Let dk = max{i|ai ≤ ek } − 1 for 1 ≤ k ≤ n, let t1 = X1d1 . . . Xndn , let t2 = X1e1 −d1 . . . Xnen −dn , and let ρij ∈ {<, >, =} be such that ei ρij ej for 1 ≤ i, j ≤ n. Then we have t = t1 t2 , and di ρij dj and (ei − di )ρij (ej − dj ) for 1 ≤ i, j ≤ n. Hence, simply by construction t1 ∈ HSG and t2 ∈ HSG . 2 Example 3.6. Let t = X15 X33 ∈ HSA3 . By Lemma 3.5, we obtain t1 = X12 X3 ∈ HSA3 , R / SAGBIA . t2 = X13 X32 ∈ HSA3 , and t = t1 t2 . Hence, orbitA3 (t) ∈ 3 R . Then either orbitG (X1e1 . . . Xnen ) Corollary 3.7. Let orbitG (X1e1 . . . Xnen ) ∈ SAGBIG = X1 . . . Xn , or there exists i 6= j, k ∈ {1, . . . , n} such that ei = ej + 1 and ek = 0.

264

M. G¨ obel

Lemma 3.8. Let G1 and G2 act on X1 , . . . , Xk and Xk+1 , . . . , Xn , respectively, and let G = G1 × G2 be the direct product of G1 and G2 acting on X1 , . . . , Xn (cf. Rotman R (1994, p. 40)). Then SAGBIG is the union of the minimal simple SAGBI basis for G1 R[X1 , . . . , Xk ] and the minimal simple SAGBI basis for R[Xk+1 , . . . , Xn ]G2 . Proof. This follows from the fact that G = G1 × G2 implies orbitG (X1 . . . Xn ) = orbitG1 (X1 . . . Xk )orbitG2 (Xk+1 . . . Xn ). 2 The symmetric group Sn and the trivial group {id} are examples of permutation groups, which have a finite SAGBI basis. R[X1 , . . . , Xn ]Sn has a finite minimal simple SAGBI basis, and the representation of any f ∈ R[X1 , . . . , Xn ]Sn w.r.t. SAGBISRn = {σ1 , . . . , σn } is f = pf (σ1 , . . . , σn ), where pf is uniquely determined by f (cf. Sturmfels (1993, proof of Theorem 1.1.1)). R[X1 , . . . , Xn ]{id} has a finite minimal simple SAGBI R basis: SAGBI{id} = {X1 , . . . , Xn }. Note that the trivial group can be viewed as a direct product of symmetric groups S1 . The alternating group An is an example of a permutation group, which has no finite SAGBI basis. R[X1 , . . . , Xn ]An has a minimal simple SAGBI basis of the following form: d+1 d R = {σ1 , . . . , σn } ∪ {orbitAn (X1d+n−1 X2d+n−2 . . . Xn−2 Xn )|1 ≤ d ∈ N} SAGBIA n

Moreover, any f ∈ R[X1 , . . . , Xn ]An has a unique representation as f = pf,0 (σ1 , . . . , σn ) +

∞ X

d+1 d pf,d (σ1 , . . . , σn ) · orbitAn (X1d+n−2 . . . Xn−2 Xn ).

d=1

This result is a generalization of Lemma 2.1 in G¨obel (1995). 4. Representation Preserving Simple SAGBI Bases It is not very difficult to present a (non-trivial) constructive description of non-minimal simple SAGBI bases B of R[X1 , . . . , Xn ]G , which are ‘representation preserving’. The additional requirement here is that any f ∈ R[X1 , . . . , Xn ]G has a representation as a finite linear combination of the G-invariant orbits in B with polynomials in multilinear Ginvariant orbits in B as coefficients (cf. G¨obel (1995, Theorem 3.11 and Algorithm 3.12)). We proceed by recalling the concept of special terms and special G-invariant orbits. 6 I ⊆ {1, . . . , n}, and let m0 and m1 denote Definition 4.1. Let t = X1e1 . . . Xnen , let ∅ = the minimum and maximum of {ei | i ∈ I}, respectively. Then t is connected w.r.t. I, if / {e1 , . . . , en }, m1 = max{e1 , . . . , en }, and {ei | i ∈ I} is the set of all integers m0 − 1 ∈ between m0 and m1 . t connected w.r.t. {1, . . . , n} is called special, if either ei = 0 for some 1 ≤ i ≤ n, or e1 = . . . = en = 1. Let t = X1e1 . . . Xnen be non-special and connected w.r.t. I. Then the reduced term of t is defined as Red(t) = X1d1 . . . Xndn with di = ei − 1 for i ∈ I and di = ei , otherwise. Note that the number of special terms in R[X1 , . . . , Xn ] is finite, and every special term has a maximal variable degree of at most max{1, n − 1} and a total degree of at }. most max{n, n(n−1) 2 Lemma 4.2. Let t ∈ HSG be non-special. Then Red(t) ∈ HSG .

SAGBI Bases for Polynomial Invariants of Permutation Groups

265

Proof. The operator Red() does not change the equalities or inequalities holding between two arbitrary exponents of t, i.e. if t is a head term of a G-invariant orbit, then the same holds for Red(t). 2 Definition 4.3. Let t0 = t be connected w.r.t. I0 , let tj = Red(tj−1 ) be connected w.r.t. Ij , let tj−1 = uj−1 tj (1 ≤ j ≤ r ∈ N) and let tr be a special term. Then the total-reduced term of t is defined as RED(t) = tr with reduction set RSRED(t) = {u0 , . . . , ur−1 }. Note that RED(t) is always a special term. Theorem 4.4. Let t ∈ HSG be non-special, and let RSRED(t) ∩ M SG 6= ∅. Then R / SAGBIG . orbitG (t) ∈ Proof. Let s ∈ RSRED(t) ∩ M SG . Then t = stˆ = HT (orbitG (s))HT (orbitG (tˆ)), and so R / SAGBIG .2 orbitG (t) ∈ R Corollary 4.5. Let B = {orbitG (t)|t ∈ HSG , RSRED(t) ∩ M SG = ∅}. Then SAGBIG G ⊆ B, and every f ∈ R[X1 , . . . , Xn ] has a representation as a finite linear combination of the G-invariant orbits in B with polynomials in multilinear G-invariant orbits in B as coefficients. B is therefore a representation preserving SAGBI basis. Moreover, a finite representation preserving SAGBI basis B can only exist if R[X1 , . . . , Xn ]G is generated (as an R-algebra) by multilinear G-invariant orbits. R ⊆ B is, in general, not representation preserving as the following example SAGBIG shows:

Example 4.6. Let G = A3 × A3 = h(231), (564)i, and let f = orbitG (X12 X3 X42 X6 ). Then we know from Lemma 3.8 that orbitG (X1e1 X2e2 X3e3 X4e4 X5e5 X6e6 ) = orbitG (X1e1 X2e2 X3e3 ) · orbitG (X4e4 X5e5 X6e6 ) R = {orbitG (X1 ), orbitG (X1 X2 ), orbitG (X1 X2 X3 ), for all (e1 , . . . , e6 ) ∈ N6 , and SAGBIG orbitG (X4 ), orbitG (X4 X5 ), orbitG (X4 X5 X6 )} ∪ {orbitG (X11+d X3d ), orbitG (X41+d X6d )|1 ≤ d ∈ N}. f factorizes into orbitG (X12 X3 )orbitG (X42 X6 ), i.e. f can be reduced by the eleR R , but there exists no proper product of elements of SAGBIG for this ments of SAGBIG reduction containing at most one non-multilinear G-invariant orbit. R is finite, if any special t ∈ HSG is a product s1 . . . sl with si ∈ Lemma 4.7. SAGBIG M SG for all 1 ≤ i ≤ l and V ar(si ) ⊂ V ar(si+1 ) for all 1 ≤ i ≤ l − 1. Moreover, the R have in this case a total degree of at most n. G-invariant orbits of SAGBIG

Proof. Any non-special t ∈ HSG can be reduced by a product of head terms of multilinear G-invariant orbits (RSRED(t) must be a subset or equal to M SG ), because RED(t) is a product s1 . . . sl with si ∈ M SG for all 1 ≤ i ≤ l and V ar(si ) ⊂ V ar(si+1 ) for all 1 ≤ i ≤ l − 1. The total degree of s ∈ M SG is at most n. 2 R is finite, if s ∈ M SG for all multilinear s and all special Corollary 4.8. SAGBIG R ⊆ {orbitG (s)|s ∈ M SG }. t ∈ HSG with t = RED(st). Moreover, we have SAGBIG

266

M. G¨ obel

If t ∈ HSG is a product of head terms of multilinear G-invariant orbits, then this does not imply that t is a product s1 . . . sl with si ∈ M SG , 1 ≤ i ≤ l such that V ar(si ) ⊆ V ar(si+1 ) for all 1 ≤ i ≤ l−1. For example, let G = h(231)(564)i, and let t = X1 X4 X52 X62 , t1 = X1 X5 X6 , and t2 = X4 X5 X6 . Then t = t1 t2 ∈ HSG with t1 , t2 ∈ HSG , but there exists no product t = s1 . . . sl with si ∈ M SG , 1 ≤ i ≤ l such that V ar(si ) ⊆ V ar(si+1 ) for all 1 ≤ i ≤ l − 1. R R / SAGBIG does not imply that orbitG (t) ∈ / SAGBIG . For example, orbitG (RED(t)) ∈ 4 3 2 2 R let G = D5 = h(23451), (52)(43)i. Then we have orbitG (X1 X2 X4 X5 ) ∈ SAGBIG but 3 2 R 2 R / SAGBIG (orbitG (X1 X2 X5 ), orbitG (X1 X2 X4 ) ∈ SAGBIG ), and orbitG (X1 X2 X4 X5 ) ∈ R R but orbitG (X14 X23 X4 X52 ) ∈ / SAGBIG (orbitG (X13 also orbitG (X15 X24 X42 X53 ) ∈ SAGBIG 2 2 R X2 X5 ), orbitG (X1 X2 X4 ) ∈ SAGBIG ). 5. Separable SAGBI Bases A generalization of the reduction technique of the last section leads to non-minimal simple SAGBI bases of R[X1 , . . . , Xn ]G , which are not representation preserving. These SAGBI bases have an additional reduction property and a combinatorial decision criterion for finiteness. Definition 5.1. Let t = se11 . . . sel l ∈ HSG be the unique representation of t as a product of multilinear terms 1 6= s1 , . . . , sl with V ar(si ) ⊂ V ar(si+1 ) for 1 ≤ i ≤ l − 1. Then t is called separable, if t = a1 a2 with a1 = sd11 . . . sdl l ∈ HSG and a2 = se11 −d1 . . . sel l −dl ∈ HSG . A simple SAGBI basis B of R[X1 , . . . , Xn ]G is called separable, if every t ∈ HSG is separable by the head terms of the elements of B. We assume in the following that writing t = se11 . . . sel l means the unique representation, and that a separable SAGBI basis B is minimal (and therefore unique) in the sense of Lemma 3.2 and Definition 3.3. Note that t = se11 . . . sel l ∈ HSG implies {s1 , . . . , sl } ∩ M SG 6= ∅. t is special, if either e1 = . . . = el = 1 and sl 6= X1 . . . Xn , or l = el = 1 and sl = X1 . . . Xn . Moreover, R implies ei = 1 for all si ∈ M SG and |V ar(sl )| < n (cf. CorolorbitG (t) ∈ SAGBIG lary 3.7). Theorem 5.2. t = se11 . . . sel l ∈ HSG is separable iff tˆ = sd11 . . . sdl l ∈ HSG with di = 1 + δ(ei − 1) for 1 ≤ i ≤ l is separable. (Notation: δ(k) = 0, if k = 0, and δ(k) = 1, if 1 ≤ k ∈ N.) Proof. ‘=⇒’ Let t be separable by a1 = sd11 . . . sdl l and a2 = se11 −d1 . . . sel l −dl . Then δ(d ) δ(d ) ˆ1 = s1 1 . . . sl l ∈ HSG , δ(di ) + δ(ei − di ) ≤ 1 + δ(ei − 1) for all 1 ≤ i ≤ l. Hence, a 1+δ(e1 −1)−δ(d1 ) 1+δ(el −1)−δ(dl ) . . . sl ∈ HSG , and tˆ = a ˆ1 a ˆ2 . a ˆ2 = s1 ˆ ˆ eˆl dˆl dˆ1 eˆ1 ˆ ˆ1 = s1 . . . sl and a ˆ2 = se1ˆ1 −d1 . . . selˆl −dl . Then ‘⇐=’ Let t = s1 . . . sl be separable by a eˆi ∈ {1, 2} and 0 ≤ dˆi ≤ eˆi for all 1 ≤ i ≤ n. Hence, t is separable by a1 = sd1 . . . sdl and a2 = se11 −d1 . . . sel l −dl with  if dˆi = 0,  0, di = if dˆi = eˆi = 1, or dˆi = eˆi = 2, ei ,  di = ei − 1, if dˆi = 1, eˆi = 2

1

l

SAGBI Bases for Polynomial Invariants of Permutation Groups

267

for 1 ≤ i ≤ n. 2 Corollary 5.3. Let t = se11 . . . sel l ∈ HSG be non-special and non-separable with ei ≥ 2 for some 1 ≤ i ≤ l. Then tˆk = se11 . . . siei +k . . . sel l ∈ HSG is non-separable for any k ∈ N. Lemma 5.4. R[X1 , . . . , Xn ]G has a finite separable SAGBI basis B, if all non-special t = se11 . . . sel l ∈ HSG with ei ∈ {1, 2} for 1 ≤ i ≤ l are separable. Moreover, the G-invariant orbits in a finite B have a maximal variable degree of at }. To decide finiteness of most max{1, n − 1} and a total degree of at most max{n, n(n−1) 2 B, it suffices to check if all G-invariant orbits with a maximal variable degree of at most max{1, 2n − 3} and a total degree of at most max{n, n(n − 1) − 1} are separable. Proof. The finiteness property is a consequence of Theorem 5.2 and Corollary 5.3. Any non-separable t = se11 . . . sel l with ei ∈ {1, 2} for all 1 ≤ i ≤ l has a maximal variable degree of at most max{1, 2n − 3} and a total degree of at most max{n, n(n − 1) − 1}. The degree bounds for the elements of a finite B, which cannot contain a non-special and non-separable G-invariant orbit, are the degree bounds for special G-invariant orbits. 2 Theorem 5.2, Corollary 5.3 and Lemma 5.4 give us a combinatorial description of a separable SAGBI basis B of R[X1 , . . . , Xn ]G . The rest of the sequel shows that the finiteness of B has much more consequences: a complete characterization of all R[X1 , . . . , Xn ]G with a finite separable SAGBI basis is — independent of R — possible. Theorem 5.5. A finite separable SAGBI basis B can only exist, if R[X1 , . . . , Xn ]G is generated (as an R-algebra) by multilinear G-invariant orbits. Moreover, any finite separable SAGBI basis B is multilinear. To decide finiteness of B, it suffices to check whether all G-invariant orbits with a maximal variable degree of } are separable. at most max{1, n − 1} and a total degree of at most max{n, n(n−1) 2 Proof. Assume that B is finite but R[X1 , . . . , Xn ]G is not generated (as an R-algebra) by multilinear G-invariant orbits. Then there exists a non-multilinear special t = s1 . . . sl ∈ HSG , which is non-separable. We choose t of smallest total degree, if more than one such G-invariant orbit exists. B can only be finite by Theorem 5.2 and Corollary 5.3, if ts is separable for all s ∈ {s1 , . . . , sl }. If ts is separable, say ts = as bs for some s ∈ {s1 , . . . , sl } \ M SG , then s must occur in the unique representation of both as and bs (otherwise t would be separable). Moreover, we have as 6= s and bs 6= s, which implies that as and bs have a smaller total degree than t, and as and bs are also special. It follows that as and bs are separable, because t was non-separable and non-multilinear of smallest total degree. Hence, t itself is separable by a product of head terms of multilinear G-invariant orbits in B (contradiction). Consequently, any finite separable SAGBI basis B is multilinear and any head term of a non-multilinear special G-invariant orbit is therefore separable. This implies the second part of the statement. 2 Lemma 5.6. R[X1 , . . . , Xn ]G has a finite separable SAGBI basis B iff G is a direct product of symmetric groups. Proof. ‘⇐=’ Obvious.

268

M. G¨ obel

‘=⇒’ If n ≤ 2, G is either S1 , S2 , or S1 × S1 , i.e. we are done. Assume now n ≥ 3 and B is a finite separable SAGBI basis but G is not a direct product of symmetric groups. If G is any other direct product of permutation groups, say G = G1 × . . . × Gl , we can make use of Lemma 3.8 and apply the argument below to any non-symmetric group of the direct product. We assume for the rest of the proof that G is not a direct product of permutation groups. / (i) |{s ∈ M SG ||V ar(s)| = 1}| = 1: As G 6= Sn there exists a transposition π ∈ G, say (n − 1, n) after an adequate renaming of variables. Furthermore, we have 2 / M SG . It follows that t = X1n−1 X2n−2 . . . Xn−2 Xn ∈ HSG , but X1 . . . Xn−2 Xn ∈ t = s1 . . . sl is non-separable by a product of the elements of M SG (contradiction). (ii) |{s ∈ M SG ||V ar(s)| = 1}| = 2: After a proper renaming of variables, we can assume that orbitG (X1 ) = X1 + . . . + Xj and orbitG (Xj+1 ) = Xj+1 + . . . + Xn . If G restricted to {X1 , . . . , Xj } ({Xj+1 , . . . , Xn }) is not equal to Sj (Sn−j ), we can argue as in (i) for one of these sets of variables. Otherwise, G consists of two symmetric groups with G 6= Sj × Sn−j . Then there exists a π ∈ G, which is Sn−j restricted to a transposition, say (n − 1, n) after an adequate renaming of variables, and which also causes an action on the variables X1 , . . . , Xj . Furthermore, we have 2 / M SG . It follows that t = X1j X2j−1 . . . Xj−1 Xj · X1 X2 . . . Xj · Xj+1 . . . Xn−2 Xn ∈ Xj+1 . . . Xn−2 Xn ∈ HSG , but t = s1 . . . sl is not a separable product of the elements of M SG (contradiction). (iii) |{s ∈ M SG ||V ar(s)| = 1}| > 2: This case is a generalization of (ii). 2 Example 5.7. Let G = h(21), (43)i, i.e. G is a direct product of two symmetric groups. Then the finite separable SAGBI basis of R[X1 , . . . , Xn ]G is {orbitG (X1 ), orbitG (X1 X2 ), orbitG (X3 ), orbitG (X3 X4 )}. Example 5.8. Let G = h(21)(43)i, i.e. G is not a direct product of two symmetric groups. Then the infinite separable SAGBI basis of R[X1 , . . . , Xn ]G is {orbitG (X1 ),orbitG (X1 X2 ), orbitG (X3 ), orbitG (X3 X4 )} ∪ {orbitG (X1 X41+i )|i ∈ N}. R[X1 , . . . , Xn ]G is generated (as an R-algebra) by the multilinear G-invariant orbits, which follows for j ≥ 2 recursively from the equation X1 X4j + X2 X3j = (X1 X4j−1 + X2 X3j−1 )(X3 + X4 ) − (X1 X4j−2 + X2 X3j−2 )X3 X4 . This shows that the condition in Theorem 5.5 is necessary but not sufficient. Example 5.9. Let G = D4 = h(2341), (42)i, i.e. G is not a direct product of symmetric groups. Then the infinite separable SAGBI basis of R[X1 , . . . , Xn ]G is {orbitG (X1 ), orbitG (X1 X2 ), orbitG (X1 X3 ), orbitG (X1 X2 X3 ), orbitG (X1 X2 X3 X4 )} ∪ {orbitG (X12+i X21+i X41+i )|i ∈ N}. 6. Minimal Simple SAGBI Bases R but A separable SAGBI basis of R[X1 , . . . , Xn ]G is, in general, not equal to SAGBIG closely related. This section deals with this most general reduction approach.

Definition 6.1. t ∈ HSG is called reducible, if t = a1 a2 for some 1 6= a1 , a2 ∈ HSG .

SAGBI Bases for Polynomial Invariants of Permutation Groups

269

Example 4.6 shows that the reducibility of t ∈ HSG does not imply that t is separable: orbitG (X1d+1 X3d X4d+1 X6d ) = orbitG (X1d+1 X3d )orbitG (X4d+1 X6d ) is reducible for 1 ≤ d ∈ N but non-separable (cf. Lemma 3.8). α

l1 1 Theorem 6.2. Let t = se11 . . . sel l ∈ HSG be reducible, say t = a1 a2 with a1 = uα 1 . . . ul1

β

∈ HSG and a2 = v1β1 . . . vl2l2 ∈ HSG and let s ∈ {u1 , . . . , ul1 , v1 , . . . , vl2 } ∪ {ui vj | 1 ≤ i ≤ l1 , 1 ≤ j ≤ l2 }. Then the following holds: (i) RED(a1 )RED(a2 ) is reducible (cf. Definition 4.3). (ii) tsk ∈ HSG is reducible for all k ∈ N. If, in addition, t is non-separable, then there exists s ∈ / {s1 , . . . , sl }. Proof. (i) By Lemma 4.2, we have RED(a1 ), RED(a2 ) ∈ HSG , i.e. RED(a1 )RED(a2 ) = HT (orbitG (RED(a1 )RED(a2 ))) = HT (orbitG (RED(a1 )))HT (orbitG (RED(a2 ))) is reducible. (ii) is a consequence of (i). For the second part of the statement assume that t is non-separable but all s ∈ {s1 , . . . , sl }. Then we have {u1 , . . . , ul1 , v1 , . . . , vl2 } = {s1 , . . . , sl }, which implies that t is separable (contradiction). 2 Definition 6.3. Let t ∈ HSG . Then the set of reduction pairs of t is defined as Mt = {(RED(a), RED(b))|RED(a) ≤lex RED(b), t = ab for all 1 6= a, b ∈ T }. Qt = {tˆ ∈ HSG |Mtˆ = Mt } denotes the set of all terms, which have Mt as the set of reduction pairs. P(a,b) = {t ∈ HSG |(a, b) ∈ Mt } denotes the set of all terms generating a reduction set Mt with (a, b) ∈ Mt . Lemma 6.4. Let t = se11 . . . sel l ∈ HSG . Then the following hold: (i) Mt is a finite, unique set. Qt and P(a,b) can be both, finite or infinite, and there exists at least one infinite set Qt and at least one infinite set P(a,b) , respectively. Furthermore, |{Mt | t ∈ HSG }| < ∞, |{Qt | t ∈ HSG }| < ∞ and |{P(a,b) | a, b special}| < ∞. (ii) (a, b) ∈ Mab and ab ∈ P(a,b) . Q Q (iii) {(si , j6=i sj ), ( j6=i sj , si )} ∩ Mt 6= ∅ for 1 ≤ i ≤ l. Q Q (iv) If (si , j6=i sj ) ∈ Mt and (si , s1 . . . sl ) ∈ Mt , or ( j6=i sj , si ) ∈ Mt and (s1 . . . sl , si ) ∈ Mt then ei ≥ 2, otherwise ei = 1. (v) Mt1 = Mt2 implies RED(t1 ) = RED(t2 ), i.e. RED(t1 ) = RED(t2 ) for any t1 , t2 ∈ Qt . (vi) Let t be reducible. Then there exists (a, b) ∈ Mt with a, b ∈ HSG . Proof. These are consequences of Definition 6.3. 2 R is finite, if any infinite set Qt contains at least one reducible Lemma 6.5. SAGBIG term.

270

M. G¨ obel

Proof. Let t1 ∈ Qt be reducible, say t1 = ab, and let t2 be any other term in Qt . Then Mt = Mt1 = Mt2 , and we have either ξ = (RED(a), RED(b)) ∈ Mt , or ξ = (RED(b), RED(a)) ∈ Mt . If follows that t2 is also reducible, because ξ ∈ Mt2 , i.e. all R terms in Qt are reducible. By Lemma 6.4, |{Qt | t ∈ HSG }| < ∞, and therefore SAGBIG itself must be finite. 2 R is — from a combinatorial point of view — determined by The structure of SAGBIG the finite set of all sets of reduction pairs Mt and by the finite set of all sets Qt . The next lemma states an upper bound for the total degree of the terms in the set Qt , which have smallest total degree, and reports how far one has to proceed to compute all sets R . Mt to obtain a full view of the structure of SAGBIG

Lemma 6.6. Any Qt contains a term with a total degree of at most max{n, n

2

(n+1) }. 2

Proof. We are done, if n = 1. Otherwise, let w ∈ Qt with RED(w) = s1 . . . sl . We have to ensure that w is such that all possible reductions of the form w = ab with a, b 6= 1 cover the full set of reduction pairs (RED(a), RED(b)) ∈ Mt , which can be constructed from {s1 , . . . , sl }. Case l = 1: We have w = se11 for some e1 ∈ N. Simply by construction, we obtain all different sets Mw for e1 ≤ |V ar(s1 )|, i.e. ei ≤ n and the total degree bound for w is at most n2 . Case 2 ≤ l ≤ n: We have w = se11 . . . sel l for some (e1 , . . . , el ) ∈ Nl . Our goal is to show that it suffices to consider terms w with ei ≤ n for all 1 ≤ i ≤ l. This implies immediately 2 . that the total degree of w is at most nn + (n − 1)n + · · · + 2n + n = n (n+1) 2 Assume w = se11 . . . sel l with ei > n for some 1 ≤ i ≤ l. W.l.o.g. choose i to be maximal such that ei > n. We are done, if Mw = Mw0 for w0 = se11 . . . sei i −1 . . . sel l simply by repeating this reduction step until all ei ≤ n. Now, take any special u with u|w. Of course, u|w0 because all exponents e1 , . . . , en are sufficiently large. Let w = ut and let w0 = ut0 . We have to show that RED(t) = RED(t0 ). The maximal variable degree of u is at most n − 1. As any exponent of t and t0 belonging to a variable in V ar(sl ) \ V ar(si ) is at most γ = ei+1 + · · · + el , and any exponent of t and t0 belonging to a variable in V ar(si ) is at least n + ei+1 + · · · + el − (n − 1) > γ, it follows that RED(t) = RED(t0 ). Hence, Mw = Mw0 . 2 It is possible to prove a similar but significantly weaker version of Theorem 5.2 and Corollary 5.3. The following lemma holds, if certain relations among the exponents in the unique representation of a term are satisfied, and therefore it does not lead to a degree R . bound to decide the finiteness of SAGBIG Lemma 6.7. Let t = se11 . . . sel l ∈ HSG with 2 ≤ l ≤ n − 1, and let ei ≥ |V ar(si )|. Then the following are equivalent: (i) tk = tski ∈ HSG is reducible for all k ∈ N. (ii) tk = tski ∈ HSG is reducible for some k ∈ N. (iii) t is reducible. If t is non-reducible, then tk = tski is non-reducible for all k ∈ N.

P

j6=i ej

+

SAGBI Bases for Polynomial Invariants of Permutation Groups

271

Proof. ‘(i) =⇒ (ii)’ Obvious. ‘(ii) =⇒ (iii)’ Let tk be reducible for some 1 ≤ k ∈ N, say tk = ak bk with ak = P αl1 β 1 and bk = v1β1 . . . vl2l2 . Then ei ≥ uα 1 . . . ul1 j6=i ej + |V ar(si )| implies that si ∈ {u1 , . . . , ul1 , v1 , . . . , vl2 } ∪ {ui vj | 1 ≤ i ≤ l1 , 1 ≤ j ≤ l2 } and the corresponding exponents αi , βj , or both are greater than 1. Hence, we can divide tk by si and ak and bk by ui and vj , respectively, without changing the structure of the reduction, i.e. tk−1 is reducible. A successive application of this argument to tk−1 , tk−2 , . . . is possible and leads finally to t = t0 is reducible. αl1 βl2 β1 1 ‘(iii) =⇒P (i)’ Let t be reducible, say t = ab with a = uα 1 . . . ul1 and b = v1 . . . vl2 . Then ei ≥ j6=i ej + |V ar(si )| implies again si ∈ {u1 , . . . , ul1 , v1 , . . . , vl2 } ∪ {ui vj | 1 ≤ i ≤ l1 , 1 ≤ j ≤ l2 }. Hence, we can obviously reduce any tk = tski . The second statement of the lemma follows from (i) – (iii). 2 R is finite iff there exists a finite separable The rest of the paper shows that SAGBIG SAGBI basis B. R R is finite iff SAGBIG is multilinear. Theorem 6.8. SAGBIG

Proof. ‘⇐=’ Obvious. ‘=⇒’ We are done, if R[X1 , . . . , Xn ]G has a finite separable SAGBI basis. R is non-multilinear, i.e. there exists no finite separable Assume now that SAGBIG R SAGBI basis B ⊇ SAGBIG . Then there exists a special t = s1 . . . sl ∈ HSG and a s ∈ {s1 , . . . , sl } \ M SG such that — according to Corollary 5.3 and the proof of Theorem 5.5 — tsk is non-separable for all k ∈ N. tsk is reducible for all k ∈ N by the head terms R {a1 , . . . , ar } of the elements of SAGBIG , i.e. for any k ∈ N there exists an r-tuple αk1 r k kr (αk1 , . . . , αkr ) ∈ N such that ts = a1 . . . aα r . Let M be the infinite set of all such r-tuples, and define a well-founded order on M as follows: (α1 , . . . , αr ) ≤ (β1 , . . . , βr ) iff α1 ≤ β1 , . . . , αr ≤ βr . Now let k1 6= k2 ∈ N be α −α α −α such that (αk1 1 , . . . , αk1 r ) ≤ (αk2 1 , . . . , αk2 r ). Then s|k2 −k1 | = a1 k2 1 k1 1 . . . ar k2 r k1 r . Moreover, s|k2 −k1 | is not a head term of a G-invariant orbit (because s ∈ / M SG ), and α −α α −α a1 k2 1 k1 1 . . . ar k2 r k1 r is a head term of a G-invariant orbit, i.e. our assumption is R must be multilinear (contradiction). 2 wrong and SAGBIG R is multilinear iff R[X1 , . . . , Xn ]G has a finite separable SAGBI Theorem 6.9. SAGBIG basis B.

Proof. ‘⇐=’ Obvious (cf. Theorem 5.5). R . Then ‘=⇒’ Assume that there exists no finite separable SAGBI basis B ⊇ SAGBIG we obtain a contradiction by precisely the same argument as in the proof of Theorem 6.8. Hence, our assumption is wrong and B must be a finite separable SAGBI. 2 R is multilinear iff t ∈ HSG is separable for all non-multiCorollary 6.10. SAGBIG linear orbitG (t) ∈ R[X1 , . . . , Xn ]G iff R[X1 , . . . , Xn ]G has a finite separable SAGBI basis B (which is multilinear) iff G is a direct product of symmetric groups.

We have shown that R[X1 , . . . , Xn ]G has a finite SAGBI basis iff G is a direct product of symmetric groups. Moreover, the seemingly weaker concept of separable SAGBI bases

272

M. G¨ obel

has w.r.t. finiteness the same power as minimal simple SAGBI bases. These results hold independent of the ground ring R. Acknowledgement This work was supported by the Deutsche Forschungsgemeinschaft. The author would like to thank Professor Bernd Sturmfels (Berkeley), Dr Amin Shokrollahi (Berkeley), Dr Michael Pesch (Passau) and the anonymous referees for their comments and remarks. References Buchberger, B. (1985). Gr¨ — obner bases: an algorithmic method in polynomial ideal theory. In Bose, N. K., ed., Multidimensional Systems Theory, pp. 184–232. Reidel. Conca, A., Herzog, J., Valla, G. (1996). SAGBI bases with applications to blow-up algebras. — J. Reine Angew. Math., 474, 113–138 Fogarty, J. (1969). Invariant Theory. W. A. Benjamin. — Go — ¨bel, M. (1995). Computing bases for permutation-invariant polynomials. J. Symb. Comput., 19, 285– 291 Huber, B., Sottile, F., Sturmfels, B. (1997). Numerical Schubert calculus. preprint, 20 pages. — Kapur, D., Madlener, K. (1989). A completion procedure for computing a canonical basis of a k-subalgebra. — In Kaltofen, E., Watt, S., eds, Proceedings of Computers and Mathematics 89, pp. 1–11. Cambridge, MA, MIT. Knuth, D. E., Bendix, P. B. (1970). Simple word problems in universal algebras. In Leech, J., ed., Com— putational Problems in Abstract Algebra, pp. 263–297. Pergamon Press. Robbiano, L., Sweedler, M. (1990). Subalgebra bases. In Bruns, W., Simis, A., eds, Commutative Algebra — (Lect. Notes Math. 1430), pp. 61–87. Springer. Rotman, J. J. (1994). An introduction to the theory of groups, Graduate Texts in Mathematics, Vol. 148, — 4th edn. Springer. Shanks, R. (1997). Formal modular seminvariants. Department of Mathematics and Statistics, Queens — University, Kingston, Ontario, Canada, K7L 3N6. preprint, 18 pages. Sturmfels, B. (1993). Algorithms in Invariant Theory, Springer. — Sturmfels, B. (1995). Gr¨ — obner Bases and Convex Polytopes, AMS University Lecture Series, Vol. 8, Providence, RI. Vasconcelos, W. (1998). Computational Methods in Commutative Algebra and Algebraic Geometry. — Springer.

Originally received 20 May 1997 Accepted 31 March 1998