An infinite family of inv-Wilf-equivalent permutation pairs

European Journal of Combinatorics 44 (2015) 57–76

Contents lists available at ScienceDirect

European Journal of Combinatorics journal homepage: www.elsevier.com/locate/ejc

An infinite family of inv-Wilf-equivalent permutation pairs Justin H.C. Chan Simon Fraser University, 8888 University Drive, Burnaby BC, V5A 1S6, Canada

article

info

Article history: Received 16 February 2014 Accepted 14 August 2014

abstract Wilf-equivalence is one of the central concepts of pattern-avoiding permutations, and has been studied for more than thirty years. The two known infinite families of Wilf-equivalent permutation pairs, due to Stankova–West and Backelin–West–Xin, both satisfy the stronger condition of shape-Wilf-equivalence. Dokos et al. recently studied a different strengthening of Wilf-equivalence called invWilf-equivalence, which takes account of the inversion number of a permutation. They conjectured that all inv-Wilf-equivalent permutation pairs arise from trivial symmetries. We disprove this conjecture by constructing an infinite family of counterexamples derived from the permutation pair 231 and 312. The key to this construction is to generalize simultaneously the concepts of shapeWilf-equivalence and inv-Wilf-equivalence. A further consequence is a proof of the recent Baxter–Jaggard conjecture on even-shapeWilf-equivalent permutation pairs. © 2014 Published by Elsevier Ltd.

1. Introduction A permutation σ avoids a permutation π = a1 a2 . . . an if there is no subsequence b1 b2 . . . bn of σ such that ai < aj

if and only if bi < bj .

For example, σ = 314256 avoids π = 1243, but σ = 214365 does not because of its subsequence 1465. Let Sn (π ) be the set of all permutations of size n (acting on {1, 2, . . . , n}) that avoid the permutation π . Permutations α and β are Wilf-equivalent if

|Sn (α)| = |Sn (β)| for all positive integers n. E-mail address: [email protected]. http://dx.doi.org/10.1016/j.ejc.2014.08.031 0195-6698/© 2014 Published by Elsevier Ltd.

58

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

Fig. 1. The Young diagram associated with permutation 1423.

Fig. 2. Permutation avoidance, demonstrating that σ1 avoids π = 231 and σ2 contains π .

The problem of classifying all permutations up to Wilf-equivalence is ‘‘the basic problem in the theory of forbidden subsequences’’ [19], and has attracted a great deal of attention since it was posed by H. Wilf in the early 1980s. The seminal paper of Simion and Schmidt [17] in 1985 proved the Wilfequivalence of 312 and 321 by giving an explicit bijection between Sn (312) and Sn (321). Previous papers by Knuth [13,14] and Rotem [16] also proved the Wilf-equivalence of 312 and 321 through implicit bijections in the context of stack–sorting algorithms. Other bijections were given by West [21] and by others, as detailed by Claesson and Kitaev [9]. The papers by Albert and Bouvel [1] and Dokos et al. [10] generalize this problem to consider pattern avoidance of more than one permutation. Up to symmetry, two infinite families of Wilf-equivalent permutation pairs and one sporadic example are known [19,2,18]. Both of the infinite families were constructed by means of the stronger property of shape-Wilf-equivalence, as we now describe. Associate a permutation σ with a square Young diagram where each cell either contains a dot or is blank, and where a cell has a dot in the ith row from the top and jth column from the left if and only if σ (j) = i; for example, the permutation 1423 is associated with the square Young diagram in Fig. 1. An equivalent formulation of pattern avoidance in permutations is that σ avoids π exactly when the square Young diagram associated with σ does not contain as a subdiagram the square Young diagram associated with π . Now extend the idea of permutation pattern avoidance to Young diagrams Y that are not necessarily square, but which contain at least one transversal (namely, a set of dots such that each row and each column of Y contains exactly one dot, as in the middle diagram of Fig. 2 for example). Use (σ , Y ) to denote a transversal σ contained in a Young diagram Y (abbreviating (σ , Y ) to σ when the context is clear). We now say that (σ , Y ) avoids the permutation π if Y does not contain as a subdiagram the square Young diagram associated with π . For example, in Fig. 2, (σ1 , Y ) avoids π , even though the rightmost three dots of σ1 form a pattern coinciding with the dots of π , because the column of the rightmost of these dots and the row of the lowermost of these dots intersect outside Y . On the other hand, (σ2 , Y ) contains (does not avoid) π , as illustrated by the shaded cells. Let SY (π ) be the set of all transversals (σ , Y ) that avoid the permutation π . Permutations α and β s

are shape-Wilf-equivalent, written α ∼ β , if

|SY (α)| = |SY (β)| for all Young diagrams Y .

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

59

Fig. 3. The ⊕ operation on two permutations, demonstrating 312 ⊕ 231 = 312564.

We can consider a permutation of size n to be a transversal of the square Young diagram SQn of order n. Shape-Wilf-equivalence implies Wilf-equivalence, by restricting the range of Y to the set {SQn }. The following two theorems are simple to state, although apparently nontrivial to prove. s

Theorem 1 (Stankova and West [19]). 231 ∼ 312. s

Theorem 2 (Backelin, West and Xin [2]). 123 . . . n ∼ n(n − 1)(n − 2) . . . 1 for all n. The power of Theorems 1 and 2 is realized in combination with Proposition 3. Given permutations

α = a1 a2 . . . am and β = b1 b2 . . . bn , let α ⊕ β be the permutation c1 c2 . . . cm+n , where ci = ai for 1 ≤ i ≤ m and cm+i = m + bi for 1 ≤ i ≤ n. For example, 312 ⊕ 231 = 312564, as shown in Fig. 3. s

s

Proposition 3 ([2]). Let α and β be permutations and suppose α ∼ β . Then α ⊕ γ ∼ β ⊕ γ for all permutations γ . Combination of Theorems 1 and 2 with Proposition 3 constructs two infinite families of shape-Wilfequivalent, and therefore Wilf-equivalent, pairs of permutations. To my knowledge, no other infinite family of Wilf-equivalent permutations has been found. Alternative proofs of Theorem 1 have been given by Jelínek [12] and by Bloom and Saracino [5]. Jelínek [12, p. 204] points out that Theorem 2 is implied by a result on pattern avoidance in matchings due to Chen et al. [8, Thm. 1.1]. An alternative proof of Theorem 2 has been given by Krattenthaler [15], and alternative proofs of special cases of Theorem 2 have been given by Jelínek [12] and by Bloom and Elizalde [4]. Recently, Dokos et al. [10] studied a different strengthening of the concept of Wilf-equivalence, based on the inversion number of a permutation. The inversion number inv(π ) of a permutation π = a1 a2 . . . an is the number of pairs of indices whose corresponding entries in π occur in descending inv(σ ) order, namely |{(i, j) : i < j and ai > aj }|. Define the generating function In (π , q) = , σ ∈Sn (π ) q which categorizes the elements of Sn (π ) according to their inversion number. Let D4 be the dihedral group of symmetries of the square. Table 1 lists In (π , q) (n ≤ 6) for all distinct π of size s ≤ n with 2 ≤ s ≤ 4, up to symmetry under the action of D4 on π . i

We say that permutations α and β are inv-Wilf-equivalent, written α ∼ β , if In (α, q) = In (β, q)

for all n.

Inv-Wilf-equivalence implies Wilf-equivalence, by setting q = 1. It is straightforward to show i

[10, p. 2763] that if α ∼ β , then inv(α) = inv(β). Let D be the subgroup of D4 comprising the identity, 180° rotation, reflection through a line of slope 1, and reflection through a line of slope −1. Every permutation π is inv-Wilf-equivalent to its image under each of the elements of D acting on the square Young diagram associated with π [10, Cor. 2.2]. This gives rise to a trivial equivalence class {f (π ) : f ∈ D} for each permutation π . Dokos et al. conjectured that all inv-Wilf-equivalences are trivial. Conjecture 1 (Dokos et al. Special Case of [10, Conj. 2.4]). Let α and β be permutations of size n. Then i

α ∼ β if and only if f (α) = β for some f ∈ D. Dokos et al. verified Conjecture 1 by computer for n ≤ 5. They also proposed a stronger version of Conjecture 1, involving pairs of sets of permutations [10, Conj. 2.4], which was disproved by

60

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

Table 1 n Examples of In (γ , q) for 2 ≤ n ≤ 6, where [a0 , a1 , . . . , an ] denotes the polynomial i=0 ai qi .

π 21 231 321 3142

I2 (π, q)

I3 (π, q)

I4 (π, q)

I5 (π, q)

I6 (π, q)

1

1 [1, 2, 1, 1]

1 [1, 3, 3, 3, 2, 1, 1] [1, 3, 5, 4, 1] [1, 3, 5, 5, 5, 3, 1] [1, 3, 5, 6, 4, 3, 1] [1, 3, 5, 5, 5, 3, 1] [1, 3, 5, 6, 4, 3, 1] [1, 3, 5, 6, 5, 2, 1] [1, 3, 5, 6, 5, 2, 1] [1, 3, 5, 6, 5, 3]

1 [1, 4, 6, 7, 7, 5, 5, 3, 2, 1, 1] [1, 4, 9, 12, 10, 4, 2] [1, 4, 9, 13, 16, 17, 16, 13, 9, 4, 1] [1, 4, 9, 15, 18, 18, 16, 11, 6, 4, 1] [1, 4, 9, 13, 17, 17, 15, 13, 9, 4, 1] [1, 4, 9, 15, 18, 18, 15, 11, 7, 4, 1] [1, 4, 9, 15, 20, 20, 16, 10, 5, 2, 1] [1, 4, 9, 15, 20, 20, 16, 10, 5, 2, 1] [1, 4, 9, 15, 20, 22, 18, 11, 3]

[1, 5, 10, 14, 17, 16, 16, 14, 11, 9, 7, 5, 3, 2, 1, 1] [1, 5, 14, 25, 31, 26, 16, 9, 4, 1] [1, 5, 14, 26, 39, 50, 57, 64, 64, 57, 50, 39, 26, 14, 5, 1] [1, 5, 14, 29, 46, 61, 70, 72, 64, 53, 41, 27, 16, 8, 5, 1] [1, 5, 14, 26, 41, 53, 61, 65, 63, 56, 45, 37, 26, 14, 5, 1] [1, 5, 14, 29, 46, 61, 68, 68, 62, 53, 40, 29, 19, 11, 5, 1] [1, 5, 14, 29, 49, 68, 80, 79, 67, 51, 32, 20, 10, 5, 2, 1] [1, 5, 14, 29, 49, 68, 80, 79, 67, 50, 33, 20, 10, 5, 2, 1] [1, 5, 14, 29, 49, 71, 87, 91, 77, 52, 26, 10, 1]

[1, 2, 2]

3412 4123 4213 4231 4312 4321

1

Trongsiriwat [20]. But we shall show that even the single-permutation version stated in Conjecture 1 fails, for n = 6. In fact, we shall construct an infinite family of counterexamples to Conjecture 1. The key to this construction lies in further strengthening the concept of Wilf-equivalence to generalize simultaneously shape-Wilf-equivalence and inv-Wilf-equivalence. First extend the definition of inversion number to a transversal (σ , Y ). We define inv(σ ) to be the number of unordered pairs of distinct dots u, v of σ such that the line through u and v has positive slope. Note that the part of the line between u and v may extend outside Y . For example, inv(σ1 ) = 22 as shown in Fig. 2. This definition of inversion number for the square Young diagram associated with a permutation π is consistent with the definition of inversion number of π ; indeed, inv(1423) = 2 as shown in Fig. 1. Then, for a permutation π , define the generating function IY (π , q) =

qinv(σ )

 σ ∈SY (π)

si

and call permutations α and β shape-inv-Wilf-equivalent, denoted as α ∼ β , if IY (α, q) = IY (β, q) for all Young diagrams Y . Shape-inv-Wilf-equivalence implies shape-Wilf-equivalence by setting q = 1, and implies inv-Wilfequivalence by restricting the range of Y to the set {SQn }. The hierarchy of these relationships is shown in Fig. 4. Our main theorem generalizes the shape-Wilf-equivalence of 231 and 312 stated in Theorem 1 to shape-inv-Wilf-equivalence. si

Theorem 4 (Main Theorem). 231 ∼ 312. We shall also prove the following extension of Proposition 3, in Section 3. si

si

Proposition 5. Let α and β be permutations and suppose α ∼ β . Then α ⊕ γ ∼ β ⊕ γ for all permutations γ . Combination of Theorem 4 with Proposition 5 constructs an infinite family of shape-inv-Wilfequivalent, and therefore inv-Wilf-equivalent, pairs of permutations all of which are counterexamples to Conjecture 1. si

Corollary 6. 231 ⊕ γ ∼ 312 ⊕ γ for all permutations γ .

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

61

Fig. 4. Hierarchy of Wilf-equivalent relationships for permutations α , β , where Y is a Young diagram and σ is a transversal.

In particular, taking γ = 231 in Corollary 6 shows that 231 ⊕ 231 = 231564 and 312 ⊕ 231 = 312564 are inv-Wilf-equivalent, even though they are not trivially related by one of the four symmetries of D. Table 2 gives all permutations γ of size at most 4 such that 231 ⊕ γ and 312 ⊕ γ form a nontrivial inv-Wilf-equivalent pair, up to symmetry under the action of D4 on the pair. Furthermore, define Sn (π ) to be the set of permutations of size n containing (rather than  inv(σ ) avoiding) π , and define In (π , q) = . We note that In (π , q) and In (π , q) are related by σ ∈Sn (π) q In (π , q) + In (π , q) = [n]q ! where [n]q ! = i=1 (1 + q + · · · + qi−1 ) is the q-analogue of the factorial. In Table 2, we list In (231 ⊕ γ , q) rather than In (231 ⊕ γ , q) for 7 ≤ n ≤ 9, since In (231 ⊕ γ , q) is a shorter polynomial. Note that the listed polynomials In (231 ⊕ γ , q) are identical for some permutation pairs (e.g. those in rows 2 and 3). However, none of the permutations pairs in distinct rows of Table 2 are inv-Wilf-equivalent, but a value of n greater than 9 is needed to demonstrate this. We now describe another corollary of Theorem 4. Let (σ , Y ) be a transversal. Let AY (π ) be the set of all transversals (σ , Y ) with inv(σ ) even, that avoid the permutation π , and write An (α) = ASQn (α). Baxter and Jaggard [3] define permutations α and β to be even-shape-Wilf-equivalent if

n

|AY (α)| = |AY (β)| for all Young diagrams Y , and even-Wilf-equivalent if

|An (α)| = |An (β)| for all n. See Fig. 4 for the relationship of these definitions to the other Wilf-equivalent definitions. Conjecture 13 of [3] states that 231 is even-shape-Wilf-equivalent to 312. Since |AY (π )| = (IY (π , −1) + IY (π , 1))/2 for all permutations π , shape-inv-Wilf-equivalence implies even-shapeWilf-equivalence. Thus, we prove this conjecture as an immediate consequence of Theorem 4. Combination with [3, Lemma 7] and [19] proves that 231564 and 312564 are even-Wilf-equivalent, as are 465132 and 465213, thus completing the classification of pairs of even-Wilf-equivalent permutations

62

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

Table 2 n Examples of nontrivial inv-Wilf-equivalent pairs, where [a0 , a1 , . . . , an ] denotes the polynomial i=0 ai qi .

γ

231 ⊕ γ

312 ⊕ γ

I7 (231 ⊕ γ , q)

I8 (231 ⊕ γ , q)

I9 (231 ⊕ γ , q)

231564

312564

q4 [3, 6, 8, 8, 6, 4, 2]

q4 [6, 21, 45, 76, 101, 123, 132, 126, 104, 75, 46, 24, 10, 3]

1423

2314756

3124756

q4

q4 [3, 7, 10, 10, 8, 6, 4, 2]

2314

2315647

3125647

q4

q4 [3, 7, 10, 10, 8, 6, 4, 2]

2341

2315674

3125674

q5

q5 [3, 6, 10, 11, 8, 6, 4, 2]

2413

2315746

3125746

q5

q5 [3, 7, 10, 10, 8, 6, 4, 2]

2431

2315764

3125764

q6

q6 [3, 7, 10, 10, 8, 6, 4, 2]

3241

2316574

3126574

q6

q6 [3, 7, 10, 10, 8, 6, 4, 2]

3421

2316754

3126754

q7

q7 [3, 7, 10, 10, 8, 6, 4, 2]

q4 [10, 48, 133, 289, 511, 796, 1126, 1463, 1762, 1968, 2038, 1961, 1750, 1446, 1098, 761, 476, 266, 130, 54, 18, 4] q4 [6, 24, 57, 103, 147, 186, 208, 211, 194, 162, 120, 79, 46, 24, 10, 3] q4 [6, 24, 57, 103, 147, 186, 208, 211, 194, 162, 120, 79, 46, 24, 10, 3] q5 [6, 21, 52, 97, 143, 184, 209, 215, 198, 166, 124, 81, 46, 24, 10, 3] q5 [6, 24, 58, 105, 147, 184, 208, 210, 194, 162, 120, 79, 46, 24, 10, 3] q6 [6, 24, 59, 105, 149, 184, 206, 209, 194, 162, 120, 79, 46, 24, 10, 3] q6 [6, 24, 59, 105, 149, 184, 206, 209, 194, 162, 120, 79, 46, 24, 10, 3] q7 [6, 24, 59, 106, 150, 185, 204, 208, 193, 162, 120, 79, 46, 24, 10, 3]

231

of size 6, as laid out in Table 3 of [3, Sect. 5.3]. Note that this new result does not contribute to the classification of even-Wilf-equivalent permutations of size 5, as laid out in Table 2 of [3, Sect. 5.2], which therefore remains an open problem. We mention, without proof, an extension to another result appearing in [19] that can be obtained using the methods of this paper. Stankova and West [19, Cor. 2] describe specific pairs of Young diagrams Y , Y ′ for which |SY (231)| = |SY ′ (231)|. Using essentially the same argument as in [19, Section 5], the stronger result IY (231, q) = IY ′ (231, q) holds for these pairs. Shape-inv-Wilf-equivalence appears to be a very strong condition. We do not know of any examples of shape-inv-Wilf-equivalent pairs of distinct permutations other than those given by Corollary 6. Even the weaker condition of shape-Wilf-equivalence has only two known infinite families of pairs, namely the family given by combination of Theorem 1 with Proposition 3 (which now occurs as a special case of Corollary 6) and the family given by combination of Theorem 2 with Proposition 3 (which does not give shape-inv-Wilf-equivalent pairs, because the inversion numbers of each pair are not the same). 2. Notation and overview A Young diagram Y that contains at least one transversal must have n rows and n columns for some n, and must contain the staircase Young diagram of size n, namely the Young diagram whose row sizes are n, n − 1, . . . , 1 when reading down the rows. In this case, all transversals of Y have size n and we define the size of Y to be |Y | = n. For a transversal σ of Y , |σ | denotes the number of dots in σ . We use the convention that the rows of a Young diagram of size n + 1 are labelled 0, 1, 2, . . . , n from top to bottom, the columns are labelled 0, 1, 2, . . . , n from left to right, and (i, j) denotes the cell at row i and column j. Let Y denote the transpose of a Young diagram (reflected in a line of slope −1). Fig. 2 uses shaded cells to illustrate that the transversal (σ2 , Y ) contains the permutation π . A subdiagram of (σ , Y ) equal to the square Young diagram associated with π is called an instance of π in (σ , Y ).

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

63

Fig. 5. Commutativity of row decomposition (RD) and column decomposition (CD), where the Z are Young diagrams smaller than Y .

We say that a Young diagram Z is smaller than a Young diagram Y if Z is a proper subdiagram of Y . We now summarize the idea of the proof method for Proposition 3 given by Backelin, West, and Xin [2]. Suppose that σ is a transversal in a Young diagram Y that avoids the permutation α ⊕ γ . We shade the cells of (σ , Y ) in such a way that the shaded portion itself forms a transversal in a Young diagram and avoids α . Then we can establish a bijection between α ⊕ γ -avoiding transversals and β ⊕ γ -avoiding transversals by simply replacing the shaded portion avoiding α with a portion having the same shape that avoids β , using the assumed bijection between α -avoiding transversals and β -avoiding transversals. We shall prove Proposition 5 in Section 3 by adapting the Backelin-West-Xin proof to take account of inversion number. Although the shading of cells is the same, we must now account for the positions of the dots. We shall consider four cases, according to whether each of the two dots of a pair lies in a shaded or an unshaded cell. We next summarize the proof method for Theorem 1 given by Stankova and West [19]. 1. Define a function T (Y ) which counts the number of 231-avoiding transversals in Young diagram Y (in other words, T (Y ) = |SY (231)|). Then Theorem 1 is equivalent to the statement that T (Y ) = T (Y ) for all Young diagrams Y . 2. Establish a functional relation called ‘‘row decomposition’’ [19, Thm. 2 and Cor. 1], which computes T (Y ) from values T (Z ) for Young diagrams Z smaller than Y . Row decomposition has a relatively straightforward combinatorial proof. The form of row decomposition given in [19, Thm. 2] is a summation of multiplied pairs of values of T (Z ), and the consequence given in [19, Cor. 1] is of the form T (Y ) = T (Z1 ) + T (Z2 )T (Z3 ). This latter result [19, Cor. 1] forms a central part of the proof. 3. Establish another functional relation called ‘‘column decomposition’’ [19, Lem. 3], which is a reflected version of row decomposition. Whereas row decomposition has a combinatorial proof, column decomposition is proved instead by induction using the commutativity of row and column decompositions shown on the left side of Fig. 5. That is, if we apply row decomposition to T (Y ) and then (using the inductive hypothesis) column decomposition to the smaller resulting parts, we obtain the same formula as if we applied column decomposition to T (Y ) and then row decomposition to the smaller resulting parts. 4. Use row decomposition and column decomposition together to prove by induction that T (Y ) = T (Y ) for all Young diagrams Y . We shall prove Theorem 4 in Section 4 by adapting the Stankova-West proof to take account of the inversion number, as follows. 1. Define a function I (Y ) which maps a Young diagram Y to a generating function in q, and which counts the number of 231-avoiding transversals σ in Y in terms of the inversion number inv(σ ). Then Theorem 4 is equivalent to the statement that I (Y ) = I (Y ) for all Young diagrams Y . 2. Establish an inversion-number version of row decomposition which computes I (Y ) from values I (Z ) for Young diagrams Z smaller than Y . We use a similar argument to that used in the combinatorial proof of [19, Thm. 2], but we must incorporate the inversion number carefully. The form of this row decomposition is the same as the form given by [19, Thm. 2] except that each term is multiplied by a power of q which depends on the index of summation. This proof, which

64

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

Fig. 6. Illustration used in the proof of Proposition 5.

depends on the shape of Y and which requires additional notation, will be motivated in Section 4. We also give in passing the inversion-number version of [19, Cor. 1], although it is not used as a central part of the proof as it is in Stankova–West. 3. Establish column decomposition as a reflected version of row decomposition. Column decomposition is likewise proved by induction using the commutativity of row and column decompositions as shown on the right side of Fig. 5. Various modifications have been applied to [19, Lem. 3] to form the proof in this paper, and we leave further details for Section 4. 4. Use row decomposition and column decomposition together to prove by induction that I (Y ) = I (Y ) for all Young diagrams Y . 3. Proof of Proposition 5 For convenience, we recall the statement of Proposition 5. si

si

Proposition 5. Let α and β be permutations and suppose α ∼ β . Then α ⊕ γ ∼ β ⊕ γ for all permutations γ . Proof of Proposition 5. (See Fig. 6.) Let (σ , Y ) be a transversal avoiding α ⊕ γ . Colour a cell of Y light grey if it lies to the upper-left of all dots of some instance of γ contained in Y . Let σ ′ be the set of dots of σ that are on light grey cells, and let δ be the set of dots of σ that are not in σ ′ . Recolour a light grey cell dark grey if there exist dots u, v ∈ σ ′ , not necessarily distinct, such that the cell is in the same row as u and in the same column as v . Let the dark grey cells be Young diagram Y ′ as a subdiagram of Y . Then σ ′ is a transversal of Y ′ . The transversal (σ ′ , Y ′ ) avoids α ; otherwise the original transversal (σ , Y ) contains α ⊕ γ . Fig. 6 illustrates this shading for a particular transversal σ and Young diagram Y , with γ = 312. si

Use the implied bijection α ∼ β to map σ ′ to transversal τ ′ of Y ′ , which avoids β and has the same inversion number as σ ′ . Form a new transversal τ = τ ′ ∪ δ of Y that avoids β ⊕ γ . We claim that inv(τ ) = inv(σ ). If so, then by mapping σ = σ ′ ∪ δ to τ = τ ′ ∪ δ , we preserve the inversion number and obtain a (β ⊕ γ )-avoiding permutation in Y . By an analogous argument, we can reverse this process to obtain a map from τ to σ . So we have a bijection between the set of (α ⊕ γ )-avoiding permutations on Y and the set of (β⊕γ )-avoiding permutations on Y that preserves inversion number. It remains to prove that inv(τ ) = inv(σ ). Let Q (u, v) = 1 if dot v is to the upper-right of dot u and Q (u, v) = 0 otherwise. Then inv(π ) =

 u,v∈π

Q (u, v)

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

65

for all transversals π , and so



inv(τ ) =



Q (u, v) =

u,v∈τ



=

Q (u, v)

u,v∈τ ′ ∪δ



Q (u, v) +

Q (u, v) +

u,v∈δ

u,v∈τ ′





Q (u, v) +

Q (u, v).

(1)

u∈τ ′ ,v∈δ

u∈δ,v∈τ ′

From the assumed inversion-preserving bijection, we have u,v∈τ ′ Q (u, v) = u,v∈σ ′ Q (u, v).   We next show that Q ( u , v) = Q ( u , v) for a given u ∈ δ . Let u ∈ δ and consider ′ ′ v∈τ v∈σ the set of cells of Y ′ (possibly empty) which are to the right of u, which we call Z . Now all cells of Z must be above u; otherwise, a cell of Z (and therefore Y ′ ) to the right or lower-right of u would mean u is in σ ′ instead of δ . Therefore, for some integer z, the cells of Z compose the z rightmost ′ columns of Y ′ . Since every exactly one dot in each   transversal of Y must have  of its columns, then Q ( u , v) = z = Q ( u , v) . It follows that ′ ′ v∈τ v∈σ u∈δ,v∈σ ′ Q (u, v). By a  u∈δ,v∈τ ′ Q (u, v) = similar argument involving rows, u∈τ ′ ,v∈δ Q (u, v) = Q ( u , v) . Thus we switch τ ′ with σ ′ ′ u∈σ ,v∈δ in (1), completing the proof. 





4. Proof of Theorem 4 We define the ‘‘non-inversion number’’ inv∗ (σ ) to be the number of unordered pairs of distinct dots u, v of σ such that the line through  uand v in the Young diagram Y has negative slope. It is

easily shown that inv(σ ) + inv∗ (σ ) =

|σ | 2

, and so for transversals σ1 , σ2 with equally many dots,

inv(σ1 ) = inv(σ2 ) if and only if inv (σ1 ) = inv∗ (σ2 ). For example,the transversal σ1 of Fig. 2 satisfies ∗

inv(σ1 ) = 22 and inv∗ (σ1 ) = 6, with inv(σ1 ) + inv∗ (σ1 ) = 28 = We also define, as an analogue of IY (π , q), IY∗ (π , q) =



8 2

.

∗ qinv (σ ) .

σ ∈SY (π)

Since inv(σ ) + inv∗ (σ ) =





IY∗ (π , q) = IY (π , q−1 )q

|σ | 2

|Y | 2





for all transversals σ , for all permutations π .

si

Therefore, α ∼ β if and only if IY∗ (α, q) = IY∗ (β, q)

for all Young diagrams Y .

We use the shorthand I (Y ) to represent IY∗ (231, q). We consider the size 0 (empty) Young diagram to be trivial, and all others to be nontrivial. We use the convention that I (Y ) = 1 when Y is trivial. Let a0 a1 . . . an denote the Young diagram of size n + 1 with ai cells in row i for 0 ≤ i ≤ n, and ∅ denote the trivial Young diagram. Table 3 gives I (Y ) = IY∗ (231, q) for all Young diagrams up to size 4. The table suggests some functional relations between generating functions I (Y ) for certain related Y . In particular, we note the following simple functional relation: Proposition 7. Let Y1 = a0 a1 . . . an−1 1 and Y2 = a0 a1 . . . an−1 2 be Young diagrams. Then I (Y2 ) = I (Y1 )(1 + q). Proof. Let Y0 = (a0 − 1)(a1 − 1) . . . (an−1 − 1) be the Young diagram formed by deleting the leftmost column of Y1 . We calculate I (Y2 ) by summing the contributions when the dot in the bottom row of Y2 lies in column 0 and when it lies in column 1. Since that dot can never play the role of 3 in an instance of 231 in Y2 , we obtain I (Y2 ) = I (Y0 ) + I (Y0 )q, where the additional factor of q when the dot lies in column 1 arises from an additional contribution of 1 to inv∗ (σ ). Similarly, I (Y1 ) = I (Y0 ). 

66

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76 Table 3 The generating function I (Y ) = IY∗ (231, q) for various Y . I (Y )

Y

∅ SQ1 = 21 SQ2 = 321 322 331 332 SQ3 = 4321 4322 4331 4332 4333 4421 4422 4431 4432 4433 4441 4442 4443 SQ4 =

1 22

333

4444

1 1 1 1+q 1 1+q 1+q 1 + 2q + q2 1 + q + 2q2 + q3 1 1+q 1+q 1 + 2q + q2 1 + q + 2q2 + q3 1+q 1 + 2q + q2 1 + 2q + q2 1 + 3q + 3q2 + q3 1 + 2q + 3q2 + 3q3 + q4 1 + q + 2q2 + q3 1 + 2q + 3q2 + 3q3 + q4 1 + q + 3q2 + 3q3 + 3q4 + q5 1 + q + 2q2 + 3q3 + 3q4 + 3q5 + q6

Example 1. As can be seen in Table 3, I (4442) = 1 + 2q + 3q2 + 3q3 + q4 = (1 + q + 2q2 + q3 )(1 + q) = I (4441)(1 + q). Other relations, however, are not so obvious. The table suggests that I (Y ) = I (Y ), a statement linked to the shape-Wilf-equivalence of 231 and 312. si

Lemma 8. 231 ∼ 312 if and only if I (Y ) = I (Y )

for all Young diagrams Y .

Proof. Since the square Young diagrams associated with 231 and 312 are transposes of each other, and transposition does not change the inversion number of a transversal, si 231 ∼ 312 ⇐⇒ IY∗ (231, q) = IY∗ (312, q)

for all Y

⇐⇒ IY∗ (231, q) = IY∗ (231, q)

for all Y

⇐⇒ I (Y ) = I (Y ) for all Y .  Thus the proof of Theorem 4 only requires the proof of the following statement. Proposition 9. I (Y ) = I (Y ) for all Young diagrams Y . The proof of Proposition 9 relies on equating the expressions obtained from Propositions 10 and 12, which we call row decomposition and column decomposition, respectively. The motivation for row decomposition, a vital part of the proof, is based on the bijection in Stankova and West’s proof of [19, Thm. 2]. The bijection here is the same, but the proof is adapted to include the inversion number in this row decomposition. We use Fig. 7 as an example for this motivation. The transversal (σ , Y ) in Fig. 7 avoids 231, and the cell C marks the location of the bottom dot. Draw a line ℓ from the top right corner of C diagonally up and right until it touches the border of Y , and form A as the light grey region in the figure with ℓ as its long diagonal.

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

67

Fig. 7. Row decomposition.

Fig. 8. Row decomposition (left) and column decomposition (right).

We shall see that A is a Young diagram, and that the part of σ that is in A forms a transversal α of A. Excluding α and (n, i), the rest of σ lies in B (the dark grey region formed by taking ‘‘the rest of Y ’’, joining parts together), and we shall see also that B is a Young diagram, and that the part of σ that is in B forms a transversal β of B. In this manner, we break up σ into two smaller transversals α and β , one which is 231-avoiding in A, and one which is 231-avoiding in B. Furthermore, the converse holds, since if transversals are chosen for subregions A and B of Y , an instance of 231 must be contained entirely in A or in B as enforced by the locations of A and B as well as the shape of Y . This method implicitly forms a bijection between the set of 231-avoiding transversals σ of Y with a given dot in C , and the set of pairs of 231-avoiding transversals α of A and β of B. Furthermore, we must account for the inversion number. We shall see that inv∗ (σ ) can be calculated by adding inv∗ (α) and inv∗ (β), and adding i(|A| + 1), where i is the number of columns of Y to the left of C . As illustrated in Fig. 7, inv∗ (α) = 2, inv∗ (β) = 2, i = 2, and |A| = 3, and so inv∗ (α) + inv∗ (β) + i(|A| + 1) = 2 + 2 + 2 · 4 = 12 = inv∗ (σ ). This motivates the definitions used for row decomposition, as illustrated in Fig. 8. Let Y be a nontrivial Young diagram of size n + 1, whose bottom row has r + 1 cells. Let i be an integer satisfying 0 ≤ i ≤ r. Draw a (possibly zero-length) line segment ℓ from the top-right corner of the cell (n, i) upward and right with slope 1 until it first touches the border of Y . Let YAi be the (possibly trivial) subdiagram obtained by deleting all rows and columns which ℓ does not cross, and YBi be the (possibly trivial) subdiagram obtained by deleting all rows and columns which ℓ crosses as well as the row and column of (n, i).

68

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

From this point on,



i

denotes summation over all i for which the summation term is defined.

Proposition 10 (Row Decomposition). I (Y ) =



I (YAi )I (YBi )qi(|YA |+1) for all nontrivial Young diagrams Y . i

i

We will delay the proof of Proposition 10 for now. Proposition 10 gives rise to another functional relation between generating functions I (Y ), as given in Corollary 11. This generalizes [19, Cor. 1], which was central to that paper’s use of row and column decompositions. In contrast, Corollary 11 is not required for our proofs, and its proof is omitted. Corollary 11. Let Y0 = a0 a1 . . . an−1 r and Y = a0 a1 . . . an−1 (r + 1) be Young diagrams, where r ≥ 1. Then r −1 I (Y ) = I (Y0 ) + I (YBr )(qr − qr −1 ) + I (YAr −1 )I (YBr −1 )q(r −1)(|YA |+1) .

Although Table 3 is small enough to be generated by hand, Proposition 10 and Corollary 11 can be used to compute I (Y ) recursively. For computing the value of I (Y ) where the values of I (Z ) are known for all Young diagrams Z smaller than Y , Corollary 11 usually has an advantage over Proposition 10, since there are only three terms to compute in Corollary 11, whereas there is a summation of potentially more terms in Proposition 10. Example 2. Let Y = 55544. Then YAi and YBi are defined only for 0 ≤ i ≤ 3. Then YA0 = 4443, YB2 = SQ3 ,

YA1 = 332,

YB0 = ∅,

YB1 = SQ1 ,

YA2 = SQ1 ,

YB3 = 4443.

YA3 = ∅,

Referring to I (Y ) when Y = ∅, SQ1 , 332, SQ3 , 4443 in Table 3, I (Y ) = I (4443)I (∅) + I (332)I (SQ1 )q1·4 + I (SQ1 )I (SQ3 )q2·2 + I (∅)I (4443)q3·1

= (1 + q3 )(1 + q + 3q2 + 3q3 + 3q4 + q5 ) + q4 (1 + 2q + q2 ) + q4 (1 + q + 2q2 + q3 ) = 1 + q + 3q2 + 4q3 + 6q4 + 7q5 + 6q6 + 4q7 + q8 . Alternatively, if we know that I (55543) = 1 + q + 4q2 + 4q3 + 7q4 + 6q5 + 4q6 + q7 , we can use Corollary 11 directly with r = 3: I (Y ) = I (55543) + I (4443)(q3 − q2 ) + I (SQ1 )I (SQ3 )q2·2

= 1 + q + 4q2 + 4q3 + 7q4 + 6q5 + 4q6 + q7 + (1 + q + 3q2 + 3q3 + 3q4 + q5 )(q3 − q2 ) + q4 (1 + q + 2q2 + q3 ) = 1 + q + 3q2 + 4q3 + 6q4 + 7q5 + 6q6 + 4q7 + q8 . We use a similar concept for column decomposition, which is the reflected version of row decomj

j

j

j

position as shown in Fig. 8. The subdiagrams YA′ and YB′ , illustrated in Fig. 8, are given by YA′ = Y A j

j

and YB′ = Y B . Unlike the proof for row decomposition, the proof for column decomposition relies on induction and use of row and column decomposition to equate the two sides. The idea of the proof, a simplified version of the proof of [19, Lem. 3], is based on ‘‘commutativity of row and column decompositions’’, as shown on the right side of Fig. 5. Proposition 12 (Column Decomposition). I (Y ) =

 j

j

j

j

j(|Y ′ |+1) A

I (YA′ )I (YB′ )q

for all nontrivial Young diagrams Y .

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

69

We will also delay the proof of Proposition 12. j

j

Example 3. Let Y = 55544. Then YA′ and YB′ are defined only for 0 ≤ j ≤ 2. Then YA0′ = SQ4 ,

YA1′ = SQ3 ,

YB0′ = ∅,

YB1′ = SQ1 ,

YA2′ = ∅,

YB2′ = SQ4 .

Then, referring to I (Y ) when Y = ∅, SQ1 , SQ3 , SQ4 in Table 3, I (Y ) = I (SQ4 )I (∅) + I (SQ3 )I (SQ1 )q1·4 + I (∅)I (SQ4 )q2·1

= (1 + q2 )(1 + q + 2q2 + 3q3 + 3q4 + 3q5 + q6 ) + q4 (1 + q + 2q2 + q3 ) = 1 + q + 3q2 + 4q3 + 6q4 + 7q5 + 6q6 + 4q7 + q8 , as previously calculated in Example 2 using row decomposition. Propositions 10 and 12 imply Proposition 9 as follows, which gives us Theorem 4. Proof of Proposition 9. We need to prove I (Y ) = I (Y ) for all Young diagrams Y . The statement is true when Y is trivial. Suppose it is true for all Young diagrams smaller than Y . By the inductive hypothesis and the fact that |Z | = |Z | for all Young diagrams Z , I (Y ) =



=



=



j

j

j

j(|Y ′ |+1) A

j

j

j(|Y ′ |+1) A

j

j

I (YA′ )I (YB′ )q

j

I (YA′ )I (YB′ )q

j

j j

I (Y A )I (Y B )qj(|Y A |+1)

j

= I (Y ).  To complete the proof of Theorem 4, it remains to prove Propositions 10 and 12. We first prove Proposition 10, based on the motivation given by Fig. 7 and the text after the statement of Proposition 9. Proof of Proposition 10. We need to prove I (Y ) =



I (YAi )I (YBi )qi(|YA |+1) i

for all nontrivial Young diagrams Y .

i

Let Y be a Young diagram of size n + 1. We use the shorthand S (Y ) to mean SY (231), the set of all transversals in Y that avoid 231, and Si (Y ) for the set of all σ ∈ S (Y ) that have a dot in (n, i). Let σ ∈ Si (Y ) and let a = |YAi |. We claim that all dots of σ in columns 0, 1, 2, . . . , i − 1 lie in rows above all those in columns i, i + 1, i + 2, . . . , i + a. If either YAi or YBi is trivial, then this holds true. Otherwise, suppose for a contradiction that the claim does not hold. Then there exists a dot t in column j < i which is lower than a dot v in column i + k with 0 < k ≤ a but higher than all dots in columns i, i + 1, . . . , i + k − 1. Then because there are k dots in columns i, . . . , i + k − 1, then at least one of these dots, which we call u, must be in row n − k + 1 or above, and below t. Since the column i + k containing v extends as far down as row n − k + 1 by definition of the Young diagram YAi , then dots t , u, v form an instance of 231 in Y , contradicting that σ avoids 231 in Y . Thus, the above claim holds. Let α be the set of dots in columns i + 1, i + 2, . . . , i + a. It follows immediately that the dots in α span the rows n − a, n − a + 1, . . . , n − 1, and all dots above the row n − a, whose set we call β , are either in columns to the left of i or to the right of i + a. Thus, if σ ∈ Si (Y ), then α ∈ S (YAi ) and β ∈ S (YBi ), and α and β are uniquely determined from σ . Conversely, for a fixed i, if we choose α ∈ S (YAi ) and β ∈ S (YBi ) in advance, then there is only one possible σ which results in the given α and β from the above process. We will show that σ avoids 231.

70

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

Suppose for a contradiction that σ contains an instance of 231 formed from dots t , u, v from left to right. We show that t , u, v are all in YAi or all in YBi , contradicting α ∈ S (YAi ) and β ∈ S (YBi ). We divide by case depending on the location of v . Clearly, v cannot be (n, i). If v is in YBi and left of column i, then t and u must also be in YBi . If v is in YBi and right of column i, then u cannot be (n, i) or in YAi ; otherwise the intersection of the row of u with the column of v would be outside Y , contradicting containment. So u, and therefore t, are in YBi . If v is in YAi , then u cannot be (n, i); otherwise, t would be above v . So u, and therefore t, are in YAi . All these cases result in a contradiction. Therefore, σ ∈ Si (Y ). Thus,

σ ∈ Si (Y ) if and only if α ∈ S (YAi ) and β ∈ S (YBi ).

(2)

We claim that inv∗ (σ ) = inv∗ (α) + inv∗ (β) + i(a + 1), where a = | Then

YAi

(3)

|. Let Q (u, v) = 1 if dot v is to the lower-right of dot u and Q (u, v) = 0 otherwise. ∗



inv∗ (π ) =

∗

Q ∗ (u, v)

u,v∈π

for all transversals π , and so



inv∗ (σ ) =

Q ∗ (u, v)

u,v∈σ



=





Q ∗ (u, v) +

u,v∈β

u,v∈α∪{(n,i)}

+



Q ∗ (u, v) +

Q ∗ (u, v)

u∈α∪{(n,i)},v∈β

Q ∗ (u, v)

u∈β,v∈α∪{(n,i)}



= inv∗ (α ∪ {(n, i)}) + inv∗ (β) +

Q ∗ (u, v)

u∈β,v∈α∪{(n,i)}

= inv∗ (α) + inv∗ (β) + i(a + 1), since the all the dots in columns 0, 1, 2, . . . , i − 1 lie in rows above all those in columns i, i + 1, i + 2, . . . , i + a. Thus, the above claim holds. By combining (2) and (3), we find that

  {σ ∈ Si (Y ) : inv∗ (σ ) = k} qk i k     {α ∈ S (Y i ), β ∈ S (Y i ) : inv∗ (α) + inv∗ (β) = k − i(|Y i | + 1)} qk = A B A i k     c i ∗ {α ∈ S (Y ) : inv (α) = c } q =

I (Y ) =

A

c

i

 ×

   {β ∈ S (Y i ) : inv∗ (β) = d} qd qi(|YAi |+1) B d

=



I (YAi )I (YBi )qi(|YA |+1) .  i

i

We now give a brief outline for the proof of Proposition 12. Let Y be a Young diagram of size n + 1. We say that YAi is inner if YAi has no cell in the rightmost j

j

column of Y ; otherwise YAi is outer. Similarly, YA′ is inner if YA′ has no cell in the bottom row of Y ; j

otherwise YA′ is outer. We note that there exists an integer i for which YAi is inner; in particular, if the bottom row of Y has r + 1 cells, then YAr is trivial. Furthermore, if k is the smallest integer such that j

YAk is inner, then YAi is inner if and only if i ≥ k, and YA′ is inner if and only if j ≥ k.

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

71

j

Fig. 9. YAi and YA′ are inner (i ≥ k, j ≥ k); see (ii).

This distinction of inner and outer is used in our motivation for column decomposition. We also require Y to be a non-square Young diagram for the following method; the case where Y = SQn+1 has a simple separate proof. In Y , mark cell (n, i) on the bottom row, and cell (j, n) on the right column, as shown in the topleft diagram of Fig. 9. We focus only on the particular term in the row and column decomposition indicated by the marked cells and we want to show that applying row and column decomposition in that order (‘‘row–column decomposition’’) gives the same particular term as applying column and row decomposition in that order (‘‘column–row decomposition’’). Now, based on whether each of YAi j

and YA′ is inner or outer, we have four important cases. j

Case 1: Suppose that both YAi and YA′ are inner, as in Fig. 9. If we first apply row decomposition, then (j, n) is in YBi and so we apply column decomposition to I (YBi ). On the other hand, if we j

first apply column decomposition, then (n, i) is in YB′ and so we apply row decomposition j YB′

to I (

). Fig. 9 illustrates that the resulting terms are the same. j

Case 2: Suppose that YAi is outer and YA′ is inner, as in Fig. 10. Unlike Case 1 above, if we first apply row decomposition, then (j, n) is in YAi instead of YBi . So we apply column decomposition to I (YAi ) using summation index j∗ = j − i. On the other hand, if we first apply column decomposition, j j then (n, i) is in YB′ and so we apply row decomposition to I (YB′ ), like the above case. Fig. 10 illustrates that the resulting terms are the same. j

Case 3: The case where YAi is inner and YA′ is outer is just a mirrored version of Case 2 above. Case 4:

j Suppose that both YAi and YA′ are outer, as in Fig. 11. This case requires more care since j j j n need not be in YAi or YBi , nor does n i need to be in YA′ or YB′ . Let i j. If we apply i row decomposition, then j n is in YA so we apply column decomposition to I YAi using

(, )

( , ) < (, ) ( ) summation index j∗ = j − i. j j Now since (n, i) is neither in YA′ nor in YB′ , then the particular term in column–row

decomposition does not exist. However, if we switch i and j so that the marked cells are (n, j) and (i, n), then although the particular term in row–column decomposition does not

72

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

j

Fig. 10. YAi is outer and YA′ is inner (i < k, j ≥ k); see (oi).

j

Fig. 11. YAi and YA′ are outer (i < k, j < k); see (oo).

exist, the particular term in column–row decomposition now exists, since (n, j) is in YAi ′ . So we apply row decomposition to I (YAi ′ ) using summation index j∗ = j − i. Fig. 11 illustrates that the resulting terms are the same. The above switch implicitly covers the case where the cells are (n, i), (j, n) with i > j. The above argument also deals with i = j.

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

73

Putting all these cases together establishes the commutativity of row and column decompositions. Example 4. Let Y = TTTTTT9855 (where T = 10), i = 1, j = 4, as shown in Fig. 10. Starting

from row decomposition, the row decomposition term of index i = 1 is I (YAi )I (YBi )qi(|YA |+1) = I (Z )I (1)q9 = I (Z )q9 , where Z = 88888763. Since (4, 9) of Y is in the subdiagram of Z in Y , apply i

j−i

j−i

j−i

(j−i)(|Y ′ |+1) A

column decomposition to I (Z ) and take the term of index j − i = 3: I (ZA′ )I (ZB′ )q I (332)I (4443)q12 . So the particular term in row–column decomposition is

=

I (332)I (4443)q12 q9 = I (332)I (4443)q21 . On the other hand, starting from column decomposition, the column decomposition term of index j = 4 is j

j

j(|Y ′ |+1) A

j

I (YA′ )I (YB′ )q

= I (332)I (W )q16 ,

where W = 666655. Since (9, 1) of Y is in the subdiagram of W in Y , apply row decomposition to

I (W ) and take the term of index i = 1: I (WAi )I (WBi )qi(|WA |+1) = I (4443)I (1)q5 = I (4443)q5 . So the particular term in column–row decomposition is i

I (332)I (4443)q16 q5 = I (332)I (4443)q21 , which is the same as above. We now give a full proof of Proposition 12. Proof of Proposition 12. We need to prove I (Y ) =



j

j

j

j(|Y ′ |+1) A

I (YA′ )I (YB′ )q

for all nontrivial Young diagrams Y .

j

The statement holds for the size 1 Young diagram. Now assume it holds for all Young diagrams smaller than Y . Let |Y | = n + 1 where n > 0. By Proposition 10, it is sufficient to prove



I (YAi )I (YBi )qi(|YA |+1) = i



j

j

j(|Y ′ |+1) A

j

I (YA′ )I (YB′ )q

.

(4)

j

i

If Y = SQn+1 , then YAi = SQn−i = YAi ′ and YBi = SQi = YBi ′ , and so (4) holds trivially. Otherwise, Y ̸= SQn+1 . We will prove (4) by applying the assumed (under induction) column j j decomposition to YAi or YBi of the LHS depending on i, applying row decomposition to YA′ or YB′ of the RHS depending on j, and comparing the terms to show that they are equal. Let k be the smallest integer such that YAk is inner. As stated above, k exists, and YAi is inner if and j

only if i ≥ k, and YA′ is inner if and only if j ≥ k. Furthermore, since Y ̸= SQn+1 , we have i, j, k < n. Now consider (4). On the LHS, if i ≥ k, then YAi is inner, so apply the assumed column decomposition to I (YBi ) using summation index j; otherwise YAi is outer, so apply it to I (YAi ) using summation index j∗ = j

j

j − i ≥ 0. On the RHS, if j ≥ k, then YA′ is inner, so apply row decomposition to I (YB′ ) using summation index i; otherwise i ,j

j YA′

j YA′

is outer, so apply it to I ( j

) using summation index i = i − j ≥ 0. For ease of ∗

presentation, YAB′ denotes (YAi )B′ , and so on. Rewriting the exponent of q, Eq. (4) is equivalent to i,j

 i≥k,all j

=

i ,j

i,j |+i+j BA′

i|YAi |+j|Y

I (YAi )I (YBA′ )I (YBB′ )q

 j≥k,all i

j

j ,i

+

 i
j ,i

j j,i j|Y ′ |+i|Y ′ |+j+i A B A

I (YA′ )I (YB′ A )I (YB′ B )q

+

i ,j − i

i,j−i

i|YAi |+(j−i)|Y

I (YAA′ )I (YAB′ )I (YBi )q

 j
j,i−j

j ,i − j

j

I (YA′ A )I (YA′ B )I (YB′ )q

i,j−i |+j AA′

j j,i−j j|Y ′ |+(i−j)|Y ′ |+i A A A

.

It is easily verified that this equation holds provided the following four identities hold. We label the identities according to whether YAi is inner (i) or outer (o) on the left side of the label, and whether

74

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

j

YA′ is inner or outer on the right. i,j

 i≥k,j≥k

i,j−i

 i
i≥k,j
i
i,j |+i+j BA′

i,j−i |+j AA′

i|YAi |+(j−i)|Y

i,j

i|YAi |+j|Y

i,j |+i+j BA′

i,j−i

i|YAi |+(j−i)|Y

j,i−j Y A′ A

I(

j
j,i−j YA′ B

)I (

i,j−i |+j AA′

=

)I (

)

j ,i

j

j,i

j ,i − j

j

I (YA′ A )I (YA′ B )I (YB′ )q

 i
j YB′

j j,i j|Y ′ |+i|Y ′ |+j+i A B A

j ,i

j,i−j j j|Y |+(i−j)|Y ′ |+i A A q A′

.

j j,i j|Y ′ |+i|Y ′ |+j+i A B A

I (YA′ )I (YB′ A )I (YB′ B )q

j,i−j

j
I (YAA′ )I (YAB′ )I (YBi )q





=



=

j ,i

j

I (YA′ )I (YB′ A )I (YB′ B )q

j≥k,i
I (YAi )I (YBA′ )I (YBB′ )q





=

j≥k,i≥k

i,j−i

i,j−i



i|YAi |+j|Y

I (YAA′ )I (YAB′ )I (YBi )q i,j



=

i,j

I (YAi )I (YBA′ )I (YBB′ )q

i,j−i

j j,i−j j|Y ′ |+(i−j)|Y ′ |+i A A A

(ii)

.

(oi)

.

(io)

i,j−i i|Y i ′ |+(j−i)|Y ′ |+j A A A

i,j−i

I (YA′ A )I (YA′ B )I (YBi ′ )q

(oo)

 .

(Note: RD and CD in Figs. 9–11 indicate row decomposition and column decomposition, respectively.) Proof of (ii): Let i ≥ k, j ≥ k; see Fig. 9. j As illustrated in Fig. 9, every cell of YAi is to the lower-left of every cell of YA′ . This is because, from the definition of k, there exist integers x, y, with 0 ≤ x ≤ n, 0 ≤ y ≤ n, such that x + y = n + k + 1 j and such that (x, y) lies outside Y . Then every cell of YAi lies on or below row x, and every cell of YA′ lies on or to the right of column y. j j ,i i,j j Therefore, YAi is completely contained in YB′ , and so YAi = YB′ A . By symmetric arguments, YBA′ = YA′ . i,j

j

Since YAi is entirely to the lower-left of YA′ , YBB′ is formed by deleting from Y all rows and columns

∪ (n, i), then deleting all rows and columns intersecting YAj ′ ∪ (j, n). YBj,′iB is formed by i ,j j ,i reversing these two operations. The operations commute, and so YBB′ = YB′ B . intersecting

YAi

This concludes the proof of (ii) Proof of (oi): Let i < k, j ≥ k; see Fig. 10. j j i ,j − i j Since YAi is outer and YA′ is inner, YA′ is completely contained in YAi . Therefore, YAA′ = YA′ . i ,j − i

j

Now consider YAB′ , which is just YAi with all rows and columns intersecting YA′ ∪ (j, n) deleted. i,j−i YAB′

j ,i YB′ A

j | − 1. Now is itself outer in YB′ , j,i j since YAi is outer. Therefore, YB′ A also lies along the columns i + 1, . . . , n − |YA′ | − 1. Since |YAi | = n − i,

Note that

j YA′

lies along the columns labelled i + 1, . . . , n − |

we have the important identity

|YAi | − |YAj ′ | − 1 = |YBj,′iA |.

(5)

i,j−i

j

j

Furthermore, YAB′ lies along the first |YAi | − |YA′ | − 1 rows above (n, i) that do not intersect YA′ ∪ (j, n), and likewise for Since

YAi

j,i YB′ A .

is outer,

Therefore YBi

i,j−i YAB′

= SQi .

j ,i YB′ A . j,i Now, YB′ B

=

j ,i

consists of the i rows above YB′ A (and thus above YAi ) j,i

j,i

intersected with the i columns to the left of (n, i), and so YB′ B = SQi . Therefore YBi = YB′ B . The above observations settle (oi), since by (5), i,j−i

i|YAi | + (j − i)|YAA′ | + j

= i|YAi | + (j − i)|YAj ′ | + j = j|YAj ′ | + i(|YAi | − |YAj ′ |) + j = j|YAj ′ | + i(|YBj,′iA | + 1) + j = j|YAj ′ | + i|YBj,′iA | + j + i.

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

75

Proof of (io): Let i ≥ k, j < k. Follows from (oi) by symmetry. Proof of (oo): Let i < k, j < k; see Fig. 11. Since YAi and YAi ′ are outer, YBi = YBi ′ , since they are both SQi . It is easy to check that |YAi | = |YAi ′ |. i,j−i

Likewise, YAB′ Now

i,j−i YAA′

= YAi,′jB−i , since they are both SQj−i .

i,j−i

j

j

is just YA′ with the bottom row and left column deleted. Likewise, YA′ A is just YA with i,j−i YAA′

i,j−i YA′ A ,

the right column and top row deleted. So = since they are both formed from Y by deleting all columns and rows other than those labelled j + 1, . . . , n − 1. This concludes the proof of (oo).  5. Conclusion Corollary 6 provides the first known infinite family of nontrivial inv-Wilf-equivalent permutation pairs. This disproves Conjecture 1, due to Dokos et al. [10]. A key aspect of our method is to combine the definitions of inv-Wilf-equivalence and shape-Wilf-equivalence into the stronger definition of shape-inv-Wilf-equivalence.  It would be interesting to find explicitly the bivariate generating function n≥0 In (231 ⊕ γ , q)t n associated with the inv-Wilf-equivalent permutation pairs of Corollary 6 for specific permutations γ . However, this appears to be a difficult problem. In particular, it would give an explicit form for the generating function of the sequence {|Sn (231 ⊕ γ )| : n ≥ 0}, by setting q = 1. However, up to symmetry, the only permutations α of size greater than 3 for which the generating function of the sequence {|Sn (α)| : n ≥ 0} is currently known are α = 1342 (see Bóna [6], and a simpler proof provided by Bloom and Elizalde [4]) and α = 12 . . . k for all k (see Gessel [11] for k = 4 and Bousquet–Mélou [7] for general k). The methods and results of this paper suggest the following questions. 1. Is there an inv-Wilf-equivalent permutation pair α and β of size n that does not arise from Corollary 6 or trivial symmetries of the dihedral group? Such a pair α, β could exist only for n ≥ 8, since for each permutation pair α, β of size n ≤ 7 we have found a suitable value of m ≤ 11 for which Im (α, q) ̸= Im (β, q), by computer search; a single such value of m demonstrates that α is not inv-Wilf-equivalent to β . 2. Can other variants of Wilf-equivalent definitions be fruitfully combined in order to address the many open questions and conjectures in the theory of forbidden subsequences? In particular, can similar methods be used to understand when two permutations are maj-Wilf-equivalent, as defined by Dokos et al. [10] by reference to the major index of a permutation? Acknowledgements I am grateful to Bruce Sagan for his interesting seminar at Simon Fraser University in May 2012 which introduced me to the study of pattern-avoiding permutations, and for his helpful correspondence. I also thank Jonathan Jedwab, Marni Mishna, and Lily Yen for their reviews and recommendations on this paper. References [1] [2] [3] [4] [5]

M. Albert, M. Bouvel, Operators of equixvalent sorting power and related Wilf-equivalences, (2013) 701–712. J. Backelin, J. West, G. Xin, Wilf-equivalence for singleton classes, Adv. Appl. Math. 38 (2007) 133–148. A. Baxter, A.D. Jaggard, Pattern avoidance by even permutations, Electron. J. Combin. 18 (2011) 15. Paper 28. J. Bloom, S. Elizalde, Pattern avoidance in matchings and partitions, Electron. J. Combin. 20 (2013) 38. Paper 5. J. Bloom, D. Saracino, 2012. A simple bijection between 231-avoiding and 312-avoiding placements arXiv:1110. 2564v2 [math.CO]. [6] M. Bóna, Exact enumeration of 1342-avoiding permutations: a close link with labeled trees and planar maps, J. Combin. Theory Ser. A 80 (1997) 257–272. [7] M. Bousquet-Mélou, Counting permutations with no long monotone subsequence via generating trees and the kernel method, J. Algebraic Combin. 33 (2011) 571–608.

76

J.H.C. Chan / European Journal of Combinatorics 44 (2015) 57–76

[8] W.Y.C. Chen, E.Y.P. Deng, R.R.X. Du, R.P. Stanley, C.H. Yan, Crossings and nestings of matchings and partitions, Trans. Amer. Math. Soc. 359 (2007) 1555–1575. [9] A. Claesson, S. Kitaev, Classification of bijections between 321- and 132-avoiding permutations, in: 20th Annual International Conference on Formal Power Series and Algebraic Combinatorics, FPSAC 2008, in: Discrete Math. Theor. Comput. Sci. Proc., AJ, Assoc. Discrete Math. Theor. Comput. Sci., Nancy, 2008, pp. 495–506. [10] T. Dokos, T. Dwyer, B.P. Johnson, B.E. Sagan, K. Selsor, Permutation patterns and statistics, Discrete Math. 312 (2012) 2760–2775. [11] I.M. Gessel, Symmetric functions and P-recursiveness, J. Combin. Theory Ser. A 53 (1990) 257–285. [12] V. Jelínek, Dyck paths and pattern-avoiding matchings, European J. Combin. 28 (2007) 202–213. [13] D.E. Knuth, The Art of Computer Programming, Vol. 1, Addison-Wesley, Reading, MA, London, Don Mills, Ont., 1969. [14] D.E. Knuth, The Art of Computer Programming, Vol. 3, Addison-Wesley, Reading, MA, London, Don Mills, Ont, 1973. [15] C. Krattenthaler, Growth diagrams, and increasing and decreasing chains in fillings of Ferrers shapes, Adv. Appl. Math. 37 (2006) 404–431. [16] D. Rotem, On correspondence between binary trees and a certain type of permutation, Inform. Process. Lett. 4 (1975) 58–61. [17] R. Simion, F. Schmidt, Restricted permutations, European J. Combin. 6 (1985) 383–406. [18] Z. Stankova, Forbidden subsequences, Discrete Math. 132 (1994) 291–316. [19] Z. Stankova, J. West, A new class of Wilf-equivalent permutations, J. Algebraic Combin. 15 (2002) 271–290. [20] W. Trongsiriwat, 2013. Permutation statistics and multiple pattern avoidance. arXiv:1309.3028 [math.CO]. [21] J. West, Generating trees and the Catalan and Schröder numbers, Discrete Math. 146 (1995) 247–262.