The number of {1243, 2134}-avoiding permutations

We show that the counting sequence for permutations avoiding both of the (classical) patterns 1243 and 2134 has the algebraic generating function supplied by Vaclav Kotesovec for sequence A164651 in The On-Line Encyclopedia of Integer Sequences.


Introduction
Several authors have developed methods to count permutations avoiding a given set of patterns; see, for example, the references in the Wikipedia entry [1].In particular, Vatter [2] and Pudwell [3] have devised enumeration schemes for automated counting.When successful, an automated method produces an enumeration scheme that yields the initial terms of the counting sequence (perhaps 20 or more) and sometimes these terms appear to have an algebraic generating function that does not follow readily from the enumeration scheme.Here, we treat one such case.The counting sequence for permutations that avoid both of the (classical) patterns 1243 and 2134 begins 1, 2, 6, 22, 87, 354, . . ., sequence A164651 in The On-Line Encyclopedia of Integer Sequences [4].In a comment on this sequence dated Oct 24 2012, Vaclav Kotesovec observed that the generating function fits the known terms of the sequence.
We show that {1243, 2134}-avoiders do indeed have this generating function.Defining a start-small permutation to be one that does not start with its largest entry, the proof rests on a bijection φ from startsmall {1243, 2134}-avoiders of length n to lists of start-small 123-avoiders whose total length is n − 1 plus the length of the list.Note that a start-small permutation has length at least 2. The Catalan numbers count 123-avoiders and the combinatorial interpretation of the Invert transform permits counting these lists of start-small 123-avoiders.The generating function for start-small {1243, 2134}-avoiders thus obtained readily yields the generating function for unrestricted {1243, 2134}-avoiders.
In Section 2, we show that φ is given by iteration of a more basic bijection ψ, which is presented in Section 3. Finally, Section 4 gives the bookkeeping details to obtain the desired generating function from φ.

The bijection φ
A mid-123 entry in a permutation is an entry that serves as the "2" in a 123 pattern.A key mid-123 entry is a mid-123 entry b whose immediate predecessor is either less than b or a right-to-left maximum (max for short).For example, the mid-123 entries in 1 6 3 2 4 5 are 3, 2, 4 but only 3 and 4 are key.
Lemma 1.A permutation with no key mid-123 entries has no mid-123 entries at all and so is a 123avoider.
Proof: We suppose that π has no key mid-123 entry and π j is the first mid-123 entry, and deduce a contradiction.Say π i π j begins a 123 pattern.Since π j is not key, we have (i) π j−1 > π j and (ii) π j−1 is not a right-to-left max.Now (i) implies i = j − 1 and (ii) implies π > π j−1 for some > j − 1.But then π i π j−1 π is a 123, contradicting the minimality of j.Let A n denote the set of start-small {1243, 2134}-avoiders on [n], and A n,k the subset with k key mid-123 entries.To produce the promised bijection φ from A n to lists of start-small 123-avoiders whose total length is n − 1 plus the length of the list, it suffices to exhibit, for 0 ≤ k ≤ n − 2, a bijection A n,k → (σ 1 , . . ., σ k+1 ) : each σ i is a start-small 123-avoider, lengths of the σ i 's sum to n + k .
For k = 0, naturally the bijection is π → (π), a singleton list, because, by Lemma 1, π is already 123-avoiding.For A n,k,j = {π ∈ A n,k : the last mid-123 entry of π is in position j}.
In the next section, for 1 ≤ k < j < n, we present a bijection ψ between A n,k,j and A j, k−1 × A n+1−j, 0 .Iteration of ψ then gives the required bijection φ.

The bijection ψ
To standardize a list τ of distinct integers means to replace the smallest by 1, the next smallest by 2, and so on, and the result is denoted St(τ ).We define ψ as follows: given π ∈ A n,k,j , we need to reversibly produce σ 1 ∈ A j, k−1 and σ 2 ∈ A n+1−j, 0 .Write π as τ 1 bτ 2 where b is the last mid-123 entry in π, and let abc be the 123 pattern in π with smallest a (c is uniquely determined by Lemma 2).Concatenate a and τ 2 and standardize to get the desired σ 2 = St(aτ 2 ) with no 123's.
Concatenate τ 1 and c to get a {1243, 2134}-avoider ρ-a candidate (after standardization) for σ 1 .This ρ may need further processing because of two glitches: ρ may still have k key mid-123's instead of the required k − 1 and ρ cannot end with its smallest entry (which must be possible in σ 1 ).But these glitches cancel out.
If b is a key mid-123 in π, then ρ has k − 1 key mid-123's because b has been lost.In this case, just standardize ρ to get σ 1 .If b is not key, the last entry of τ 1 exceeds b but is not a right-to-left max of π.Delete from ρ the longest right factor (terminal string) of τ 1 that is decreasing but does not start with a right-to-left max of π, equivalently, does not start with an entry > c; say t ≥ 1 entries are deleted.Add t to each remaining entry of ρ, append the entries t, t − 1, . . ., 1 and standardize to obtain σ 1 .
Before proving the invertibility of ψ, we give 3 examples according as  Lemma 3. Suppose b is key.Take minimal in Then (i) π , . . ., π j are consecutive integers, and (ii) For h < , π h < b implies h = i.
Proof: (i) If not, say π h < x < π h+1 with ≤ h < j and consider the position of x in π.If x occurs after c, then aπ h cx is a forbidden 1243.If x occurs between b and c, then axc is a 123 contradicting the assumption that b is the last mid-123.Otherwise, since π h , π h+1 , . . ., π j = b are contiguous entries in π, x must occur before π h and xπ h π h+1 c is a forbidden 2134.
(ii) If not, by (i) there exists h < with π h < π and h = i.Now π −1 = a or π −1 > b by the definition of , and π h > a by the definition of a.In case π −1 = a, π h abc is a 2134, and in case π −1 > b, π h abc is a 2134 if π h occurs before a and aπ h π −1 b is a 1243 if π h occurs after a. Lemma 4. Suppose b is not key.Take r ≥ 1 maximal such that π j−r is not a right-to-left max and π j−r > π j−r+1 > • • • > π j .Then (i) π j , π j−1 , . . ., π j−r+1 are consecutive integers, and (ii) the integers in the interval π j−r+1 + 1, . . ., π j−r − 1 (if any) all occur immediately before π j−r and in increasing order.
Proof: Take u ≥ 0 maximal such that π j , π The construction of σ 1 and σ 2 to define ψ can now be described more explicitly: σ 2 is obtained from aτ 2 by replacing c with the length of aτ 2 and leaving all other entries intact, and σ 1 is obtained from τ 1 c as follows.Proposition 5.For 1 ≤ k < j < n, the mapping ψ is a bijection.
Proof: In view of the preceding paragraph, to retrieve π from ψ(π) = (σ 1 , σ 2 ), we need to recover n, i, j, k, a, b, c, to determine whether b is key or not, and to find r, s, t in the appropriate cases.Then we need to distinguish in σ 1 the entries increased by s − 1 from those increased by s − 2 when b is key, and to distinguish the entries increased by b − 1 from those increased by b − 2 when b is not key.Clearly The index i is the position of 1 in σ 1 .Also, k = (position in σ 2 of its maximum entry) + j − 1, and a is the first entry of σ 2 .The cases (1) π j = b is key and π j−1 < b, (2) π j is key and π j−1 is a right-to-left max, (3) π j is not key, can be distinguished according as (1) σ 1 ends with an ascent, (2) σ 1 ends with a descent to a number other than 1, (3) σ 1 ends with a descent to 1.In all cases, the entries in π after b other than c are evident in σ 2 .
• Case b is key.Here, s is retrieved as the length of σ 2 and i is the position of 1 in σ 1 .Write σ 1 as σ 11 σ 12 where σ 12 is the longest increasing right factor of σ 1 .Then b = s − 1 + (length of σ 12 ).In σ 1 , add s − 1 to each entry of σ 11 and to the last entry of σ 12 and add s − 2 to all other entries of σ 12 to obtain τ 1 c.
• Case b is not key.Here, b is retrieved as the length of σ 2 , r is the length of the longest right factor of σ 1 of the form r(r − 1) . . . 1, and i is the position of r + 1 in σ 1 .Write σ 1 as σ 1 r (r − 1) . . .

Putting it all together
It is well known that 123-avoiders are counted by the Catalan numbers C n = 2n n − 2n n−1 and so the number of start-small 123-avoiders of length n is 3 where C(x) = n≥0 C n x n is the generating function for the Catalan numbers.Say a k-list is a list of length k and the weight of a start-small 123-avoider is one less than its length.
From the bijection φ, the number of start-small {1243, 2134}-avoiders of length n is ( Recall that if A is a class (species) of combinatorial structures with a n structures of weight n (n ≥ 1), and a compositional A-structure of weight n is one obtained by taking a composition (n 1 , n 2 , . . ., n k ) of n and forming a k-list of A-structures of respective weights n 1 , n 2 , . . ., n k , then the counting sequence (b n ) n≥1 for compositional A-structures by total weight has generating function B(x) := n≥1 b n x n given by , where A(x) is the generating function for A-structures (this is the INVERT transform).Applying it to the class of nonempty start-small 123-avoiders with weight measured as "length minus 1" so that A(x) = xC(x) 3 , we get From ( 1) and ( 2), the generating function G(x) for nonempty start-small {1243, 2134}-avoiders (with x marking length) is G(x) = xB(x).Now let (u n ) n≥0 = (1, 1, 2, 6, . . . ) and (v n ) n≥0 = (1, 0, 1, 4, . . . ) denote the counting sequences for {1243, 2134}-avoiders and start-small {1243, 2134}-avoiders respectively.Clearly, v n = u n − u n−1 for n ≥ 1, (consider deletion of the first entry from a {1243, 2134}avoider on [n] that starts n).So the generating functions F (x) = n≥0 u n x n and G(x) = n≥1 v n x n are related by F (x) = (1 + G(x))/(1 − x).Thus x which, after expansion, agrees with Kotesovec's formula.
Losonczy [5] has counted permutations that avoid 3421, 4312 and 4321 or equivalently (by reversal) both of the patterns treated here and 1234.

Lemma 2 .
Suppose b is the last mid-123 entry in a {1243, 2134}-avoider.Then there is only one entry that occurs after b in π and is larger than b.Proof: Suppose abc and abc are 123 patterns with c = c , say c < c .If c precedes c in the permutation, then c is a mid-123, violating the hypothesis on b.If c follows c , then abc c is an instance of a forbidden 1243 pattern.

•
Case b is key.Here, B consists of s, s + 1, . . ., b with 2 ≤ s ≤ b.Replace a with 1, subtract s − 1 from each entry in X and from the sole entry c in C (to account for s − 1 missing entries {b} ∪ (τ 2 \{c}) ), and subtract s − 2 from each of the entries s, s + 1, . . ., b − 1 in B (to account for τ 2 \{c}).• Case b is not key.Here B consists of b + r, b + r + 1, . . ., b + t, b + r − 1, b + r − 2, . . ., b. Replace a with r + 1, subtract b − 1 from each entry in X and from the sole entry c in C, subtract b − 2 from each of the entries b + r, b + r + 1, . . ., b + t − 1 in B, and then append r, r − 1, . . ., 1.