Exponential bounds and tails for additive random recursive sequences

Exponential bounds and tail estimates are derived for additive random recursive sequences, which typically arise as functionals of recursive structures, of random trees or in recursive algorithms. In particular they arise as parameters of divide and conquer type algorithms. We derive tail bounds from estimates of the Laplace transforms and of the moment sequences. For the proof we use some classical exponential bounds and some variants of the induction method. The paper generalizes results of Rosler (% \citeyearNPRoesler:91, % \citeyearNPRoesler:92) and % \citeNNeininger:05 on subgaussian tails to more general classes of additive random recursive sequences. It also gives sufficient conditions for tail bounds of the form \exp(-a t^p) which are based on a characterization of \citeNKasahara:78.


Introduction
A large number of functionals (X n ) n≥0 of random recursive structures, random trees, and recursive algorithms satisfy recurrences of the form (1) Here X n is the interesting parameter of a problem of size n, which is split into M ≥ 1 subproblems r of random sizes k ) are distributional copies of (X k ), that correspond to the contribution of subgroup r, b n is a random toll function term and A r (n) are random factors weighting the subproblems.Further, (X In typical examples X n ranges from depth, size, and path length of random trees, the number of various substructures or components of combinatorial structures, the number of comparisons, space requirement, and other cost measures of algorithms to various parameters of communication models and in particular 1365-8050 c 2007 Discrete Mathematics and Theoretical Computer Science (DMTCS), Nancy, France to typical instances of the 'divide and conquer' paradigm.For numerous examples of this type we refer to the books of Mahmoud (1992), Sedgewick and Flajolet (1996), Szpankowski (2001), and Arratia et al. (2003).
A general distributional limit theorem for this type of recurrences was given together with many examples in Neininger and Rüschendorf (2004a,b) and Rösler (2001) by means of the contraction method.The limit of X n after normalization is characterized as unique solution of some fixed point equation of the form where the distributions are restricted to some subclass as e.g. to M 2,c , the class of all distributions with finite second moment and first moment equal to c.Here X (1) , . . ., X (M ) are independent copies of X and the random vector (A 1 , . . ., A M , b) arises as the limit of the corresponding normalized vector There has been quite a lot of work going on for establishing concentration results and tail bounds for (randomized) algorithms in general, which serve in particular for establishing approximation bounds and error estimates.Besides classical tail bounds from statistics like Chernoff's, Hoeffding's, Bennett's, Bernstein's bounds, and martingale bounds more recently techniques like induction methods, entropy, methods, Talagrand's convex-distance inequality and others have been developed and applied.For references we refer to McDiarmid (1998), Motwani and Raghavan (1995), Boucheron et al. (2000Boucheron et al. ( , 2003)), and many references given in these papers.The concentration results supplement the asymptotic distributional analysis.They allow sharp error estimates.In some cases they allow to establish laws of large numbers for related parameters like the height of trees (see e.g.Devroye (2002) and Broutin and Devroye (2005)).
Exponential bounds for solutions of the fixed point equation in (2) were given by an induction method in Rösler (1992).For the Quicksort algorithm exponential bounds were established in Rösler (1991), see also Fill and Janson (2002) for some improvements in this case.For some classes of recurrences as in (1) bounds were given in Karp (1994) while Neininger (2005) established subgaussian bounds for (X n ) in cases, where the rescaled quantities converge to a normal limit.In this paper we aim to develop some exponential tail bounds for more general classes of additive recursive sequences.
In the first part of this paper we establish exponential bounds for Laplace transforms leading to exponential tail bounds for recursive algorithms of type (1) and for solutions of recursive equations of type (2).The proof is based on an induction argument.In the case that the convergence theorem for recursive algorithms is applicable we alternatively also indicate a proof which is based on a uniform approximation argument for Laplace transforms.Under stronger assumptions on the coefficients we establish a stronger exponential bound valid on all finite intervals.
In the second part we derive various exponential inequalities for the Laplace transform and for the tail probabilities on the whole real line.In particular we derive exponential bounds of subgaussian type for a general class of recursive sequences extending previous results in Neininger (2005).By alternative methods of proof based on Bernstein's inequality and on a moment characterization of subgaussianity we derive under corresponding conditions on the coefficients subgaussian behaviour resp.linear exponential bounds on the whole real line.
In the final part of the paper we establish general asymptotic bounds for the moments of X n .A characterization of exponential tails by the asymptotics of moments due to Kasahara (1978) allows to establish some general conditions for exponential tails of any order.
The paper gives a series of results on exponential (tail) bounds for recursive algorithms.Depending on the strength of the conditions on the coefficients of the recursive sequence we obtain a broad spectrum of exponential bounds of different strength.For the proofs we use a variety of different techniques, making use in particular of induction arguments, approximation arguments for Laplace transforms, classical inequalities as Hoeffding's and Bernstein's inequalities, and various characterization results of exponential tails by moment conditions and by bounds on the Laplace transforms.All results in this paper make use in some sense of the 'contradiction' condition E i A 2 i < 1 resp.E r A 2 r (n) < 1 for the recursive sequence.Thus the exponential bounds in this paper are usually not well suited for (asymptotically) normal sequences, which are 'typically' characterized by the limit equation ( 2) with b = 0 and i A 2 i = 1.Our results apply only in some of these normal cases, when special properties of the expansion of the mean hold true (see for example Remark 3.2).

Bounds on the Laplace transform
The fixed point equation ( 2) can be written in operator form as where S(F ) is the distribution of the right-hand side of (2).Under suitable conditions on A i , b it is known that a unique solution F * of (3) with finite second moment and specified first moment exists and that the iterated operator S n F converges to F * w.r.t. the minimal ℓ 2 -metric (see Rösler (1992), Rachev and Rüschendorf (1995)).Rösler (1992) established the following exponential bound for a solution Z of the fixed point equation (2).
Theorem 2.1 (Exponential bound for fixed point equation (Rösler (1992))) Consider the fixed point equation ( 2) and assume the following conditions: (5) This implies by Chernoff's bounding technique the tail estimates for all λ ∈ [−L, L] This becomes minimal for λ = t/(2 K) if this is less or equal to L and in the other case we set λ = L.
Since the same bound holds for P (EZ − Z ≥ t) we obtain For recursive sequences general limit theorems have been established by means of the contraction method.The following is a typical result for this approach (see Rösler (2001) or Neininger and Rüschendorf (2004a)).We consider convergence of X n without further normalization.
Theorem 2.3 (Convergence theorem) Let (X n ) be a recursive sequence as in (1), square-integrable, and with expectation 0. We assume that stabilization: Then ℓ 2 (X n , X * ) → 0 as n → ∞, where X * is the unique solution of the fixed point equation ( 2) in the class M 2,0 of all distributions with finite second moment and expectation 0 (as in Theorem 2.1).
The convergence theorem does in general not imply that for large n X n has a similar tail behaviour as the limit X.Some boundedness condition however allows to derive uniform convergence of the Laplace transforms, which can be used to derive exponential tails of (X n ) (see Remark 2.6).
Proof: Without loss of generality let X n , X be a.s.convergent versions To prove uniformity of the convergence for |λ| ≤ L observe that and, therefore, by the triangle inequality Thus we can restrict to the case X n ≥ 0, X ≥ 0 since Ee λX + n ≤ Ee λXn + 1 and Ee λX + n ≤ 1 for λ < 0 and similarly for X − n , X − .For 0 ≤ λ ≤ L and X n ≥ 0, X ≥ 0 holds by monotonicity in λ of the integrand.Further, for λ > 0 Now following the proof of Theorem 4.5 in Rösler (1991) for w.l.g.X n , X ≥ 0 and N > 0 we have By the bounded convergence theorem the first term on the right-hand side converges towards zero.For the second term we use Hölder's inequality to obtain for N → ∞, since the first factor on the right-hand side is uniformly bounded and the second converges towards zero.Similarly we obtain Ee LX 1 (X>N ) → 0 as N → ∞. ✷ In the following theorem we establish some general conditions on the coefficients which imply exponential bounds for recurrences as in (1).
Theorem 2.5 (Laplace transform, exponential bounds) Let (X n ) be a recursive sequence as in (1) with EX n = 0, n ∈ N 0 .Assume that for some constant K and for λ in some open interval for all m ∈ N 0 and for all λ in some open interval V = V (0) around zero, where K is a finite constant.
Proof: The proof is given by induction.The assertion is true for m ≤ n 0 − 1 by assumption with K = K.For the induction step we consider the case n ≥ n 0 and assume that ( 9) is true for m ≤ n − 1.
Define the function f for some positive constant K and n ≥ n 0 .We will write (A(n), I (n) , b n ) for short for the vector we obtain by the induction hypothesis and boundedness of A r (n) Therefore we have proven the induction step if we can show that for some K ≥ K, independent of n, holds: By differentiation we obtain for K ≥ K large enough using condition b) and noting that EX n = 0, ∀n ∈ N 0 , implies Eb n = 0, ∀n ≥ n 0 .We also note that K ≥ max{ K, C} can be chosen independently of n.
It remains to show f For the proof we establish that is bounded in some neighbourhood of zero independently of n.Therefore consider By the uniform boundedness of the Laplace transform of b n and assumptions b) and c), we obtain that this expression is bounded for K fixed and |λ| ≤ K 0 independently of n.It follows that the second derivative is less than 0 in an open interval around zero uniformly in n and therefore f The neighbourhood of zero thus can be chosen independently of n.Thus we obtain some constant L such that (10) holds.✷ Remark 2.6 In the case that additionally to the assumptions in Theorem 2.5 the conditions of the convergence theorem hold a direct proof of the exponential bounds in Theorem 2.5 is possible without referring to the induction argument in the proof of Theorem 2.5 but using instead the convergence of the Laplace transforms in Proposition 2.4 applied to f ) as well as to its first and second derivative.This approximation argument allows to derive from the limiting case the crucial bound f (n) K (λ) ≤ 1 for λ in a neighbourhood of zero and uniformly for all n ≥ N .Together with the corresponding inequality for the finitely many indices n = 1, . . ., N in a possibly smaller neighbourhood the exponential inequality follows.
In the following theorem we derive exponential bounds for |λ| < L for any given L > 0. In order to obtain this stronger conclusion we strengthen the condition on the coefficients A r (n).For the purpose of ease of application we formulate this result for the normalized version (Y n ) of the recursive sequence (X n ).Let (X n ) be a recursive sequence as in (1) and denote by a normalized version of X n with nondecreasing scalings s n = s(n) > 0. We define µ n := EX n , n ∈ N 0 .Then (Y n ) satisfies the recursive equation where Thus (Y n ) again is a recursive sequence of the form (1) with modified coefficients b (n) and sn .The following theorem gives under stronger conditions on the coefficients compared to Theorem 2.5 a stronger result on exponential bounds for the Laplace transform of the scaled version Theorem 2.7 (Laplace transforms of the normalized sequence) Let (X n ) satisfy the recurrence (1) with normalized version (Y n ) as in (11) Then for all L > 0 there exists a finite constant K L such that for all n ∈ N 0 and λ ∈ [−L, L] Ee λYn ≤ e λ 2 K L . (13) We establish in the following by induction that for any L > 0 there exists a K L such that Step 1: There exist constants K 1 > 0 and For the proof of step 1 we note that f (K, λ, n) is well defined and twice differentiable (as shown in the proof of Theorem 2.1 of Rösler (1992)) and it holds Choosing dλ 2 f (K 1 , λ, n) < 0 in some neighbourhood V = V (0) independent of n, the neighbourhood U can be chosen also independently of n (similar as is shown in the proof of Theorem 2.5).
Step 2: For L > 0 there exists For the proof of step 2 note that by assumption b) and thus f (K, λ, n) is monotonically nonincreasing in K for λ, n fixed.
To prove ( 14) we define Now we prove (13) by induction.
By assumption E exp(λY k ) ≤ exp(λ 2 K), k = 0, . . ., n 0 − 1.For the induction step define K L = max{ K, K L }. Then using the induction hypothesis The conditions of Theorems 2.5 and 2.7 are in many examples not difficult to check.The Quicksort algorithm satisfies the recurrence (1) in the form where converges to the Quicksort distribution and the exponential bounds of Theorem 2.5 are established by Rösler (1991) in this case.For Quicksort type recursions with a monomial toll term C n = n α exponential bounds have been established in Neininger (2002).For several further examples of a similar type as ( 15) which fit our conditions, see Hwang and Neininger (2002).

Subgaussian behaviour of recurrences
A random variable X has subgaussian right (left) tails if for some constants L > 0, C > 0 X has subgaussian tails if it has subgaussian right and left tails.In this section we aim to sharpen the bounds in Section 2 to imply that a recurrence (X n ) or its normalized form Y n = Xn−EXn sn has subgaussian tails, i.e. it holds independently of n ∈ N 0 : where As consequence this implies improved exponential bounds compared to (6), ( 9), and ( 13) We assume that the normalizing sequence s n is nondecreasing, s n ≥ ε > 0, ∀n and that s 2 n is the order of magnitude of the expectation:

E)
µ n := EX n = µs 2 n + r n for some µ > 0 where r n = o(s 2 n ) is the remainder term.The normalized sequence (Y n ) satisfies the recursive equation Thus the bounds in Section 2 apply under corresponding conditions on the modified coefficients b The following result on subgaussian tails extends Theorem 1 in Neininger (2005), who considers the special case where s 2 n = n, A r (n) = 1 and r n = O(1).In comparison to the results in Section 2 we postulate some stronger assumptions.In particular we assume boundedness of b n where the bounds are of order O(r n ), further we strengthen condition b) of Theorem 2.5 resp.conditions b) and c) of Theorem 2.7.
Theorem 3.1 Let (X n ) be a recursive sequence as in ( 1) satisfying condition E) and assume that for some b n ∞ ≤ M 1 rn , rn := max{r n , 1}, for n ≥ n 0 and some 17) holds.
Proof: By assumption we have for some Q > 0. We next establish by induction that for some finite constant Let ( 23) be true for j ≤ n − 1 and let Υ n denote the distribution of (I (n) , b (n) , A(n)).Then by conditioning and by the induction hypothesis we obtain Next we apply Hoeffding's inequality to b (n) : If X is a random variable with a ≤ X ≤ b and EX = 0, then for all λ ∈ R it holds (see e.g.Lugosi (2005), Hoeffding (1963)) As consequence we obtain and large enough such that it holds true also for n = 0, . . ., n 0 − 1. ✷ Remark 3.2 a) Condition ( 20) can be replaced by the following conditions: Let for some generic constants In the particular case r n = O(1) we thus get the following sufficient conditions for the conclusion of Theorem 3.1: If e.g.X n denotes the number of leaves in a random binary tree, then (X n ) satisfies the recursive equation and The expectation has the expansion The conditions A2) are satisfied with s n = √ n, µ = 1 3 , r n = O(1), A r (n) = 1, and thus we get subgaussian behaviour.This example fits also the conditions of Neininger's Theorem 1 2005.Further similar examples fit in this scheme like the size of m-ary search trees, m ≤ 13, several parameters of binary search trees with bounded toll functions and some others (see Hwang and Neininger (2002), Neininger (2002), and Neininger and Rüschendorf (2004a)).
For the size of m-ary search trees subgaussian behaviour is known to hold for m ≤ 26 as follows from the moment expansions in Chern and Hwang (2001) and Lemma 3.3.But as mentioned in the introduction our methods do not suit well to this normal case and thus we get subgaussian behaviour due to the special properties of the expansion of the mean in this case for m ≤ 13 only.b) If the condition s n > 0 does not hold true for all n ≥ 0 as in the case s n = n or s n = log n, then we may choose s ε (n) := max(ε, s n ) and obtain with this modification analogous bounds.
An example for this situation are recursive equations of the form where s 2 n = log n, n ≥ 2, r n = O(1).It suffices in that case to assume Recursions of the type (30) are considered in Neininger (2005, Theorem 2).Conditions ( 31) and ( 32) are not fulfilled for I n ∼ unif{0, . . ., n−1} but fit to more concentrated subgroup sizes I n where e.g.
The following characterization of subgaussian right tails allows to dismiss with the boundedness of the toll terms b n in Theorem 3.1.The conditions on the weight coefficients A r (n) in Theorem 3.1 and in the following Theorem 3.4 are however not comparable.For the proof we shall make use of the following lemma which can be found in Lugosi (2005) (up to a slight modification).
Lemma 3.3 Let X be a real random variable.
2. If for some c, C > 0 we have Ee λX ≤ ce Cλ 2 , ∀λ > 0, then there exists an universal constant K, so that for all k ∈ N Theorem 3.4 Let (X n ) be a recursive sequence as in ( 1) with E) and with normalized sequence Y n = Xn−EXn sn . We assume the following conditions: a) Then for all t > 0 Proof: We prove, that for some constant c Using the definition of b (n) in ( 19) and condition d) we estimate Because of condition a) we have For the induction step now assume that (35) holds for Y n0 , . . ., Y n−1 with a suitable constant c ≥ max{c, η 2 c b } (therefore where ( * ) follows from Thus by Lemma 3.3 Y n has subgaussian tails and therefore similar to (18) with Chernoff's bounding technique we obtain the assertion.✷ Under somewhat weaker growth conditions on the moments of bn sn than those in Theorem 3.4 we still obtain exponential tails of linear order instead of subgaussian tails.The proof is based on Bernstein's inequality which we state for the ease of reference.
If for some constant W > 0 and all i ≤ n and all r ∈ N holds Remark 3.6 Actually the Bernstein inequality in Bennet (1962) is formulated with σ i , 1 ≤ i ≤ n, respectively σ instead of upper bounds L i , 1 ≤ i ≤ n respectively L. But the proof reveals immediately that the more general version formulated here is also valid.
Theorem 3.7 Let (X n ) be a recursive sequence as in ( 1) with condition E) and let Y n = Xn−EXn sn be the normalized sequence.Assume that σ 2 i = Var(X i ) < ∞, i = 0, . . ., n 0 −1 and assume for some constants as well as conditions c), d) from Theorem 3.4.Then there exists some σ ≥ σ := max j≤n0 σ j such that Proof: As in the proof of Theorem 3.4 we obtain in the first step the estimate for some constants W b and σb which can be given in explicit form.Then we establish the induction step and obtain after some calculations similar to those in the proof of Theorem 3.4 with some explicit constants c, W and in fact σ = 2 W .These estimates allow to apply the Bernstein inequality in (37) to the r.v.s (Y n ) to obtain which yields the result replacing s by t e s and with W = e σ √ 2 .For details of the argument see Schopp (2005).✷ 4 Exponential tail bounds; the moment method In this section we derive exponential tail bounds for recursive sequences using a characterization of general asymptotic exponential tails by the asymptotics of moments.This method uses essentially the following lemma which is based on a theorem of Kasahara (1978).Define the functions f (x), g(x) on R + , Lemma 4.1 Let X be a random variable, p, a > 0 and b := 2) For general X and p > 1 a) is further equivalent to: c) ln Ee tX ≤ as ct q (46) where c = q −1 (pa) −(q−1) and 1 p + 1 q = 1.
3) The statements in 1), 2) remain valid also if ≤ as is replaced by asymptotic equivalence ∼.
Remark 4.2 In the paper of Kasahara (1978) the statement of Lemma 4.1 was given for the asymptotic equivalence case (as in part 3)).The method of proof in that paper however also allows to cover the ≤ as -bounds as in parts 1), 2) of Lemma 4.1.For details see Schopp (2005).The characterization of asymptotic exponential tails in part 3) of Lemma 4.1 was used in Janson and Chassaing (2004) to derive asymptotic exponential tails for the Wiener index of simply generated trees.
In the following theorem we apply Lemma 4.1 to recurrences as in (1).
For the proof of ( * ) we obtain from the induction hypothesis This implies (47) and as consequence from Lemma 4.1 we obtain the exponential tail bounds in (48).The uniformity is clear, since a * (q) is independent of n 0 .✷ I (n) , b n , A(n)) are independent, where I (n) n) = (A 1 (n), . . ., A M (n))denote the corresponding vectors.Finally d = denotes equality in distribution.
1, let Eb = 0 and let c ∈ R 1 be any fixed number.Then for any random variable X d = F with EX = c, Ee λX ≤ e cλ+Kλ 2 for some K > 0 and all λ ∈ [−L, L] holds: 1) S n F converges to the unique fixed point F * in M 2,c w.r.t.ℓ 2 and 2) for Z d = F * or Z d = S n F holds uniformly in n ∈ N for some constant K ≥ 0 Ee λZ ≤ e cλ+ e Kλ 2 , ∀λ ∈ [−L, L].As consequence of Theorem 2.1 one obtains for Z d = F * or Z d = S n F uniformly in n Ee λ(Z−EZ) ≤ e e Kλ 2 , λ ∈ [−L, L].