Generating functions and the satisﬁability threshold

The 3-SAT problem consists in determining if a boolean formula with 3 literals per clause is satisﬁable. When the ratio between the number of clauses and the number of variables increases, a threshold phenomenon is observed: the probability of satisﬁability appears in simulations to decrease sharply from 1 to 0 in the neighbourghood of a threshold value, conjectured to be close to 4 . 25. Although the threshold has been proved to exist for the 2-SAT formulæ and for closely related problems like 3-XORSAT, there is still no precise mathematical characterization of the threshold phenomena involved in the 3-SAT problem. Recent works have provided so far upper and lower bounds for the threshold’s potential location. We present here a uniﬁed approach to upper bounds that is based on urn models, generating functions, and saddle-point bounds. No new upper bound is presented here but instead, we show that several existing results (with long proofs) can be organized in a simpler and uniform manner.


Introduction
We consider boolean formulae over a set of variables x 1 , . . ., x n (where the x j range over {0, 1} or, equivalently, {true, false}).A literal is either a variable x j or a negated variable ¬x j .It is known that each boolean formula admits a conjunctive normal form, being a conjunction of clauses, themselves disjunctions of literals.A 3-SAT formula is then such a formula with exactly 3 literals per clause.A typical formula is then for example: We will choose the standard probabilistic model where each clause is composed of a set of three literals composed of distinct variables.There are then 8 n 3 distinct clauses and (8 n 3 ) m formulae with m clauses.The quantities n, m are the fundamental parameters of the model.From previous studies, we know that the "interesting" region is when n and m are linearly related m = rn.(Alternative models have occasionally been used for convenience in calculations, for example, the three literals may be ordered and not necessarily distinct so that there would be 8n 3 clauses.All such models are easily proved to be equivalent with respect to the asymptotic probability of satisfiability).In Figure 1, taken from Ludovic Meunier's master's thesis [19], a phase transition phenomenon can be observed regarding the satisfiability of these formulae when they are drawn at random.As the ratio r = m/n of the number m of clauses to the number n of variables increases, the probability of satisfiability drops abruptly from nearly 1 to nearly 0. From these experiments, it is believed that there exists a critical value r 3 such that for any ε > 0, the probability of satisfiability tends to 1 for r < r 3 − ε (as m and n tend to infinity), and tends to 0 for r > r 3 + ε.Experiments suggest for r 3 the value 4.25 ± 0.05.However, so far, only successive upper and lower bounds on the threshold's potential location have been obtained.The table below lists the bounds successively established for the 3-SAT threshold.The bounds marked with a star admit an extension to k-SAT for any k.A survey by Olivier Dubois explains the techniques used to obtain them [7].For the lower bounds, one can refer to the surveys by Franco [11] and Achlioptas [2].

Lower bounds for 3-SAT threshold
Upper bounds for 3-SAT threshold 2.9 *   Chao and Franco (1986Franco ( ,1990) )  Apart from these works, Friedgut [13] also proved that there exists a sequence γ n such that for any ε > 0, if m/n < γ n − ε then, the probability of satisfiability tends to 1 as m and n increase while it tends to 0 if m/n > γ n + ε.But it has not been proved that the sequence γ n converges to some limit value γ, that would then be equal to the threshold r 3 .
The aim of the present paper is to present some of the most significant upper bounds on the satisfiability threshold and organize them within a unified framework.We will specifically focus on enumerative proofs, based on generating functions.

Expectation of the number of solutions
The first bound for the 3-SAT threshold has been obtained by several authors as a direct application of the first moment method to the random variable giving the number of solutions of a random formula.Under an enumerative perspective, it can be regarded as a direct application of the following simple remark: since positive integer are ≥ 1, one has the inequality: where |E| denotes the cardinality of the set E. There, S represents an assignment of the n variables to values in {0, 1}.Let C = ±x i ∨ ±x j ∨ ±x k be a clause and fix some assignment C (we identify negation ¬ with the minus sign).There is only one way to choose the signs of the three literals in order to have the value of C be false under S: each literal must have the opposite sign of its assignment.Then, there are 7 ways to choose the signs in order to render C true.The number of clauses satisfied by any given S is then 7 n 3 .Since S is a solution of a 3-SAT formula Φ iff all clauses of Φ are satisfied by S, for any assignment, there are exactly (7 n 3 ) m formulas with m clauses which admit S as a solution.The cardinality of the pairs (Φ, S) such that S is a solution of Φ is then given by 2 n (7 n 3 ) m .Dividing each term of (1) by the total number of formulae (8 n 3 ) m gives (with r = m/n): For r > ln(2)/ ln(8/7) ≈ 5.191, the right side of (2) tends to 0 as n tends to infinity, and so does the probability of satisfiability.This gives the first upper bound obtained by Franco and Paull as early as 1983.
This argument can easily be extended to the problem k-SAT for any k.For any fixed assignment, the number of clauses it satisfies which are disjunctions of k literals is given this time by The probability of satisfiability of k-SAT formulae tends to 0 when r > − ln(2)/ ln(1 − 2 −k ) and the following proposition holds Proposition 1 (Franco and Paull [12]) Let m, n be integers and Ω m,n the set of k-SAT formulae with m clauses constructed over n variables.Let m, n → ∞, while m/n ≥ r.Then, for any r > − ln(2)/ ln(1−2 −k ), the expectation of the number of solutions of a random formula tends to 0 and hence For k = 3, the bound is 5.191.
Note that this majorization of the threshold also holds for k = 2, but the value − ln(2)/ ln(3/4) ≈ 2.4 is appreciably greater than the exact value of the 2-SAT threshold, which has been proved to be 1.It also suggests 2 k ln(2) as an approximate value of the threshold for large k of the potential threshold location.

Prime implicants
In the previous section, we have bounded the number of satisfiable formulae by the number of formulasolution pairs.Since a satisfiable formula may have from 1 to almost 2 n solutions, the upper bound obtained may be very coarse.The next idea is to group some of the solutions which look very close to each other, and enumerate only these groups for each formula.In this way, it may be possible to get an improved upper bound on the satisfiability threshold.
This leads to the idea of using partial assignments and prime implicants, first introduced in this context by Olivier Dubois and Yacine Boufkhad [4].A partial assignment A is simply an assignment to a subset of the n variables (possibly all, so that complete assignment are also considered partial assignments).Let us say that A satisfies a formula Φ iff all complete assignments A ′ extending A are solutions of Φ.A necessary and sufficient condition for this is that in each clause of Φ, at least one of the three literals is true under A. Note that this implies that at least one variable is assigned.If there are i missing variables in a partial assignment A, then A "groups" 2 i solutions together.
A natural order may be placed on partial assignments.We say that A is smaller than B if we can remove some assigned variables from B to get A. A prime implicant of a formula φ is then defined as a partial assignment which is minimal with respect to this order.Any satisfiable formula has then at least one prime implicant since it has at least one solution and the set of partial assignments is then non empty.As in the previous section, we get from there the inequality (see (1)): Note that the sets of solutions grouped together by two different partial assignments are not necessarily disjoint and some formulae may even have more prime implicants than solutions.But in fact, on average, the number of prime implicants of a random formula appears to be smaller by an exponential factor than the number of solutions.
At this point, we start an original re-derivation of the upper bound value 4.883 obtained by O. Dubois and Y. Boufkhad for the 3-SAT threshold, with the help of generating functions, which provides a much shorter proof of their estimate.
Let us consider the partial assignment I which assigns all variables x 1 , . . ., x ℓ to 0. Then, all clauses in a formula Φ that admits I as a prime implicant must contain at least one literal satisfied by I. Any such clause is hence of the form ¬x i ∨ a ∨ b with 1 ≤ i ≤ ℓ.Let A n,ℓ be the set of such clauses and α n,ℓ their number.Since the complement of A n,ℓ is the set of clauses which do not contain any literal Recall that I has to be minimal with respect to the order defined earlier.Remove the variable x i from the set of assigned variables.Then at least one clause in Φ no longer contains a positive literal.Hence Φ must contains a clause of the form ¬x i ∨ a ∨ b with a and b either of the form Let B n,ℓ,i be the set of these clauses and β n,ℓ,i their number.It is easy to see that these sets are mutually disjoint and have the same cardinality Finally, to build a formula which admits I as a prime implicant, one must draw m elements in the set A n,ℓ so that in the subsets at least one element is taken.The number of such draws is then the number I m,n,ℓ whose generating function is given by The proof of this statement is given in the first part of the appendix.It is based on generating functions and the enumeration of throws in urns.Now, it is clear that for any other partial assignment of ℓ variables, the number of formulae of length m for which it is a prime implicant is the same as the one where the ℓ first variables are assigned to 0 and thus, it only depends on m, n and ℓ.Finally, since there are n ℓ 2 ℓ possible partial assignments of ℓ variables, the total number of pairs (Φ, I) such that I is a prime implicant for Φ is given by: The next step depends on the following general remark: if ( f k ) is a sequence of non negative reals and f (z) = ∑ f k z k then for all s > 0 within the domain of convergence of f (z): This is a very classical majorization called the saddle-point bound (the minimum of the function s → f (s)s −k is called the saddle point and the shape of the complex function f (s)s −k in its neighbourhood looks like a horse's saddle).The majorization may seem coarse since but in many cases it gives the exact exponential rate of growth of the coefficient f k knowing the generating function f (z).
From now on, we set ℓ = αn, m = rn and make use of the majorizations From the previous remark, it follows that for any u α The same remark proves that for any integer s, one has 1 s! ≤ e s s s .Combining this inequality (for the denominator) with Stirling's formula (for the numerator) ensures that there exists a constant C such that, for each integer n and each α such that αn is an integer, one has Naturally, stronger asymptotic estimates are possible, e.g.n αn ∼ (2πnα(1 − α)) −1/2 α α (1 − α) 1−α −n follows obviously from three applications of Stirling formula.However, equation ( 7) wholy suffices for our purposes while making subsequent argumentations simpler.
Finally, since (rn)!n 2rn (8( n 3 )) rn ∼ √ 2πre 3r n 1/2 3r 4e rn , ( 4) and (6) give and where δ α can be any strictly positive real.In order to get the best majorization, δ α has to minimize the expression between parentheses.If 0 < α < r, this expression has a unique minimum δ α given by Assume that r > 1.Then one gets Any real r > 1 such that the maximum of g r over ]0, 1] is strictly smaller than 1 is hence an upper bound on the satisfiability threshold.The value r = 4.883 is then the best value computed with Maple which has this property (details are given in the second part of appendix).[4]) Let m, n be integers and Ω m,n the set of 3-SAT formulae with m clauses constructed over n variables.Let m, n → ∞, while m/n ≥ r.Then, for any r > 4.883, the expectation of the number of prime implicants of a random formula tends to 0 and

Proposition 2 (Dubois and Boufkhad
The same analysis can easily be extended to k-SAT for any integer k.

Negatively prime solutions
The next idea is to introduce a partial order on the set of solutions.Define B to be an assignment smaller than A if one can change the values of some of B's variables from 0 to 1 to get A. We could envisage to enumerate only pairs (Φ, S) where S is a maximal solution with respect to this order; however, it is very difficult to find for any given assignment a simple characterization of formulae for which it is a maximal solution.In order to remedy this situation, Olivier Dubois and Yacine Boufkhad introduced a weaker definition of local maximal solution (also called negatively prime solution or NPSs).This is a solution for which changing the value of any variable from 0 to 1 no longer gives a solution.This amounts to considering solutions which do not admit a greater solution that differs in exactly one variable.Once more, there holds the inequality and the probability of satisfiability is smaller than the expectation of the number of NPSs.What follows now is a new calculation of this expectation.
Let A be an assignment setting ℓ variables to the value 0 and Φ a formula for which A is an NPS .Then, all clauses of Φ must belong to the set A n of all 7 n 3 clauses satisfied by A (as seen in Section 2).Now, if any variable x i assigned to 0 is changed to 1, there must be at least one clause in Φ that is no longer satisfied by this new assignment: at least one clause must be thus of the form ¬x i ∨ a ∨ b, where a and b are false under A. If we denote by C x i this last set of clauses (for each variable assigned to 0), then all these sets have the same number n−1 2 of elements and are mutually disjoint.As in the previous section, since there are n ℓ solutions with ℓ variables assigned to 0, we get: The linearity of coefficient extraction gives a closed-form expression: The same use of saddle-bound as in the previous section, the same majorization provided by Stirling's formula, and the change of variable δ = z n−1 2 provide that, for any δ > 0 with m = rn, This expression is minimized by δ 7 3 − 1 2e δ −1 = r and, with such a δ, is strictly smaller than 1 as soon as r > 4.643.Hence Proposition 3 (Dubois and Boufkhad [8]) Let m, n be integers and Ω m,n the set of 3-SAT formulae with m clauses constructed over n variables.Let m, n → ∞, while m/n ≥ r.Then, for any r > 4.643, the expectation of the number of NPSs of a random formula tends to 0 and Once more, this upper bound can be extended to k-SAT for any k.For any assignment giving the value 0 to ℓ variables, the number of satisfied clauses which are disjunction of k literals is this time (2 k − 1) n k .Once more, for all variable x i assigned to 0, define the set C x i of clauses which forbid the flip of x i from 0 to 1.Then, all these sets are mutually disjoint and have the same cardinality n−1 k−1 .The same arguments then provide the exact formula which gives the general majorization for any δ > 0, considering k-SAT formulae of length m = rn, With r, one determines that the probability of satisfiability of k-SAT formulae of length m = rn tends to 0 as soon as r > 10.217 for k = 4 and r > 43.508 for k = 6 for instance.This general upper bound has first been obtained by Olivier Dubois and is still so far the best one for general k.However, the initial proof took 25 pages where ours only needs about 2.

Throws into urns and generating functions
Let us consider a general problem.Let M 1 , . . ., M p be subsets of N. One has a collection of p urns and wants to throw in these urns m distinguishable balls (say by integer labels 1, 2, 3, . ..), such that in the end, the number of balls in urn i is in M i .For instance, if M 1 = {0, 1, 2} and M 2 = {2, 3, . ..}, one want to throw at most two balls in the first urn and at least two balls in the second one.The problem is to enumerate the number of such throws.
Let F m be this number.Then clearly, one has where [[P ]] equals 1 if P is true and equals 0 otherwise.The binomial coefficient enumerates all ways to put n 1 balls in urn 1, n 2 balls in urn 2 and so on.The product is then equal to 1 if and only if all conditions concerning the final number of balls in each urn are satisfied.Introduce the exponential generating function of the sequence (F m ).Then, (14) immediately implies For instance, the generating function of the example above is F(z) = ( z 2 2 + z + 1)(e z − z − 1).Let us now consider a slightly more complex problem.We now have p subsets of urns, say E 1 , . . ., E p of cardinality e 1 , . . ., e p , and p subsets of N M 1 , . . ., M p .This time, we want to throw m distinguishable balls in all these urns such that in the end, for any i, the total number of balls in the subset E i of urns is an element of M i .The formula (14) now becomes since there is now (e i ) n i ways to throw n i balls in the subset E i of urns, and the translation of this formula into generating function gives In section 3, we had to pick elements in one set A n,ℓ , so that in ℓ subsets B n,ℓ,i , at least 1 element is picked.Let Ãn,ℓ = A n,ℓ − (∪ 1≤i≤ℓ B n,ℓ,i ).This is equivalent to picking any number of elements in Ãn,ℓ and at least one in each of the B n,ℓ,i .
"Replace" now each clause by an urn.Picking m clauses with replacement to construct a formula is then equivalent to throwing m distinguishable balls in the set of urns.The enumeration of such draws can thus be regarded as an urn problem just defined, where we dispose of ℓ + 1 sets of urns.The first set contains αn,ℓ = α n,ℓ − ℓβ n,ℓ urns and we may throw any number of balls in it (hence M 1 = N).The other sets each contain β nℓ urns and we must throw at least one ball in one urn of the set (hence The equation ( 17) finally gives the desired result.

Maple computations
In this section, we prove with the help of Maple that for r = 4.883, the maximum of g r defined in section 3 is strictly smaller than 1.The idea is to get a coarse majorization of g ′ r and then to ask Maple to evaluate g r at sufficiently many points x 0 < • • • < x p .With the mean value theorem, the condition ensures that for any t ∈ / {x 0 , . . ., x p }, it also holds that g r (t) < 1.The purpose of this section is hence to replace many pages of analysis by Dubois and Boufkhad by a combination of straightforward inequalities and trivial computation.
A minor difficulty arises from the fact that the function α → 2 α α α (1−α) 1−α has infinite derivatives at 0 and 1.Hence, a rough majorization of g r on intervals of the form ]0, ε] and [1 − η, 1] must be obtained first.Monotonicity properties : The function g r can be written as a product of one constant 3r 2e r and two functions which are: First of all, one can prove that δ α and f 2 are both monotonic functions.Let us define Since h(α, δ α ) = r, differentiating this expression with respect to α yields It now easy to verify that both ∂h ∂x and ∂h ∂y are positive function on [0, 1]×]0, +∞[ and hence, δ α is a decreasing function of α.Concerning the function f 2 , the condition h(α, δ α ) = r also implies The function α → δ α being decreasing, for any α, ln(e δ α − 1) ≥ ln(e δ 1 − 1) ≈ 1.8017 > 0. Hence, f ′ 2 is positive and f 2 is an increasing function of α.
Finally, the function f 1 satisfies The expression between parentheses is smaller than ln(2) − ln(α) + ln(1 − α) − r, which is negative as soon as α > 2 e r +2 ∼ 0.0149.The function f 1 is thus decreasing on [0.015, 1].Majorization of g r on ]0, 0.897] and [0.957, 1]: First of all, we make use of the majorization which implies that for any α ∈]0, 0.414], g r (α) ≤ 3 3r By subdividing [0.015, 1] into many small intervals, it is shown with this majorization that g r is smaller than 1 on both intervals ]0, 0.897] and [0.957, 1].It is easy to see that both ∂h ∂x and ∂h ∂y are positive increasing functions of x and y so their respective greatest and lowest values are ∂h ∂x (0.957, δ 0.897 ) and ∂h ∂y (0.897, δ 0.957 ).To majorize ∂ f r ∂α and ∂ f r ∂δ α , we introduce a r such that f r = exp(a r ).Then ∂y .The derivatives of a r are simple functions, and finally, a coarse majorization of f r , ∂a r ∂x and ∂a r ∂x gives the bound:

Numerical evaluations:
We set r = 4.883.We got Maple to compute the greatest value of g r with an accuracy of 15 digits for points in the interval [0.897, 0.957] at distance at most 0.6 • 10 −7 of each other (10 6 values).The greatest value returned was 1 − 0.2721 • 10 −5 .Hence, the condition ( 18) is verified and the value 4.883 is proved to be a valid upper bound of the satisfiability threshold.