A Semantic Framework for the Declarative Debugging of Wrong and Missing Answers in Declarative Constraint Programming

The current book is a nice blend of number of great ideas, theories, mathematical models, and practical systems in the domain of Semantics. The book has been divided into two volumes. The current one is the first volume which highlights the advances in theories and mathematical models in the domain of Semantics. This volume has been divided into four sections and ten chapters. The sections include: 1) Background, 2) Queries, Predicates, and Semantic Cache, 3) Algorithms and Logic Programming, and 4) Semantic Web and Interfaces. Authors across the World have contributed to debate on state-of-the-art systems, theories, mathematical models in the domain of Semantics. Subsequently, new theories, mathematical models, and systems have been proposed, developed, and evaluated.


Introduction
Debugging tools are a practical need for helping programmers to understand why their programs do not work as intended. Declarative programming paradigms involving complex operational details, such as constraint solving and lazy evaluation, do not fit well to traditional debugging techniques relying on the inspection of low-level computation traces. As a solution to this problem, and following a seminal idea by Shapiro (Shapiro, 1982), declarative debugging (a.k.a. declarative diagnosis or algorithmic debugging) uses Computation Trees (shortly, CTs) in place of traces. CTs are built a posteriori to represent the structure of a computation whose top-level outcome is regarded as a symptom of the unexpected behavior by the user. Each node in a CT represents the computation of some observable result, depending on the results of its children nodes, using a program fragment also attached to the node. Declarative diagnosis explores a CT looking for a so-called buggy node which computes an unexpected result from children whose results are all expected. Each buggy node points to a program fragment responsible for the unexpected behavior. The search for a buggy node can be implemented with the help of an external oracle (usually the user with some semiautomatic support) who has a reliable declarative knowledge of the expected program semantics, the so-called intended interpretation.
The generic description of declarative diagnosis in the previous paragraph follows (Naish, 1997). Declarative diagnosis was first proposed in the field of Logic Programming (LP) (Ferrand, 1987;Lloyd, 1987;Shapiro, 1982), and it has been successfully extended to other declarative programming paradigms, including (lazy) Functional Programming (FP) (Nilsson, 2001;Nilsson & Sparud, 1997;Pope, 2006;Pope & Naish, 2003), Constraint Logic Programming (CLP) (Boye et al., 1997;Ferrand et al., 2003;Tessier & Ferrand, 2000), and Functional-Logic Programming (FLP) (Caballero & Rodríguez, 2004;Naish & Barbour, 1995). The nature of unexpected results differs according to the programming paradigm. Unexpected results in FP are mainly incorrect values, while in CLP and FLP an unexpected result can be either a single computed answer regarded as incorrect, or a set of computed answers (for one and the same goal with a finite search space) regarded as incomplete. These two possibilities give rise to the declarative debugging of wrong and missing computed answers, respectively. The case of unexpected finite failure of a goal is a particular symptom of missing answers with special relevance. However, diagnosis methods must consider the most general case, since finite 2 Will-be-set-by-IN-TECH failure of a goal is often caused by non-failing subgoals that do not compute all the expected answers.
In contrast to recent approaches to error diagnosis using abstract interpretation (e.g., (Alpuente et al., 2003;Comini et al., 1999;Hermenegildo, 2002), and some of the approaches described in (Deransart et al., 2000)), declarative diagnosis often involves complex queries to the user. This problem has been tackled by means of various techniques, such as user-given partial specifications of the program's semantics (Boye et al., 1997), safe inference of information from answers previously given by the user (Caballero & Rodríguez, 2004), or CTs tailored to the needs of a particular debugging problem over a particular computation domain (Ferrand et al., 2003). Another practical problem with declarative diagnosis is that the size of CTs can cause excessive overhead in the case of computations that demand a big amount of computer storage. As a remedy, techniques for piecemeal construction of CTs have been considered; see (Pope, 2006) for a recent proposal in the FP field. However, current research in declarative diagnosis has still to face many challenges regarding both the foundations and the development of practical tools.
In spite of the above mentioned difficulties, we are confident that declarative diagnosis methods can be useful for detecting programming bugs by observing computations whose demand of computer storage is modest. The aim of this chapter is to present a logical and semantic framework for diagnosing wrong and missing computed answers in CFLP(D) (López et al., 2006), a newly proposed generic programming scheme for lazy Constraint Functional-Logic Programming which can be instantiated by any constraint domain D given as parameter, and supports a powerful combination of functional and constraint logic programming over D. Sound and complete goal solving procedures for the CFLP(D) scheme have been obtained (López et al., 2004). Moreover, useful instances of this scheme have been implemented in the TOYsystem (López & Sánchez, 1999) and tested in practical applications (Fernández et al., 2007). Borrowing ideas from CFLP(D) declarative semantics we obtain a suitable notion of intended interpretation, as well as a kind of abridged proof trees with a sound logical meaning to play the role of CTs. Our aim is to achieve a natural combination of previous approaches that were independently developed for the CLP(D) scheme (Tessier & Ferrand, 2000) and for lazy functional-logic languages (Caballero & Rodríguez, 2004). We give theoretical results showing that the proposed debugging method is logically correct for any sound CFLP(D)-system whose computed answers are logical consequences of the program in the sense of CFLP(D) semantics. We also present a practical debugger called DDT , developed as an extension of previously existing but less powerful tools (Caballero, 2005;Caballero & Rodríguez, 2004). DDT implements the proposed diagnosis method for CFLP(R)-programming in the TOYsystem (López & Sánchez, 1999) using the domain R of arithmetic constraints over the real numbers.
The rest of the chapter is organized as follows: Section 2 motivates our approach by presenting debugging examples which are used as illustration of the main features of our diagnosis method. Section 3 recalls the CFLP(D) scheme from (López et al., 2006) to the extent needed for understanding the theoretical results in this chapter. Section 4 presents a correct method for the declarative diagnosis of wrong computed answers in any soundly implemented CFLP(D)-system. Section 5 describes the debugging tool DDT for wrong answers. Section 6 presents the abbreviated proof trees used as CTs in our method for debugging missing 122 Semantics -Advances in Theories and Mathematical Models www.intechopen.com A Semantic Framework for the Declarative Debugging of Wrong and Missing Answers in Declarative Constraint Programming 3 computed answers, as well as the results ensuring the logical correctness of the diagnosis. Section 7 presents a prototype debugger for diagnosing missing answers. Section 8 concludes and points to some plans for future work.

Motivating examples
As a motivation for our declarative debugging method of wrong answers in the CFLP(D) scheme, we consider the following program fragment written in TOY (López & Sánchez, 1999), a programming system which supports several instances of the CFLP(D) scheme:

Example 1. (Debugging Wrong Answers in TOY)
infixr 40 && (&&) :: bool -> bool -> bool In this example (see Fig. 1), TOYis used to implement the instance CFLP(R) of the CFLP(D) scheme, with the parameter D replaced by the real numbers domain R, which provides real numbers, arithmetic operations and various arithmetic constraints, including equalities, disequalities and inequalities. The type figure is intended to represent geometric figures as boolean functions, the function rect is intended to represent rectangles (more precisely, (rect (X,Y) LX LY) is intended to represent a rectangle with leftmost-bottom vertex (X,Y) and rightmost-upper vertex (X+LX,Y+LY)); and the function ladder is intended to build an infinite list of rectangles in the shape of a ladder. Although the text of the program seems to include no constraints, it uses arithmetic and comparison operators that give rise to constraint solving in execution time. More precisely, consider the following session in TOY: The goal asks for the membership of a generic point (X,Y) to the intersection of the two rectangles (rect (20,20) 50 20) and (rect (5,5) 30 40), computed indirectly as the first steps of two particular ladders. The diagram included in Fig. 1 shows these two rectangles as well as the rectangle corresponding to their intersection (highlighted in black). The TOYsystem has solved the goal by a combination of lazy narrowing and constraint solving; the computed answer consists of the substitution R-> true and three constraints imposed on the variables X and Y 1 . The only constraint imposed on Y (namely Y <= 5) allows for arbitrarily small values of Y, which cannot correspond to points belonging to the rectangle expected as intersection. Therefore, the user will view the computed answer as wrong with respect to the intended meaning of the program. As we will see in Sections 4 and 5, the declarative debugging technique presented in this chapter leads to the diagnosis of the program rule for the function rect as responsible for the wrong answer. Indeed, this program rule is incorrect with respect to the intended program semantics; as shown in Fig. 1, the third inequality at the right hand side After this correction, no more wrong computed answers will be observed for the goal discussed above. As any debugging technique, declarative diagnosis has limitations. A "corrected" program fragment can still include more subtle bugs that can be observed in the computed answers for other goals. In our case, we can consider the goal Toy > /cflpr Toy(R)> intersect (head (ladder (70,40) -50 -20)) (head (ladder (35,45) -30 -40)) (X,Y) == R whose meaning with respect to the intended semantics is the same as for the previous goal, except that the rectangles playing the role of initial steps of the two ladders are represented differently. Since the boolean expression at the right hand side of the "corrected" program rule for function rect yields the result false whenever LX or LY is bound to a negative number, wrong answers including the substitution R-> false will be computed. Moreover, other answers including the substitution R-> true will be expected by the user but missing to occur among the computed answers.
The traditional approach to declarative debugging in the CLP(D) scheme includes the diagnosis of both wrong and missing computed answers (Tessier & Ferrand, 2000). Now, we motivate our approach for the declarative debugging of missing answers in the CFLP(D) scheme by means of the following example, intended to illustrate the main features of our diagnosis method.
1 There are other five computed answers consisting of the substitution R-> false and various constraints imposed on X and Y.

Example 2. (Debugging Missing Answers in TOY)
The following small CFLP(H)-program P fD , written in TOY syntax over the Herbrand domain H with equality (==) and disequality (/=) constraints, includes program rules for the non-deterministic functions (//) and fDiff, and the deterministic functions gen and even. Note the infix syntax used for (//), as well as the use of the equality symbol = instead of the rewrite arrow -> for the program rules of those functions viewed as deterministic by the user. This is just meant as user given information, not checked by the TOYsystem, which treats all the program defined functions as possibly non-deterministic. Function fDiff is intended to return any element belonging to the longest prefix Xs of the list given as parameter such that Xs does not include two identical elements in consecutive positions. In general, there will be several of such elements, and therefore fDiff is non-deterministic. Function gen is deterministic and returns a potentially infinite list of the form [d 1 , d 2 , d 2 , d 1 , d 1 , d 2 ,...], where the elements d 1 and d 2 are the given parameters. Therefore, the lazy evaluation of (fDiff (gen 1 2)) is expected to yield the two possible results 1 and 2 in alternative computations, and the initial goal G fD : even (fDiff (gen 1 2)) == true for P fD is expected to succeed, since (fDiff (gen 1 2)) is expected to return the even number 2. However, if the third program rule for function fDiff were missing in program P fD , the expression (fDiff (gen 1 2)) would return only the numeric value 1, and therefore the goal G fD would fail unexpectedly. At this point, a diagnosis for missing answers could take place, looking for a buggy node in a suitable CT in order to detect some incomplete function definition (that of function fDiff, in this case) to be blamed for the missing answers. As we will see in Sections 6 and 7, this particular incompleteness symptom could be mended by placing again the third rule for fDiff within the program.

The CFLP(D) programming scheme
In this section we summarize the essentials of the CFLP(D) scheme (López et al., 2006) for lazy Constraint Functional-Logic Programming over a parametrically given constraint domain D, which serves as a logical and semantic framework for the declarative diagnosis method presented in the chapter.

125
A Semantic Framework for the Declarative Debugging of Wrong and Missing Answers in Declarative Constraint Programming www.intechopen.com

Preliminary notions
We consider a universal signature Σ = DC, FS , where DC = n∈N DC n and FS = n∈N FS n are countably infinite and mutually disjoint sets of data constructors resp. evaluable function symbols, indexed by arities. Evaluable functions are further classified into domain dependent primitive functions PF n ⊆ FS n and user defined functions DF n = FS n \ PF n for each n ∈ N.W e write Σ ⊥ for the result of extending DC 0 with the special symbol ⊥, intended to denote an undefined data value and we assume that DC includes the two constants true and false and the usual list constructors. We use the notations c, d ∈ DC, f , g ∈ FS, and h ∈ DC ∪ FS. We also assume a countably infinite set V of variables X, Y, . . . and a set U of primitive elements u, v,...
(as e.g. the set R of the real numbers) mutually disjoint and disjoint from Σ ⊥ . Expressions e ∈ Exp ⊥ (U ) have the following syntax: where u ∈U, X ∈V, h ∈ DC ∪ FS. An important subclass of expressions is the set of patterns s, t ∈ Pat ⊥ (U ), whose syntax is defined as follows: where u ∈U, X ∈V, c ∈ DC n with m ≤ n, and f ∈ FS n with m < n. Patterns are used as representations of possibly functional data values. For instance, the rectangle (rect (5, 5) 30 40) we met when discussing Example 1 is a functional data value represented as pattern 2 .
As usual, we define substitutions σ ∈ Sub ⊥ (U ) as mappings σ : V→Pat ⊥ (U ) extended to σ : Exp ⊥ (U ) → Exp ⊥ (U ) in the natural way. By convention, we write eσ instead of σ(e) for any e ∈ Exp ⊥ (U ), and σθ for the composition of σ and θ. A substitution σ such that σσ = σ is called idempotent.

Constraints over a constraint domain
Intuitively, a constraint domain provides a set of specific data elements, along with certain primitive functions operating upon them. Primitive predicates can be modelled as primitive functions returning boolean values. Formally, a constraint domain with primitive elements U and primitive functions PF ⊆ FS is any structure D = D U , {p D | p ∈ PF} with carrier set D U the set of ground patterns (i.e., without variables) over U and interpretations p D ⊆ D n U × D U of each p ∈ PF n satisfying the technical monotonicity, antimonotonicity, and radicality requirements given in (López et al., 2006). We use the notation p D t n → t to indicate that (t n , t) ∈ p D .
Constraints over a given constraint domain D are logical statements built from atomic constraints by means of logical conjunction ∧ and existential quantification ∃. Atomic constraints can have the form ♦ (standing for truth), (standing for falsity), or p e n →! t , meaning that the primitive function p ∈ PF n with parameters e n ∈ Exp ⊥ (U ) returns a total result t ∈ Pat ⊥ (U ) (i.e, with no occurrences of ⊥). Constraints whose atomic parts have the 2 Note that (5, 5) can be seen as syntactic sugar for (pair 55), pair being a constructor for ordered pairs. form ♦, or p t n →! t with t n ∈ Pat ⊥ (U ) are called primitive constraints. In the sequel, we use the notation PCon ⊥ (D) for the set of primitive constraints over D and DCon ⊥ (D) for the set of user defined constraints over D.

Example 3. (Constraint Domain R)
The constraint domain R has the carrier set D R of ground patters over R and the primitives defined below: 1. eq R , equality primitive for real numbers, such that: eq R R uu→ true for all u ∈ R;e q R R uv→ f alse for all u, v ∈ R, u = v; eq R R ts→⊥otherwise. 2. seq, strict equality primitive for ground patterns over the real numbers, such that: seq R tt→ true for all total t ∈ D R ; seq R ts→ f alse for all t, s ∈ D R such that t, s have no common upper bound with respect to the information ordering introduced in (López et al., 2006); seq R ts→⊥ otherwise. In the sequel, e 1 == e 2 abbreviates seq e 1 e 2 →! true. 3. +, −, * , for addition, subtraction and multiplication, such that: x + R y → x + R y for all x, y ∈ R;t + R s →⊥whenever t / ∈ R or s / ∈ R; and analogously for − R and * R . 4. <, ≤, >, ≥, for numeric comparisons, such that: x < R y → true for all x, y ∈ R with x < R y; x < R y → f alse for all x, y ∈ R with x ≥ R y; t < R s →⊥whenever t / ∈ R or s / ∈ R; and analogously for ≤ R , > R , ≥ R . In the sequel, e 1 < e 2 abbreviates e 1 < e 2 →! true and e 1 ≥ e 2 abbreviates e 1 < e 2 →! f alse (analogously for other comparison primitives).
The set of valuations over a constraint domain D is defined as the set Val ⊥ (D) of ground substitutions (i.e., mappings from variables to ground patterns). The semantics of constraints relies on the idea that a given valuation can satisfy or not a given constraint. Therefore, the set of solutions of π ∈ PCon ⊥ (D) can be defined in a natural way as a subset Sol D (π) ⊆ Val ⊥ (D); see (López et al., 2006) for details. Moreover, the set of solutions of Π ⊆ PCon ⊥ (D) is defined as Sol D (Π)= π∈Π Sol D (π).

Constraint functional-logic programming
For any given constraint domain D,aCFLP(D)-program P is presented as a set of constrained rewrite rules, called program rules, that define the behavior of user-defined functions. More precisely, a constrained program rule R for f ∈ DF n has the form R : f t n → r ⇐ Δ (abbreviated as f t n → r if Δ is empty) and is required to satisfy the conditions listed below 3 : 1. The left-hand side f t n is a linear expression (i.e, there is no variable having more than one occurrence), and for all is a finite set of total atomic constraints, intended to be interpreted as conjunction, and possibly including occurrences of user defined functions.
Program defined functions can be higher-order and/or non-deterministic. For instance, the TOY program presented in Example 1 can be viewed as an example of CFLP(R)-program written in TOY's syntax. The reader is referred to (López et al., 2006) for more explanations and examples in other constraint domains. The intended use of programs is to perform computations by solving goals proposed by the user. An admissible goal for a given CFLP(D)-program must have the form G : ∃U. (P ✷ Δ), where U is a finite set of so-called existential variables of the goal G (the rest of variables in G are called free variables and denoted by fvar(G)), P is a finite conjunction of so-called productions of the form e → s fulfilling the admissibility conditions given in (López et al., 2006), with e ∈ Exp ⊥ (U ) and s ∈ Pat ⊥ (U ) intended to mean that e can be evaluated to s, and Δ ⊆ DCon ⊥ (D) is a finite conjunction of total user defined constraints. Two special kinds of admissible goals are interesting. Initial goals, where U and P are both empty (i.e., G has only a constrained part Δ without occurrences of existential variables), and solved goals (also called solved forms)ofthe form S : ∃U. (σ ✷ Π), where σ is a finite set of productions X → t or s → Y interpreted as the variable bindings of an idempotent substitution and Π ⊆ PCon ⊥ (D) is a finite conjunction of total primitive constraints. Finally, a goal solving system for CFLP(D) is expected to accept a program P and an initial goal G from the user, and to obtain one or more solved forms S i as computed answers. As explained in Section 2, an initial goal G for the CFLP(R)-program shown in Example 1 can be intersect (head (ladder (20, 20) 50 20)) (head (ladder (5, 5) 30 40) Goal solving systems can be implementations of CFLP languages such as Curry (Hanus, 2003) or TOY(López & Sánchez, 1999), or formal goal solving calculi including recent proposals such as the CDNC(D) calculus (López et al., 2004), which is sound and complete with respect to the declarative semantics discussed in the next subsection, and behaves as a faithful formal model for actual computations in the TOYsystem.

Standardized programs and negative theories
Let P be a CFLP(D)-program. Its associated Negative Theory P − is obtained in two steps.
for each function symbol f whose standardized program rules are { f X n → Y ⇐ R i } i∈I . By convention, we may use the notation D f for the disjunction ( i∈I R i ) ∨ (⊥→Y), and we may leave the universal quantification of the variables X n , Y implicit. Intuitively, the axiom ( f ) − P says that any result computed for f must be obtained by means of some of the rules for f in the program. The last alternative (⊥→Y) within D f says that Y is bound to the undefined result ⊥ in case that no program rule for f succeeds to compute a more defined result. For example, let P fD be the CFLP(H)-program given in Section 2, with the third program rule for fDiff omitted. Then P − fD includes (among others) the following axiom for the function symbol fDiff:

Answer collection assertions
In this work we propose to use computation trees for missing answers whose nodes have attached so-called answer collection assertions, briefly acas. The aca at the root node has the form G ⇒ i∈I S i , where G is the initial goal and i∈I S i (written as the failure symbol if I = ∅) is the disjunction of computed answers observed by the user. This root aca asserts that the computed answers cover all the solutions of the initial goal, and will be regarded as a false statement in case that the user misses computed answers. For example, the root aca corresponding to the initial goal G fD for program P fD is even (fDiff (gen 1 2)) == true ⇒ stating that this goal has (unexpectedly) failed. The acas at internal nodes in our computation trees have the form f t n → t ✷ S ⇒ i∈I S i , asserting that the disjunction of computed answers i∈I S i covers all the solutions for the intermediate goal Note that G ′ asks for the solutions of the production f t n → t which satisfy the constraint store S. The acas of this form correspond to the intermediate calls to program defined functions f needed for collecting all the answers computed for the initial goal G. Due to lazy evaluation, the parameters t n and the result t will appear in the most evaluated form demanded by the topmost computation. When these values are functions, they are represented in terms of partial applications of top-level function names. This is satisfactory under the assumption that no local function definitions are allowed in programs, as it happens in TOY.

Declarative semantics
In this subsection we recall some notions and results on the declarative semantics of CFLP(D)-programs which were developed in (López et al., 2006) and are needed for the rest of this work. Given a constraint domain D we consider two different kinds of constrained statements (briefly, c-statements) involving partial patterns t, t i ∈ Pat ⊥ (U ), partial expressions e, e i ∈ Exp ⊥ (U ), and a finite set Π ⊆ PCon ⊥ (D) of primitive constraints: 1. c-productions e → t ⇐ Π, with e ∈ Exp ⊥ (U ) and t ∈ Pat ⊥ (U ), intended to mean that e can be evaluated to t if Π holds (if Π is empty they boil down to unconstrained productions written as e → t). As a particular kind of c-productions useful for debugging we distinguish c-facts f t n → t ⇐ Π with f ∈ DF n . A c-production is called trivial iff t = ⊥ or Sol D (Π)=∅. 2. c-atoms p e n →! t ⇐ Π, with p ∈ PF n and t total (if Π is empty they boil down to unconstrained atoms written as p e n →! t ). A c-atom is called trivial iff Sol D (Π)=∅.
In the sequel, we use ϕ to denote any c-statement. A c-interpretation over D is defined as any set I of c-facts including all the trivial c-facts and closed under D-entailment, a generalization of the entailment notion introduced in (Caballero & Rodríguez, 2004) to arbitrary constraint domains. We write I⊢ ⊢ D ϕ to indicate that the c-statement ϕ (not necessarily a c-fact) is semantically valid in the c-interpretation I. This notation relies on a formal definition given in (López et al., 2006). Now we are in a position to define various semantics notions which rely on a given c-interpretation I over D.

129
A Semantic Framework for the Declarative Debugging of Wrong and Missing Answers in Declarative Constraint Programming www.intechopen.com The set of solutions of a set of constraints Δ⊆DCon ⊥ (D) is defined as Sol I (Δ)= δ∈Δ Sol I (δ).

The set of solutions of a production e → t is a subset Sol
The set of solutions of a set of productions P is defined as Sol I (P)= (e→t)∈P Sol I (e → t).
3. The set of solutions of an admissible goal G : ∃U.
For primitive constraints one can easily check that Sol I (Π)=Sol D (Π). Moreover, we note that Sol I (S)=Sol D (S) for every solved form S.

Definition 2. (Model-Theoretic Semantics)
Let P a CFLP(D)-program and I a c-interpretation.
A solved form S is a semantically valid answer for a goal G with respect to a program P (in When this inclusion holds, we say that ( f ) − P is valid in I, or also that f 's definition as given in P is complete with respect to I. 4. The aca G ⇒ i∈I S i is a logical consequence of P − iff Sol I (G) ⊆ i∈I Sol D (S i ) for any model I of P − . When this happens, we also say that the disjunction of answers i∈I S i is complete for G with respect to P.

Declarative debugging of wrong answers in CFLP(D)
In this section, we present the logical and semantic framework of the declarative diagnosis method of wrong answers for CFLP(D) and prove its logical correctness. In what follows, we assume that a constraint domain D and a CFLP(D)-program P are given.

Wrong answers and intended interpretations
Declarative diagnosis techniques rely on a declarative description of the intended program semantics. We will assume that the user knows (at least to the extent needed for answering queries during the debugging session) a so-called intended model I, which is a c-interpretation expected to satisfy I| = D P, unless P is incorrect. For instance, rect (X, Y) LX LY (A, B) → false ⇐ A < X ∧ LX > 0 ∧ LY > 0 could belong to the intended model I for the program fragment shown in Example 1. As explained in Subsection 3.6, the c-facts belonging to c-interpretations can be non-ground. Nevertheless, the model notion I| = D P used here (see Definition 2 above) corresponds to the so-called weak semantics from (López et al., 2006), which depends just on the ground c-facts valid in I. Therefore, different presentations of the intended model will be equivalent for the purposes of this work, as long as the ground c-facts valid in them are the same.
The aim of declarative diagnosis of wrong answers is to start with an observed symptom of erroneous program behavior, and detect some error in the program. The proper notions of symptom and error in our setting are as follows: Definition 3. (Symptoms and Errors) Assume I is the intended interpretation for program P, and consider a solved form S produced as computed answer for the initial goal G by some goal solving system. We define: 1. S is a wrong answer w.r.t I (serving as symptom) iff Sol D (S) ⊆ Sol I (G). 2. P is incorrect with respect to I iff there exists some program rule ( f t n → r ⇐ Δ) ∈P (manifesting an error) that is not valid in I (in the sense of Definition 2).
For instance, the computed answer shown in Example 1 is wrong with respect to the intended model of the program assumed in that example, for the reasons already discussed in Section 2. As illustrated by that example, computed answers typically include constraints on the variables occurring in the initial goal. However, goal solving systems for CFLP(D) programs also maintain internal information on constraints related to variables used in intermediate computation steps, but not occurring in the initial goal. Such information is relevant for declarative debugging purposes. Therefore, in the rest of this section we will assume that computed answers S include also constraints related to intermediate variables.

A logical calculus for witnessing computed answers
Assuming that S is a computed answer for an initial goal G using a program P, the declarative diagnosis of wrong answers needs a suitable Computation Tree (shortly, CT) representing the computation. In our setting we will obtain the CT from a logical proof P⊢ CPPC(D) G ⇐ S which derives the statement G ⇐ S from the program P in the Constraint Positive Proof Calculus (shortly CPPC(D)) given by the inference rules in Fig. 2. We will say that the CPPC(D)-proof witnesses the computed answer.
Most of these inference rules have been borrowed from the proof theory of CRWL(D),a Constraint ReWriting Logic which characterizes the semantics of CFLP(D) programs (López et al., 2006). The main novelties in CPPC(D) are the addition of rule EX (to deal with existential quantifiers in computed answers) and a reformulation of rule DF P , which is presented as the consecutive application of two inference steps named AR f and FA f , which cannot be applied separately. The purpose of this composite inference is to introduce the c-facts f t n → t ⇐ Π at the conclusion of inference FA f , called boxed c-facts in the sequel. As we will see, only boxed c-facts will appear at the nodes of CTs obtained from CPPC(D)-proofs. Therefore, all the queries asked to the user during a declarative debugging session will be about the validity of c-facts in the intended model of the program, which is itself represented as a set of c-facts. We also agree that the premises Gσ ⇐ Π in rule EX (resp. Δ ⇐ Π in rule DF P ) must be understood as a shorthand for several premises α ⇐ Π, one for each atomic ϕ in Gσ (resp. Δ). Moreover, rule PF depends on the side condition Sol D (Π) ⊆ Sol D (pt n → t) which is true iff p D t n η → tη holds for all η ∈ Sol D (Π). Some other inference rules in Fig. 2 head (ladder (20, 20) 50 20)) (head (ladder (5, 5) 30 40)) (X, Y)==true

Fig. 3. A Positive Proof Tree in CPPC(R)
Any CPPC(D)-derivation P⊢ CPPC(D) G ⇐ S can be depicted in the form of a Positive Proof Tree over D (shortly, PPT(D)) with G ⇐ S at the root and c-statements at the internal nodes, and such that the statement at any node is inferred from the statements at its children using some CPPC(D) inference rule. In particular, the statement at the root must be inferred using rule EX, which is then applied nowherelse in the proof tree. Fig. 3 shows a PPT(R) representing a CPPC(R)-derivation which witnesses the computed answer from Example 1, which is wrong with respect to the intended model of the program. We say that a goal solving system is called CPPC(D)-sound iff for any computed answer S obtained for an initial goal G Proof. Straightforward adaptation of the soundness theorem for CDNC(D) presented in (López et al., 2004).
In addition to the goal solving calculus CDNC(D), other formal goal solving calculi known for CFLP(D) are also CPPC(D)-sound. Moreover, it is also reasonable to assume CPPC(D)-soundness for implemented goal solving systems such as Curry (Hanus, 2003) and TOY (López & Sánchez, 1999) whose computation model is based on constrained lazy narrowing. Moreover, any CPPC(D)-sound goal solving system is semantically sound in the sense of item 2 in Definition 2:

Theorem 2. (Semantic Correctness of the CPPC(D) Calculus)
If G is an initial goal for P and S is a solved goal s.t. P⊢ CPPC(D) G ⇐ S then P| = D G ⇐ S.
Proof. For each of the inference rules EX, AR f , and FA f , we prove that an arbitrary model I| = D P such that the premises of the rule are valid in I, also verifies that the conclusion of the rule is valid in I. Similar proofs for the other inference rules in CFLP(D) can be found in (López et al., 2006).

Declarative diagnosis using positive proof trees
Now we are ready to present a declarative diagnosis method of wrong answers and to prove its correctness. Our results apply to any CPPC(D)-sound goal solving system. First we prove that the observation of an error symptom implies the existence of some error in the program:

Theorem 3. (Wrong Answers Are Caused By Erroneous Program Rules)
We assume that a CPPC(D)-sound goal solving system computes S as an answer for the initial goal G using program P. If S is wrong with respect to the user's intended interpretation I then some program rule belonging to P is incorrect with respect to I.
Proof. Because of CPPC(D)-soundness of the goal solving system, we know that P⊢ CPPC(D) G ⇐ S. Then, from Theorem 2 we obtain P| = D G ⇐ S, i.e., Sol D (S) ⊆ Sol J (G) for each model J| = D P. Since S is wrong with respect to the user's intended model I, it must be the case that Sol D (S) ⊆ Sol I (G) because of Definition 3. Therefore, we can conclude that the intended model I is not a model of P. Then, by Definition 2, some program rule belonging to P is not valid in I.
The previous theorem does not yet provide a practical method for finding an erroneous program rule. As explained in the Introduction, a declarative diagnosis method is expected to find the erroneous program rule by inspecting a CT. We propose to use abbreviated CPPC(D) proof trees as CTs. Since DF P is the only inference rule in the CPPC(D) calculus that depends on the program, abbreviated proof trees will omit the inference steps related to all the other CPPC(D) rules. More precisely, given a PPT(D) T , its associated Abbreviated Positive Proof Tree over D (shortly, APPT(D)) AT is defined as follows: • The root of AT is the root of T .
• The children of a node N in AT are the closest descendants of N in T corresponding to boxed c-facts introduced by DF P inference steps.

135
A Semantic Framework for the Declarative Debugging of Wrong and Missing Answers in Declarative Constraint Programming

www.intechopen.com
A node in an APPT(D) is called a buggy node iff the c-statement at the node is not valid in the intended interpretation I, while all the c-statements at the children nodes are valid in I. Our last theorem guarantees that declarative diagnosis with APPT(D)s used as CTs leads to the correct detection of program errors.

Theorem 4 (Declarative Diagnosis of Wrong Answers).
Under the assumptions of Theorem 3, any APPT(D) witnessing P⊢ CPPC(D) G ⇐ S (which must exist due to CPPC(D)-soundness of the goal solving system) has some buggy node. Moreover, each buggy node points to a program rule belonging to P which is incorrect in the user's intended interpretation.  Before starting a debugging session, the user may inspect and simplify the tree using several facilities. For instance the user could mark any node corresponding to the infix function && as trusted, indicating that the definition of && is surely not erroneous. This makes all the nodes corresponding to && automatically valid. Valid nodes can be removed from the tree safely (the set of buggy nodes doesn't change) by using a suitable menu option.

A declarative debugging tool of wrong answers in TOY
Next, the user can start a debugging session by selecting one of the two possible strategies included in DDT : the top-down or the divide and query strategy (see (Caballero & Rodríguez, 2004) for a comparative between both strategies in an older version of DDT which did not yet support constraints). After selecting the divide and query strategy, which usually leads to shorter sessions, DDT asks about the validity of the following node: The intended program model corresponds to the intuitions explained in Section 2. Therefore, the question must be understood as: Is (X, Y) a point in the intersection of the two rectangles for all possible values of X, Y satisfying X ≤ 35, X ≥ 20, Y ≤ 5 is (X, Y)? The answer is no, because with these constraints Y can take any value less than 5 and some of these values would yield a pair (X, Y) out of the intersection for every X. Therefore the user marks the cross meaning that the c-fact is non-valid. The next question is: which is also reported as non-valid by the user. At this point a buggy node is found by the tool, pointing out to the incorrect program rule and ending the debugging session: The current version of the debugger supports programs using the constraint domain R, which provides arithmetic constraints over the real numbers as well as strict equality and disequality constraints over data values of any type; see Example 3 and (López et al., 2006) for details. The tool is as an extension of older versions which did not yet support constraints over the domain R (Caballero, 2005;Caballero & Rodríguez, 2004), and it is part of the public distribution of the functional-logic programming system TOY, available at http://toy.sourceforge.net.
The APPT(R) associated to a wrong answer is constructed by means of a suitable program transformation. The yielded tree is then displayed through a graphical debugging interface implemented in Java. More detailed explanations on the practical use of DDT can be found in (Caballero, 2005;Caballero & Rodríguez, 2004).

IM IMitation
if he m is not a pattern, X ∈V, and X m / ∈ var(he m → X ✷ S).

(AR) f Argument Reduction for Defined Functions
, and X n , Y / ∈ var( f e n a k → t ✷ S).

Declarative debugging of missing answers in CFLP(D)
The declarative debugging of missing answers also assumes an intended interpretation I P of the CFLP(D)-program P, starts with the observation of an incompleteness symptom and ends with an incompleteness diagnosis. A more precise definition of this debugging scenario of missing answers is as follows: Definition 4. (Debugging Scenario of Missing Answers) For any given CFLP(D)-program P: 1. An incompleteness symptom occurs if the goal solving system computes finitely many solved goals {S i } i∈I as answers for an admissible initial goal G, and the programmer judges that Sol I P (G) i∈I Sol D (S i ), meaning that the aca G ⇒ i∈I S i is not valid in the intended interpretation I P , so that some expected answers are missing.
2. An incompleteness diagnosis is given by pointing to some defined function symbol f such that the axiom ( f ) − P : ( f X n → Y ⇒ D f ) for f in P − is not valid in I P , which means Sol I P ( f X n → Y) ⊆ Sol I P (D f ), showing that f 's definition as given in P is incomplete w.r.t. I P .
Some concrete debugging scenarios have been discussed in Section 2. Assume now that an incompleteness symptom has been observed by the programmer. Since the goal solving system has computed the disjunction of answers D = i∈I S i , the aca G ⇒ D asserting that the computed answers cover all the solutions of G should be derivable from P − . The Constraint Negative Proof Calculus CNPC(D) consisting of the inference rules displayed in Fig.  5 has been designed with the aim of enabling logical proofs P − ⊢ CNPC(D) G ⇒ D of acas. We use a special operator & in order to express the result of attaching to a given goal G a solved goal S ′ resulting from a previous computation, so that computation can continue from the new goal G & S ′ .
Formally, assuming G = ∃U. (R ✷ (Π ✷ σ)) and (Rσ ′ ✷ (Π ′ ✷ σ ′ )). The inference rule CJ infers an aca for a goal with composed kernel (R 1 ∧ R 2 ) ✷ S from acas for goals with kernels of the form R 1 ✷ S and (R 2 & S i ), respectively; while other inferences deal with different kinds of atomic goal kernels.
Any CNPC(D)-derivation P − ⊢ CNPC(D) G ⇒ D can be depicted in the form of a Negative Proof Tree over D (shortly, NPT) with acas at its nodes, such that the aca at any node is inferred from the acas at its children using some CNPC(D) inference rule. We say that a goal solving system for CFLP(D) is admissible iff whenever finitely many solved goals {S i } i∈I are computed as answers for an admissible initial goal G, one has P − ⊢ CNPC(D) G ⇒ i∈I S i with some witnessing NPT. The next theorem is intended to provide some plausibility to the pragmatic assumption that actual CFLP systems such as Curry (Hanus, 2003) or TOY(López & Sánchez, 1999) are admissible goal solving systems.  (López et al., 2004), and a certain selection strategy that only selects atoms descendants of the part R) then P − ⊢ CNPC(D) R & S ⇒ D with some witnessing NPT. The proof proceeds by induction on p, using an auxiliary lemma to deal with compound goals whose kernel is a conjunction.
We have also proved the following theorem, showing that any aca which has been derived by means of a NPT is a logical consequence of the negative theory associated to the corresponding program. This result will be used below for proving the correctness of our diagnosis method of missing answers.
Theorem 6. (Semantic Correctness of the CNPC(D) Calculus) Let G ⇒ D be any aca for a given CFLP(D)-program P.IfP − ⊢ CNPC(D) G ⇒ D then G ⇒ D is a logical consequence of P − in the sense of Definition 2.

Declarative diagnosis of missing answers using negative proof trees
We are now prepared to present a declarative diagnosis method for missing answers which is based on NPTs and leads to correct diagnosis for any admissible goal solving system. First, we show that incompleteness symptoms are caused by incomplete program rules. This is guaranteed by the following theorem:

Theorem 7. (Missing Answers are Caused by Incomplete Program Rules)
Assume that an incompleteness symptom has been observed for a given CFLP(D)-program P as explained in Definition 4, with intended interpretation I P , admissible initial goal G, and finite disjunction of computed answers D = i∈I S i . Assume also that the computation has been performed by an admissible goal solving system. Then there exists a defined function symbol f such that the axiom ( f ) − P for f in P − is not valid in I P , so that f 's definition as given in P is incomplete with respect to I P .
Proof. Because of the admissibility of the goal solving system, we can assume P − ⊢ CNPC(D) G ⇒ D. Then the aca G ⇒ D is a logical consequence of P − because of Theorem 6. By Definition 2, we conclude that Sol I (G) ⊆ Sol D (D) holds for any model I of P − . However, we also know that Sol I P (G) Sol D (D), because the disjunction D of computed answers is an incompleteness symptom with respect to I P . Therefore, we can conclude that I P is not a model of P − , and therefore the completeness axiom ( f ) − P of some defined function symbol f must be invalid in I P .
The previous theorem does not yet provide a practical method for finding an incomplete function definition. As explained in Section 2, a declarative diagnosis method is expected to find the incomplete function definition by inspecting a CT. We propose to use abbreviated NPTsa sCTs. Note that (DF) f is the only inference rule in the CNPC(D) calculus that depends on the program, while all the other inference rules are correct with respect to arbitrary interpretations. For this reason, abbreviated proof trees will omit the inference steps related to the CNPC(D) inference rules other than (DF) f . More precisely, given a NPT T witnessing a CNPC(D) proof P − ⊢ CNPC(D) G ⇒ D, its associated Abbreviated Negative Proof Tree (shortly, ANPT) AT is constructed as follows: (1) The root of AT is the root of T .
(2) The children of any node N in AT are the closest descendants of N in T corresponding to boxed acas introduced by (DF) f inference steps. Fig. 6. NPT for the declarative diagnosis of missing answers As already explained, declarative diagnosis methods search a given CT looking for a buggy node whose result is unexpected but whose children's results are all expected. In our present setting, the CTsa r eANPTs, the "results" attached to nodes are acas, and a given node N is buggy iff the aca at N is invalid (i.e., it represents an incomplete recollection of computed answers in the intended interpretation I P ) while the aca at each children node N i is valid (i.e., it represents a complete recollection of computed answers in the intended interpretation I P ).
As a concrete example, Fig. 6 displays a NPT which can be used for the diagnosis of missing answers in the Example 2. Buggy nodes are highlighted by encircling the acas attached to them within double boxes. The CT shown in Fig. 7 is the ANPT constructed from this NPT. In this case, the programmer will judge the root aca as invalid because he did not expect finite failure. Moreover, from him knowledge of the intended interpretation, he will decide to consider the acas for the functions gen, even, and (//) as valid. However, the 141 A Semantic Framework for the Declarative Debugging of Wrong and Missing Answers in Declarative Constraint Programming www.intechopen.com aca fDiff (2:2:1:G) → F 2 ⇒ (F 2 →⊥ ) asserts that the undefined value ⊥ is the only possible result for the function call fDiff (2:2:1:G), while the user expects also the result 2. Therefore, the user will judge this aca as invalid. The node where it sits (enclosed within a double box in Fig. 7) has no children and thus becomes buggy, leading to the diagnosis of fDiff as incomplete. This particular incompleteness symptom could be mended by placing the third rule for fDiff within the program. Our last result is a refinement of Theorem 7. It Fig. 7. CT for the declarative diagnosis of missing answers guarantees that declarative diagnosis with ANPTs used as CTs leads to the correct detection of incomplete program functions.

Theorem 8. (ANPTs Lead to the Diagnosis of Incomplete Functions)
As in Theorem 7, assume that an incompleteness symptom has been observed for a given CFLP(D)-program P as explained in Definition 4, with intended interpretation I P , admissible initial goal G, and finite disjunction of answers D = i∈I S i , computed by an admissible goal solving system. Then P − ⊢ CNPC(D) G ⇒ D, and the ANPT constructed from any NPT witnessing this derivation, has some buggy node. Moreover, each such buggy node points to an axiom ( f ) − P which is incomplete with respect to the user's intended interpretation I P .

A declarative debugging tool of missing answers in TOY
In this section, we discuss the implementation in the TOY system of a tool based on the debugging method presented in the previous section. The current prototype only supports the Herbrand constraint domain H, although the same principles can be applied to other constraint domains D.
We summarize first the normal process followed by the TOYsystem when compiling a source program P.toy and solving an initial goal G with respect to P. During the compilation process the system translates a source program P.toy into a Prolog program P.pl including a predicate for each function in P. For instance the function even of our running example is transformed into a predicate where the variable N corresponds to the input parameter of the function, R to the function result, and IC,OC represent, respectively, the input and output constraint store. Moreover, each goal G of P is also translated into a Prolog goal and solved with respect to P.pl by the underlying Prolog system. The result is a collection of answers which are presented to the user in a certain sequence, as a result of Prolog's backtracking.
If the computation of answers for G finishes after having collected finitely many answers, the user may decide that there are some missing answers (incompleteness symptom, in the explained in Section 6, as responsible for the missing answers. The current implementation of the prototype is available at http://gpd.sip.ucm.es/rafav/. Fig. 8. Snapshots of the prototype of missing answers Fig. 8 shows how the tool displays the CT corresponding to the debugging scenario discussed in Section 2. The initial goal is not displayed, but the rest of the CT corresponds to Fig.  7, whose construction as ANPT has been explained in Section 6. When displaying an aca f t n → t ✷ S ⇒ i∈I S i , the tool uses list notation for representing the disjunction i∈I S i and performs some simplifications: useless variable bindings within the stores S and S i are dropped, as in the aca displayed as gen 2 1 -> A ==> [A = 2:1:_] in Fig. 7; and if t happens to be a variable X, the case {X →⊥ }is omitted from the disjunction i∈I S i ,s o that the user must interpret the aca as a collection of the possible results for X other than the undefined value ⊥. The tool also displays the underscore symbol _ at some places. Within any aca, the occurrences of _ at the right hand side of the implication ⇒ must be understood as different existentially quantified variables, while each occurrence of _ at the left hand side of ⇒ must be understood as ⊥. For instance, 1/ /_->A= = >[ A=1 ] is the aca 1/ / ⊥→A ⇒{ A → 1} as displayed by the tool. Understanding the occurrences of _ at the left hand side of ⇒ as different universally quantified variables would be incorrect. For instance, the aca 1/ / ⊥→A ⇒{ A → 1} is valid with respect to the intended interpretation I P fD of P fD , while the statement ∀X. (1/ / X → A ⇒{ A → 1}) has a different meaning and is not valid in I P fD .
In the debugging session shown in Fig. 8 the user has selected the Divide & Query strategy (Silva, 2006) in order to find a buggy node. The lower part of the left-hand side snapshot shows the first question asked by the tool after selecting this strategy, namely the aca fDiff 1:2:2:1:_ -> A ==> [A = 1]. According to her knowledge of I P fD the user marks this aca as invalid. The strategy now prunes the CT keeping only the subtree rooted by the invalid aca at the previous step (every CT with an invalid root must contain at least one buggy node). The second question, which can be seen at the right-hand side snapshot, asks about the validity of the aca fDiff 2:2:1:_ -> A ==> [] (which in fact represents fDiff 2:2:1:⊥→A ⇒{ A →⊥ } , as explained before). Again, her knowledge of I P fD leads the user to expect that fDiff 2:2:1:⊥ can return some defined result, and the aca is marked as invalid. After this question the debugger points out at fDiff as an incomplete function, and the debugging session ends.
Regarding the efficiency of this debugging method our preliminary experimental results show that: 1. Producing the transformed P T . pl from P.pl is proportional in time to the number of functions of the program, and does require an insignificant amount of system memory since each predicate is transformed separately. 2. The computation of the goal for P T . pl requires almost the same system resources as for P.pl because writing the trace causes no significant overhead in our experiments. 3. Producing the CT from the trace is not straightforward and requires several traverses of the trace. Although more time-consuming due to the algorithmic difficulty, this process only keeps portions of the trace in memory at each moment. 4. The most inefficient phase in our current implementation is the graphical interface.
Although it would be possible to keep in memory only the portion of the tree displayed at each moment, our graphical interface loads the whole CT in main memory. We plan to improve this limitation in the future. However the current prototype can cope with CTs containing thousands of nodes, which is enough for medium size computations. 5. As usual in declarative debugging, the efficiency of the tool depends on the computation tree size, which in turn usually depends on the size of the data structures required and not on the program size.
A different issue is the difficulty of answering the questions by the user. Indeed in complicated programs involving constraints the acas can be large and intricate, as it is also the case with other debugging tools for CLP languages. Nevertheless, our prototype works reasonably well in cases where the goal's search space is relatively small, and we believe that working with such goals can be useful for detecting many programming bugs in practice. Techniques for simplifying CTs should be worked out in future improvements of the prototype. For instance, asking the user for a concrete missing instance of the initial goal and starting a diagnosis session for the instantiated goal might be helpful.

Conclusions and future work
We have presented a logical and semantic framework for the declarative diagnosis of wrong and missing computed answers in CFLP(D), a generic scheme for Constraint Functional-Logic Programming over a given constraint domain D which combines the expressivity of lazy FP and CLP languages. The diagnosis technique of wrong answers represents the computation which has produced a wrong computed answer by means of an abridged proof tree whose inspection leads to the discovery of some erroneous program rule responsible for the wrong answer. The logical correctness of the method can be formally proved thanks to the connection between abbreviated proof trees and program semantics. The method for missing answers relies on computation trees whose nodes are labeled with answer collection assertions (acas). As in declarative diagnosis for FP languages, the values displayed at acas are shown in the most evaluated form demanded by the topmost computation. Following the CLP tradition, we have shown that our computation trees for missing answers are abbreviated proof trees in a suitable inference system, the so-called Constraint Negative Proof Calculus. Thanks to this fact, we can prove the correctness of our diagnosis method for any admissible goal solving system whose recollection of computed answers can be represented by means of a proof tree in the constraint negative proof calculus. As far as we know, no comparable result was previously available for such an expressive framework as CFLP.

145
A Semantic Framework for the Declarative Debugging of Wrong and Missing Answers in Declarative Constraint Programming www.intechopen.com Intuitively, the notion of aca bears some loose relationship to programming techniques related to answer recollection, as e.g., encapsulated search (Brassel et al., 2004). However, acas in our setting are not a programming technique. Rather, they serve as logical statements whose falsity reveals incompleteness of computed answers with respect to expected answers. In principle, one could also think of a kind of logical statements somewhat similar to acas, but asserting the equality of the observed and expected sets of computed answers for one and the same goal with a finite search space. We have not developed this idea, which could support the declarative diagnosis of a third kind of unexpected results, namely incorrect answer sets as done for Datalog. In fact, we think that a separate diagnosis of wrong and missing answers is pragmatically more convenient for users of CFLP languages.
On the practical side, our method can be applied to actual CFLP systems such as Curry or TOY, leading to correct diagnosis under the pragmatic assumption that they behave as admissible goal solving systems. This assumption is plausible in so far as the systems are based on formal goal solving procedures that can be argued to be admissible. A debugging tool called DDT , which implements the proposed technique for wrong answers over the domain R of arithmetic constraints over the real numbers has been implemented as a non-trivial extension of previously existing debugging tools. DDT provides several practical facilities for reducing the number and the complexity of the questions that are presented to the user during a debugging session. Moreover, a prototype debugger for missing answers under development is available, which implements the method in TOY.
As future work, we plan several improvements of DDT , such as enabling the diagnosis supporting finite domain constraints (Estévez et al., 2009;Fernández et al., 2007), and providing new facilities for simplifying the presentation of queries to the user. In this sense, some important pragmatic problems well known for declarative diagnosis tools in FP and CLP languages also arise in our context: both the CTs and the acas at their nodes may be very big in general, causing computation overhead and difficulties for the user in answering the questions posed by the debugging tool. In spite of these difficulties, the prototype works reasonably well in cases where the goal's search space is relatively small, and we believe that working with such goals can be useful for detecting many programming bugs in practice.