1.1 --- a/doc/loupe-test.txt	Mon Jan 30 09:11:07 2012 +0000
     1.2 +++ b/doc/loupe-test.txt	Sat May 25 17:01:36 2013 +0100
     1.3 @@ -142,7 +142,64 @@
     1.4         │    Line 3 column 29 - Query possible scanno arid              │
     1.5         └───────────────────────────────────────────────────────────────┘
     1.6  
     1.7 -Caveats
     1.8 --------
     1.9 +False-positives
    1.10 +---------------
    1.11  
    1.12 -There is no support yet for testcases which are expected to fail.
    1.13 +Most of the time, the input can be tweaked so that all warnings bookloupe
    1.14 +reports represent real errors in the text. Sometimes, however, this either
    1.15 +cannot be done and still test what we need to. In these cases we need a
    1.16 +means to describe these false-positives (warnings that do not describe
    1.17 +a real error). This is important so that a later version of bookloupe can
    1.18 +be improved to not issue the false-positive warning and still pass the
    1.19 +test. In order to do this, we need to describe the warnings in a more
    1.20 +structures manner, like this:
    1.21 +
    1.22 +       ┌───────────────────────────────────────────────────────────────┐
    1.23 +       │**************** OPTIONS ****************                      │
    1.24 +       │-s                                                             │
    1.25 +       │**************** INPUT ****************                        │
    1.26 +       │'In a moment,' Peter replied,' I'm just coming.'               │
    1.27 +       │                                                               │
    1.28 +       │'Underneath the girls' scarves.                                │
    1.29 +       │                                                               │
    1.30 +       │**************** WARNINGS ****************                     │
    1.31 +       │<expected>                                                     │
    1.32 +       │  <error>                                                      │
    1.33 +       │    <at line="1" column="30"/>                                 │
    1.34 +       │    <text>Wrongspaced singlequotes?</text>                     │
    1.35 +       │  </error>                                                     │
    1.36 +       │  <false-positive>                                             │
    1.37 +       │    <at line="2"/>                                             │
    1.38 +       │    <text>Mismatched singlequotes?</text>                      │
    1.39 +       │  </false-positive>                                            │
    1.40 +       │  <false-negative>                                             │
    1.41 +       │    <at line="3" column="1"/>                                  │
    1.42 +       │    <at line="4"/>                                             │
    1.43 +       │    <text>Mismatched singlequotes?</text>                      │
    1.44 +       │  </false-negative>                                            │
    1.45 +       │</expected>                                                    │
    1.46 +       └───────────────────────────────────────────────────────────────┘
    1.47 +
    1.48 +Here, we use the "WARNINGS" tag instead of "EXPECTED" to denote that we
    1.49 +wish to use structured warnings and the list of warnings is enclosed in
    1.50 +an <expected> ... </expected> node.
    1.51 +
    1.52 +Each warning, or potential warnings is then described using either an
    1.53 +"error" node (for warnings that represent real errors in the text), a
    1.54 +"false-positive" node (for warnings that do not represent real errors),
    1.55 +or a "false-negative" node (for warnings that should be issued, but that
    1.56 +are not yet detected by bookloupe).
    1.57 +
    1.58 +Within each warning node, there are then one or more "at" nodes which
    1.59 +list the acceptable locations for the warning to be reported at (the
    1.60 +first listed should be the preferred location) and exactly one "text"
    1.61 +node which must match the text of the warning issued.
    1.62 +
    1.63 +A testcase will pass if all the warnings marked as errors were issued and
    1.64 +if no warnings were issued that are not listed in one form or another.
    1.65 +If the testcase passes with an expected failure (ie., issues a warning
    1.66 +for a false positive or does not issue a warning for a false negative),
    1.67 +then the test is counted as passed, but a note will be printed describing
    1.68 +this, eg.:
    1.69 +
    1.70 +sample: PASS (with 1 of 1 false positives and 1 of 1 false negatives)