# HG changeset patch # User ali # Date 1369942065 -3600 # Node ID 411867e8e20bd6bd0db301e168f5864fa93b295e # Parent cffa80824f8cf7fe977a5e290564c8d4650de3bd Update documentation diff -r cffa80824f8c -r 411867e8e20b configure.ac --- a/configure.ac Thu May 30 19:26:24 2013 +0100 +++ b/configure.ac Thu May 30 20:27:45 2013 +0100 @@ -1,7 +1,7 @@ # -*- Autoconf -*- # Process this file with autoconf to produce a configure script. -AC_INIT([bookloupe],[1.52],[ali@juiblex.co.uk]) +AC_INIT([bookloupe],[1.90],[ali@juiblex.co.uk]) AC_PREREQ(2.59) AC_CONFIG_AUX_DIR([config]) AC_CONFIG_SRCDIR([bookloupe/bookloupe.c]) diff -r cffa80824f8c -r 411867e8e20b doc/bookloupe.txt --- a/doc/bookloupe.txt Thu May 30 19:26:24 2013 +0100 +++ b/doc/bookloupe.txt Thu May 30 20:27:45 2013 +0100 @@ -1,30 +1,26 @@ - Gutcheck documentation + Bookloupe documentation -gutcheck: lists possible common formatting errors in a Project -Gutenberg candidate file. It is a command line program and can be used -under Win32 or Unix (gutcheck.c should compile anywhere; if it doesn't, -tell me). For Windows-only people, there is an appendix at the end -with brief instructions for running it. +bookloupe: lists possible common formatting errors in a Project +Gutenberg candidate file. Bookloupe is based on gutcheck, written +by Jim Tinsley. It is a command line program and can be used under +Microsoft Windows, Mac or Unix. For Windows-only people, there is +an appendix at the end with brief instructions for running it. +Current version: 1.90, a beta version leading up to version 2.0 -Current version: 0.99. Users of 0.98 see end of file for changes. +This software is Copyright Jim Tinsley 2000-2005 and +J. Ali Harlow 2012 onwards. -You should also have received the licence file COPYING, a README file, -gutcheck.c, the source code, and gutcheck.exe, a DOS executable, with -this file. - -This software is Copyright Jim Tinsley 2000-2005. - -Gutcheck comes wih ABSOLUTELY NO WARRANTY. For details, read the file COPYING. +Bookloupe comes wih ABSOLUTELY NO WARRANTY. For details, read the file COPYING. This is Free Software; you may redistribute it under certain conditions (GPL). -See http://gutcheck.sourceforge.net for the latest version. +See http://www.juiblex.co.uk/pgdp/bookloupe/ for the latest version. -Usage is: gutcheck [-setopxlywm] filename +Usage is: bookloupe [-setopxlywm] filename where: -s checks Single quotes -e switches off Echoing of lines @@ -40,36 +36,43 @@ -u warns about words in a user-defined typo file gutcheck.typ -d ignores some DP-specific markup -Running gutcheck without any parameters will display a brief help message. +Running bookloupe without any parameters will display a brief help message. Sample usage: - gutcheck warpeace.txt + bookloupe warpeace.txt More detail: + Character encoding + + Bookloupe will handle e-texts encoded in UTF-8 (preferred), + ISO-8859-1 (also known as Latin-1), or WINDOWS-1252 (also known, + incorrectly, as ansi). The output will be in the same encoding + as the input e-text. + Echoing lines (-e to switch off) - You may find it convenient, when reviewing Gutcheck's - suggestions, to see the line that Gutcheck is questioning. + You may find it convenient, when reviewing Bookloupe's + suggestions, to see the line that Bookloupe is questioning. That way, you can often see at a glance whether it is a real error that needs to be fixed, or a false positive - that should be in the text, but Gutcheck's limited + that should be in the text, but Bookloupe's limited programming doesn't understand. - By default, gutcheck echoes these lines, but if you don't + By default, bookloupe echoes these lines, but if you don't want to see the lines referred to, -e will switch it OFF. Quotes (-s and -p switches) - Gutcheck always looks for unbalanced doublequotes in a + Bookloupe always looks for unbalanced doublequotes in a paragraph. It is a common convention for writers not to close quotes in a paragraph if the next paragraph opens with quotes and is a continuation by the same speaker. - Gutcheck therefore does not normally report unclosed quotes + Bookloupe therefore does not normally report unclosed quotes if the next paragraph begins with a quote. If you need to see all unclosed quotes, even where the next paragraph begins with a quote, you should use the -p switch. @@ -84,7 +87,7 @@ for contractions rather than quotes, but it isn't easy to get a program to recognize that. - Since Gutcheck makes too many mistakes when trying to match + Since bookloupe makes too many mistakes when trying to match singlequotes, it doesn't look for unbalanced singlequotes unless you specify the -s switch. @@ -102,7 +105,7 @@ Typos (-t switch) - It's not Gutcheck's job to be a spelling checker, but it + It's not bookoupe's job to be a spelling checker, but it does check for a list of common typos and OCR errors if you use the -t switch. (The -x switch also turns typo checking on.) @@ -113,7 +116,7 @@ playing the odds - a few false positives for many errors found. Similarly with "ii", which is a very common OCR error. - Gutcheck suppresses multiple reporting of the first 40 "typos" + Bookloupe suppresses multiple reporting of the first 40 "typos" found. This is to remove the annoyance of seeing something like "FN" (footnote) or "LK" (initials) flagged as a typo 147 times in a text. @@ -127,11 +130,11 @@ and Mac have different conventions, but the final text should always use a CR/LF pair as its line terminator. - By default, Gutcheck verifies that every line does have + By default, bookloupe verifies that every line does have the correct terminator, but if you're on a work-in-progress in Linux, you might want to convert the line-ends as a final step, and not want to see thousands of errors every time you - run Gutcheck before that final step, so you can turn off + run bookloupe before that final step, so you can turn off this checking with the -l switch. @@ -143,101 +146,103 @@ Overview mode (-o switch) - This mode just gives a count of queries found - instead of a detailed list. + This mode just gives a count of queries found + instead of a detailed list. Header quote (-h switch) - If you use the -h switch, gutcheck will also display - the Title, Author, Release and Edition fields from the - PG header. This is useful mostly for the automated - checks we do on recently-posted texts. + If you use the -h switch, bookloupe will also display + the Title, Author, Release and Edition fields from the + PG header. This is useful mostly for the automated + checks we do on recently-posted texts. Errors to stdout (-y switch) - If you're just running gutcheck normally, you can ignore - this. It's only there for programs that provide a front - end to gutcheck. It makes error messages appear within - the output of gutcheck so that the front end knows whether - gutcheck ran OK. + If you're just running bookloupe normally, you can ignore + this. It's only there for programs that provide a front + end to bookloupe. It makes error messages appear within + the output of bookloupe so that the front end knows whether + bookloupe ran OK. Verbose reporting (-v switch) - Normally, if gutcheck sees lots of long lines, short lines, - spaced dashes, non-ASCII characters or dot-commas ".," it - assumes these are features of the text, counts and summarizes - them at the top of its report, but does not list them - individually. If the -v switch is on, gutcheck will list them all. + Normally, if bookloupe sees lots of long lines, short lines, + spaced dashes, non-ASCII characters or dot-commas ".," it + assumes these are features of the text, counts and summarizes + them at the top of its report, but does not list them + individually. If the -v switch is on, bookloupe will list them all. Markup interpretation (-m switch) - Normally, gutcheck flags anything it suspects of being HTML - markup as a possible error. When you use the -m switch, - however, it matches anything that looks like markup against - a short list of common HTML tags and entities. If the markup - is in that list, it either ignores the markup, in the case - of a tag, or "interprets" the markup as its nearest ASCII - equivalent, in the case of an entity. So, for example, using - this switch, gutcheck will "see" + Normally, bookloupe flags anything it suspects of being HTML + markup as a possible error. When you use the -m switch, + however, it matches anything that looks like markup against + a short list of common HTML tags and entities. If the markup + is in that list, it either ignores the markup, in the case + of a tag, or "interprets" the markup as its nearest ASCII + equivalent, in the case of an entity. So, for example, using + this switch, bookloupe will "see" - “He went thataway!” + “He went thataway!” - as + as - "He went thataway!" + "He went thataway!" - and report accordingly. + and report accordingly. - This switch does not, not, NOT check the validity of HTML; - it exists so that you can run gutcheck on most HTML texts - for PG, and get sane results. It does not support all tags. - It does not support all entities. When it sees a tag or entity - it does not recognize, it will query it as HTML just as if - you hadn't specified the -m switch. + This switch does not, not, NOT check the validity of HTML; + it exists so that you can run bookloupe on most HTML texts + for PG, and get sane results. It does not support all tags. + It does not support all entities. When it sees a tag or entity + it does not recognize, it will query it as HTML just as if + you hadn't specified the -m switch. - Gutcheck 0.99 will automatically switch on markup interpretation - if it sees a lot of tags that appear to be markup, so mostly, you - won't have to specify this. + Bookloupe will automatically switch on markup interpretation + if it sees a lot of tags that appear to be markup, so mostly, you + won't have to specify this. User-defined typos (-u switch) - If you have a file named gutcheck.typ either in your current - working directory or in the directory from which you explicitly - invoked gutcheck, but not necessarily on your path, and if you - specify the -u switch, gutcheck will query any word specified - in that file. The file is simple: one word, in lower case, per - line. 999 lines are allowed for. Be careful not to put multiple - words onto a line, or leave any rubbish other than the word on - the line. You should have received a sample file gutcheck.typ - with this package. + If you have a file named bookloupe.typ or gutcheck.typ either + in your current working directory or in the directory from + which you explicitly invoked bookoupe, but not necessarily on + your path, and if you specify the -u switch, bookloupe will + query any word specified in that file. The file is simple: one + word, in lower case, per line. Be careful not to put multiple + words onto a line, or leave any rubbish other than the word on + the line. You should have received a sample file bookloupe.typ + with this package. The file may be encoded in UTF-8 (preferred), + ISO-8859-1 (also known as Latin-1), or WINDOWS-1252 (also known, + incorrectly, as ansi). Ignore DP markup (-d switch) - Distributed Proofreaders (http://www.pgdp.net) is currently - (2005) the main source of PG texts, and proofers there use - special conventions. This switch understands those conventions, - so that people can use gutcheck on files in process that still - haven't had the special conventions removed yet. The special - conventions supported in 0.99 are page-separators and - "", "", "/*", "*/", "/#", "#/", "/$", "$/". + Distributed Proofreaders (http://www.pgdp.net) has for some + time been the main source of PG texts, and proofers there use + special conventions. This switch understands those conventions, + so that people can use bookloupe on files in process that still + haven't had the special conventions removed yet. The special + conventions supported are page-separators and + "", "", "/*", "*/", "/#", "#/", "/$", "$/". -You will probably only run gutcheck on a text once or maybe twice, +You will probably only run bookloupe on a text once or maybe twice, just prior to uploading; it usually finds a few formatting problems; it also usually finds queries that aren't problems at all - it often questions Tables of Contents for having short lines, for example. -These are called "false positives", and need a human to decide on +These are called "false positives," and need a human to decide on them. The text should be standard prose, and already close to PG normal format (plain text, about 70 characters per line with blank lines between paragraphs). -Gutcheck merely draws your attention to things that might be errors. +Bookloupe merely draws your attention to things that might be errors. It is NOT a substitute for human judgement. Formatting choices like short lines may be for a reason that this program can't understand. @@ -246,8 +251,8 @@ them. Of these, spellchecking (with _very_ careful human judgement) is the most important and most useful. -Gutcheck does perform some basic typo-checking if you ask it to, -but its focus is on formatting errors specific to PG texts - +Bookloupe does perform some basic typo-checking if you ask it to, +but its focus is on formatting errors specific to PG texts— mismatched quotes, non-ASCII characters, bad spacing, bad line length, HTML tags perhaps left from a conversion, unbalanced brackets. @@ -258,7 +263,7 @@ - How do _I_ use it? + How does Jim Tinsley use gutcheck? Practically everyone I give gutcheck to asks me how _I_ use it. Well, when I get a text for posting, say filename.txt, I run @@ -291,33 +296,23 @@ - Future development of gutcheck + Future development of bookloupe -Gutcheck has gone about as far as it can, given its current -structure. In order to add better singlequotes checking, -sentence checking, better he/be checking and other good stuff -that I'd like to see, I'll have to rewrite it from a different -angle - looking at the syntax instead of the lines. And I'll -probably get around to that sooner or later. +Bookloupe version 2.0 is intended to add UTF-8 support to +gutcheck. All the functionality should already be implemented +in the beta versions leading up to version 2.0, although +some bugs may well remain. -Meantime, I'm just trying to get this version stabilized, so -please report any bugs you find. When it is stable, I'll run -up a Windows port for those timid souls who can't look a -command line in the eye. :-) +Future versions will add support for UTF-8 characters that +are not in ISO-8859-1 (eg., curled quotation marks); +characters that do not have a composed form (version 2 +treats these as taking 2 or more columns); zero width and +wide characters (version 2 treats these as taking 1 column). -And I've started work on gutspell, a companion to gutcheck -which will concentrate on spelling problems. PG spelling -problems are unusual, since the range of texts we cover is -so wide, and I'll be taking a somewhat unorthodox approach -to writing this spelling-checker _specifically_ for texts -containing a lot of dialect and uncommon words that have -probably already been spell-checked against a standard -modern dictionary. - -Explanations of common gutcheck messages: +Explanations of common bookloupe messages: --> 74 lines in this file have white space at end @@ -330,7 +325,7 @@ --> 84 lines in this file are long. Not reporting long lines. --> 8 lines in this file are VERY long! - If there are a lot of long or short lines, Gutcheck won't list + If there are a lot of long or short lines, bookloupe won't list them individually. The short lines version of this message is commonly seen when gutchecking poetry and some plays, where the normal line length is shorter than the standard for prose. @@ -345,7 +340,7 @@ The PG standard for an emdash--like these--is two minus signs with no spaces before or after them. However, some older texts used spaced dashes - like these -- and if there are very many - such spaced dashes in the file, gutcheck just draws your + such spaced dashes in the file, bookoupe just draws your attention to it and doesn't list them individually. @@ -357,12 +352,11 @@ represented according to several different non-ASCII encoding schemes, using values over 127. If you have a plain English text with a few accented characters in words like cafe or tete-a-tete, - you should replace the accented characters with their unaccented + you might replace the accented characters with their unaccented versions. The English pound sign is another commonly-seen non-ASCII character. If you have enough non-ASCII characters in - your text that you feel removing them would degrade your text - unacceptably, you should probably consider doing an 8-bit text - as well as a plain-ASCII version. + your text that you feel removing them would degrade your text, + you should probably consider doing a UTF-8 text. @@ -391,7 +385,7 @@ The tilde character (~) might be legitimately used, but it's the character commonly used by OCR software to indicate a place where - it couldn't make out the letter, so gutcheck flags it. + it couldn't make out the letter, so bookloupe flags it. @@ -407,7 +401,7 @@ PG texts should have lines shorter than 76. There may be occasions where you decide that you really have to go out to 79 characters, - but the sample above says that line 1451 is 129 characters long - + but the sample above says that line 1451 is 129 characters long— probably two lines run together. @@ -416,12 +410,12 @@ PG texts should have lines longer than 54 characters. However, there are special cases like poetry and tables of contents where - the lines _should_ be shorter. So treat Gutcheck warnings about + the lines _should_ be shorter. So treat bookloupe warnings about short lines carefully. Sometimes it's a genuine formatting problem; sometimes the line really needs to be short. - Hint: gutcheck will not flag lines as short if they are indented - - if they start with a space. I like to start inserted stanzas + Hint: bookloupe will not flag lines as short if they are indented + —if they start with a space. I like to start inserted stanzas and other such items indented with a couple of spaces so that they stand out from the main text anyway. @@ -437,7 +431,7 @@ Line 1850 - Spaced em-dash? The PG standard for an em-dash--like these--is two minus signs - with no spaces before or after them. Gutcheck flags non-PG + with no spaces before or after them. Bookloupe flags non-PG em-dashes - like this one. Normally, you will replace it with a PG-standard em-dash. @@ -445,7 +439,7 @@ Line 1904 - Query he/be error? - Gutcheck makes a very minor effort to look for that scourge of all + Bookloupe makes a very minor effort to look for that scourge of all proofreaders, "be" replacing "he" or vice-versa, and draws your attention to it when it thinks it has found one. @@ -454,7 +448,7 @@ Line 2017 - Query digit in a1most The digit 1 is commonly OCRed for the letter l, the digit 0 for - the letter O, and so on. When gutcheck sees a mix of digits and + the letter O, and so on. When bookloupe sees a mix of digits and letters, it warns you. It may generate a false positive for something like 7am. @@ -462,7 +456,7 @@ Line 2083 - Query standalone 0 - In paranoid mode (see -x) only, gutcheck warns about the digit 0 + In paranoid mode (see -x) only, bookloupe warns about the digit 0 and the number 1 standing alone as a word. This can happen if the OCR misreads the words O or I. @@ -470,7 +464,7 @@ Line 2115 - Query word whetber - If you have switched typo-checking on, gutcheck looks for + If you have switched typo-checking on, bookloupe looks for potential typos, especially common h/b errors. It's not infallible; it sometimes queries legit words, but it's always worth taking a look. @@ -504,10 +498,9 @@ Line 2385 column 27 - Wrongspaced quotes? - As of version 0.98, gutcheck adds extra checks on whether a quote - seems to be a start or end quote, and queries those that appear to - be misplaced. This does give rise to false positives when quotes are - nested, for example: + Bookloupe checks whether a quote seems to be a start or end quote, + and queries those that appear to be misplaced. This does give rise + to false positives when quotes are nested, for example: "And how," she asked, "will your "friends" help you now?" @@ -540,14 +533,14 @@ Line 2540 - Mismatched quotes - Another gutcheck mainstay - unclosed doublequotes in a paragraph. + Another bookloupe mainstay—unclosed doublequotes in a paragraph. See the discussion of quotes in the switches section near the start of this file. - Since the mismatch doesn't occur on any one line, gutcheck quotes + Since the mismatch doesn't occur on any one line, bookloupe quotes the line number of the first blank line following the paragraph, since this is the point where it reconciles the count of quotes. - However, if gutcheck is echoing lines, that is, you haven't used + However, if bookloupe is echoing lines, that is, you haven't used the -e switch, it will show the _first_ line of the paragraph, to help you find the place without using line numbers. The offending paragraph is therefore between the quoted line and @@ -611,36 +604,36 @@ For Windows-only users who are unfamiliar with DOS: If you're a Windows-only user, you need to save - gutcheck.exe into the folder (directory) where the + bookloupe.exe into the folder (directory) where the text file you want to check is. Let's say your - text file is in C:\GUT, then you should save - GUTCHECK.EXE into C:\GUT. + text file is in C:\gut, then you should save + bookloupe.exe into C:\gut. - Now get to a DOS prompt. You can do this by + Now get to a console. You can do this by selecting the "Command Prompt" or "MS-DOS Prompt" option that will be somewhere on your Start/Programs menu. - Now get into the C:\GUT directory. - You can do this using the CD (change directory) + Now get into the C:\gut directory. + You can do this using the cd (change directory) command, like this: - CD \GUT + cd \gut and your prompt will change to - C:\GUT> + C:\gut> so you know you're in the right place. Now type - gutcheck yourfile.txt - and you'll see gutcheck's report + bookloupe yourfile.txt + and you'll see bookloupe's report - By default, gutcheck prints its queries to screen. + By default, bookloupe prints its queries to screen. If you want to create a file of them, to edit against the text, you can use the greater-than sign (>) to tell it to output the report to a file. For example, if you want its report in a - file called QUERIES.LST, you could type - - gutcheck yourfile.txt > queries.lst + file called queries.lst, you could type + + bookloupe yourfile.txt > queries.lst The queries.lst file will then contain the listing of possible formatting errors, and you can @@ -649,7 +642,7 @@ Whatever you do, DON'T make the filename after the greater-than sign the name of a file already on your disk that you want to keep, because - the greater-than sign will cause gutcheck to + the greater-than sign will cause bookloupe to replace any existing file of that name. So, for example, if you have two Tolstoy files @@ -658,85 +651,11 @@ is ever used following the greater-than sign. To check these correctly, you might do: - gutcheck warpeace.txt >war.lst + bookloupe warpeace.txt > war.lst and - gutcheck annak.txt > annak.lst + bookloupe annak.txt > annak.lst separately. Then you can look at war.lst and annak.lst - to see the gutcheck reports. - - * * * * - - -For existing 0.98 users upgrading to 0.99: - - If you run on old 16-bit DOS or Windows 3.x, I'm afraid - you're out of luck. I'm not saying it _can't_ be compiled - to run on 16-bit, but the executable with the package is - for Win32 only. *nix users won't notice the change at all. - - - There are two new switches: -u and -d. - See above for full rundown. - - -Here's a list of the new errors: - - Line 1456 - Carat character? - - I^ve found a few. - - - Line 1821 - Forward slash? - - Common error for italicized "I", or so /'ve found. - - - Line 2139 - Query missing paragraph break? - - "Come here, son." "Do I _have_ to go, dad?" - Like that. False positives in some texts. Sorry 'bout that, - but these are often errors. - - - Line 2200 - Query had/bad error? - - Clear enough. Doesn't catch as many as I'd like it to, - but rarely gives false alarms. - - - Line 2268 - Query punctuation after the? - - Some words, like "the", very rarely have punctuation - following them. Others, like "Mrs", usually have a - period, but never a comma. Occasional false positives. - - - Line 2380 - Query possible scanno arid - - It found one of your user-defined typos when you - used the -u switch. - - - Line 2511 - Capital "S"? - - Surprisingly common specific case, like: Jane'S - - - Line 3469 - endquote missing punctuation? - - OK. This one can really cause a lot of false positives - in some books, but it switches itself off if it finds - more than 20 in a text, unless you force it to list them - all with the -v switch. - "Hey, dad" Johnny said, "can we go now?" - is a common punctuation-missing error. - - - Line 4266 - Mismatched underscores? - - Like mismatched anything else! - - + to see the bookloupe reports.