# Check documents against each other (experimental)

Source: https://docs.littlebigbrain.com/guides/check-documents-against-each-other/

One purchase leaves several documents. The order says 10 kg at 12.50 EUR.
The delivery note says 8 kg arrived. The invoice bills 10 kg at 14.00 EUR.

Each document is correct on its own. The differences show only when someone
reads two of them side by side.

This guide builds that check from LBB's own parts. Upload the documents.
LBB reads the values, with the quote that states each. SHACL shapes state the
rules between the documents.

Each difference becomes a [review item](https://docs.littlebigbrain.com/guides/review-in-workflows/) for a
person. The item shows the quotes of both documents.

The output is a list of candidates for a person to check. The pipeline makes
no payment, approval or compliance decision. It writes no fact about the
purchase. A person reads the quotes and decides what each difference means.

The pipeline is the `documents-check`
[template](https://docs.littlebigbrain.com/guides/workflow-templates/). All names and numbers on this page
are invented.

## The pipeline

| Step | Part | Runs in |
| --- | --- | --- |
| 1 | [Files](https://docs.littlebigbrain.com/guides/documents-to-graph/#1-upload-the-files): an upload of a PDF is a `Document` | LBB |
| 2 | `documents.parse` reads the PDF into pages, with every line and its box | LBB |
| 3 | `documents-texts` writes the text of each parsed PDF as an instance of a class | your worker |
| 4 | `values.extract` reads the text of each instance and writes its values, each with a quote | LBB |
| 5 | SHACL shapes state the rules between the documents; LBB validates them after each publication | LBB |
| 6 | `documents-check` files one review item per difference | your worker |

Your worker runs two small workflows. Everything else is LBB-run.

`documents-texts` is glue. It finds the class of a PDF in its first line.
The line reads "PURCHASE ORDER", "DELIVERY NOTE" or "INVOICE".
`documents-check` reads the conformance report of the shapes.

## Run it on a local stack

In the LBB repository, one command starts a stack with mock models. It needs
no keys. Step 2 needs `pdftotext` from Poppler.

```sh
npm run dev:stack -- --worker documents-check
```

The command starts a data plane and creates the project from the template.
It puts the pipeline on graph `main` and uploads three sample PDFs. Then it
starts the worker.

After about a minute, open the console and choose **Review**. Three items
wait for you.

From your own project, use any stack that has the workflow engine and models:

```sh
npm create @littlebigbrain -- --template documents-check my-checks
cd my-checks && npm install
npm test                                      # both workflows on the samples, with mock models
LBB_GRAPH=try-documents-check npm run try     # the pipeline and the sample PDFs on your stack
LBB_GRAPH=try-documents-check npm start       # serve the two workflows
```

`npm run setup` creates the pipeline on a graph of yours. It uploads nothing.

### What setup changes

Setup publishes `shapes.ttl` as the graph's shape set. Publishing replaces
the whole set, and it can change the enforcement mode. So setup reads the
graph's shapes first.

- A graph with shapes of its own stops the setup before anything is written.
  The error names the shapes. Merge `shapes.ttl` into yours and publish the
  union in the mode you want.
- After the merge, set `publishShapes: false` in the settings. Setup then
  reads and writes no shape set, and it creates the rest.
- A graph that holds these shapes keeps its mode.
- A graph with no shapes gets `warn`.

The workflows own the classes `PurchaseOrder`, `DeliveryNote` and `Invoice`.
They own the properties `document_text` and `read_sha256`, and the relation
`TEXT_OF`. They drop only the instances they wrote.

The `rows` and `sections` templates own other names (`READ_FROM`, `text`).
All three can run on one graph.

## The sample documents

Three PDFs, one page each. The lines have the form `label: value`, which the
mock models read.

| Document | Holds |
| --- | --- |
| `purchase-order.pdf` | `reference: PO-1001`, `quantity: 10 kg`, `price: 12.50 EUR`, `total: 125.00 EUR` |
| `delivery-note.pdf` | `reference: DN-1001`, `order: PO-1001`, `quantity: 8000 g` |
| `invoice.pdf` | `reference: INV-1001`, `order: PO-1001`, `delivery: DN-1001`, `quantity: 10 kg`, `price: 14.00 EUR`, `total: 139.50 EUR` |

The delivery note gives its weight in grams. The ontology declares `kg` as
the unit of `quantity`. So `values.extract` writes 8, with the quote
`quantity: 8000 g`.

The two quantities compare in one unit without any code of yours.

## The rules

A shape selects the documents of a class. It reports a result for each one
that breaks the rule.

A rule that compares two documents is a SHACL-SPARQL constraint
(`sh:sparql`). Its `sh:select` returns two variables:

- `$this`, the document that breaks the rule;
- `?value`, the document it was compared with.

This rule says an invoice must not bill more than the delivery note it names
shows:

```turtle
ex:InvoiceQuantityVsDelivery a sh:NodeShape ;
  sh:targetClass dclass:invoice ;
  sh:severity sh:Warning ;
  sh:sparql [
    sh:message "The invoice bills more than the delivery note shows" ;
    sh:select """
      PREFIX dclass: <https://littlebigbrain.com/class/>
      PREFIX dp: <https://littlebigbrain.com/p/>
      SELECT $this ?value WHERE {
        $this dp:delivery ?reference ; dp:quantity ?billed .
        ?value a dclass:deliverynote ; dp:reference ?reference ; dp:quantity ?delivered .
        FILTER (?billed > ?delivered)
      }""" ] .
```

A document names another by its reference. The invoice says
`delivery: DN-1001`, and the delivery note has `reference: DN-1001`. The
shape joins on that value.

The template has four rules:

| Rule | Reports |
| --- | --- |
| `invoice-quantity-vs-delivery` | The invoice bills more than the delivery note shows, to the gram |
| `invoice-quantity-vs-order` | The invoice bills more than the order asks for, to the gram |
| `invoice-price-vs-order` | The invoice price differs from the order price, compared in cents |
| `total-matches-quantity-times-price` | A total differs from quantity times price by more than one cent |

The last rule has a tolerance. A total that is a cent off from rounding
passes.

The shapes compare money in whole cents and quantities to the gram. So a
floating-point error never decides. The samples break three rules and
satisfy the other two on purpose.

The shapes are published in `warn` mode:

```ts
import { readFileSync } from "node:fs";

await lbb.schema.publish({
  desired_mode: "warn",
  shapes: { source: readFileSync("shapes.ttl", "utf8"), format: "turtle" },
});
```

In `warn` mode every write is accepted. The conformance report lists the
differences.

A rule with `sh:sparql` has no bounded set of affected records. On a records
graph, `reject` mode would check the whole graph on each commit. An
RDF-native graph refuses `reject` mode for such a rule.

Use `warn` for rules that compare documents. See
[Published SHACL conformance](https://docs.littlebigbrain.com/guides/sparql-and-shacl/#published-shacl-conformance).

## What lands in the Review inbox

`documents-check` runs in three cases:

- a field of a document changes;
- a document is removed;
- a person answers one of its items.

It watches the class `BusinessDocument`, which the three document classes
extend.

Each run reads the conformance report. When the message is about a commit,
the run waits until the report covers that commit. The wait is a durable
timer, so no worker is held.

The run then finds each quote in the parsed pages of its file. It files one
item of kind `choice` per result.

```json
{
  "kind": "choice",
  "title": "Invoice INV-1001 bills 10 kg. Delivery note DN-1001 shows 8 kg.",
  "subject": { "type": "Invoice", "name": "invoice.pdf" },
  "question": {
    "options": [
      { "id": "discrepancy", "label": "A real difference: follow it up" },
      { "id": "misread", "label": "A value was read wrongly: correct it" },
      { "id": "acceptable", "label": "Acceptable: no action" }
    ],
    "proposed": "discrepancy"
  },
  "evidence": [
    {
      "quote": "quantity: 10 kg",
      "source_id": "https://littlebigbrain.com/e/357244d5…#page=1",
      "region": { "page": 0, "x0": 0.121, "y0": 0.3232, "x1": 0.2413, "y1": 0.3352, "confidence": 1 }
    },
    {
      "quote": "quantity: 8000 g",
      "source_id": "https://littlebigbrain.com/e/19a8c14d…#page=1",
      "region": { "page": 0, "x0": 0.121, "y0": 0.2947, "x1": 0.2526, "y1": 0.3067, "confidence": 1 }
    }
  ]
}
```

Each quote names its file and page, and the box of its line. The inbox
shows each quote with its source. A client of your own can open the page at
the box.

List the items from code:

```ts
const { data } = await lbb.reviews.list({
  originId: "starter:documents-check",
  status: "open",
  kind: "choice",
});
for (const item of data) console.log(item.title);
```

```sh
curl "$LBB_URL/v1/reviews?graph=main&kind=choice&origin_id=starter:documents-check" \
  -H "Authorization: Bearer $LBB_API_KEY"
```

## Answer an item

A person chooses one option, or **Can't tell**. A choice is a label. It
writes no fact.

| Answer | Meaning |
| --- | --- |
| A real difference | The person follows it up outside LBB. |
| A value was read wrongly | The model read a document wrongly. The person corrects the value in the `value` item that `values.extract` filed, when the value was below the gate, or deletes the file and uploads the right one. |
| Acceptable | The difference is known and needs no action. |

A change of a value, or the deletion of a file, wakes `documents-check`
again. When the report no longer lists a difference, the workflow withdraws
its open item. An item that a person answered stays as answered.

The key of an item holds a hash of the document IRIs and the quotes:

- The same difference is the same item, however often the check runs.
- A difference with other quotes is a new item.
- A difference that returns reopens its withdrawn item.

## Use your own documents

Change four things, all in the template:

1. **The kinds.** `KINDS` in `texts.ts` lists each class. It also lists the
   first line that names the class, and the properties `values.extract`
   fills. `PROPERTIES` in `workflow.ts` gives their types and units.
2. **The rules.** Write a shape in `shapes.ttl`. Add its entry to `checks`
   in `checks.ts`. The entry holds the sentences of the review item and the
   quotes it shows. A test fails when a shape has no entry or an entry has
   no shape.
3. **The context.** `defaults.context` in `workflow.ts` is one sentence about
   your documents, for the model.
4. **The samples.** `npm test` reads three things. They are
   `samples/records.json` (the text of each document), the parsed pages in
   `samples/pages/` and the answer key `samples/expected.json`. Replace the
   three together with your own.

With real models, the quote of a value can differ from the quote in the key.

The `records` of `samples/expected.json` have the shape of the
`values-extract` answer key. They also load as an
[Evals dataset](https://docs.littlebigbrain.com/guides/datasets/) of kind `values`. Use it to compare models
on your documents.

## Limits

- Each class has one value per property. A document with several line items
  needs one instance per line. Change `documents-texts` to write one instance
  per row.
- A rule skips a document whose reference names no document of the graph. An
  invoice that names no delivery note is never compared with one. The same
  holds when the delivery note does not exist.
- The first line of a PDF must equal the title of a kind, ignoring case.
  `documents-texts` skips "INVOICE REMINDER".
- The price is read as a price per kg, as the samples say. The shapes do not
  compare currencies.
- The text of a document is the lines of its pages, up to 20,000 characters.
- A quantity is compared in the unit the ontology declares. `values.extract`
  converts a quantity in another unit of the same kind. It refuses a unit of
  another kind.
- The shapes compare what the model read. A wrongly read value can start a
  difference or hide one. The quotes let a person see which.
- A value below the gate is a `value` item, as in any `values.extract`
  workflow. The shapes see the value only after it is written.
- A turn files at most 25 new items (`candidatesPerTurn`, at most 40). A
  continuation files the rest, in up to 20 more turns. An item that exists
  already, open or answered, takes no share of a turn. A turn reads a report
  window of 150 results. The state counts the results past it as skipped.
- A turn that leaves a result out withdraws nothing. A value with no quote
  gives no item, so it leaves a result out too. Withdrawals wait for a turn
  that builds every item.
- A turn waits 60 seconds for the report to cover its commit. If it does not,
  the message continues in a new turn. After 30 minutes in all, the message
  ends as not checked. Nothing is filed or withdrawn, and the state counts it
  as `unchecked`. The next change checks again.
- The check workflow skips a turn that failed for good (`on_exhausted: skip`).
  The next message runs the whole check again.
- `values.extract` does not clean up after a deleted file. Its `ValueSource`
  entities stay, and so do the open `value` items of the deleted file's
  instance. Answer or withdraw those items by hand.