# Workflow templates (experimental)

Source: https://docs.littlebigbrain.com/guides/workflow-templates/

Most [starter workflows](https://docs.littlebigbrain.com/guides/starter-workflows/) run in LBB. You set
the watch and the settings, and LBB runs the work. When you want to change
the work itself (a prompt, a rule, what gets written), start from a
**template**: the code of a starter workflow, written for your own
workflow worker. The worker calls LBB's models through **activities**, so
your code needs no model keys and LBB meters the calls.

| Template | Does | LBB-run version |
| --- | --- | --- |
| `documents-link` | Connects mentions of the same thing across the documents: each reference links to where that thing is defined or described | `documents.link` |
| `values-extract` | Fills properties of a class from the text of its instances, each value with a quote | `values.extract` |
| `ontology-fit` | Proposes the ontology changes the texts of a class support, as review items | `ontology.fit` |
| `documents-map` | Maps the links of `documents-link` into your own ontology, as fact proposals | `documents.map` |
| `review` | Writes the fact proposals at or above a gate as facts, and sends the others to a person | `review` |
| `link` | Connects the mentions of one thing across your records (contacts, leads, deals, notes) into one thing, and proposes records of one thing as an identity for a person to accept | none yet |
| `rows` | Reads each CSV, TSV or XLSX file into one instance per row, each linked to the file with its row as the evidence | none |
| `sections` | Reads each Markdown file into its sections, each linked to its parent with its lines as the evidence | none |
| `documents-check` | Checks PDFs against each other: SHACL rules over the values `values.extract` reads, and a review item with the quotes of both documents for each difference | none |

A template's id is the id of its starter workflow, with dashes for dots.
`values-extract` also has a Python port, in `examples/values_extract` of the
Python SDK. It has the same rules and the same samples.

`documents-check` is an example that joins LBB's own parts into one
pipeline. [Check documents against each other](https://docs.littlebigbrain.com/guides/check-documents-against-each-other/)
walks through it.

Embeddings (`search.embed`) and the parse of PDF files
(`documents.parse`) have no template: they stay LBB-run.

A template is a starting point. The LBB-run version is a separate
implementation that also handles large graphs and long documents; each
template's README names what that version adds. `values-extract`,
`review` and `ontology-fit` follow the rules of their LBB-run version: the
same prompts, checks, review items and counters. An eval runs each of them
and its LBB-run version on the same recorded model answers and finds the
same outputs, and each README names where the two differ.

## Create a project from a template

```sh
npm create @littlebigbrain -- --template documents-link my-links
cd my-links
npm install
npm test
```

The project holds the template's code in `src/`, its samples in
`samples/`, and three commands:

| Command | Does |
| --- | --- |
| `npm test` | Runs the workflow on its samples with mock models. No server. |
| `npm run try` | Creates a graph of your stack (`LBB_GRAPH`, default `try-<template>`), puts the samples on it, and creates the workflow there |
| `npm run setup`, `npm start` | Creates the workflow on your graph, then serves it |

`npm run try`, `npm run setup` and `npm start` read `LBB_URL` (your
stack's endpoint) and `LBB_API_KEY` (a key of the stack). The server needs
the workflow engine, and the models need a plan with managed models.

`--lang python` creates the Python port of a template instead.
`values-extract` has one.
[Run a worker in Python](https://docs.littlebigbrain.com/guides/python-worker/#start-from-a-template) shows
its commands.

The `link` template has a fourth command: `npm run grade -- --gold <file>`
grades the run on a graph against your gold groups (link precision and
recall, cluster purity and completeness, identity precision). Its
`npm run setup` creates one workflow per class it reads, and the parts
that group the links into things. By default no model reads a record
value: an equal email, web address, phone number or code links the
mentions, and every other candidate goes to a person. Set
`modelsReadRecords: true` in the settings to let the decision model
decide.

## Read files by their type

`rows` and `sections` start when a file arrives. Each is a named workflow
with a **content trigger**: LBB sends it each upload of its media types
within seconds, one file per message. `rows` takes `text/csv` and
`text/tab-separated-values`. `sections` takes `text/markdown`, and
`text/plain` when the name ends with `.md`. Both are in the starter
workflow catalog, under Documents, with the media types of their trigger.
In the console, their **Use template** shows the code and the command
below.

```sh
npm create @littlebigbrain -- --template rows my-rows
cd my-rows && npm install
npm test
LBB_GRAPH=try-rows npm run try && LBB_GRAPH=try-rows npm start
```

`rows` writes one instance per row of the class its settings name (the
samples' `Customer`). A column whose header is a property of the graph
fills that property: `Customer ID` fills `customer_id`. Each instance has
an edge `READ_FROM` to the file's `Document`. The edge's evidence is the
row's text, and its source is the file's IRI with the row:
`https://littlebigbrain.com/e/41ab…#row=5`, as RFC 7111 counts rows (the
header is row 1). `sections` writes one instance per heading, with its
title, level, text and path of headings, and an edge `PART_OF` to its
parent section or the file. Its evidence names the lines:
`#line=39,44` is lines 40 to 44 (RFC 5147). The console opens the file at
those places.

Both read a file once per version and settings. Before a file, the
workflow checks the file's **derivation** with
`client.workflows.derivations.check`, and skips the file when one exists.
After its last write it records one, for the upload the check named. The
same bytes under a second name are then read once. A file deleted while a
run reads it gets no derivation from that run. The run counts it as
skipped. The run of a new upload reads the file again. Say the file was
deleted and not uploaded again. Then the run ends at the next read
(`404`), and counts the file as skipped too. `rows` reads a file by byte range
(`client.files.content(sha256, { range })`), at most 4 MiB per read. Each
turn reads on from the byte where the last one stopped, so a large file is
read about once over all its turns, and a quoted field that spans two
reads is still one row. With the setting `represents: { class: "Report" }`,
the workflow also writes `Document REPRESENTS <report>` when the file's
name is the label of exactly one report: `q3-customers.csv` and the
report `Q3 customers`.

Read the rows of a file with SPARQL:

```sparql
PREFIX p: <https://littlebigbrain.com/p/>
PREFIX r: <https://littlebigbrain.com/r/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX lbb: <https://littlebigbrain.com/ns#>
SELECT ?customer ?email ?source WHERE {
  ?doc p:name "q3-customers.csv" .
  ?customer r:read_from ?doc .
  OPTIONAL { ?customer p:email ?email }
  ?st rdf:subject ?customer ; rdf:predicate r:read_from ; lbb:evidenceSource ?source .
}
```

## The activities

`lbbActivities(client)` from `@littlebigbrain/client` returns the nine calls
a workflow makes. The first three are model calls:

| Activity | Does |
| --- | --- |
| `reason(request, { effectId, label, instance, attempt })` | The reasoning model answers a prompt with JSON that follows a JSON Schema. `attempt` (1 to 10) names the call's attempt in the call log: send `2` for a retry of a failed answer, under its own `effectId` |
| `decide({ state, questions }, { effectId, label, instance })` | The decision model answers choice questions about a state, each with a choice, a confidence and the probability of each option |
| `embed({ texts }, { effectId, label, instance })` | The embedding model embeds up to 256 texts |
| `pages(sha256, { from, to })` | The parsed pages of a file, each line and word with its box |
| `sparql(query, options)` | A SPARQL query, its rows flattened |
| `writeOwned(ctx, stepKey, write)` | An owned write, committed inside a step |
| `review({ workflow, title, rationale, facts, change })` | A review item for a person to accept |
| `evolve(ops, { dryRun })` | An ontology change, or what it would do |
| `ontology()` | The graph's classes, relations and properties |

A model call runs inside a step and passes the step's `effectId`. When a
worker stops after the model answered and before the step was saved, the
next attempt sends the same effect id, and LBB returns the first answer at
no cost:

```ts
import { isActivityRefused, lbbActivities, splitInHalves } from "@littlebigbrain/client";

const lbb = lbbActivities(client);
const answer = await ctx.step("define-p3", (s) =>
  lbb.reason<{ definitions: { row_id: string; identifier: string }[] }>(
    { prompt, schema },
    { effectId: s.effectId, label: "equipment list p3" },
  ),
);
```

Each model call names the turn's run, so the run's page lists it, and the
worker adds up the calls' usage for the turn's run summary. A call also
names its step (`workflow.step`, the step's key). That is the step whose
`effectId` the call passes, or whose `effectId` starts the call's id, as in
`${s.effectId}.2`. Pass
`instance` in the options (an instance IRI or a document's name) to show
what a call works on.

The calls count against the stack's model budget for workflows. A request
that is too large for the model fails with `422 activity_request_refused`:
`isActivityRefused(error)` names that error, and `splitInHalves(items)`
splits the items of the request so each half goes in its own call. In
Python, `lbb_activities(client)` of `lbb.workflows` has the same calls, with
`is_activity_refused` and `split_in_halves`
([Run a worker in Python](https://docs.littlebigbrain.com/guides/python-worker/)).

## Walk through documents-link

The samples are three PDFs of an invented school building tender: an
equipment list that defines codes such as `S 1.1`, a specification with
numbered sections, and two room sheets that use the codes, some in other
spellings (`S1.1`, `S.2.2`).

1. `npm run try` creates the graph `try-documents-link`, creates LBB's
   `documents.parse` workflow there, uploads the three PDFs and waits until
   they are parsed. Then it adds the classes `DocEntry` and `DocMention` to
   the ontology and creates the triggered workflow `documents-link`, which
   watches `Document` and its field `parse_status`.
2. `LBB_GRAPH=try-documents-link npm start` serves the workflow. The first
   trigger message names the three documents.
3. The workflow reads each document in windows of pages, one window per
   turn. It reads the definitions of the equipment list and the
   specification first and writes them as `DocEntry` entities. Then, for
   the room sheets, it asks which spans are codes, finds candidates for
   each span among the entries (by characters and by meaning), asks the
   decision model which entry each span names, and writes each mention as
   a `DocMention` with a `REFERS_TO` link to the entry it names.
4. A link whose confidence is below the gate (0.8) goes to the review
   inbox instead of the graph.

Read the links with SPARQL:

```sparql
PREFIX p: <https://littlebigbrain.com/p/>
PREFIX r: <https://littlebigbrain.com/r/>
SELECT ?span ?page ?identifier WHERE {
  ?doc p:name "raumblaetter.pdf" .
  ?m r:mentioned_in ?doc ; p:span ?span ; p:page ?page ; r:refers_to ?entry .
  ?entry p:identifier ?identifier .
}
ORDER BY ?page ?span
```

The answer key of the samples lists 15 links, among them `S1.1` to
`S 1.1` and `S.2.2` to `S 2.2`. `npm test` runs the same workflow with
mock models and checks that it makes those 15 links and no other.

To change the work, edit `src/workflow.ts` (the steps and the writes) or
`src/link.ts` (the prompts, the candidates and the link rule), run
`npm test`, and restart the worker. A turn that runs again replays its
saved steps in their order, and fails when the code's steps differ: change
the order of the steps while the workflow has no turn in progress.

Do not run a template and its LBB-run version on one graph: they own the
same relations, and the second one is refused with
`409 workflow_owns_conflict`. `rows` and `sections` can both own
`REPRESENTS` on one graph: no upload reaches both of their content
triggers.

## Map the links, then review the facts

A link says "this line cites that entry". `documents-map` turns the links
into facts of your own ontology, such as `room 0.12 HAS_EQUIPMENT T 1`. For
each entry, the decision model picks the instance it is, or a new instance
and its class. For each link, it picks the statement the line makes. Each
statement becomes a `FactProposal` with a confidence and its evidence.

`review` then decides. A proposal at or above the gate (0.8) becomes a
fact, with the proposal as its evidence. A proposal below the gate goes to
a person. So does every statement about a new instance, by default. The
person gets one review item per instance, with each fact and its line. A
statement the graph cannot hold is refused, with a note.

The two templates chain after `documents-link`, or after LBB's own
workflows. You can mix them with LBB's own versions.

```sh
npm create @littlebigbrain -- --template documents-map my-map
npm create @littlebigbrain -- --template review my-review
```

In `documents-map`, `npm run try` uploads the three sample PDFs and lets
LBB parse them. Then it commits the links a correct `documents-link` run
makes on them. So the map runs on known links, without a link run first.
The answer key holds 15 facts and 15 new instances: two rooms, eleven
pieces of equipment and two sections of the specification.

In `review`, `npm run try` commits a small graph and 14 proposals, one for
each case. The answer key gives the status each proposal gets: `written`,
`in_review` or `refused`, and the three review items, one per instance.

The templates write through `POST /v1/graph/commit`. So they take the
relations, properties and classes of the records ontology, and labels.
Each README lists what the LBB-run version adds.