# From documents to a graph (experimental)

Source: https://docs.littlebigbrain.com/guides/documents-to-graph/

Documents point at each other. A service manual says to replace the filter
`P-114` every 500 hours. A parts catalogue describes `P-114` as an air
filter of 120 mm. A contract cites "clause 4.2", and clause 4.2 is on
another page. Following such references by hand across hundreds of pages
is slow, and a missed one goes unnoticed.

Linking connects the mentions of one thing across your documents. Each
reference goes to where that thing is defined or described, in the same
document or in another one.

Four [starter workflows](https://docs.littlebigbrain.com/guides/starter-workflows/) do this work on the
files of a graph. You create a workflow from each:

| Starter workflow | What it does |
| --- | --- |
| `documents.parse` | Reads the text layer of each uploaded PDF file into pages, with every line and word and its box on the page. |
| `documents.link` | Finds the identifiers each document defines or describes (entries) and the spans that refer to one (mentions). It links each mention to the entry of the same thing, with a confidence. |
| `documents.map` | Maps the links into your own ontology: which instance each entry is, and which statement each link makes. It proposes each statement with a confidence. |
| `values.extract` | Reads the text of each instance of a class and fills the properties you choose, each value with a quote from the text. |

Every link keeps the page and the box of its mention, so you can check it
on the page it came from. A link below a confidence gate waits for a person
in the review inbox.

These workflows run on records graphs and on RDF-native graphs. On an
RDF-native graph they write the same statements, under the same IRIs, so
the SPARQL queries below return the same answers. A rerun replaces what the
workflow wrote for the document before.

## What lands in the graph

| Fact | Holds |
| --- | --- |
| `Document` | One per file: `name`, `file_sha256`, `file_bytes`, `content_type`, `uploaded_at`, `language` and `schema` when the upload gives them, and after the parse `parse_status`, `page_count`, `ocr_pages` (only above 0), `parser` and `parse_error`. Linked to its `Content` by `HAS_CONTENT`. |
| `Content` | One per file: the bytes' `sha256` and `byte_length`. Its key is `content:<sha256>`. |
| `REPRESENTS` | From a `Document` to the thing the file carries, of any class. The upload adds the relation to the ontology. A workflow that maps the file, or your own commit, writes it. |
| `DocEntry` | One per identifier a document defines: `identifier`, `context` (the text up to the next definition), `page`, the box `x0`, `y0`, `x1`, `y1` and `line_id`. Linked to its document by `DEFINED_IN`. |
| `DocMention` | One per span that refers to an identifier: `span`, `kind`, `page`, the span's box, `line_id` and `line_text`. Linked to its document by `MENTIONED_IN`. |
| `REFERS_TO`, `SAME_ITEM_AS` | From a mention to the entry of the same thing. `REFERS_TO` when the span names the entry. `SAME_ITEM_AS` when the number of the item a page is about describes the same thing as the entry. Each carries its confidence and evidence: the line's text, the source, and the span's box as region. The source is the `Document`'s IRI with the page, as in `https://littlebigbrain.com/e/41ab…#page=3`. |

A box is `[x0, y0, x1, y1]` normalized to the page: from 0 to 1, with the
origin at the top left. `page` is 0-based. A line id is `p<page>-l<line>`,
both from 1, so `p1-l114` is line 114 of the first page.

## 1. Upload the files

The request body is the file, at most 1 GiB by default. `name` is the name the graph
lists. It is stored composed (Unicode NFC), so a name a Mac sends with a
separate combining mark (`u` and U+0308 for `ü`) is found by a query that
writes `ü`.

```sh
curl -X POST "$LBB/v1/files?name=manual.pdf" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/pdf' \
  --data-binary @manual.pdf
```

The server stores the bytes once under their SHA-256. It commits a
`Document` for them, with a `Content` that holds the bytes' SHA-256 and
length. The answer holds the file's `sha256`, its `document` IRI and
`created`. The same bytes under the same name again change nothing, so a
retry is safe. The same bytes under another name rename the file.

`Content-Type` names the file's type. Without one, or with
`application/octet-stream`, the server reads the type from the bytes and
the name: PDF, PNG, JPEG, GIF and WebP by how the bytes start, `.xlsx`,
`.docx` and `.pptx` files as their Office types, and UTF-8 text named
`.csv`, `.tsv`, `.md`, `.json` or `.txt` as `text/csv`,
`text/tab-separated-values`, `text/markdown`, `application/json` or
`text/plain`. A stated type always wins.

Two optional parameters describe the content further. `language` is the
natural language of the content, a BCP 47 tag such as `en` or `de-CH`, at
most 64 characters. `schema` names the schema
it follows (a name or a URL), at most 256 characters. The `Document` stores
each one given. A later upload that gives none keeps the earlier value.

```sh
curl "$LBB/v1/files" -H "Authorization: Bearer $LBB_API_KEY"
```

lists the newest 1,000 files, the newest upload first, each with its `parse`
once the parse workflow has read it: `status` (`parsed`, `unsupported` for a
file that is not a PDF or is empty, or `failed`), `page_count`, `ocr_pages`, `parser` and
`error`. `?limit=50` reads 50 at a time: pass the answer's `next` as `after`
for the next page. `GET /v1/files/counts` counts the files by parse status,
as of the commit the published generation holds.

`?parse_status=failed` lists the files the parse could not read
(`waiting` lists the ones it has not read yet), and `?name=manual` the files
whose name holds `manual`, in any case. A filtered list reads the
`Document` facts of the published generation, and `as_of_commit_seq` names
the commit it read: a file uploaded or parsed since shows once a generation
holds it.

```sh
curl "$LBB/v1/files?parse_status=failed&name=manual" -H "Authorization: Bearer $LBB_API_KEY"
```

From the SDKs:

```ts
import { readFile } from "node:fs/promises";

const bytes = await readFile("manual.pdf");
const { file } = await lbb.files.upload(bytes, { name: "manual.pdf", contentType: "application/pdf", language: "en" });
console.log(file.sha256, file.document);
```

```python
answer = lbb.files.upload("manual.pdf", language="en")  # a path, or bytes with name=
print(answer["file"]["sha256"])
```

In the console, **Sources → Files → Upload files** takes several files at
once. **Integrations → File upload → Choose files** sends Turtle, N-Triples
and NDJSON to the import and every other file to the graph's files.

## 2. Parse the files

Create a workflow from the starter workflow `documents.parse`:

```sh
curl -X PUT "$LBB/v1/workflows/triggered" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
  --data '{"starter": "documents.parse"}'
```

From the SDKs, that is `lbb.workflows.starters.use({ starter: "documents.parse" })`
in TypeScript and `lbb.use_workflow_starter("documents.parse")` in Python.
The workflow's name defaults to `documents-parse`. For each new file it runs
the text-layer parser (Poppler `pdftotext`, at most 2,000 pages and 120
seconds per file) and stores the pages. It calls no model.

Read the parsed pages, `from` up to `to` (0-based, `to` excluded, at most 50
pages per call):

```sh
curl "$LBB/v1/files/pages?sha256=$SHA&from=0&to=1" -H "Authorization: Bearer $LBB_API_KEY"
```

Each page has its `width` and `height` in PDF points and its `lines`, each
with its `id`, `text`, `box` and `words`. A file the parser has not read
answers `404 file_not_parsed`.

A scanned page is a picture of text. Its text layer is empty or nearly so.
The parser flags a page `ocr_required: true` when two things hold:

- It has fewer than 16 characters of text.
- Its images cover at least 40% of it.

The file's `ocr_pages` counts the flagged pages. The console warns on the
page and in the file list. A cover with a short title and a small logo has no
flag. A blank page has none either.

OCR itself is not built. A flagged page has few or no lines. The parser lists
the images with `pdfimages -list` only when a page has little text. If that
listing fails, the parse of the file stops and stores nothing. The file goes
to the back of the queue, and the files behind it are read. A file whose parse
stops three times is stored as `failed`, with the reason.

The Rust version of the workflow records a file with scanned pages in its
run results. A file where every page needs OCR is a result with no value. A
file where only some do is a success with an `ocr_pages` value. The hosted
workers record no run results. They write the flags and `ocr_pages` only.

If you run LBB yourself, read this about the parse programs (`pdftotext`
and `pdfimages`). They run with an empty environment and limits on memory,
CPU time and file size. An empty environment keeps keys and tokens out of
them. They still run as the user of the server. They still reach the network of the host. For files you do not trust, parse them on a host that
holds no credentials.

## Spreadsheets

An XLSX workbook is a file you can read by sheet and cell. The workflow
templates and the console use the same route:

```sh
curl "$LBB/v1/files/sheets?sha256=$SHA" -H "Authorization: Bearer $LBB_API_KEY"
curl "$LBB/v1/files/sheets?sha256=$SHA&sheet=Q3%20Sales&from=1&to=100" -H "Authorization: Bearer $LBB_API_KEY"
```

Without `sheet`, the answer lists every sheet with its name, state and size.
With `sheet`, it holds that sheet's rows. Rows are A1 rows from 1, and both
ends are included. The default is 100 rows. A cell has its stored `value` as
text and its `formula` apart.

LBB never calculates a formula. A formula cell has the value that the file
stored when a spreadsheet program saved it. It has none when the file stored
none. A date has its ISO value and keeps the stored serial in `raw`.

The first read of a workbook parses it. A child process of the server does
this, with an empty environment. The server stores the result.

Every limit refuses the whole workbook with `400 xlsx_refused`. The message
names the reason. The limits are:

- 64 MiB for the file, and 192 MiB after expansion
- 32 sheets, 50,000 rows and 512 columns (up to `SR`)
- 500,000 cells, and 32,767 characters in a cell
- 4 million XML elements
- 20 seconds of CPU time

The reader also refuses a workbook with macros. It refuses an encrypted or
older Excel file. It refuses a part that declares a DTD.

A node parses two workbooks at once, and a tenant parses one. A read that
finds no slot gets `503 xlsx_busy`.

If you run LBB yourself, the child runs as the user of the server. An empty
environment keeps keys and tokens out of it. A flaw in the reader could still
read what that user can read. For workbooks you do not trust, run LBB on a
host that holds no credentials.

Evidence names a place in a sheet with `<sheet>!<range>` after the
`Document` IRI: `…/e/41ab…#Sales!B2:D9`. The range is a cell (`B2`), a
rectangle (`B2:D9`), whole rows (`5:7`) or whole columns (`B:D`). A sheet
name with a space or a symbol is single quoted and percent encoded
(`'Q3%20Sales'!B2`). The console opens the file at the range. The `rows`
template writes the source `…#Sales!A5:F5` for row 5 of an XLSX file.

```ts
import { spreadsheetFragment } from "@littlebigbrain/client";

const outline = await lbb.files.sheets(sha256);
const sheet = await lbb.files.sheets(sha256, { sheet: "Q3 Sales", from: 1, to: 100 });
const source = `${document}#${spreadsheetFragment({ kind: "cells", sheet: "Q3 Sales", row: 2, col: 2, toRow: 2, toCol: 2 })}`;
```

## 3. Link the documents

```sh
curl -X PUT "$LBB/v1/workflows/triggered" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
  --data '{"starter": "documents.link", "gate": {"below": 0.8}}'
```

The name defaults to `documents-link`. Once a document is parsed, the
workflow:

1. joins a code that wraps in a narrow table column (`W` above `125`)
   into one line;
2. groups each page's lines into rows, and leaves out running headers and
   footers, also a footer whose page number changes (`Page 69 of 196`).
   A code in a header or footer does not link;
3. asks the reasoning model for the identifiers each page defines and the
   spans that may refer to one, and drops a definition or a span that is
   not in the line it cites. A long page is read in parts. A line that
   holds only a code labels the text below it. A page with lines written
   like the identifiers the document defines elsewhere (`PL 002` when other
   sheets define `PL 001` and `PL 004`), which its answer left out, is
   asked once more;
4. builds one catalogue of entries over every parsed document of the graph;
5. picks candidate entries for each mention, by characters, by context and,
   when the server has an embedding model, by meaning;
6. asks the decision model which entry the span names, if any, and with what
   probability. When the model picks a group (`MU`) for a code that the
   catalogue holds exactly (`MU 1.1`), the exact entry wins. A code of the
   same form with another number (`E1` for `E6`) is never the answer, and a
   place number (a room, floor or grid number) links only to an entry with
   that number as its code.

A link at or above `gate.below` (default 0.8) is written. A link below it
goes to review. Each document's outputs are stored per stage, so a rerun
pays only for what changed. When a new document adds entries, the other
documents are resolved again from their stored mentions.

The workflow's status reports, per loaded document, the pages, entries,
mentions, links written, links for review and mentions not linked, and the
links by confidence band (`>=0.9`, `0.8-0.9`, `0.5-0.8`, `<0.5`). It lists
the 20 documents with the most review items; `documents_total` counts all
of them and `documents_totals` adds up their numbers:

```sh
curl "$LBB/v1/workflows/triggered?starter=documents.link&name=documents-link" \
  -H "Authorization: Bearer $LBB_API_KEY"
```

To read every document, page through them. `q` finds documents by name,
and `sort` is `review` (the default), `name` or `recent`:

```sh
curl "$LBB/v1/workflows/named/documents?name=documents-link&q=schedule&limit=50" \
  -H "Authorization: Bearer $LBB_API_KEY"
```

The answer is one page in the [list envelope](https://docs.littlebigbrain.com/sdks/http-api/#lists), with
`total`, the documents that match `q`. Pass `next_cursor` as `cursor` for
the next page. The SDKs read every page with
`workflows.named.allDocuments("documents-link")` (TypeScript) and
`workflows.named.all_documents("documents-link")` (Python).

`documents.link` and `values.extract` call models. Their workflows share the
stack's monthly model budget for workflows, and each workflow can set a
lower one with `budget_usd_per_month`. The status shows the month's `spend`, and
`details.stack_budget` shows the stack's budget and spend.

## 4. Fill values from text

`values.extract` writes properties that the graph's property registry holds, so
add them first. Give each number property its unit:

```sh
curl -X POST "$LBB/v1/ontology/evolve" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
  --data '{"ops": [
    {"op": "add_property", "name": "width_mm", "value_type": "i64", "unit": "mm"},
    {"op": "add_property", "name": "weight_kg", "value_type": "f64", "unit": "kg"},
    {"op": "add_property", "name": "material", "value_type": "text"}
  ]}'
```

Then create a workflow of `values.extract` on the entries:

```sh
curl -X PUT "$LBB/v1/workflows/triggered" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
  --data '{
    "starter": "values.extract",
    "name": "parts",
    "watch": {"class": "DocEntry", "fields": ["identifier", "context"]},
    "params": {
      "properties": ["width_mm", "weight_kg", "material"],
      "context": "Entries of a parts catalogue for industrial pumps"
    }
  }'
```

For each entry the reasoning model proposes values, each with a quote. A
value whose quote is not in the text, or that does not fit its property's
type, is dropped. The decision model scores how well each quote supports its
value: at or above the gate the value is written, from 0.5 to the gate it
goes to review, and below 0.5 it is dropped. Each written value gets a
`ValueSource` linked by `HAS_VALUE_SOURCE`, with the quote as evidence and
the entry's box as region.

The model gives each number with the unit the text states. When the
property declares a unit, LBB converts the number with a fixed factor. So
`60 cm` is written as 600 for `width_mm`. A number with no unit in its quote
goes to review. A number in a unit of another kind, such as `12 kg` for a
width, is dropped. A property without a unit takes the number as stated.

## 5. Read mentions, entries and boxes with SPARQL

The properties are `https://littlebigbrain.com/p/<name>` and the relations
`https://littlebigbrain.com/r/<name>`. The mentions on the first page of the
manual, with their boxes and the entries they link to:

```sparql
PREFIX p: <https://littlebigbrain.com/p/>
PREFIX r: <https://littlebigbrain.com/r/>
SELECT ?span ?kind ?x0 ?y0 ?x1 ?y1 ?identifier ?entryPage ?entryDocument ?context WHERE {
  ?doc p:name "manual.pdf" .
  ?m r:mentioned_in ?doc ; p:page 0 ; p:span ?span ; p:kind ?kind ;
     p:x0 ?x0 ; p:y0 ?y0 ; p:x1 ?x1 ; p:y1 ?y1 .
  OPTIONAL {
    ?m r:refers_to|r:same_item_as ?e .
    ?e p:identifier ?identifier ; p:page ?entryPage ; p:context ?context ;
       r:defined_in ?d .
    ?d p:name ?entryDocument .
  }
}
ORDER BY ?y0
```

A mention without `?identifier` links to no entry, or its link waits for
review. The confidence of a link is on its evidence statement node:

```sparql
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX lbb: <https://littlebigbrain.com/ns#>
PREFIX r: <https://littlebigbrain.com/r/>
SELECT ?m ?e ?confidence WHERE {
  ?m r:refers_to ?e .
  ?st rdf:subject ?m ; rdf:predicate r:refers_to ; rdf:object ?e ;
      lbb:confidence ?confidence .
}
```

The entries a page defines:

```sparql
PREFIX p: <https://littlebigbrain.com/p/>
PREFIX r: <https://littlebigbrain.com/r/>
SELECT ?identifier ?context ?x0 ?y0 ?x1 ?y1 WHERE {
  ?doc p:name "parts-catalogue.pdf" .
  ?e r:defined_in ?doc ; p:page 0 ; p:identifier ?identifier ;
     p:x0 ?x0 ; p:y0 ?y0 ; p:x1 ?x1 ; p:y1 ?y1 .
  OPTIONAL { ?e p:context ?context }
}
```

## 6. Review the held links

Links below the gate are review items with facts, filed by the workflow
with the origin `starter:<name>`. Each item groups the mentions of one span
in one document that would link to one entry.

```sh
curl "$LBB/v1/ontology/suggestions?origin_id=starter:documents-link&status=open" \
  -H "Authorization: Bearer $LBB_API_KEY"
```

Accepting an item commits its links; later runs write an accepted item's
links again. Dismissing writes nothing. The items of `values.extract` have the
origin `starter:parts` (the workflow's name). See
[Ontology suggestions](https://docs.littlebigbrain.com/guides/ontology-suggestions/).

## 7. Map the links into your ontology

A link says "this line cites that entry". Your graph wants the fact:
procedure 4.2 of the manual uses the part P-114. Create a workflow of
`documents.map`:

```sh
curl -X PUT "$LBB/v1/workflows/triggered?graph=manuals" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
  --data '{"starter": "documents.map"}'
```

It reads the classes and properties of your ontology. For each linked entry,
the decision model picks the instance it is. It can also pick a new
instance, with its class. For each link, the model picks the statement the
line makes. It chooses among the properties that fit the two ends. It can
also choose "same item", "no fact" or "a property the ontology lacks".

The subject of a link is the entry its line belongs to. On a page about
one item, such as a product sheet, that is the item in the head of the
page. Without a list of those items, the number in the head becomes its own
entry. In a bill or a list, the subject is the position above the line. A
page with neither gives its links no subject. A new instance takes part in statements too, under the
IRI the workflow made for it.

Each statement becomes a `FactProposal` with these properties:

| Property | Holds |
| --- | --- |
| `fact_subject`, `fact_predicate`, `fact_object` | The statement, as IRIs (a label or an identifier is a literal). |
| `fact_confidence` | The model's probability for the statement. When the link waits for review, the link's confidence times that probability. |
| `fact_evidence` | JSON: the line, the page, the box, the link, and every option with its probability. |
| `proposed_by` | `documents.map/<name>`. |

A `review` workflow writes the proposals at or above its gate. It sends the
others to a person: one review item per instance, such as "Add P-114 Air
filter as Part?". The item shows the line of the page it comes from, then
the instance's type, name, code and links in plain words, and the links of
other items that wait for it. Uncheck a fact to leave it out,
then add the rest to the graph. A link that goes away takes its proposal
with it. Both workflows run on records graphs and on RDF-native graphs.

Read the proposals with SPARQL:

```sparql
PREFIX p: <https://littlebigbrain.com/p/>
SELECT ?subject ?predicate ?object ?confidence WHERE {
  ?x a <https://littlebigbrain.com/class/factproposal> ;
     p:fact_subject ?subject ; p:fact_predicate ?predicate ;
     p:fact_object ?object ; p:fact_confidence ?confidence .
}
ORDER BY DESC(?confidence)
```

Sometimes the ontology has no class or no property for a pattern. Then the
workflow files one ontology suggestion for the pattern, and the suggestion
says why. Its rationale quotes the lines (or the entries), what the model
answered, and with what probability. Its evidence lists each line with
**View on page**. A class suggestion names its entries in its title, such
as "Add a class for 4.1 Filters". Those entries get no type, label or
identifier until the class exists. A relation suggestion takes its draft
name from the words next to the span on its lines, such as `SEE_SECTION`
for "Filters: see section 3.2".

When a relation of the ontology already fits the two classes, a new one is
suggested only from 2 lines, or from one line at 0.8 or more. Its rationale
then names that relation and the probability the model gave it. Edit the
name with **Edit, then accept**, and accept. The next run maps those links.

A link that makes no proposal and no suggestion stays visible. The status
of the `documents.map` workflow lists it under `unmapped`, with its line,
the model's answer and the reason.

You can also declare a mapping. Then no model is asked for those links:

```sh
curl -X PUT "$LBB/v1/workflows/triggered?graph=manuals" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
  --data '{"starter": "documents.map", "params": {"mappings": {"parts": {
    "from": {"document": "manual.pdf", "entry_class": "Procedure"},
    "to": {"entry_class": "Part"},
    "property": "usesPart"}}}}'
```

The model calls count under the job `map` in the call log. Their price is
Jev's list price for input tokens, in the workflow's spend.

To change the questions or the rules, start from a template. The templates
`documents-map` and `review` hold the code of these two workflows, for your
own worker. See [Workflow templates](https://docs.littlebigbrain.com/guides/workflow-templates/).

## 8. See the pages in the console

With starter workflows on, the console shows the files and their pages:

- **Sources → Files** lists the files with their size, pages, parse status
  and upload time. A parsed file opens on its pages. A CSV or TSV file opens
  as a table, Markdown and text as numbered lines, JSON with its nodes, and
  an image as itself. **Read by** shows a PDF's parse, and for a file of
  another type the workflows whose content trigger reads it. **Types**
  lists the media types of the files, each with the workflows that read
  it; a type filters the list. An upload whose browser names no type takes
  the type of its extension (`.csv`, `.md`, `.json`, …). **Delete** asks
  for the file's name. Its option **Also retract the facts that cite this
  file** sends `purge=true`, after a warning that counts those facts. **Upload files** adds files, and when the graph lacks a
  document workflow the page offers **Create Parse documents workflow** or
  **Create Link documents workflow**.
- A file opens at the place its evidence names: `?row=5` or `?cell=14,3`
  in a table (rows from 1, the header is row 1), `?line=39,44` in text
  (lines 40 to 44), `?pointer=/items/3` in JSON, `?box=` on an image. The
  place is highlighted and scrolled into view, and a view shows at most a
  few hundred rows or lines around it.
- The page view (`#/files/<sha256>?page=<n>&box=<x0,y0,x1,y1>`, `page`
  0-based) draws the page's lines at their boxes. Each mention is a box:
  green when linked, amber when a review item holds its link (a link below
  the gate, or a no-link answer the model was unsure of), grey when not
  linked. The entries the page defines have blue dotted boxes. Point at a
  mention to see its span, kind, confidence and entry; click a linked
  mention to open the entry's page with the entry's box highlighted.
- **Operate → Workflows** lists the document workflows under Documents and
  the workflow of `values.extract` under Values from text. A workflow's page
  shows the parse per file, the links per document and by confidence, the
  values filled, and the model budget.
- In **Suggestions**, a held link with a page region has **View on page**.

## Write to the entries from your own code

A [developer workflow](https://docs.littlebigbrain.com/guides/starter-workflows/#developer-workflows) can
watch `DocEntry` or `DocMention` and write facts of its own. Its messages
name instances by IRI. An entry has a key (`entry:<sha256>:<identifier>`),
so a commit that names it by type and name would reach another record. Name
it by its id instead: a `key` of `"\u0000lbb-eid:"` followed by the 32
hexadecimal characters of its IRI addresses that record directly. The SDKs
build that reference:

```ts
import { entityRef, writeOwned } from "@littlebigbrain/client";

// In onInstances: `fixtures` is what your code found for each entry,
// as { iri, identifier, fixture }.
await writeOwned(lbb, ctx, "fixtures", {
  subjects: message.instances.map((iri) => entityRef(iri, "DocEntry")),
  relations: ["HAS_FIXTURE"],
  triplets: fixtures.map(({ iri, identifier, fixture }) => ({
    source: entityRef(iri, "DocEntry", identifier),
    relation: "HAS_FIXTURE",
    target: { type: "Fixture", name: fixture },
  })),
}, { graph: message.graph });
```

```python
from lbb import entity_ref

subjects = [entity_ref(iri, "DocEntry") for iri in message["instances"]]
```

The third argument is the record's display name. In `replace_owned` only
the id counts. A link endpoint or a field row writes its name as the
record's display name, so pass the entry's identifier there.

## Agents

The MCP tool `lbb_files` reads the files: `list` shows 50 files a call with
their parse status (`after` reads the next 50). `parse_status` and `name`
filter the list, for example the files whose parse `failed`. `pages` reads
the lines of a page range, each as
`<line id> [x0,y0,x1,y1] <text>`. `contains` keeps the lines that hold a
text, and `max_lines` caps the answer. `lbb_workflows` reads the workflows'
status, and `lbb_query` runs the SPARQL above.

## A measured run

One run of 2026-10-10 read five PDF files: 345 pages in all. The reasoning
model was Claude Sonnet 5.5 and the decision model Jev, with no embedding
model.

| Step | Time | Result |
| --- | --- | --- |
| `documents.parse` | 27 s | 5 files parsed |
| `documents.link` | 7 min 38 s | 544 entries, 4,917 mentions, 1,799 links written, 342 links held for review (171 review items), 2,181 mentions not linked |
| `values.extract` | 4 min 9 s | 544 entries read, 81 values proposed, 55 written, 13 held for review |

Links by confidence: 1,540 at 0.9 or more, 259 from 0.8 to 0.9, 289 from
0.5 to 0.8, and 53 below 0.5. The model calls cost $5.42: $4.86 for the
links and $0.56 for the values.

## Errors

| Error | When |
| --- | --- |
| `400 invalid_request` | An upload without `name`, a `language` over 64 characters or a `schema` over 256, a `sha256` that is not 64 hexadecimal characters, a delete without the right `confirm`. |
| `413 file_too_large` | An upload above the server's file limit, 1 GiB by default. |
| `404 file_not_found` | The graph has no file with this SHA-256. |
| `404 file_not_parsed` | `GET /v1/files/pages` of a file the parse workflow has not read. |
| `429 file_reclaim_in_progress` | An upload of the bytes of a deleted file while they go, after the 30 days; retry after `Retry-After`. |
| `403 free_managed_models_unavailable` | A Free account creates a workflow of `documents.link`, `documents.map` or `values.extract`. `documents.parse` calls no model and is open on Free. |
| `429 workflow_budget_exceeded` | A preview with `propose: true` when the model budget for workflows is spent. |
| `503 parser_unavailable` | The server cannot run `pdftotext`. |
| `503 fit_models_unavailable` | The server has no reasoning or no decision model. |

A delete (`DELETE /v1/files?sha256=<hex>&confirm=<hex>`) removes the file
from the list and retracts its `Document` and its `Content`. The link workflow then retracts
the entries and mentions it wrote for that document. The bytes stay for 30
days, with the parsed pages and what the workflows made of the file. An
upload of the same bytes in that time sends nothing again. After the 30
days they go, unless a fact still names the file in its evidence. Such a
fact is an edge a person accepted in review, or one your own code committed
with the file as its source. It keeps the bytes, and the console's view of
the file lists it. `purge=true` on the delete retracts those facts too:

```sh
curl -X DELETE "$LBB/v1/files?sha256=$SHA&confirm=$SHA&purge=true" \
  -H "Authorization: Bearer $LBB_API_KEY"
```

```ts
await lbb.files.delete(sha256, { purge: true });
```

```python
lbb.files.delete(sha256, purge=True)
```

A file deleted and uploaded again is a new upload. The parse and the link
workflows read it again.