Build a workflow

View Markdown

A workflow does work on your records after each published commit. The one on this page reads product data sheets. It fills four properties: the width, the depth, the material and the color. Each value comes with the words of the text that state it.

You test the workflow before it writes. A test runs it on two records and writes nothing to the graph. You read each value with its quote, correct one, and keep your answers as a dataset. Then you create the workflow, and it writes the values.

The workflow comes from values.extract, one of LBB’s templates. LBB runs it, so you need no worker of your own.

1. Connect

In the console, open your stack’s Connect view and copy its endpoint and an API key. To run on your machine instead, run npm run dev:stack in the LBB repository. Its endpoint is http://127.0.0.1:7400, and its key is lbb_sk_test_local.

Set both in your terminal. The TypeScript and Python examples read them from there. The cURL steps use jq to read fields of the answers.

In cURL, RUN holds the time you start. It makes this run’s idempotency keys unique. A request sent again with the same key does its work once. Each cURL step also sends Lbb-Version: 2026-10-21, the API version of this page. The SDKs send the version and a key for each write.

export LBB_URL="http://127.0.0.1:7400"
export LBB_API_KEY="lbb_sk_test_local"
export RUN="$(date +%s)"

import { LbbClient, LbbError } from "@littlebigbrain/client";

const client = new LbbClient({ baseUrl: process.env.LBB_URL!, apiKey: process.env.LBB_API_KEY, graph: "main" });

import os
import time

from lbb import LbbClient, LbbError

client = LbbClient(os.environ["LBB_URL"], api_key=os.environ["LBB_API_KEY"], graph="main")

Each example on this page adds to the one before it. In TypeScript and Python, keep them in one file.

2. Add the class and two records

A workflow fills properties that the graph declares. Create the graph main first. If it exists, the create answers 409, and you go on. Declare the class Product, its text, and the four properties to fill. A number property names its unit. The workflow then converts a number in another unit: 1.6 m is 160 for width_cm.

Then write two records from the template’s samples. They are the data sheets of a conference table and of a drawer unit. The second one is in German. A test reads published records, so wait until the commit is published.

curl -sS -X POST "$LBB_URL/v1/graph/create?graph=main" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21'

curl -sS -X POST "$LBB_URL/v1/ontology/evolve?graph=main" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21' \
  -H 'Content-Type: application/json' \
  --data '{"ops": [{"op": "add_entity_type", "name": "Product"}, {"op": "add_property", "name": "text", "value_type": "text"}, {"op": "add_property", "name": "width_cm", "value_type": "f64", "unit": "cm"}, {"op": "add_property", "name": "depth_cm", "value_type": "f64", "unit": "cm"}, {"op": "add_property", "name": "material", "value_type": "text"}, {"op": "add_property", "name": "color", "value_type": "keyword"}]}'

curl -sS -X POST "$LBB_URL/v1/graph/commit?graph=main" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21' \
  -H 'Content-Type: application/json' -H "Idempotency-Key: product-records-$RUN" \
  --data '{"entity_properties": [{"type": "Product", "name": "T-160", "properties": {"text": "T-160 Conference table of oak veneer, width 1.6 m, depth 800 mm. Accessory: a cable tray of steel under the top."}}, {"type": "Product", "name": "R-3", "properties": {"text": "R-3 Rollcontainer aus Stahlblech, Breite 43, Höhe 60 cm, mit drei Schubladen. Farbe: lichtgrau."}}]}'

for i in $(seq 120); do
  [ "$(curl -sS "$LBB_URL/v1/graph/publication-status?graph=main" \
    -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21' | jq -r '.state')" = current ] && break
  [ "$i" = 120 ] && echo "The commit is not published after 2 minutes." >&2
  sleep 1
done

R3=$(curl -sS "$LBB_URL/v1/graph/entity?graph=main&type=Product&name=R-3" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21' | jq -r '.entity.id')

The loop asks every second if the commit is published, for at most two minutes. The last command keeps the id of record R-3 for step 4.

try {
  await client.createGraph();
} catch (error) {
  if (!(error instanceof LbbError && error.status === 409)) throw error;
}

await client.ontology.evolve({
  ops: [
    { op: "add_entity_type", name: "Product" },
    { op: "add_property", name: "text", value_type: "text" },
    { op: "add_property", name: "width_cm", value_type: "f64", unit: "cm" },
    { op: "add_property", name: "depth_cm", value_type: "f64", unit: "cm" },
    { op: "add_property", name: "material", value_type: "text" },
    { op: "add_property", name: "color", value_type: "keyword" },
  ],
});

const written = await client.graph("main").facts.create({
  entity_properties: [
    {
      type: "Product",
      name: "T-160",
      properties: {
        text: "T-160 Conference table of oak veneer, width 1.6 m, depth 800 mm. Accessory: a cable tray of steel under the top.",
      },
    },
    {
      type: "Product",
      name: "R-3",
      properties: {
        text: "R-3 Rollcontainer aus Stahlblech, Breite 43, Höhe 60 cm, mit drei Schubladen. Farbe: lichtgrau.",
      },
    },
  ],
});
await client.waitForPublished(written.commit_seq);

try:
    client.create_graph()
except LbbError as error:
    if error.status_code != 409:
        raise

client.ontology.evolve({
    "ops": [
        {"op": "add_entity_type", "name": "Product"},
        {"op": "add_property", "name": "text", "value_type": "text"},
        {"op": "add_property", "name": "width_cm", "value_type": "f64", "unit": "cm"},
        {"op": "add_property", "name": "depth_cm", "value_type": "f64", "unit": "cm"},
        {"op": "add_property", "name": "material", "value_type": "text"},
        {"op": "add_property", "name": "color", "value_type": "keyword"},
    ]
})

written = client.graph("main").facts.create(
    {
        "entity_properties": [
            {
                "type": "Product",
                "name": "T-160",
                "properties": {
                    "text": "T-160 Conference table of oak veneer, width 1.6 m, depth 800 mm. Accessory: a cable tray of steel under the top."
                },
            },
            {
                "type": "Product",
                "name": "R-3",
                "properties": {
                    "text": "R-3 Rollcontainer aus Stahlblech, Breite 43, Höhe 60 cm, mit drei Schubladen. Farbe: lichtgrau."
                },
            },
        ]
    }
)
client.wait_for_published(written["commit_seq"])

3. Test the template

A model that reads values from text can read a wrong one. It can read the size of an accessory, or a height as a depth. The test shows what the workflow would write.

Send the workflow as you will create it, in install. It names the template and the workflow’s name, product. It names the class, the field to watch and the properties to fill. sample: 2 runs it on the first two records of the class. The test goes from queued to running, then ends done.

TEST=$(curl -sS -X POST "$LBB_URL/v1/workflows/named/tests?graph=main" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21' \
  -H 'Content-Type: application/json' -H "Idempotency-Key: product-test-$RUN" \
  --data '{"install": {"starter": "values.extract", "name": "product", "watch": {"class": "Product", "fields": ["text"]}, "params": {"properties": ["width_cm", "depth_cm", "material", "color"]}}, "sample": 2}' | jq -r '.id')

for i in $(seq 300); do
  [ "$(curl -sS "$LBB_URL/v1/workflows/named/tests/get?graph=main&id=$TEST" \
    -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21' | jq -r '.status')" = done ] && break
  [ "$i" = 300 ] && echo "The test is not done after 5 minutes. Read its status and error." >&2
  sleep 1
done

curl -sS "$LBB_URL/v1/workflows/named/tests/get?graph=main&id=$TEST" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21'

const install = {
  starter: "values.extract",
  name: "product",
  watch: { class: "Product", fields: ["text"] },
  params: { properties: ["width_cm", "depth_cm", "material", "color"] },
};

const started = await client.workflows.tests.create({ install, sample: 2 });
const test = await client.workflows.tests.wait(started.id);
for (const result of test.results ?? []) {
  const label = test.inputs.find((input) => input.iri === result.iri)?.label;
  for (const value of result.values ?? []) {
    console.log(label, value.property, value.value, `"${value.quote}"`, value.outcome);
  }
}

install = {
    "starter": "values.extract",
    "name": "product",
    "watch": {"class": "Product", "fields": ["text"]},
    "params": {"properties": ["width_cm", "depth_cm", "material", "color"]},
}

started = client.workflows.tests.create(install=install, sample=2)
test = client.workflows.tests.wait(started.id)
labels = {each.iri: each.label for each in test.inputs}
for result in test.results or []:
    for value in result.values or []:
        print(labels[result.iri], value.property, value.value, repr(value.quote), value.outcome)

For each record, the test keeps the text it read and the values. Each value has a quote and a support. The support says how well the quote supports the value, from 0 to 1. Its outcome says what the workflow does with it. written means the workflow writes the value. review means it asks a person first.

On the local stack, T-160 gets three values. width_cm is 160 from width 1.6 m, and depth_cm is 80 from depth 800 mm. Its material is oak veneer, and the steel of the cable tray is left out. The text names no color, so color gets no value.

R-3 also gets three values, and no depth: Höhe 60 cm is its height. Its width_cm 43 quotes Breite 43, which states no unit. So its outcome is review: a person confirms the unit before the value is written.

4. Correct a value and save the answers

Your answers turn the test into examples. A dataset of examples lets you compare a later version of the workflow with this one.

Say your product list is in English and names the material of R-3 sheet steel. Correct the value. The decisions accept and reject take the same route. An answer writes nothing to the graph.

Then save the answers as the test’s dataset. Each record you answered becomes an example. It holds the text, the values a correct run writes and the values it leaves out.

A record’s IRI is https://littlebigbrain.com/e/ followed by its id. In the bodies below, '"$TEST"' ends the single quotes. The shell then puts in the value of TEST.

curl -sS -X POST "$LBB_URL/v1/workflows/named/tests/answer?graph=main" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21' \
  -H 'Content-Type: application/json' -H "Idempotency-Key: product-answer-$RUN" \
  --data '{"id": "'"$TEST"'", "iri": "https://littlebigbrain.com/e/'"$R3"'", "property": "material", "answer": {"decision": "correct", "value": "sheet steel"}}'

curl -sS -X POST "$LBB_URL/v1/evals/datasets?graph=main" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21' \
  -H 'Content-Type: application/json' -H "Idempotency-Key: product-dataset-$RUN" \
  --data '{"name": "product answers", "kind": "values", "source": {"kind": "test", "id": "'"$TEST"'"}}'

const r3 = test.inputs.find((input) => input.label === "R-3")!;
await client.workflows.tests.answer({
  id: test.id,
  iri: r3.iri,
  property: "material",
  answer: { decision: "correct", value: "sheet steel" },
});

const dataset = await client.evals.datasets.create({
  name: "product answers",
  kind: "values",
  source: { kind: "test", id: test.id },
});
console.log(dataset.id, dataset.item_count);

r3 = next(each for each in test.inputs if each.label == "R-3")
client.workflows.tests.answer(test.id, r3.iri, "material", {"decision": "correct", "value": "sheet steel"})

dataset = client.evals.datasets.create("product answers", source={"kind": "test", "id": test.id})
print(dataset.id, dataset.item_count)

The dataset holds one example, R-3. Its material is sheet steel, and Stahlblech is a value to leave out. Answer more values and save again. A test has one dataset, and each save brings it up to date.

5. Create the workflow

Create the workflow with the same body you sent as install. This is how you create a workflow from one of LBB’s templates.

The workflow first reads every record of the class. After that, it reads each record that a published commit adds or changes. Each step it takes is a run, with its outcome, its model calls and their cost. The list of runs has the list fields data, has_more and next_cursor, newest first.

The workflow writes what the model reads. Your answers stay in the test and its dataset, so R-3 gets the material Stahlblech. The width of R-3 is not written: it waits for a person, as in the test.

curl -sS -X PUT "$LBB_URL/v1/workflows/triggered?graph=main" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21' \
  -H 'Content-Type: application/json' \
  --data '{"starter": "values.extract", "name": "product", "watch": {"class": "Product", "fields": ["text"]}, "params": {"properties": ["width_cm", "depth_cm", "material", "color"]}}'

for i in $(seq 300); do
  [ "$(curl -sS "$LBB_URL/v1/workflows/named/runs?graph=main&name=product" \
    -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21' | jq -r '.data[0].outcome')" = done ] && break
  [ "$i" = 300 ] && echo "The workflow has no done run after 5 minutes. Read its runs." >&2
  sleep 1
done

curl -sS "$LBB_URL/v1/workflows/named/runs?graph=main&name=product&limit=5" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21'

curl -sS "$LBB_URL/v1/workflows/named?graph=main&name=product" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21'

for i in $(seq 120); do
  [ "$(curl -sS "$LBB_URL/v1/graph/publication-status?graph=main" \
    -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21' | jq -r '.state')" = current ] && break
  [ "$i" = 120 ] && echo "The values are not published after 2 minutes." >&2
  sleep 1
done

curl -sS "$LBB_URL/v1/graph/entity?graph=main&type=Product&name=R-3" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Lbb-Version: 2026-10-21'

await client.workflows.starters.use(install);

let workflow = await client.workflows.named.get("product");
for (let second = 0; (workflow.status.counters?.instances ?? 0) < 2; second++) {
  if (second === 300) throw new Error("The workflow has not read both records after 5 minutes.");
  await new Promise((resolve) => setTimeout(resolve, 1_000));
  workflow = await client.workflows.named.get("product");
}
console.log(workflow.status.counters);

const runs = await client.workflows.named.runs("product", { limit: 5 });
console.log(runs.data, runs.has_more, runs.next_cursor);

const { head_seq } = await client.publicationStatus();
await client.waitForPublished(head_seq);
const record = await client.entityDetail({ type: "Product", name: "R-3" });
console.log(record.attributes);

client.use_workflow_starter(
    "values.extract",
    name="product",
    class_="Product",
    fields=["text"],
    params={"properties": ["width_cm", "depth_cm", "material", "color"]},
)

workflow = client.workflows.named.get("product")
for second in range(301):
    if (workflow.status.counters or {}).get("instances", 0) >= 2:
        break
    if second == 300:
        raise TimeoutError("The workflow has not read both records after 5 minutes.")
    time.sleep(1)
    workflow = client.workflows.named.get("product")
print(workflow.status.counters)

runs = client.workflows.named.runs("product", limit=5)
print(runs.data, runs.has_more, runs.next_cursor)

client.wait_for_published(client.publication_status_model().head_seq)
record = client.entities.detail(type="Product", name="R-3")
print(record["attributes"])

The workflow’s status.counters count its work. instances counts the records it read: 2. written counts the values it wrote: 5. review counts the values it sent to a person: 1, the width of R-3.

The record’s attributes hold the values. Each value also has a link HAS_VALUE_SOURCE to its source. The link’s evidence is the quote. Add a third record, and the workflow reads it after the commit is published.

6. Where to go next

Review. A value goes to review when its support is from 0.5 up to the gate. A number whose quote states no unit goes to review too. The gate is 0.8 unless you set gate. A person confirms or corrects the value, and the answer is written. GET /v1/reviews?workflow=product lists the items: here, the width of R-3. See Ask a person.
Spend and Quality. In the console, open Operate → Workflows and the workflow. Spend shows its model calls this month and its budget. Quality shows its gate, the model checks of its uses and its open review items.
Your own code. To change the prompt or the rules, run the template’s code in your own worker. See Workflow templates and Run a worker in Python.
More on tests. Every field is in Test a workflow before it writes, Datasets of examples and Starter workflows.

If something goes wrong

What you see	What to do
`400 workflow_param_invalid`: the class has no published instances	The commit is not published yet. Wait for it, then start the test again.
`400 workflow_param_invalid` that names a property	The graph lacks the property, or its type. Declare it as in step 2.
`409 conflict` when you start a test	A test of this workflow is still open. Wait for it to end, or cancel it.
A test that ends `failed`	Its `error` says why. The records it read keep their results.
A wait that prints its message	The step did not end in time. Read the test’s `status` and `error`, or the workflow’s runs.
`429 workflow_budget_exceeded`	The test may cost more than the monthly workflow budget has left.
`503 workflows_disabled`	The server runs without workflows. On the local stack, use `npm run dev:stack`.