Ask a person: review in workflows (experimental)

A workflow sometimes needs a person. A value can sit below the gate. A label can be unclear. An answer can need a grade. Review is one inbox for these questions.

A question is a review item. Its id starts with rev_. Its result is an answer. Each answer does three things:

• A confirmed or corrected value goes into the graph.
• The answer becomes a label for the workflow’s gate.
• The workflow that asked receives the answer.

Kinds of questions

Kind	The person	The answer
value	confirms one value, or corrects it	accept, correct, reject or cant_tell
choice	picks one option	choose or cant_tell
grade	grades an output on a scale	choose or cant_tell
answer	writes a short answer	accept with the text, or cant_tell
change	accepts an ontology change	accept, reject or cant_tell

A change is an ontology suggestion. The other kinds are new.

Ask from a workflow

Use lbb.review() in a step. The call returns at once, and the run ends.

await ctx.step("review", () =>
  lbb.review({
    kind: "value",
    workflow: message.workflow,
    key: `starter:${message.workflow}:value:${roomId}:area`,
    title: "Is the area of R 1.04 12.5 m2?",
    subject: { type: "Room", name: "R 1.04" },
    question: { property: "area", value: 12.5, unit: "m2", confidence: 0.62 },
    evidence: [{ quote: "Fläche 12,5 m²", source_id: documentIri }],
  }),
);

The key makes the call safe to repeat. The same key updates the item. An answered item comes back with its answer.

Receive the answer

Add a review trigger beside the workflow’s watch:

{
  "name": "areas",
  "workflow": { "workflow_type": "areas.check", "version": "v1" },
  "triggers": [
    { "watch": { "class": "Room", "fields": ["area"] } },
    { "review": {} }
  ]
}

Each answer then arrives as one event of type review.answered. The event holds the answered item and the id of the request that answered it. Handle it in onReviewAnswered:

triggeredWorkflow({
  name: "areas.check",
  version: "v1",
  async onInstances(ctx, message, state) { /* ask */ },
  async onReviewAnswered(ctx, message, state) {
    const item = message.review_item;
    if (item.answer?.decision === "correct") {
      // The corrected value is in the graph at item.commit_seq.
    }
    return { state };
  },
});

A value’s accept or correction is in the graph before the message arrives. The workflow does not write it again.

Wait for the answer

A run with its own identity can wait. The worker is free while it waits.

const result = await lbb.waitForReview(
  ctx,
  "floor",
  {
    kind: "choice",
    key: `floor:${roomId}`,
    title: "Which floor covering does R 1.04 have?",
    question: {
      options: [
        { id: "parquet", label: "Parquet" },
        { id: "tiles", label: "Tiles" },
      ],
    },
  },
  { timeoutMs: 7 * 24 * 3_600_000 },
);
if (result.answered) console.log(result.answer.option);

An answer that arrives before the wait is kept. The wait then returns at once. When the wait times out, it reads the item again: an answer whose signal did not arrive still counts.

A trigger shard serves many inputs, and a wait would hold them all. So waitForReview refuses to run there. Use a review trigger instead.

The Python worker has the same call, lbb.wait_for_review. See Run a worker in Python.

Answer in the console

Open Review in the sidebar. The badge counts open items. Filter by kind, by status and by source. Workflows shows what workflows asked.

A value shows the record, the property, the proposed value and the quote. Choose Accept value, Correct…, Can’t tell or Reject…. A correction is a number in the property’s unit. It is saved as entered. A choice shows its options as buttons. An item links to the workflow that asked.

Answer with the API

const page = await client.reviews.list({ status: "open", kind: "value" });
const item = page.data[0];
await client.reviews.answer(
  item.id,
  {
    answer: { decision: "correct", value: 14.25, by: "ana" },
    revision: item.revision,
  },
  { idempotencyKey: `answer-${item.id}` },
);

revision is the revision you read. An item changed since then answers 409 review_stale.

A corrected number may name its unit, such as "450 mm". LBB converts it to the property’s unit. A unit of another kind answers 400 review_answer_value. A second, different answer answers 409 review_answered.

A list returns one page. Pass next_cursor as cursor for the next page, or read every item with client.reviews.all(), with the same filters.

When the graph refuses a value that a person accepted or corrected, the value is not written. The item is open again, and effects.commit_error says why. The person can answer again. Your workflow must not write that value itself.

A run that waits for the item gets no signal for a refused answer: it waits for the next answer. Your workflow’s review trigger, if it has one, gets the refused answer instead, as a review.answered message with review_item.commit_error.

Each answer sends an Idempotency-Key: yours, or one the SDK makes. A retry sends the same key, returns the item and writes nothing again. The same key with another answer answers 409 idempotency_conflict.

The Python SDK has the same names: client.reviews.list(), client.reviews.all(), client.reviews.get() and client.reviews.answer().

Gates and labels

Every answer is a label for the gate of the workflow that asked. A value is right when a person accepts it as proposed. A correction or a rejection counts as wrong. “Can’t tell” gives no label.