# Ask a person: review in workflows (experimental)

Source: https://docs.littlebigbrain.com/guides/review-in-workflows/

A workflow sometimes needs a person. A value can sit below the gate. A label
can be unclear. An answer can need a grade. **Review** is one inbox for these
questions.

A question is a **review item**. Its id starts with `rev_`. Its result is
an **answer**. Each answer does three things:

- A confirmed or corrected value goes into the graph.
- The answer becomes a label for the workflow's gate.
- The workflow that asked receives the answer.

## Kinds of questions

| Kind | The person | The answer |
| --- | --- | --- |
| `value` | confirms one value, or corrects it | `accept`, `correct`, `reject` or `cant_tell` |
| `choice` | picks one option | `choose` or `cant_tell` |
| `grade` | grades an output on a scale | `choose` or `cant_tell` |
| `answer` | writes a short answer | `accept` with the text, or `cant_tell` |
| `change` | accepts an ontology change | `accept`, `reject` or `cant_tell` |

A `change` is an [ontology suggestion](https://docs.littlebigbrain.com/guides/ontology-suggestions/). The
other kinds are new.

## Ask from a workflow

Use `lbb.review()` in a step. The call returns at once, and the run ends.

```ts
await ctx.step("review", () =>
  lbb.review({
    kind: "value",
    workflow: message.workflow,
    key: `starter:${message.workflow}:value:${roomId}:area`,
    title: "Is the area of R 1.04 12.5 m2?",
    subject: { type: "Room", name: "R 1.04" },
    question: { property: "area", value: 12.5, unit: "m2", confidence: 0.62 },
    evidence: [{ quote: "Fläche 12,5 m²", source_id: documentIri }],
  }),
);
```

The `key` makes the call safe to repeat. The same key updates the item. An
answered item comes back with its answer.

## Receive the answer

Add a review trigger beside the workflow's watch:

```json
{
  "name": "areas",
  "workflow": { "workflow_type": "areas.check", "version": "v1" },
  "triggers": [
    { "watch": { "class": "Room", "fields": ["area"] } },
    { "review": {} }
  ]
}
```

Each answer then arrives as one event of type `review.answered`. The event
holds the answered item and the id of the request that answered it. Handle
it in `onReviewAnswered`:

```ts
triggeredWorkflow({
  name: "areas.check",
  version: "v1",
  async onInstances(ctx, message, state) { /* ask */ },
  async onReviewAnswered(ctx, message, state) {
    const item = message.review_item;
    if (item.answer?.decision === "correct") {
      // The corrected value is in the graph at item.commit_seq.
    }
    return { state };
  },
});
```

A value's accept or correction is in the graph before the message arrives.
The workflow does not write it again.

## Wait for the answer

A run with its own identity can wait. The worker is free while it waits.

```ts
const result = await lbb.waitForReview(
  ctx,
  "floor",
  {
    kind: "choice",
    key: `floor:${roomId}`,
    title: "Which floor covering does R 1.04 have?",
    question: {
      options: [
        { id: "parquet", label: "Parquet" },
        { id: "tiles", label: "Tiles" },
      ],
    },
  },
  { timeoutMs: 7 * 24 * 3_600_000 },
);
if (result.answered) console.log(result.answer.option);
```

An answer that arrives before the wait is kept. The wait then returns at
once. When the wait times out, it reads the item again: an answer whose
signal did not arrive still counts.

A trigger shard serves many inputs, and a wait would hold them all. So
`waitForReview` refuses to run there. Use a review trigger instead.

The Python worker has the same call, `lbb.wait_for_review`. See
[Run a worker in Python](https://docs.littlebigbrain.com/guides/python-worker/#ask-a-person).

## Answer in the console

Open **Review** in the sidebar. The badge counts open items. Filter by
kind, by status and by source. **Workflows** shows what workflows asked.

A value shows the record, the property, the proposed value and the quote.
Choose **Accept value**, **Correct…**, **Can't tell** or **Reject…**.
A correction is a number in the property's unit. It is saved as entered. A
choice shows its options as buttons. An item links to the workflow that
asked.

## Answer with the API

```ts
const page = await client.reviews.list({ status: "open", kind: "value" });
const item = page.data[0];
await client.reviews.answer(
  item.id,
  {
    answer: { decision: "correct", value: 14.25, by: "ana" },
    revision: item.revision,
  },
  { idempotencyKey: `answer-${item.id}` },
);
```

`revision` is the revision you read. An item changed since then answers
`409 review_stale`.

A corrected number may name its unit, such as `"450 mm"`. LBB converts it
to the property's unit. A unit of another kind answers `400
review_answer_value`. A second, different answer answers `409
review_answered`.

A list returns one page. Pass `next_cursor` as `cursor` for the next page,
or read every item with `client.reviews.all()`, with the same filters.

When the graph refuses a value that a person accepted or corrected, the
value is not written. The item is open again, and `effects.commit_error`
says why. The person can answer again. Your workflow must not write that
value itself.

A run that waits for the item gets no signal for a refused answer: it waits
for the next answer. Your workflow's review trigger, if it has one, gets
the refused answer instead, as a `review.answered` message with
`review_item.commit_error`.

Each answer sends an `Idempotency-Key`: yours, or one the SDK makes. A
retry sends the same key, returns the item and writes nothing again. The
same key with another answer answers `409 idempotency_conflict`.

The Python SDK has the same names: `client.reviews.list()`,
`client.reviews.all()`, `client.reviews.get()` and
`client.reviews.answer()`.

## Gates and labels

Every answer is a label for the gate of the workflow that asked. A value is
right when a person accepts it as proposed. A correction or a rejection
counts as wrong. "Can't tell" gives no label.