Skip to content
little big braindocs

Add links with an LLM

View Markdown

Some facts live only in text. A manual says which account a booking rule is for. An advisory says which library a bug is in. An LLM can read each section and write those facts as links.

This guide continues Load your data. It uses the same graph, kb, the same client and the same files.

How it works

Section titled “How it works”
  1. You send the model one section and the records it may link to.
  2. The model answers with links as JSON.
  3. Your code drops each link to a record you do not have.
  4. A dry run checks the rest against the schema. Then you write them.

The schema refuses unknown types, links and fields. It cannot know which records your model meant. Step 3 checks that.

Pick your use case

Section titled “Pick your use case”

The examples follow the use case you pick. The choice is the same as on Load your data.

Your sources. A ticket database and your release notes.

The question. Which release fixed the login bug?

The graph.

Ticket → REPORTED_AS → Bug ← COVERS ← Section → PART_OF → Document

1. Ask the model

Section titled “1. Ask the model”

Send one prompt per section. The prompt names the link, the records the link may point to, and the section:

You link document sections to records in a knowledge graph.


Add a COVERS link from the section to each record below that the section
states a rule or a fact about. Use only these records:


<the contents of candidates.json>


Section key: <key>
Section name: <name>
Section text: <text>


Answer with JSON only, in this form:
{"triplets": [{"source": {"type": "Section", "key": "<key>", "name": "<name>"},
  "relation": "COVERS",
  "target": {"type": "<type>", "key": "<key>", "name": "<name>"},
  "evidence": "<key>"}]}
Answer {"triplets": []} when the section is about none of these records.

candidates.json lists the records the link may point to. Your database already has them, so export them from there. Use any model that can answer with JSON.

Collect the answers for all sections in one file, links.json. A model can make mistakes. In each example below, one link points to a record that does not exist:

candidates.json
[
  { "type": "Bug", "key": "AUTH-112", "name": "session token expiry" },
  { "type": "Bug", "key": "PDF-31", "name": "PDF fonts missing" }
]
links.json
{
  "triplets": [
    {
      "source": { "type": "Section", "key": "release-notes-3.2#1", "name": "Sign-in fixes" },
      "relation": "COVERS",
      "target": { "type": "Bug", "key": "AUTH-112", "name": "session token expiry" },
      "evidence": "release-notes-3.2#1"
    },
    {
      "source": { "type": "Section", "key": "release-notes-3.2#2", "name": "Faster exports" },
      "relation": "COVERS",
      "target": { "type": "Bug", "key": "EXPORT-7", "name": "export runs on one thread" },
      "evidence": "release-notes-3.2#2"
    }
  ]
}

The last link is wrong: EXPORT-7 is not in your bugs table.
Section titled “2. Drop links to unknown records”

A link to an unknown key creates a new record. So keep only the links whose target is in candidates.json:

const known = new Set(read("candidates.json").map((r: any) => `${r.type}/${r.key}`));
const links = read("links.json");
links.triplets = links.triplets.filter((t: any) =>
  known.has(`${t.target.type}/${t.target.key}`),
);
console.log(links.triplets.length, "links kept");

3. Check, then write

Section titled “3. Check, then write”

A dry run checks the links against the schema and writes nothing. It fails like a real write on an unknown type, link or field. Say the model names a link EXPLAINS. The error lists the links your schema allows:

unknown relation 'EXPLAINS'; allowed relations: REPORTED_AS, PART_OF, COVERS

Send such an error back to the model and ask again. When the dry run passes, it reports the number of links it would write. Then write them:

await lbb.commitDryRun(links);


const written = await lbb.graph("kb").facts.create(links, { idempotencyKey: "kb-links-1" });
await lbb.waitForPublished(written.commit_seq);

Each link keeps the section key in evidence. An answer can then cite the section it came from.

4. Follow the path

Section titled “4. Follow the path”

Your database records and your document sections are one graph now. This query starts at a record and follows its links to the section that explains it:

path.rq
PREFIX dclass: <https://littlebigbrain.com/class/>
PREFIX dr: <https://littlebigbrain.com/r/>
PREFIX dp: <https://littlebigbrain.com/p/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>


SELECT ?ticket ?bug ?section ?document WHERE {
  ?t a dclass:ticket ; rdfs:label ?ticket ; dr:reported_as ?b .
  ?b rdfs:label ?bug .
  ?s dr:covers ?b ; rdfs:label ?section ; dr:part_of ?d .
  ?d rdfs:label ?document .
}

The result has 1 row:

ticket bug section document
login fails after 3.1 session token expiry Sign-in fixes Release notes 3.2

Run the query:

const { rows } = await lbb.sparqlRows({ query: readFileSync("path.rq", "utf8") });
console.log(rows);

Search finds a section by meaning. This query then proves which records it is about. How search works shows the two steps together.

Run it again

Section titled “Run it again”

The same files give the same links. A rerun with the same idempotency key returns the first write and stores nothing new.

When a document changes, load its sections again and ask the model again. Use a new idempotency key for the new links.

Section titled “Related”