Skip to content
little big braindocs

Build with JSON records

View Markdown

You can build a complete application on little big brain with JSON. This guide goes further than the quickstart. It adds typed fields, a chain of links of any length, rules and a replayed decision.

Underneath, each record is stored as RDF and each graph serves SPARQL. You do not need to know RDF for this guide. The one query and the one rules document each take a few lines.

The running example is a medicine recall. A supplier recalls one ingredient lot. Which released batches contain material from that lot? Which active study sites received them?

The answer runs through intermediate lots. It is a chain of links, so one join cannot answer it.

The four terms below are the whole vocabulary you need. The API calls them entity types, entities, relations and properties.

Term What it is JSON example
Type A kind of record ReleasedBatch
Record One thing, identified by its type plus a key or a name { "type": "ReleasedBatch", "name": "Cardiovex batch CVX-A17", "key": "CVX-A17" }
Link A named connection from one record to another PRODUCED_FROM
Field A typed value on one record "batch_number": "CVX-A17"

  1. Declare the schema. A graph accepts only the types, links and fields it declares, so declare them first. One define call creates the graph with its types and links. One evolve call adds the typed fields. Both calls are safe to repeat.

    A type can include super_types: ["ParentClass"]. Here an ActiveStudySite is a kind of StudySite. Declare each parent in the same document, in any order. Unknown fields and unknown parents are rejected, so the hierarchy is never lost without an error.

    The graph stores each parent as an rdfs:subClassOf statement. A study-site query with entailment: "subclass" then matches the active sites too. Use RDF import when you also need OWL axioms, labels and comments.

    import { LbbClient } from "@littlebigbrain/client";
    

    const lbb = new LbbClient({
      baseUrl: process.env.LBB_URL!,
      apiKey: process.env.LBB_API_KEY!,
      graph: "recall",
    });
    const graph = lbb.graph("recall");
    

    await graph.ontology.define({
      format: "spec",
      merge_default: false,
      source: JSON.stringify({
        entity_types: [
          { name: "IngredientLot" }, { name: "IntermediateLot" }, { name: "ReleasedBatch" },
          { name: "StudySite" }, { name: "ActiveStudySite", super_types: ["StudySite"] },
          { name: "RecallNotice" }, { name: "AgentDecision" },
        ],
        relation_types: [
          { name: "PRODUCED_FROM", source: ["IntermediateLot", "ReleasedBatch"], target: ["IngredientLot", "IntermediateLot"] },
          { name: "RECALLS", source: "RecallNotice", target: "IngredientLot", inverse: "RECALLED_BY" },
          { name: "RECEIVED_BATCH", source: ["StudySite", "ActiveStudySite"], target: "ReleasedBatch", inverse: "SHIPPED_TO" },
          { name: "DECIDED_ABOUT", source: "AgentDecision", target: "RecallNotice" },
        ],
      }),
    });
    

    await graph.ontology.evolve({
      ops: [
        { op: "add_property", name: "lot_number", value_type: "keyword" },
        { op: "add_property", name: "batch_number", value_type: "keyword" },
        { op: "add_property", name: "recall_date", value_type: "date_time" },
        { op: "add_property", name: "based_on_commit_seq", value_type: "i64" },
        { op: "add_property", name: "decision_outcome", value_type: "keyword",
          allowed_values: ["affected", "clear"] },
      ],
    });

    allowed_values is an enforced value set. A later write with decision_outcome: "maybe" is rejected at commit time. No rules file is needed. The message is:

    value for 'decision_outcome' is not one of its allowed values [affected, clear]

    inverse declares a second name for the same link. A query can ask SHIPPED_TO or RECEIVED_BATCH and read the same fact.

    GET /v1/ontology?graph=recall returns what the graph declares. Its property_defs list the field names and types a write may use.

  2. Write records. A commit carries triplets (links) and entity_properties (values). Give a record a key when a source system already identifies it. A rename then keeps the same record. Two records with the same display name stay distinct.

    Each triplet can carry a plain evidence string, the reason the fact was recorded.

    const lot = { type: "IngredientLot", name: "Valsartan API lot ZH-VAL-2018-07", key: "ZH-VAL-2018-07" };
    const granulation = { type: "IntermediateLot", name: "Granulation lot G-17", key: "G-17" };
    const batch = { type: "ReleasedBatch", name: "Cardiovex batch CVX-A17", key: "CVX-A17" };
    const notice = { type: "RecallNotice", name: "FDA valsartan recall" };
    

    const written = await graph.facts.create(
      {
        triplets: [
          { source: notice, relation: "RECALLS", target: lot, evidence: "FDA recall notice, 2018-07-13" },
          { source: granulation, relation: "PRODUCED_FROM", target: lot, evidence: "Batch record CVX-A17, page 3" },
          { source: batch, relation: "PRODUCED_FROM", target: granulation, evidence: "Batch record CVX-A17, page 9" },
          { source: { type: "ActiveStudySite", name: "Berlin Cardiology Study Site" },
            relation: "RECEIVED_BATCH", target: batch, evidence: "Shipment receipt, Berlin site" },
          { source: { type: "StudySite", name: "Oslo Cardiology Study Site" },
            relation: "RECEIVED_BATCH", target: batch, evidence: "Shipment receipt, Oslo site (closed 2018-03-31)" },
        ],
        entity_properties: [
          { ...lot, properties: { lot_number: "ZH-VAL-2018-07" } },
          { ...batch, properties: { batch_number: "CVX-A17" } },
          { ...notice, properties: { recall_date: "2018-07-13" } },
        ],
      },
      { idempotencyKey: "recall-load-1" },
    );
    await graph.waitForPublished(written.commit_seq);
    console.log(written.commit_seq);

    The response reports commit_seq. The code then waits until the commit is published. The reads in the next steps then see it.

    Field writes merge. A later commit that sends only recall_date keeps lot_number in place. A value of the wrong type fails the whole commit before anything is written. So does an unknown field or an unknown type.

    For a large load, stream the same records as newline-delimited JSON. Use the durable import job.

  3. Query a single hop without query text. The structured query takes link names and fields as JSON. This one asks which records were produced directly from the recalled lot:

    const direct = await lbb.entities.filterByAttributes({
      patterns: [{ subject: { var: "made" }, predicate: "PRODUCED_FROM", object: { var: "lot" } }],
      where: [{ var: "lot", field: "lot_number", value: "ZH-VAL-2018-07" }],
      select: ["made"],
    });

    That returns the granulation lot, one hop away, with its type and name. The released batch is two hops away, and a real genealogy can have many. SPARQL property paths follow a chain of unknown length. So the next step uses query text.

  4. Follow the chain with one SPARQL query. Each name you declared in JSON has a fixed IRI. A query needs three prefixes and nothing else:

    You declared SPARQL IRI Rule
    Entity type ReleasedBatch <https://littlebigbrain.com/class/releasedbatch> lowercase
    Relation PRODUCED_FROM <https://littlebigbrain.com/r/produced_from> lowercase
    Property batch_number <https://littlebigbrain.com/p/batch_number> lowercase
    An entity’s name rdfs:label plain string

    dr:produced_from+ means “one or more PRODUCED_FROM links in a row”. The query finds the released batches whose genealogy reaches the recalled lot. It also finds the active site that received each one:

    PREFIX dclass: <https://littlebigbrain.com/class/>
    PREFIX dr: <https://littlebigbrain.com/r/>
    PREFIX dp: <https://littlebigbrain.com/p/>
    PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
    

    SELECT ?batchNumber ?site WHERE {
      ?lot dp:lot_number "ZH-VAL-2018-07" .
      ?batch a dclass:releasedbatch ;
             dr:produced_from+ ?lot ;
             dp:batch_number ?batchNumber .
      OPTIONAL {
        ?siteNode a dclass:activestudysite ;
                  dr:received_batch ?batch ;
                  rdfs:label ?site .
      }
    }
    ORDER BY ?batchNumber ?site

    In the code below, genealogyQuery (genealogy_query in Python) holds this query text. Pass the write’s commit_seq as the floor. The response names the commit it answered from, as snapshot.served_at_seq.

    Publication may not cover the floor yet. Then the server answers with a retryable 429 and a Retry-After. The SDKs wait and retry. With cURL, --retry does the same.

    const exposure = await lbb.sparqlRows(
      { query: genealogyQuery },
      { consistency: "eventual", minIndexedSeq: written.commit_seq },
    );
    const readAt = exposure.snapshot!.served_at_seq!;   // keep this for step 6

    The result is one row: batch CVX-A17, site Berlin Cardiology Study Site. The Oslo site received the same batch. It is a StudySite, so the activestudysite pattern leaves it out. That is the reportable answer.

    A strong read is exact as soon as the commit is acknowledged. Pass consistency: "strong" with the same floor. Its response has no snapshot block.

    Use it when you need the answer now. Use the eventual read with a floor when you must record which commit you read.

  5. Validate writes. allowed_values already guards one field. For rules across fields and links, publish SHACL shapes. This is the one Turtle document in the guide. It uses the same three prefixes:

    @prefix sh: <http://www.w3.org/ns/shacl#> .
    @prefix dclass: <https://littlebigbrain.com/class/> .
    @prefix dr: <https://littlebigbrain.com/r/> .
    @prefix dp: <https://littlebigbrain.com/p/> .
    

    <https://example.org/shapes/ReleasedBatch> a sh:NodeShape ;
      sh:targetClass dclass:releasedbatch ;
      sh:property [ sh:path dp:batch_number ; sh:maxCount 1 ;
                    sh:message "A released batch carries at most one batch number." ] ;
      sh:property [ sh:path dr:produced_from ; sh:minCount 1 ;
                    sh:message "A released batch names the lot it was produced from." ] .

    Publish in warn mode first and read the conformance report. Then publish the same source in reject mode. In the code, shapesTurtle (shapes_turtle in Python) holds the Turtle document above.

    await graph.schema.publish({ desired_mode: "warn", shapes: { source: shapesTurtle, format: "turtle" } });
    const report = await graph.ontology.conformance({ consistency: "strong" });
    if (report.conforms) {
      await graph.schema.publish({ desired_mode: "reject", shapes: { source: shapesTurtle, format: "turtle" } });
    }

    The strong conformance read returns a retryable 429 until the report for the new shapes exists. Then it returns conforms, result_count and validated_at_seq.

    In reject mode, a write that breaks a shape returns HTTP 400. Nothing is stored. A shipment to a new batch that names no lot fails like this:

    schema validation failed in reject mode with 1 SHACL violation(s):
    http://www.w3.org/ns/shacl#MinCountConstraintComponent
    on '<https://littlebigbrain.com/e/4b3d…>' path '<https://littlebigbrain.com/r/produced_from>':
    A released batch names the lot it was produced from.
    Preflight with dry_run=true to get the full structured report (schema_validation.results)

    To get that report as data before a real write, use a dry run. Send the same body to commitDryRun / commit_dry_run.

    Then schema_validation.conforms is false. schema_validation.results lists each violation with its component, focus_node, result_path and message. An agent can read that list, fix its own output and retry.

  6. Record the decision and replay it later. An agent that acts on a query result should store what it read. Write the decision as a record. Set its based_on_commit_seq to the served_at_seq from step 4. Link it to the notice it was about:

    const decision = { type: "AgentDecision", name: "Valsartan lot impact assessment", key: "decision-2018-07-13-01" };
    await graph.facts.create(
      {
        triplets: [{ source: decision, relation: "DECIDED_ABOUT", target: notice,
                     evidence: "genealogy query, batch CVX-A17 and Berlin site exposed" }],
        entity_properties: [{ ...decision, properties: { based_on_commit_seq: readAt, decision_outcome: "affected" } }],
      },
      { idempotencyKey: "decision-2018-07-13-01" },
    );
    

    // Weeks later: rerun the same query against the state the agent read.
    const replay = await lbb.sparqlRows({ query: genealogyQuery, as_of_commit_seq: readAt });

    Suppose a later commit retracts the Berlin shipment. The receipt was filed against the wrong site. A query of the latest data then returns batch CVX-A17 with no site.

    The replay still returns CVX-A17 with Berlin Cardiology Study Site. Its snapshot echoes as_of_commit_seq. The decision and the rows it rested on stay together.

    Reads of a retained commit are exact. A commit may no longer be retained. Then the read fails with historical_generation_unavailable. It never returns a nearby state. Replay an answer covers retention and the change log.

Where the JSON path stops

Section titled “Where the JSON path stops”

The type hierarchy stays on the JSON path. Step 1 gives ActiveStudySite the parent StudySite. The graph stores that link as an rdfs:subClassOf statement. A query for dclass:studysite with entailment: "subclass" also matches the active sites. See Reasoning & inference.

Three things need RDF statements. Each is optional, and each works on the graph you built above without changing it.

  • OWL axioms. OWL can declare that a batch with a recalled lot has a reportable exposure. One plain pattern then returns the answer. The JSON schema has no field for such rules, so load them with RDF import. See Answer a question with OWL.
  • Records that already are RDF. A dump from another triple store loads into the same graph with facts.importRdf. Its statements keep their source predicates, next to the JSON-declared ones.
  • Load the JSON records first and the RDF second. A graph whose first write is RDF is RDF-native. It refuses the JSON path with 400 rdf_native_graph. See Load RDF data.
  • Standard tooling. YASGUI, Protégé and RDFLib connect to the graph’s /sparql endpoint with the stack API key. A CONSTRUCT query there returns Turtle. That is the export path for the whole graph.
Section titled “Related”
  • Use cases: five more agent decisions with the same modeling and query patterns.
  • Coming from SQL: the mapping from tables and joins to types and links.
  • Query & validate: the full structured query surface and SHACL enforcement modes.
  • HTTP API: every route this guide called.