# Build with JSON records Source: https://docs.littlebigbrain.com/guides/without-rdf/ You can build a complete application on little big brain with JSON. This guide goes further than the [quickstart](https://docs.littlebigbrain.com/start/quickstart/). It adds typed fields, a chain of links of any length, rules and a replayed decision. Underneath, each record is stored as RDF and each graph serves SPARQL. You do not need to know RDF for this guide. The one query and the one rules document each take a few lines. The running example is a medicine recall. A supplier recalls one ingredient lot. Which released batches contain material from that lot? Which active study sites received them? The answer runs through intermediate lots. It is a chain of links, so one join cannot answer it. The four terms below are the whole vocabulary you need. The API calls them entity types, entities, relations and properties. | Term | What it is | JSON example | | --- | --- | --- | | Type | A kind of record | `ReleasedBatch` | | Record | One thing, identified by its type plus a `key` or a `name` | `{ "type": "ReleasedBatch", "name": "Cardiovex batch CVX-A17", "key": "CVX-A17" }` | | Link | A named connection from one record to another | `PRODUCED_FROM` | | Field | A typed value on one record | `"batch_number": "CVX-A17"` | 1. **Declare the schema.** A graph accepts only the types, links and fields it declares, so declare them first. One `define` call creates the graph with its types and links. One `evolve` call adds the typed fields. Both calls are safe to repeat. A type can include `super_types: ["ParentClass"]`. Here an `ActiveStudySite` is a kind of `StudySite`. Declare each parent in the same document, in any order. Unknown fields and unknown parents are rejected, so the hierarchy is never lost without an error. The graph stores each parent as an `rdfs:subClassOf` statement. A study-site query with `entailment: "subclass"` then matches the active sites too. Use RDF import when you also need OWL axioms, labels and comments. ### TypeScript ```ts import { LbbClient } from "@littlebigbrain/client"; const lbb = new LbbClient({ baseUrl: process.env.LBB_URL!, apiKey: process.env.LBB_API_KEY!, graph: "recall", }); const graph = lbb.graph("recall"); await graph.ontology.define({ format: "spec", merge_default: false, source: JSON.stringify({ entity_types: [ { name: "IngredientLot" }, { name: "IntermediateLot" }, { name: "ReleasedBatch" }, { name: "StudySite" }, { name: "ActiveStudySite", super_types: ["StudySite"] }, { name: "RecallNotice" }, { name: "AgentDecision" }, ], relation_types: [ { name: "PRODUCED_FROM", source: ["IntermediateLot", "ReleasedBatch"], target: ["IngredientLot", "IntermediateLot"] }, { name: "RECALLS", source: "RecallNotice", target: "IngredientLot", inverse: "RECALLED_BY" }, { name: "RECEIVED_BATCH", source: ["StudySite", "ActiveStudySite"], target: "ReleasedBatch", inverse: "SHIPPED_TO" }, { name: "DECIDED_ABOUT", source: "AgentDecision", target: "RecallNotice" }, ], }), }); await graph.ontology.evolve({ ops: [ { op: "add_property", name: "lot_number", value_type: "keyword" }, { op: "add_property", name: "batch_number", value_type: "keyword" }, { op: "add_property", name: "recall_date", value_type: "date_time" }, { op: "add_property", name: "based_on_commit_seq", value_type: "i64" }, { op: "add_property", name: "decision_outcome", value_type: "keyword", allowed_values: ["affected", "clear"] }, ], }); ``` ### Python ```python import json, os from lbb import LbbClient lbb = LbbClient(os.environ["LBB_URL"], api_key=os.environ["LBB_API_KEY"], graph="recall") graph = lbb.graph("recall") lbb.ontology.define({ "format": "spec", "merge_default": False, "source": json.dumps({ "entity_types": [ {"name": "IngredientLot"}, {"name": "IntermediateLot"}, {"name": "ReleasedBatch"}, {"name": "StudySite"}, {"name": "ActiveStudySite", "super_types": ["StudySite"]}, {"name": "RecallNotice"}, {"name": "AgentDecision"}, ], "relation_types": [ {"name": "PRODUCED_FROM", "source": ["IntermediateLot", "ReleasedBatch"], "target": ["IngredientLot", "IntermediateLot"]}, {"name": "RECALLS", "source": "RecallNotice", "target": "IngredientLot", "inverse": "RECALLED_BY"}, {"name": "RECEIVED_BATCH", "source": ["StudySite", "ActiveStudySite"], "target": "ReleasedBatch", "inverse": "SHIPPED_TO"}, {"name": "DECIDED_ABOUT", "source": "AgentDecision", "target": "RecallNotice"}, ], }), }) lbb.ontology.evolve({ "ops": [ {"op": "add_property", "name": "lot_number", "value_type": "keyword"}, {"op": "add_property", "name": "batch_number", "value_type": "keyword"}, {"op": "add_property", "name": "recall_date", "value_type": "date_time"}, {"op": "add_property", "name": "based_on_commit_seq", "value_type": "i64"}, {"op": "add_property", "name": "decision_outcome", "value_type": "keyword", "allowed_values": ["affected", "clear"]}, ], }) ``` ### cURL ```sh curl --fail-with-body -sS -X POST "$LBB_URL/v1/ontology/define?graph=recall" \ -H "Authorization: Bearer $LBB_API_KEY" -H "Content-Type: application/json" \ -d '{ "format": "spec", "merge_default": false, "source": "{\"entity_types\":[{\"name\":\"IngredientLot\"},{\"name\":\"IntermediateLot\"},{\"name\":\"ReleasedBatch\"},{\"name\":\"StudySite\"},{\"name\":\"ActiveStudySite\",\"super_types\":[\"StudySite\"]},{\"name\":\"RecallNotice\"},{\"name\":\"AgentDecision\"}],\"relation_types\":[{\"name\":\"PRODUCED_FROM\",\"source\":[\"IntermediateLot\",\"ReleasedBatch\"],\"target\":[\"IngredientLot\",\"IntermediateLot\"]},{\"name\":\"RECALLS\",\"source\":\"RecallNotice\",\"target\":\"IngredientLot\",\"inverse\":\"RECALLED_BY\"},{\"name\":\"RECEIVED_BATCH\",\"source\":[\"StudySite\",\"ActiveStudySite\"],\"target\":\"ReleasedBatch\",\"inverse\":\"SHIPPED_TO\"},{\"name\":\"DECIDED_ABOUT\",\"source\":\"AgentDecision\",\"target\":\"RecallNotice\"}]}" }' curl --fail-with-body -sS -X POST "$LBB_URL/v1/ontology/evolve?graph=recall" \ -H "Authorization: Bearer $LBB_API_KEY" -H "Content-Type: application/json" \ -d '{"ops": [ {"op": "add_property", "name": "lot_number", "value_type": "keyword"}, {"op": "add_property", "name": "batch_number", "value_type": "keyword"}, {"op": "add_property", "name": "recall_date", "value_type": "date_time"}, {"op": "add_property", "name": "based_on_commit_seq", "value_type": "i64"}, {"op": "add_property", "name": "decision_outcome", "value_type": "keyword", "allowed_values": ["affected", "clear"]} ]}' ``` `allowed_values` is an enforced value set. A later write with `decision_outcome: "maybe"` is rejected at commit time. No rules file is needed. The message is: ```text value for 'decision_outcome' is not one of its allowed values [affected, clear] ``` `inverse` declares a second name for the same link. A query can ask `SHIPPED_TO` or `RECEIVED_BATCH` and read the same fact. `GET /v1/ontology?graph=recall` returns what the graph declares. Its `property_defs` list the field names and types a write may use. 2. **Write records.** A commit carries `triplets` (links) and `entity_properties` (values). Give a record a `key` when a source system already identifies it. A rename then keeps the same record. Two records with the same display name stay distinct. Each triplet can carry a plain `evidence` string, the reason the fact was recorded. ### TypeScript ```ts const lot = { type: "IngredientLot", name: "Valsartan API lot ZH-VAL-2018-07", key: "ZH-VAL-2018-07" }; const granulation = { type: "IntermediateLot", name: "Granulation lot G-17", key: "G-17" }; const batch = { type: "ReleasedBatch", name: "Cardiovex batch CVX-A17", key: "CVX-A17" }; const notice = { type: "RecallNotice", name: "FDA valsartan recall" }; const written = await graph.facts.create( { triplets: [ { source: notice, relation: "RECALLS", target: lot, evidence: "FDA recall notice, 2018-07-13" }, { source: granulation, relation: "PRODUCED_FROM", target: lot, evidence: "Batch record CVX-A17, page 3" }, { source: batch, relation: "PRODUCED_FROM", target: granulation, evidence: "Batch record CVX-A17, page 9" }, { source: { type: "ActiveStudySite", name: "Berlin Cardiology Study Site" }, relation: "RECEIVED_BATCH", target: batch, evidence: "Shipment receipt, Berlin site" }, { source: { type: "StudySite", name: "Oslo Cardiology Study Site" }, relation: "RECEIVED_BATCH", target: batch, evidence: "Shipment receipt, Oslo site (closed 2018-03-31)" }, ], entity_properties: [ { ...lot, properties: { lot_number: "ZH-VAL-2018-07" } }, { ...batch, properties: { batch_number: "CVX-A17" } }, { ...notice, properties: { recall_date: "2018-07-13" } }, ], }, { idempotencyKey: "recall-load-1" }, ); await graph.waitForPublished(written.commit_seq); console.log(written.commit_seq); ``` ### Python ```python lot = {"type": "IngredientLot", "name": "Valsartan API lot ZH-VAL-2018-07", "key": "ZH-VAL-2018-07"} granulation = {"type": "IntermediateLot", "name": "Granulation lot G-17", "key": "G-17"} batch = {"type": "ReleasedBatch", "name": "Cardiovex batch CVX-A17", "key": "CVX-A17"} notice = {"type": "RecallNotice", "name": "FDA valsartan recall"} written = graph.facts.create({ "triplets": [ {"source": notice, "relation": "RECALLS", "target": lot, "evidence": "FDA recall notice, 2018-07-13"}, {"source": granulation, "relation": "PRODUCED_FROM", "target": lot, "evidence": "Batch record CVX-A17, page 3"}, {"source": batch, "relation": "PRODUCED_FROM", "target": granulation, "evidence": "Batch record CVX-A17, page 9"}, {"source": {"type": "ActiveStudySite", "name": "Berlin Cardiology Study Site"}, "relation": "RECEIVED_BATCH", "target": batch, "evidence": "Shipment receipt, Berlin site"}, {"source": {"type": "StudySite", "name": "Oslo Cardiology Study Site"}, "relation": "RECEIVED_BATCH", "target": batch, "evidence": "Shipment receipt, Oslo site (closed 2018-03-31)"}, ], "entity_properties": [ {**lot, "properties": {"lot_number": "ZH-VAL-2018-07"}}, {**batch, "properties": {"batch_number": "CVX-A17"}}, {**notice, "properties": {"recall_date": "2018-07-13"}}, ], }, idempotency_key="recall-load-1") graph.wait_for_published(written["commit_seq"]) print(written["commit_seq"]) ``` ### cURL ```sh curl --fail-with-body -sS -X POST "$LBB_URL/v1/graph/commit?graph=recall" \ -H "Authorization: Bearer $LBB_API_KEY" -H "Idempotency-Key: recall-load-1" \ -H "Content-Type: application/json" \ -d '{ "triplets": [ {"source": {"type": "RecallNotice", "name": "FDA valsartan recall"}, "relation": "RECALLS", "target": {"type": "IngredientLot", "name": "Valsartan API lot ZH-VAL-2018-07", "key": "ZH-VAL-2018-07"}, "evidence": "FDA recall notice, 2018-07-13"}, {"source": {"type": "IntermediateLot", "name": "Granulation lot G-17", "key": "G-17"}, "relation": "PRODUCED_FROM", "target": {"type": "IngredientLot", "name": "Valsartan API lot ZH-VAL-2018-07", "key": "ZH-VAL-2018-07"}, "evidence": "Batch record CVX-A17, page 3"}, {"source": {"type": "ReleasedBatch", "name": "Cardiovex batch CVX-A17", "key": "CVX-A17"}, "relation": "PRODUCED_FROM", "target": {"type": "IntermediateLot", "name": "Granulation lot G-17", "key": "G-17"}, "evidence": "Batch record CVX-A17, page 9"}, {"source": {"type": "ActiveStudySite", "name": "Berlin Cardiology Study Site"}, "relation": "RECEIVED_BATCH", "target": {"type": "ReleasedBatch", "name": "Cardiovex batch CVX-A17", "key": "CVX-A17"}, "evidence": "Shipment receipt, Berlin site"}, {"source": {"type": "StudySite", "name": "Oslo Cardiology Study Site"}, "relation": "RECEIVED_BATCH", "target": {"type": "ReleasedBatch", "name": "Cardiovex batch CVX-A17", "key": "CVX-A17"}, "evidence": "Shipment receipt, Oslo site (closed 2018-03-31)"} ], "entity_properties": [ {"type": "IngredientLot", "name": "Valsartan API lot ZH-VAL-2018-07", "key": "ZH-VAL-2018-07", "properties": {"lot_number": "ZH-VAL-2018-07"}}, {"type": "ReleasedBatch", "name": "Cardiovex batch CVX-A17", "key": "CVX-A17", "properties": {"batch_number": "CVX-A17"}}, {"type": "RecallNotice", "name": "FDA valsartan recall", "properties": {"recall_date": "2018-07-13"}} ] }' ``` The response reports `commit_seq`. The code then waits until the commit is published. The reads in the next steps then see it. Field writes merge. A later commit that sends only `recall_date` keeps `lot_number` in place. A value of the wrong type fails the whole commit before anything is written. So does an unknown field or an unknown type. For a large load, stream the same records as newline-delimited JSON. Use the [durable import job](https://docs.littlebigbrain.com/sdks/http-api/#bulk-ndjson-ingest). 3. **Query a single hop without query text.** The structured query takes link names and fields as JSON. This one asks which records were produced directly from the recalled lot: ### TypeScript ```ts const direct = await lbb.entities.filterByAttributes({ patterns: [{ subject: { var: "made" }, predicate: "PRODUCED_FROM", object: { var: "lot" } }], where: [{ var: "lot", field: "lot_number", value: "ZH-VAL-2018-07" }], select: ["made"], }); ``` ### Python ```python direct = lbb.entities.filter_by_attributes( patterns=[{"subject": {"var": "made"}, "predicate": "PRODUCED_FROM", "object": {"var": "lot"}}], where=[{"var": "lot", "field": "lot_number", "value": "ZH-VAL-2018-07"}], select=["made"], ) ``` That returns the granulation lot, one hop away, with its type and name. The released batch is two hops away, and a real genealogy can have many. SPARQL property paths follow a chain of unknown length. So the next step uses query text. 4. **Follow the chain with one SPARQL query.** Each name you declared in JSON has a fixed IRI. A query needs three prefixes and nothing else: | You declared | SPARQL IRI | Rule | | --- | --- | --- | | Entity type `ReleasedBatch` | `` | lowercase | | Relation `PRODUCED_FROM` | `` | lowercase | | Property `batch_number` | `` | lowercase | | An entity's `name` | `rdfs:label` | plain string | `dr:produced_from+` means "one or more `PRODUCED_FROM` links in a row". The query finds the released batches whose genealogy reaches the recalled lot. It also finds the active site that received each one: ```sparql PREFIX dclass: PREFIX dr: PREFIX dp: PREFIX rdfs: SELECT ?batchNumber ?site WHERE { ?lot dp:lot_number "ZH-VAL-2018-07" . ?batch a dclass:releasedbatch ; dr:produced_from+ ?lot ; dp:batch_number ?batchNumber . OPTIONAL { ?siteNode a dclass:activestudysite ; dr:received_batch ?batch ; rdfs:label ?site . } } ORDER BY ?batchNumber ?site ``` In the code below, `genealogyQuery` (`genealogy_query` in Python) holds this query text. Pass the write's `commit_seq` as the floor. The response names the commit it answered from, as `snapshot.served_at_seq`. Publication may not cover the floor yet. Then the server answers with a retryable `429` and a `Retry-After`. The SDKs wait and retry. With cURL, `--retry` does the same. ### TypeScript ```ts const exposure = await lbb.sparqlRows( { query: genealogyQuery }, { consistency: "eventual", minIndexedSeq: written.commit_seq }, ); const readAt = exposure.snapshot!.served_at_seq!; // keep this for step 6 ``` ### Python ```python exposure = lbb.sparql(genealogy_query, consistency="eventual", min_indexed_seq=written["commit_seq"]) read_at = exposure.snapshot["served_at_seq"] # keep this for step 6 ``` ### cURL ```sh curl --retry 10 --fail-with-body -sS -X POST \ "$LBB_URL/v1/query/sparql-text?graph=recall&consistency=eventual&min_indexed_seq=$COMMIT_SEQ" \ -H "Authorization: Bearer $LBB_API_KEY" -H "Content-Type: application/json" \ -d "{\"query\": $(jq -Rs . < genealogy.sparql)}" ``` The result is one row: batch `CVX-A17`, site `Berlin Cardiology Study Site`. The Oslo site received the same batch. It is a `StudySite`, so the `activestudysite` pattern leaves it out. That is the reportable answer. A strong read is exact as soon as the commit is acknowledged. Pass `consistency: "strong"` with the same floor. Its response has no `snapshot` block. Use it when you need the answer now. Use the eventual read with a floor when you must record which commit you read. 5. **Validate writes.** `allowed_values` already guards one field. For rules across fields and links, publish SHACL shapes. This is the one Turtle document in the guide. It uses the same three prefixes: ```turtle @prefix sh: . @prefix dclass: . @prefix dr: . @prefix dp: . a sh:NodeShape ; sh:targetClass dclass:releasedbatch ; sh:property [ sh:path dp:batch_number ; sh:maxCount 1 ; sh:message "A released batch carries at most one batch number." ] ; sh:property [ sh:path dr:produced_from ; sh:minCount 1 ; sh:message "A released batch names the lot it was produced from." ] . ``` Publish in `warn` mode first and read the conformance report. Then publish the same source in `reject` mode. In the code, `shapesTurtle` (`shapes_turtle` in Python) holds the Turtle document above. ### TypeScript ```ts await graph.schema.publish({ desired_mode: "warn", shapes: { source: shapesTurtle, format: "turtle" } }); const report = await graph.ontology.conformance({ consistency: "strong" }); if (report.conforms) { await graph.schema.publish({ desired_mode: "reject", shapes: { source: shapesTurtle, format: "turtle" } }); } ``` ### Python ```python lbb.schema.publish({"desired_mode": "warn", "shapes": {"source": shapes_turtle, "format": "turtle"}}) report = lbb.ontology.conformance(consistency="strong") if report.conforms: lbb.schema.publish({"desired_mode": "reject", "shapes": {"source": shapes_turtle, "format": "turtle"}}) ``` The strong conformance read returns a retryable `429` until the report for the new shapes exists. Then it returns `conforms`, `result_count` and `validated_at_seq`. In `reject` mode, a write that breaks a shape returns HTTP 400. Nothing is stored. A shipment to a new batch that names no lot fails like this: ```text schema validation failed in reject mode with 1 SHACL violation(s): http://www.w3.org/ns/shacl#MinCountConstraintComponent on '' path '': A released batch names the lot it was produced from. Preflight with dry_run=true to get the full structured report (schema_validation.results) ``` To get that report as data before a real write, use a dry run. Send the same body to `commitDryRun` / `commit_dry_run`. Then `schema_validation.conforms` is `false`. `schema_validation.results` lists each violation with its `component`, `focus_node`, `result_path` and `message`. An agent can read that list, fix its own output and retry. :::note[Put required links in the same commit] With `sh:minCount` on `dr:produced_from`, a new released batch needs its first `PRODUCED_FROM` link. Both must arrive in one commit. A batch written alone is a violation the moment it lands. ::: 6. **Record the decision and replay it later.** An agent that acts on a query result should store what it read. Write the decision as a record. Set its `based_on_commit_seq` to the `served_at_seq` from step 4. Link it to the notice it was about: ### TypeScript ```ts const decision = { type: "AgentDecision", name: "Valsartan lot impact assessment", key: "decision-2018-07-13-01" }; await graph.facts.create( { triplets: [{ source: decision, relation: "DECIDED_ABOUT", target: notice, evidence: "genealogy query, batch CVX-A17 and Berlin site exposed" }], entity_properties: [{ ...decision, properties: { based_on_commit_seq: readAt, decision_outcome: "affected" } }], }, { idempotencyKey: "decision-2018-07-13-01" }, ); // Weeks later: rerun the same query against the state the agent read. const replay = await lbb.sparqlRows({ query: genealogyQuery, as_of_commit_seq: readAt }); ``` ### Python ```python decision = {"type": "AgentDecision", "name": "Valsartan lot impact assessment", "key": "decision-2018-07-13-01"} graph.facts.create({ "triplets": [{"source": decision, "relation": "DECIDED_ABOUT", "target": notice, "evidence": "genealogy query, batch CVX-A17 and Berlin site exposed"}], "entity_properties": [{**decision, "properties": {"based_on_commit_seq": read_at, "decision_outcome": "affected"}}], }, idempotency_key="decision-2018-07-13-01") # Weeks later: rerun the same query against the state the agent read. replay = lbb.sparql(genealogy_query, as_of_commit_seq=read_at) ``` Suppose a later commit retracts the Berlin shipment. The receipt was filed against the wrong site. A query of the latest data then returns batch `CVX-A17` with no site. The replay still returns `CVX-A17` with `Berlin Cardiology Study Site`. Its `snapshot` echoes `as_of_commit_seq`. The decision and the rows it rested on stay together. Reads of a retained commit are exact. A commit may no longer be retained. Then the read fails with `historical_generation_unavailable`. It never returns a nearby state. [Replay an answer](https://docs.littlebigbrain.com/guides/time-travel-audit/) covers retention and the change log. ## Where the JSON path stops The type hierarchy stays on the JSON path. Step 1 gives `ActiveStudySite` the parent `StudySite`. The graph stores that link as an `rdfs:subClassOf` statement. A query for `dclass:studysite` with `entailment: "subclass"` also matches the active sites. See [Reasoning & inference](https://docs.littlebigbrain.com/guides/reasoning/#add-a-type-hierarchy-to-json-records). Three things need RDF statements. Each is optional, and each works on the graph you built above without changing it. - **OWL axioms.** OWL can declare that a batch with a recalled lot has a reportable exposure. One plain pattern then returns the answer. The JSON schema has no field for such rules, so load them with [RDF import](https://docs.littlebigbrain.com/guides/load-rdf/). See [Answer a question with OWL](https://docs.littlebigbrain.com/guides/reasoning/#answer-a-question-with-owl). - **Records that already are RDF.** A dump from another triple store loads into the same graph with `facts.importRdf`. Its statements keep their source predicates, next to the JSON-declared ones. - Load the JSON records first and the RDF second. A graph whose **first** write is RDF is RDF-native. It refuses the JSON path with `400 rdf_native_graph`. See [Load RDF data](https://docs.littlebigbrain.com/guides/load-rdf/). - **Standard tooling.** YASGUI, Protégé and RDFLib connect to the graph's `/sparql` endpoint with the stack API key. A `CONSTRUCT` query there returns Turtle. That is the export path for the whole graph. ## Related - [Use cases](https://docs.littlebigbrain.com/start/use-cases/): five more agent decisions with the same modeling and query patterns. - [Coming from SQL](https://docs.littlebigbrain.com/start/database-comparison/#if-you-use-a-relational-database): the mapping from tables and joins to types and links. - [Query & validate](https://docs.littlebigbrain.com/guides/sparql-and-shacl/): the full structured query surface and SHACL enforcement modes. - [HTTP API](https://docs.littlebigbrain.com/sdks/http-api/): every route this guide called.