Concepts
This document explains what Linked Data Store (LDS) is and the ideas it is built on: document, entity type, namespace, id, version and valid time, link, the three kinds of resource, model, provider, and what happens to a write. It is for people who use LDS, and it is the document to read before the references. A few requests are shown to make an idea concrete. The complete description of every request is in the REST API and the GraphQL API. Each term has one meaning, given in the glossary.
What LDS is
LDS is a server that keeps JSON documents and gives access to them over HTTP: a REST API that reads and writes, and a GraphQL API that reads. It stands between the applications that work with the documents and the database that keeps them. Four ideas make it what it is:
- A model. LDS reads a model when it starts. The model declares the entity types, the kinds of document there are, and what a document of each entity type holds. The addresses of the REST API and the queries of the GraphQL API follow from the model.
- Time. LDS is a time machine. A write adds a version of a document that holds from its valid time, and leaves the other versions as they are. A read is an as-of read: it gives the document as it was at the time asked for.
- Links. A document can point at other documents: a link is a typed reference. A link can be followed from both ends.
- Providers. LDS does not keep the documents itself. A persistence provider keeps them, and a search index provider finds them by text. Each is chosen by configuration, and the requests stay the same.
Version 3.0.0 of LDS is in development and not released. Where something does not work as a user would
expect, this document says so in a paragraph that starts with "Not finished:".
About the examples
The examples were run against LDS 3.0.0-SNAPSHOT with nothing configured but its port. LDS then has the example
model, with the entity types Person and Company, and keeps everything in memory.
Getting started tells how to start LDS that way.
The examples start with an LDS that has no documents. LDS in memory has none when it starts, so start it anew if it has the documents of another guide.
In the commands $LDS stands for the address of LDS. Set it first. This is the address when the port is not
changed:
LDS=http://localhost:9090The examples were run in the order they stand, on 20 November 2026, and a later example uses the documents of an
earlier one. LDS sends a JSON answer on one line; here it is shown formatted, with the fields in the order LDS
sent them. A GraphQL query is sent as application/graphql, the query itself as the body, so that it can be shown
formatted. A query with variables is sent as JSON.
Document, entity, namespace and id
A document is a JSON object: what LDS keeps and gives back.
An entity type is a kind of document, named in the model: Person, Company. Every
document belongs to one entity type. The name is written as the model writes it, and upper and lower case letters count:
Company is an entity type of the example model, company is not. In an address its place is {Entity}.
A namespace is the first part of every address of a document. One LDS serves one
namespace. Its name is the value of the configuration key namespace.default, which is ns when nothing else is set.
An address with another namespace, such as /other/Company/acme, is answered 400 with
Unsupported namespace: "other". The namespace is kept with every document, and with PostgreSQL and Neo4j one database
can hold the documents of several namespaces.
An id names one document among the documents of its entity type. The one who writes the document chooses the id. LDS does not make ids.
Together they are the document key, and the address of a document:
/{namespace}/{Entity}/{id}| Part | In /ns/Company/acme |
What it is |
|---|---|---|
| namespace | ns |
The namespace the running LDS serves. |
| entity | Company |
The entity type, named as the model names it. |
| id | acme |
Chosen by the one who writes the document. |
Write a document of the entity type Company with the id acme, and read it:
curl -X PUT "$LDS/ns/Company/acme" -H 'Content-Type: application/json' -d '{"name": "Acme"}'{"saga-execution-id": "01a1f9de-1074-7733-a1ad-9ebe08cc9a57"}curl "$LDS/ns/Company/acme"{"name": "Acme"}The write is answered 201. Its answer is not the document. It is the id of the saga that carried the write,
which is explained in What a write goes through.
Not finished: a write with a body that LDS cannot read is answered 500 with no body. That is the answer to a
body that is not JSON. It is also the answer to curl -d without the header Content-Type: application/json,
because curl then names another content type.
LDS gives a document back with the values it was written with. Two things can differ. The fields can come in
another order than they were written in. A whole number that does not fit in 64 bits, and a decimal number with
more digits than a 64-bit floating point number holds, come back changed: 12345678901234567890 was read back as
1.2345678901234567E19.
Version and time
LDS is a time machine. It does not keep one state of a document, it keeps the document through time:
- A write does not change the document that is there. It adds a version: the whole document as it is from its valid time on.
- The valid time comes with the write, in the query parameter
timestamp. A write without it gets the time at which LDS receives the request. - The valid time can be in the past, the present or the future. A version for a time in the future is kept now, and becomes the document that is read when that time has come.
- A version holds from its valid time until the valid time of the next version.
- An as-of read gives the version that is valid at the time asked for: the latest version whose valid time is not after it. A read without a time is as of now. At a time before the first version there is no document.
- Versions can be written in any order of valid time. A write at the exact valid time of a version that is there replaces that version. No other version is changed.
Three writes of the company fjord, each with its valid time. The last one written is the one in the middle:
curl -X PUT "$LDS/ns/Company/fjord?timestamp=2020-01-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{"name": "Fjord Fish"}'
curl -X PUT "$LDS/ns/Company/fjord?timestamp=2030-01-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{"name": "Fjord Group"}'
curl -X PUT "$LDS/ns/Company/fjord?timestamp=2023-01-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{"name": "Fjord Seafood"}'Each was answered 201 with a saga-execution-id. What an as-of read gives depends on the time it asks for:
| Request | Answer |
|---|---|
curl "$LDS/ns/Company/fjord?timestamp=2019-06-01T00:00:00Z" |
404, no body |
curl "$LDS/ns/Company/fjord?timestamp=2021-06-01T00:00:00Z" |
{"name":"Fjord Fish"} |
curl "$LDS/ns/Company/fjord" (in November 2026) |
{"name":"Fjord Seafood"} |
curl "$LDS/ns/Company/fjord?timestamp=2031-06-01T00:00:00Z" |
{"name":"Fjord Group"} |
On the timeline:
%%{init: {"gantt": {"leftPadding": 150, "fontSize": 15, "sectionFontSize": 15, "barHeight": 28, "barGap": 8}}}%%
gantt
title Company/fjord on the timeline
dateFormat YYYY-MM-DD
axisFormat %Y
todayMarker off
section Versions
Fjord Fish, written 1st : 2020-01-01, 2023-01-01
Fjord Seafood, written 3rd : 2023-01-01, 2030-01-01
Fjord Group, written 2nd : 2030-01-01, 2033-01-01
section A read at
2019-06 gives 404 : milestone, 2019-06-01, 0d
2021-06 gives Fjord Fish : milestone, 2021-06-01, 0d
now (2026-11) gives Fjord Seafood : milestone, 2026-11-20, 0d
2031-06 gives Fjord Group : milestone, 2031-06-01, 0d
Every version of a document, ordered by valid time, is its timeline:
curl "$LDS/ns/Company/fjord?timeline"[
{
"version": "2020-01-01T00:00Z[Etc/UTC]",
"document": {"name": "Fjord Fish"}
},
{
"version": "2023-01-01T00:00Z[Etc/UTC]",
"document": {"name": "Fjord Seafood"}
},
{
"version": "2030-01-01T00:00Z[Etc/UTC]",
"document": {"name": "Fjord Group"}
}
]A time is written with an offset or with Z, as in 2020-01-01T00:00:00Z. A time without one is refused with
400. LDS keeps the valid time of a version in UTC, so the same instant written with another offset is the same
version: a write to fjord with timestamp=2020-01-01T01:00:00%2B01:00 replaced the first version, and the
timeline still had three. (In an address the plus sign of an offset is written %2B.)
The GraphQL API makes as-of reads too. There the time is given once for a whole query, in the query parameter
snapshot, and every document the query reaches is read as of that time.
Not finished: the GraphQL API does not read a time as the REST API does. A snapshot that is not a time, or that
has no offset, is not refused: the query is answered as of now. And with LDS in memory, a snapshot with another
offset than Z gave a document at a time before its first version, where the REST API answered 404 for the
same time. Write a snapshot with Z.
A delete in time
A delete does not take a document away. It adds a version, a tombstone, that says: from
this time the document is deleted. The versions before that time stay and an as-of read still finds them at their times,
and so do the versions after it. From the time of the delete until the next version, the document is not there: a read
of it is answered 404, and it is not in the list of the documents of its entity type.
Delete fjord from the start of 2025:
curl -X DELETE "$LDS/ns/Company/fjord?timestamp=2025-01-01T00:00:00Z"The answer is 204 with no body. After it:
| Request | Answer |
|---|---|
curl "$LDS/ns/Company/fjord?timestamp=2024-06-01T00:00:00Z" |
{"name":"Fjord Seafood"} |
curl "$LDS/ns/Company/fjord" (in November 2026) |
404, no body |
curl "$LDS/ns/Company/fjord?timestamp=2031-06-01T00:00:00Z" |
{"name":"Fjord Group"} |
On the timeline:
%%{init: {"gantt": {"leftPadding": 150, "fontSize": 15, "sectionFontSize": 15, "barHeight": 28, "barGap": 8}}}%%
gantt
title Company/fjord with a delete from 2025
dateFormat YYYY-MM-DD
axisFormat %Y
todayMarker off
section Versions
Fjord Fish : 2020-01-01, 2023-01-01
Fjord Seafood : 2023-01-01, 2025-01-01
deleted : crit, 2025-01-01, 2030-01-01
Fjord Group : 2030-01-01, 2033-01-01
section A read at
2024-06 gives Fjord Seafood : milestone, 2024-06-01, 0d
now (2026-11) gives 404 : milestone, 2026-11-20, 0d
2031-06 gives Fjord Group : milestone, 2031-06-01, 0d
In the timeline the tombstone is a version with an empty document:
[
{
"version": "2020-01-01T00:00Z[Etc/UTC]",
"document": {"name": "Fjord Fish"}
},
{
"version": "2023-01-01T00:00Z[Etc/UTC]",
"document": {"name": "Fjord Seafood"}
},
{
"version": "2025-01-01T00:00Z[Etc/UTC]",
"document": {}
},
{
"version": "2030-01-01T00:00Z[Etc/UTC]",
"document": {"name": "Fjord Group"}
}
]A delete at the valid time of a version replaces that version, as a write does, and a write at the time of a delete replaces the tombstone. A delete is accepted for an id that was never written: the timeline of that id then has a tombstone and nothing else. No request of LDS takes a version away. A version changes only when a write or a delete at its exact valid time replaces it.
Link
A link is a typed reference from a field of a document to another document. The model says which fields are links,
and which entity types each of them may point at. In the document a link is a text of the form /Entity/id, for example
/Company/acme. A field holds one link, or a list of links. A link has no namespace: it points at a document of the
same namespace.
When a document is written, LDS checks that a link has this form and names an entity type that the field may link to. LDS does not check that the document it points at exists, so a link may be a dangling reference. This is on purpose:
- Documents can be written in any order. The one that points can come before the one it points at.
- Documents live in time. The document a link points at may get its first version at a later time than the document that points at it, and may be deleted for a while.
Not finished: a link inside a part of a document is not always checked. With a model that is a GraphQL schema, a
document was written whose link, in a part that is an element of a list, was the text not a link.
The model says what is checked.
A link is resolved when it is read, at the time of the read. It leads to the document it points at as that document was at that time, and it leads nowhere when there is no such document at that time.
Write a person whose employer is a company that does not exist:
curl -X PUT "$LDS/ns/Person/ola?timestamp=2020-01-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{
"name": "Ola",
"employer": "/Company/nordlys"
}'The write is answered 201. The REST API gives the link as the text it is. The GraphQL API resolves it, and here it
leads nowhere:
curl "$LDS/graphql" -H 'Content-Type: application/graphql' \
-d '{
PersonById(id: "ola") {
name
employer {
name
}
}
}'{
"data": {
"PersonById": {"name": "Ola", "employer": null}
}
}Now write the company, valid from 2021, and send the same query again:
curl -X PUT "$LDS/ns/Company/nordlys?timestamp=2021-01-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{"name": "Nordlys"}'The write is answered 201, and the query now gives:
{
"data": {
"PersonById": {
"name": "Ola",
"employer": {"name": "Nordlys"}
}
}
}The same query as of the middle of 2020, when Ola had the link and the company was not there yet:
curl "$LDS/graphql?snapshot=2020-06-01T00:00:00Z" -H 'Content-Type: application/graphql' \
-d '{
PersonById(id: "ola") {
name
employer {
name
}
}
}'{
"data": {
"PersonById": {"name": "Ola", "employer": null}
}
}A link can be read from the other end too, as a reverse link. The example model gives the
other end of employer the name employees, so a company can be asked who links to it:
curl "$LDS/graphql" -H 'Content-Type: application/graphql' \
-d '{
CompanyById(id: "nordlys") {
name
employees {
edges {
node {
name
}
}
}
}
}'{
"data": {
"CompanyById": {
"name": "Nordlys",
"employees": {
"edges": [
{
"node": {"name": "Ola"}
}
]
}
}
}
}On the timeline:
%%{init: {"gantt": {"leftPadding": 150, "fontSize": 15, "sectionFontSize": 15, "barHeight": 28, "barGap": 8}}}%%
gantt
title A link is followed at the time of the read
dateFormat YYYY-MM-DD
axisFormat %Y
todayMarker off
section Person/ola
employer /Company/nordlys : 2020-01-01, 2027-06-01
section Company/nordlys
Nordlys : 2021-01-01, 2027-06-01
section A query at
2020-06 gives employer null : milestone, 2020-06-01, 0d
now (2026-11) gives Nordlys : milestone, 2026-11-20, 0d
| A query at | ola → employer leads to |
Answer |
|---|---|---|
| 2020-06-01 | nordlys, which has no version yet |
"employer": null |
| now | nordlys as it is now |
"employer": {"name": "Nordlys"} |
Asked from the other end, nordlys has ola among its employees: that is the reverse link.
A delete does not look at links either. A document that other documents link to can be deleted. The links stay in those documents as they are, and lead nowhere from the time of the delete: they are dangling references.
The three kinds of resource
The REST API has three kinds of resource. All three are read with an as-of read and written at a valid time, as described above.
| Kind | Address | What it is |
|---|---|---|
| The whole document | /{namespace}/{Entity}/{id} |
One document. Without the id, the address is the list of the documents of the entity type. |
| A part of a document | /{namespace}/{Entity}/{id}/{field} |
One field of the document, or a field deeper in it, addressed by its path. It is a value, an embedded object or a list. |
| A link | /{namespace}/{Entity}/{id}/{link field}/{TargetEntity}/{target id} |
One link from the document to another document. It is there or it is not. |
In its own messages LDS calls them a managed resource, an embedded resource and a reference resource.
flowchart LR
subgraph kari["/ns/Person/kari — the whole document: read, written and deleted as one"]
direction TB
name["name: Kari"]:::value
address["address: {street: Storgata 1, city: Oslo}"]:::value
employer["employer: /Company/acme"]:::link
name ~~~ address ~~~ employer
end
part["<b>/ns/Person/kari/address</b><br/>a part of the document, by its path"]:::addr
link["<b>/ns/Person/kari/employer/Company/acme</b><br/>a link: there or not"]:::addr
part --> address
link --> employer
classDef value fill:#f1f5f9,stroke:#64748b,color:#0f172a
classDef link fill:#dbeafe,stroke:#2563eb,color:#0f172a
classDef addr fill:#ffffff,stroke:#334155,color:#0f172a
linkStyle 2,3 stroke:#64748b,stroke-width:2px
A write of a part or of a link is a write of the whole document:
flowchart LR
put["PUT /ns/Person/kari/employer/Company/acme"] --> read["read kari as it is<br/>at the time of the write"] --> change["put /Company/acme<br/>in employer"] --> write["write the whole of kari<br/>as a new version from that time"]
linkStyle default stroke:#64748b,stroke-width:2px
Write the person kari and read a part of that document:
curl -X PUT "$LDS/ns/Person/kari" -H 'Content-Type: application/json' \
-d '{
"name": "Kari",
"address": {"street": "Storgata 1", "city": "Oslo"}
}'
curl "$LDS/ns/Person/kari/address"{"saga-execution-id": "01a1f9de-1286-7220-912c-c551bc2ca161"}
{
"city": "Oslo",
"street": "Storgata 1"
}Then add a link to the document and read the whole document. (sync=true makes the answer to the write of the
link wait until the write is done; see What a write goes through.)
curl -X PUT "$LDS/ns/Person/kari/employer/Company/acme?sync=true"
curl "$LDS/ns/Person/kari"{"saga-execution-id": "01a1f9de-12a0-72ac-8c2a-d60619db82d6"}
{
"employer": "/Company/acme",
"name": "Kari",
"address": {"city": "Oslo", "street": "Storgata 1"}
}A write to a part or to a link is a write of the whole document. LDS reads the document as it is at the valid time of
the write, changes it, and writes it as a new version. After the write of the document and the write of the link, the
timeline of kari had two versions: one without the link and one with it. So a part or a link can be written only to a
document that is there at that time; otherwise the answer is 404 with no body.
Not finished: writes to parts and links of one document are not kept apart. Two that run at the same time each
read the document, and each writes it without the change of the other. With LDS in memory, 20 links added to one
document at the same time were all answered 200, and the document then had 12 of them. Sent one after the
other, all 20 were there.
Not finished: a path does not reach every field of a document. With the example model,
/ns/Person/kari/address/city is answered 400 with Not a valid path element: "city".
Which paths a model gives, and every method and answer, is in the REST API.
The model
The model says what LDS keeps: the entity types, the fields of the documents of each entity type and the kind of value of each field, the fields a document must have, and which fields are links and to which entity types. A model is written in one of two forms:
- a GraphQL schema, in one file;
- JSON Schema 2020-12 files, one file for each entity type.
This is the entity type Person of the example model, a GraphQL schema. @domain makes the type an entity type.
! marks a field that a document must have. @link makes a field a link: employer is one link to a Company,
friends is a list of links to documents of Person. Address is a type without @domain, an embedded type: it
is not an entity type, and its values are embedded objects, parts of the documents they are used in.
type Person @domain {
name: String!
born: DateTime
address: Address
employer: Company @link(reverseName: "employees")
friends: [Person] @link
}What a field holds
This is the heart of a model. A field of a document holds one of three things:
flowchart LR
subgraph ola["Person/ola — a document: an id, an address and versions of its own"]
direction TB
values["<b>values</b><br/>name: Ola<br/>born: 1990-05-17T00:00:00Z"]:::value
subgraph address["address — an embedded object, part of ola"]
addr["street: Storgata 1<br/>city: Oslo"]:::value
end
links["<b>links</b><br/>employer: /Company/acme<br/>friends: [/Person/kari]"]:::link
values ~~~ address ~~~ links
end
acme["<b>Company/acme</b><br/>a document of its own"]:::doc
kari["<b>Person/kari</b><br/>a document of its own"]:::doc
links -- employer --> acme
links -- friends --> kari
classDef value fill:#f1f5f9,stroke:#64748b,color:#0f172a
classDef link fill:#dbeafe,stroke:#2563eb,color:#0f172a
classDef doc fill:#dcfce7,stroke:#16a34a,color:#0f172a
style address fill:#fef9c3,stroke:#ca8a04,color:#0f172a
linkStyle 2,3 stroke:#64748b,stroke-width:2px
- A value: a text, a number,
trueorfalse, a date-time, the name of an enum, or a list of them. - An embedded object, when the type of the field is an embedded type, a type
without
@domain:addressis anAddress. A list of them is written[Chapter]. It is a value object, and the document holds it by composition: the object is part of the document it is in. It has no id and no versions of its own; it is written and read with its document, and it can be reached by its path, as the three kinds of resource shows. An embedded type has fields of the same three kinds in its turn. - A link, when the type of the field is an entity type and the field has
@link: the text/Entity/id, or a list of them. A link is a typed reference: the document it points at is not copied into this one. It is a document of its own, with versions of its own, so between the two documents there is association, not composition. A read resolves the link to that document as it is at the time the read asks for (see snapshots). A field may also link to a document of one of several entity types: in a GraphQL schema the type of the field is then a union of them, and in JSON Schema files the field names each.
A field whose type is an entity type can also hold the values of that entity type in the document itself, as an embedded
object: a copy, not a link. That is a list of an entity type without @link, such as support: [Support], where the
entity type Support has a link of its own (technical: [Contact] @link). The values are then part of the document
they are in, as an embedded object is, and are reached by their path: /ns/Agreement/a1/support, and the link inside it
at /ns/Agreement/a1/support/technical/Contact/c1. They have no id and no versions of their own, and a later version of
a Support document does not change them. Any other field whose type is an entity type is read as a link when @link
is left out: owner: Support holds the text /Support/id.
Not finished: the copy does not work as written. The field holds one object, not a list: a list is refused with
$.support: array found, object expected. Of the fields of the entity type each list is read as a list of links, also a
list of texts, and the others are not checked: level is taken as it comes. The GraphQL API still reads the field as a
list, and answers it with null and the error
Can't resolve value (/AgreementById/support) : type mismatch error, expected type LIST.
The model lists it with the other shapes.
Not finished: a field whose type is an interface is none of the three. The fields of the interface become fields of the entity type, and the field itself is not checked. The model lists this with other shapes LDS does not read as they are written.
Checked when a document is written
LDS reads the model once, when it starts. To change the model, LDS is started again with the new one.
Before a document is written, LDS checks it against the model (schema validation). A
document that does not fit is refused with 400 and a text that says what is wrong, and nothing is written:
curl -X PUT "$LDS/ns/Person/per" -H 'Content-Type: application/json' -d '{"name": 7}'
curl -X PUT "$LDS/ns/Person/per" -H 'Content-Type: application/json' \
-d '{
"name": "Per",
"employer": "/Person/ola"
}'Schema validation error: [$.name: integer found, string expected]
Schema validation error: Illegal linked-domain: "Person". Navigation: employer: "/Person/ola"Not finished: the check does not cover everything a model says. With the example model the value of address is
not checked: a person was written with "address":7, and another with an address that has no city.
The addresses follow the model as well. A request for an entity type the model does not have, such as /ns/Robot/r2, is
answered 400 with Not a managed resource name: "Robot".
The two forms are not equal in one thing: LDS makes its GraphQL API from a GraphQL schema, so a model given as JSON Schema files has the REST API only.
How to write a model, what exactly LDS checks in a document and what it does not check, is in the model.
Snapshots: the time machine
A snapshot is what LDS keeps, the whole namespace, as of one time T: every document in
the version that is valid at T, and every link resolved as of T. An as-of read asks for a snapshot by its time:
timestamp in the REST API, snapshot in the GraphQL API, and now when no time is given. T1, T2 and Tn are three
snapshots of one history:
%%{init: {"gantt": {"leftPadding": 150, "fontSize": 15, "sectionFontSize": 15, "barHeight": 28, "barGap": 8}}}%%
gantt
title Five writes, each valid from its time
dateFormat YYYY-MM-DD
axisFormat %Y
todayMarker off
section Person/ola
Bergen, employer nordlys : 2020-01-01, 2023-01-01
Oslo, employer acme : 2023-01-01, 2027-06-01
section Company/acme
Acme : 2020-01-01, 2027-06-01
section Company/nordlys
Nordlys : 2021-01-01, 2025-01-01
deleted : crit, 2025-01-01, 2027-06-01
section Snapshots
T1 : milestone, 2020-06-01, 0d
T2 : milestone, 2022-06-01, 0d
Tn = now (2026-11) : milestone, 2026-11-20, 0d
The snapshots of that history: every document as it was at T, and every link resolved as of T.
flowchart TB
model[["<b>The model</b> — one, read when LDS starts"]]
subgraph T1["T1 = 2020-06-01"]
direction TB
ola1["<b>ola</b><br/>city: Bergen"]:::doc
none1["no nordlys yet"]:::gone
acme1["<b>acme</b><br/>employees: none"]:::doc
ola1 -. employer .-> none1
end
subgraph T2["T2 = 2022-06-01"]
direction TB
ola2["<b>ola</b><br/>city: Bergen"]:::doc
nordlys2["<b>nordlys</b><br/>employees: ola"]:::doc
acme2["<b>acme</b><br/>employees: none"]:::doc
ola2 -- employer --> nordlys2
end
subgraph Tn["Tn = now"]
direction TB
ola3["<b>ola</b><br/>city: Oslo"]:::doc
acme3["<b>acme</b><br/>employees: ola"]:::doc
nordlys3["nordlys: deleted, 404"]:::gone
ola3 -- employer --> acme3
end
model -. checks and serves .-> T1
model -. checks and serves .-> T2
model -. checks and serves .-> Tn
classDef doc fill:#dcfce7,stroke:#16a34a,color:#0f172a
classDef gone fill:#f8fafc,stroke:#94a3b8,color:#475569,stroke-dasharray: 4 3
style model fill:#fef9c3,stroke:#ca8a04,color:#0f172a
linkStyle default stroke:#64748b,stroke-width:2px
These are the five writes, and the GraphQL query that was sent at T1, T2 and Tn:
curl -X PUT "$LDS/ns/Company/acme?sync=true×tamp=2020-01-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{"name": "Acme"}'
curl -X PUT "$LDS/ns/Person/ola?sync=true×tamp=2020-01-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{
"name": "Ola",
"address": {"city": "Bergen"},
"employer": "/Company/nordlys"
}'
curl -X PUT "$LDS/ns/Company/nordlys?sync=true×tamp=2021-01-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{"name": "Nordlys"}'
curl -X PUT "$LDS/ns/Person/ola?sync=true×tamp=2023-01-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{
"name": "Ola",
"address": {"city": "Oslo"},
"employer": "/Company/acme"
}'
curl -X DELETE "$LDS/ns/Company/nordlys?sync=true×tamp=2025-01-01T00:00:00Z"
curl "$LDS/graphql?snapshot=2020-06-01T00:00:00Z" -H 'Content-Type: application/graphql' \
-d '{
PersonById(id: "ola") {
name
address {
city
}
employer {
name
}
}
CompanyById(id: "acme") {
name
employees {
edges {
node {
name
}
}
}
}
}'The writes were answered 201, and the delete 204. The answers to the query, at T1, at T2
(snapshot=2022-06-01T00:00:00Z) and now (no snapshot):
{
"data": {
"PersonById": {
"name": "Ola",
"address": {"city": "Bergen"},
"employer": null
},
"CompanyById": {
"name": "Acme",
"employees": {
"edges": []
}
}
}
}
{
"data": {
"PersonById": {
"name": "Ola",
"address": {"city": "Bergen"},
"employer": {"name": "Nordlys"}
},
"CompanyById": {
"name": "Acme",
"employees": {
"edges": []
}
}
}
}
{
"data": {
"PersonById": {
"name": "Ola",
"address": {"city": "Oslo"},
"employer": {"name": "Acme"}
},
"CompanyById": {
"name": "Acme",
"employees": {
"edges": [
{
"node": {"name": "Ola"}
}
]
}
}
}
}A snapshot is whole: a link is not resolved to the document as it is now, but as it was at the time of the
snapshot. At T2 nordlys was there and ola worked for it; at Tn nordlys is deleted, its version is a
tombstone (GET /ns/Company/nordlys is answered 404), and the earlier link leads nowhere, while T2 still gives
it.
The model is the same for every snapshot. LDS reads one model when it starts, and uses it for every time: a write
is checked against it whatever valid time the write has, and the GraphQL API is made from it, the same for T1 as
for Tn.
A person written for 2019 without the name that the model asks for is refused as one written for now would be:
curl -X PUT "$LDS/ns/Person/per?sync=true×tamp=2019-01-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{"born": "1970-01-01T00:00:00Z"}'Schema validation error: [$: required property 'name' not found]The versions that are stored are not checked again when LDS is started with another model. They are given back as they were written.
Not finished: the model is not kept on the timeline. A snapshot holds the documents of its time, not the model
they were written under. Every write goes into the transaction log with a field schema, and LDS writes the text
TODO in it, so nothing says which model a write was checked against. When LDS is started with another model,
every snapshot, T1 as well as Tn, is checked and served with that model.
Providers
LDS leaves two jobs to providers, parts that can be exchanged by configuration: keeping the documents (persistence) and finding them by text (search).
Persistence
The persistence provider keeps every version of every document. There are three, chosen with the configuration
key persistence.provider:
| Provider | Where the documents are |
|---|---|
mem |
In the memory of the LDS process. Nothing is left when LDS stops. It is for trying LDS and for tests, and it is what LDS uses when nothing else is configured. |
postgres |
In a PostgreSQL database. |
neo4j |
In a Neo4j database, as a graph: documents are nodes and links are relationships between them. |
The addresses and the requests are the same with every provider, and so are the rules for versions, deletes and links that this document describes: every provider passes the same tests for them, the shared suite. The choice decides where the documents are, whether they are there after LDS has stopped, and what has to run beside LDS. Configuration has the keys of each provider, and operations says what each one stores and what to keep safe.
Search
Search is optional, and it is off when nothing else is configured (the key graphql.search.enabled). There is one
search index provider, solr, which uses Apache Solr. With search on, every write and every delete of a document also
goes to the search index, and the GraphQL API gets a field Search. Search is reached through the GraphQL API only, so
it needs a model that is a GraphQL schema.
Search finds a document by the texts of its fields named languageText, and by nothing else in the document.
What search covers is under review.
What a write goes through
Every write of the REST API is carried by a saga: a short list of steps that LDS records as it does them, so that a write that was begun is also completed. That holds for a write and a delete of a whole document, of a part and of a link. The steps are:
- An entry for the write is put in the transaction log, a log with one entry for every write.
- The document is written to the persistence provider, or marked as deleted there.
- When search is on, the search index is updated.
Step 1 is done first. Steps 2 and 3 both come after it, and neither waits for the other.
flowchart TB
put["PUT /ns/Person/ola"] --> check{"checked against<br/>the model"}
check -- "does not fit" --> refused["400, nothing is written"]
check -- fits --> saga["saga, recorded in the saga log"]
saga -. "the answer: when the saga is recorded,<br/>or when its steps have ended (sync)" .-> answer["{#quot;saga-execution-id#quot;: #quot;...#quot;}"]
saga --> txlog["1 — transaction log"]
txlog --> store["2 — persistence provider"]
txlog --> search["3 — search index, when search is on"]
classDef bad fill:#fee2e2,stroke:#dc2626,color:#0f172a
class refused bad
linkStyle default stroke:#64748b,stroke-width:2px
LDS records the start of a saga, with the whole request, and the start and the end of each step in a saga log. A saga that does not complete, because a step failed or because LDS stopped, stays in the saga log and is run again later, with the request as it was first sent: the same document and the same time. LDS goes forward only (forward recovery): a step that was done is not undone when a later step fails.
Not finished: a saga is not run again without end. When it is run again because a later write needs its place in the saga log, and it fails once more, LDS moves it to a log that nothing reads, and its write is not done. Operations says when a saga is run again.
The answer to a write holds the id of its saga, {"saga-execution-id":"..."}. It is the id of the entry in the
transaction log too. A few answers have no body, such as the 204 to a delete of a whole document;
the REST API has the answer of every request.
The saga log and the transaction log have providers of their own. With nothing configured, the saga log is in memory, so a saga that has not completed when LDS ends is lost, and the transaction log is not kept at all. Both can be kept in PostgreSQL; see configuration and operations.
Not finished: LDS writes the transaction log and reads it back for one thing only: to tell a source of writes which of its writes was the last. Nothing in LDS builds the documents again from a transaction log.
What sync means
The query parameter sync of a write says whether the answer waits for the saga.
- The answer waits, a synchronous write. It is sent when every step has ended. A read sent after the answer gives what was written (read-your-writes).
- The answer does not wait. It is sent when the saga, with the request, is in the saga log. The steps are done shortly after. A read sent right after the answer may not give what was written yet (eventual consistency), and when a step fails, the one who sent the write is not told.
When sync is not given, a write or a delete of a whole document waits, and a write or a delete of a part or
of a link does not wait. sync=true makes every write wait, and sync=false makes none wait.
The REST API has the table of every case.
Not finished: a write whose saga fails while the answer waits is answered 500 with no body, and the 500 does
not say whether the document was written. A saga that failed is run again later, so the write may also be done
after the 500. The REST API shows what a failed write can leave behind.
A call is safe to send again
Every call of the HTTP API is idempotent: a call can be sent a second time, and a read then gives what it gave
after the first. So when no answer comes, or the answer is 500, the call is sent again. No call needs to be
looked up first to find out whether it was done.
- A read changes nothing. A read with a
timestampgives the same answer every time, until a write adds or replaces a version at or before that time. - A write of a document with a
timestampwrites the version of that time. Sent again, it replaces that version with the same document. - A delete with a
timestampwrites the same tombstone again. A delete of a document that is not there is answered204, as a delete of a document that is there. - A write of a link that the document has is answered
200and writes nothing. A delete of a link that the document does not have is answered200and writes nothing.
The same write sent twice with a timestamp, and the timeline after it:
curl -X PUT "$LDS/ns/Company/hav?timestamp=2024-03-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{"name": "Hav"}'
curl -X PUT "$LDS/ns/Company/hav?timestamp=2024-03-01T00:00:00Z" -H 'Content-Type: application/json' \
-d '{"name": "Hav"}'
curl "$LDS/ns/Company/hav?timeline"{"saga-execution-id": "01a1f9de-132e-7ff3-881c-4244047f7318"}
{"saga-execution-id": "01a1f9de-133b-7fe7-a137-63ec95e5e356"}
[
{
"version": "2024-03-01T00:00Z[Etc/UTC]",
"document": {"name": "Hav"}
}
]One thing follows from time. A write without a timestamp gets as valid time the time at which LDS receives it, so each
time it is sent it adds a version. A read gives the same document after the second as after the first, and the timeline
has one version more. A delete without a timestamp adds a tombstone each time in the same way. The same write sent
twice without a timestamp:
curl -X PUT "$LDS/ns/Company/vind" -H 'Content-Type: application/json' -d '{"name": "Vind"}'
curl -X PUT "$LDS/ns/Company/vind" -H 'Content-Type: application/json' -d '{"name": "Vind"}'
curl "$LDS/ns/Company/vind?timeline"{"saga-execution-id": "01a1f9de-1353-7e41-830b-28eb4c54befc"}
{"saga-execution-id": "01a1f9de-1360-7964-9445-947a2c18689a"}
[
{
"version": "2026-11-20T16:50:43.155Z[Etc/UTC]",
"document": {"name": "Vind"}
},
{
"version": "2026-11-20T16:50:43.168Z[Etc/UTC]",
"document": {"name": "Vind"}
}
]A caller that wants a write to be exactly the same when it is sent again gives it a timestamp.
Where to read next
- Getting started: start LDS and write a first document.
- The model: how to write a model, and what LDS checks.
- The REST API: every address, method, parameter and answer.
- The GraphQL API: the queries LDS makes from a model.
- Configuration: every key, and how to choose providers.
- Operations: running LDS, the saga log and recovery.