Business continuity

OBSCURA Case: .obscase

What it is

A zip.

Standard inside

Zip; JSON and JSON Lines (one JSON record a line); plain UTF-8 text; each original file exactly as it was loaded; SHA-256.

Its own

The names of the fields in the JSON, which are Case's. Every case file carries a README.txt that explains them.

Reading it without OBSCURA. Unzip it. sha256sum -c manifest.sha256 checks the manifest, and each entry's hash in manifest.json checks the rest. The originals in natives/ open in whatever opened them before. A production Case makes for the other side is also a zip, with standard Concordance (.dat) and Opticon (.opt) load files, TIFF or JPEG pages, text, the originals, and its own manifest.

Written by pro-code/case-share.js; the build checks this note against it.

The Photo Editor: .obphoto

What it is

A zip, with a different name.

Standard inside

Zip; JSON; every layer's pixels as a PNG, at full size; lookup tables as the .cube text they came in as.

Its own

How the layers are put together: blend modes (named as the browser's canvas names them, such as source-over and multiply), adjustment layers, effects and editable text are described in document.json in the editor's own terms. A program that does not know them gets every layer's pixels, and has to stack them itself.

Reading it without OBSCURA. Rename it to .zip, or open it with any zip program. Each layer is an ordinary PNG; stacked bottom to top at the position document.json gives, with its opacity, they make the picture. To keep a picture any program opens, export a PNG, JPEG, WebP, TIFF or PSD from the editor as well.

Written by pro-code/photo-editor-file.js; the build checks this note against it.

Foundry (the Engineering Suite's 2D drawings): .foundry

What it is

One JSON text file.

Standard inside

JSON in UTF-8. Pictures traced over are inside it as data: URLs.

Its own

The drawing's shapes are Foundry's own records: lines, polylines, arcs, circles, splines, text, dimensions, hatches, blocks, sheets and viewports, with points as numbers in the drawing's units. Nothing but Foundry draws them from this file.

Reading it without OBSCURA. Any text editor or JSON reader opens it. To keep a drawing any CAD program opens, export DXF (AutoCAD's exchange format, version 2000) from Foundry as well, or PDF and SVG for a picture of it.

Written by pro-code/foundry-io.js; the build checks this note against it.

The Engineering Suite (3D parts and assemblies): .json (a project file)

What it is

One JSON text file.

Standard inside

JSON in UTF-8.

Its own

The part is kept as its history: sketches, and the features made from them (extrusions, revolves, holes, fillets and the rest), with their numbers and the variables they use. It is the suite's own parametric description; no other CAD program rebuilds a solid from it.

Reading it without OBSCURA. Any JSON reader opens it. To keep the shape itself readable by any CAD program, export STEP (a faceted solid), 3MF, STL, OBJ or glTF from the suite as well; a drawing of the part exports as DXF or PDF.

Written by pro-code/eng-io.js; the build checks this note against it.

OBSCURA Roots: .oroots

What it is

A zip archive.

Standard inside

The tree exports as GEDCOM 5.5.1 or 7.0, the genealogy standard every family history program and site reads, and as GEDZIP (GEDCOM 7.0 with its pictures in one zip). Pictures and documents inside the project are kept byte for byte, in their own formats.

Its own

tree.json, the whole tree as Roots holds it: the GEDCOM it was read from, with every tag Roots does not show kept on its record, and what GEDCOM has no place for: DNA matches, the research log and to-do lists, and faces marked in pictures.

Reading it without OBSCURA. Any zip tool opens it and any JSON reader reads tree.json. To keep the tree readable by every genealogy program, export GEDCOM from Roots as well.

Written by pro-code/roots-store.js; the build checks this note against it.

OBSCURA Vector: .ovector

What it is

A zip, with the mimetype first and uncompressed, as an EPUB or an OpenDocument file has it.

Standard inside

Zip, JSON in UTF-8, every picture as the PNG or JPEG it was, and preview.svg: the first artboard as an ordinary SVG file that any browser or drawing program opens.

Its own

The shape of the JSON: artboards, layers, and each object with its geometry and style. Paths are lists of anchor points with their two handles; rectangles, ellipses, polygons and stars keep their sizes and a matrix; text keeps its face, size and spacing; parts of an imported SVG that Vector does not edit are kept as the SVG they were.

Reading it without OBSCURA. Unzip it and open preview.svg in any browser, or read document.json in any JSON reader. To keep every artboard in a standard file, export SVG for each from Vector, and PDF for print.

Written by pro-code/vector-out.js; the build checks this note against it.

OBSCURA Erase: .oerase

What it is

The line OBSCURA-ERASE, a four-byte length and a JSON header, then a zip sealed with AES-256-GCM under a key made from the file’s own passphrase with Argon2id (or PBKDF2-SHA-256 at 600,000 rounds where Argon2 cannot run); the header holds the parameters.

Standard inside

AES-256-GCM, Argon2id, zip and JSON in UTF-8; each request’s letters as PDF; each file that came back as the bytes it arrived as.

Its own

The shape of the JSON: the person’s details, and each request with its organisation, law, right, the text it was asked with, and its timeline of dated events.

Reading it without OBSCURA. With the passphrase, the header gives the key derivation’s parameters and the rest is one AES-GCM message: any tool with Argon2id and AES-GCM opens it, and the zip inside opens anywhere. The PDFs read in any PDF reader.

Written by pro-code/erase-store.js; the build checks this note against it.

OBSCURA Umbra: a folder

What it is

Not a file of its own: a project is a folder of ordinary files.

Standard inside

Everything. A folder you opened is read and written in place, so the files on your disk are the project. A project kept in the browser exports as an ordinary zip, with its git history (a normal .git folder) if you ask.

Its own

Nothing.

Reading it without OBSCURA. Open the folder, or unzip the export, in any editor. A project kept only in the browser is lost if the browser's data is cleared, so export a zip now and then.

Written by pro-code/umbra-fs.js; the build checks this note against it.

OBSCURA Books: .obooks

What it is

A zip.

Standard inside

Zip; JSON; the receipts and bills as the pictures or PDFs they were; SHA-256.

Its own

The shape of the JSON: the company, its chart, contacts, documents, bank lines and journals, as Books keeps them.

Reading it without OBSCURA. Unzip it. company.json opens in any JSON reader or spreadsheet's import; the attachments are ordinary files. Books also exports every journal line, document and bank line as CSV for an accountant's own software.

Written by pro-code/books-store.js; the build checks this note against it.

OBSCURA Keep: .okeep

What it is

One file: the line OBSCURA-KEEP, a JSON header, then each sealed part as a name and its bytes. A package left for one person is the same, with a format of obscura-keep-package.

Standard inside

JSON; base64; Argon2id (or PBKDF2-SHA-256, as the header says); AES-256-GCM; HKDF-SHA-256.

Its own

The way the parts are strung together, and the names of the fields inside the index, which are Keep's. Every part is sealed, so nothing in it reads without the passphrase.

Reading it without OBSCURA. The Python below, with the cryptography package (version 45 or later), opens one with its passphrase and writes the index and every document into a folder. A package somebody left you also opens at /keep-open, with no account.

import base64, getpass, json, os, struct, sys
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
from cryptography.hazmat.primitives.kdf.argon2 import Argon2id
from cryptography.hazmat.primitives.kdf.hkdf import HKDF
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC

data, out = open(sys.argv[1], "rb").read(), sys.argv[2]
assert data.startswith(b"OBSCURA-KEEP\n"), "not a Keep file"
at = 13
n, = struct.unpack(">I", data[at:at + 4]); at += 4
header = json.loads(data[at:at + n]); at += n
parts = {}
while True:
    nl, = struct.unpack(">H", data[at:at + 2]); at += 2
    if not nl: break
    name = data[at:at + nl].decode(); at += nl
    ln, = struct.unpack(">I", data[at:at + 4]); at += 4
    parts[name] = data[at:at + ln]; at += ln
kid, k = header["id"], header["kdf"]
salt = base64.b64decode(k["salt"])
secret = (sys.argv[3] if len(sys.argv) > 3 else getpass.getpass("Passphrase: ")).encode()
if k["alg"] == "argon2id":
    kek = Argon2id(salt=salt, length=32, iterations=k["t"], lanes=k["p"], memory_cost=k["m"]).derive(secret)
else:
    kek = PBKDF2HMAC(hashes.SHA256(), 32, salt, k["i"]).derive(secret)
def unseal(key, sealed, aad):
    assert sealed[0] == 1
    return AESGCM(key).decrypt(sealed[1:13], sealed[13:], aad.encode())
vault = unseal(kek, base64.b64decode(header["wraps"]["pass"]), "obscura-keep|v1|wrap|pass|" + kid)
key = HKDF(hashes.SHA256(), 32, ("obscura-keep|" + kid).encode(), b"obscura-keep v1 data").derive(vault)
index = json.loads(unseal(key, parts["index.bin"], "obscura-keep|v1|index|%s|index" % kid))
os.makedirs(out, exist_ok=True)
json.dump(index, open(os.path.join(out, "index.json"), "w"), indent=1, ensure_ascii=False)
for item in index["items"].values():
    for f in item.get("files", []):
        body = unseal(key, parts["b-%s.bin" % f["id"]], "obscura-keep|v1|blob|%s|%s" % (kid, f["id"]))
        open(os.path.join(out, f["id"] + " " + f["name"].replace("/", "-")), "wb").write(body)
print(len(index["items"]), "items written to", out)

Written by pro-code/keep-store.js; the build checks this note against it.

OBSCURA Research: .oresearch (a project file)

What it is

A zip archive.

Standard inside

Zip, holding JSON in UTF-8, the pictures as PNG, JPEG, GIF or WebP files and the recordings as the audio and video files they were. The data also save as SPSS .sav, Stata .dta, CSV and Excel .xlsx, the output as Word and PDF, the codebook as CSV and Word, transcripts as WebVTT, and the coding as a REFI-QDA project, all standard files.

Its own

The project's description in project.json: the data set by column with each variable's label, value labels, missing values and measure; the sources' text; the codes, the coded passages (a code, a source and the characters from and to), memos, annotations and cases; the output viewer's tables; and the syntax log.

Reading it without OBSCURA. Unzip it and open project.json in any JSON reader; the coded passages are character offsets into each source's text. To take the data elsewhere, save them as .sav, CSV or Excel from the File menu.

Written by pro-code/research-project.js; the build checks this note against it.

OBSCURA Write: .owrite

What it is

A zip, with the mimetype first and uncompressed, as an EPUB or an OpenDocument file has it.

Standard inside

Zip, JSON in UTF-8, and the pictures and PDFs in the research as the files they were.

Its own

The shape of the JSON: the binder as items with their children, and the bible, labels, targets, writing history, comments and submissions. The text of each document is a small subset of HTML (paragraphs, emphasis, footnotes and notes as spans); a screenplay's scene marks each paragraph with its element.

Reading it without OBSCURA. Unzip it and read project.json in any JSON reader; every document's text is there as HTML a browser shows. To have the writing itself in a standard file, compile to Word, EPUB, Markdown or plain text, or a screenplay to Final Draft or Fountain.

Written by pro-code/write-out.js; the build checks this note against it.

OBSCURA Authorship: .oauthor (a record of how a work was made)

What it is

A zip of ordinary files.

Standard inside

Zip, JSON lines, JSON, Markdown, HTML, PDF, Word (.docx), PNG, RFC 3161 timestamp tokens (DER) and an ECDSA P-256 signature. The work itself is in its own Markdown, and as HTML, PDF and Word beside it.

Its own

The record (chain.jsonl): blocks of events chained by SHA-256, laid out in full at the top of oauthor.js, which is free code anyone can read and run.

Reading it without OBSCURA. Unzip it with anything. /verify-authorship checks it, free; the work in document/ opens in any word processor or browser.

Written by pro-code/authorship-export.js; the build checks this note against it.

OBSCURA Sound: .osound

What it is

A zip, with the mimetype first and uncompressed, as an EPUB or an OpenDocument file has it.

Standard inside

Zip, JSON in UTF-8, and the audio as the files it came in as (an MP3 stays an MP3, a FLAC a FLAC), or as 32-bit float WAV for audio made in Sound: a recording, or the result of an effect.

Its own

The shape of the JSON: the tracks with their gain, pan, mute and solo; each clip's place on the timeline in samples, the stretch of its file it plays, its gain and its fades; the markers; and a noise profile if one was learned.

Reading it without OBSCURA. Unzip it: every recording in audio/ plays in any audio player or editor. To have the mix itself in a standard file, export it to WAV, which any editor opens; project.json says where each clip goes if the edit is to be rebuilt elsewhere.

Written by pro-code/sound-store.js; the build checks this note against it.

PDF Studio: standard files only

What it is

PDF Studio has no file format of its own.

Standard inside

It writes PDF; comments as XFDF; form data as FDF, XFDF or JSON; measurements as CSV; Word, Excel and PowerPoint files; certificates as PKCS#12 (.p12).

Its own

Nothing.

Reading it without OBSCURA. Whatever reads PDF and those formats: Acrobat, a browser, LibreOffice, Microsoft Office.

Written by pro-code/pdfs-comments-ui.js; the build checks this note against it.

The Sovereign Suite: .sovereign (a backup)

What it is

Sealed one of two ways: with a passphrase, as a small JSON envelope, or to age public keys, as a standard age file.

Standard inside

JSON; base64; PBKDF2-SHA256 and AES-256-GCM; age (age-encryption.org/v1).

Its own

The names of the registers inside, which are the Suite's. Each record carries its SHA-256.

Reading it without OBSCURA. One sealed to age keys opens with the age program: age -d -i your-key.txt backup.sovereign. One sealed with a passphrase opens with this Python, which needs the cryptography package, and checks every record's hash as it goes:

import base64, hashlib, json, sys
from cryptography.hazmat.primitives.ciphers.aead import AESGCM

envelope = json.load(open(sys.argv[1]))
assert envelope["format"] == "sovereign-backup" and envelope["method"] == "passphrase"
blob = base64.b64decode(envelope["sealed"])
salt, iv, sealed = blob[:16], blob[16:28], blob[28:]
key = hashlib.pbkdf2_hmac("sha256", sys.argv[2].encode(), salt, 310000, 32)
payload = json.loads(AESGCM(key).decrypt(iv, sealed, None))
for name, value in payload["items"].items():
    assert hashlib.sha256(value.encode()).hexdigest() == payload["sha256"][name]
json.dump(payload, sys.stdout, indent=1)

Written by pro-code/sovereign-svc-data.js; the build checks this note against it.