Portable Knowledge for AI with the Open Knowledge Format

What you'll build

By the end of this lab you'll have authored, validated, consumed, and generated knowledge in the Open Knowledge Format (OKF) — the vendor-neutral specification Google Cloud introduced on June 12, 2026 for representing the metadata and context that AI systems need. OKF's whole premise is radical simplicity: a knowledge corpus is "just a directory of markdown files with YAML frontmatter." That means you can do real work with it using nothing but Python's standard library plus a YAML parser.

You'll play both ends of the format:

• As a producer: read a hand-authored bundle, then write a small program that generates a conformant bundle from a data source.
• As a consumer: validate a bundle against the v0.1 conformance rules, then build a knowledge graph and query layer over it.

Everything runs offline — no cloud account, no API keys, no LLM required.

Why this matters (the 5-minute business case)

Read this section before handing the lab to your team.

The problem OKF targets is that your organization's most valuable AI context is trapped in incompatible silos. The information an AI agent needs to be useful is overwhelmingly internal: what a table's columns actually mean, how your business defines "active user," the runbook for an incident, the join path between two systems, why an API was deprecated. Today that knowledge is scattered across metadata catalogs (each with its own API), wikis, shared drives, code comments, notebooks — and the heads of a few senior engineers. Every time someone builds a new agent, they re-solve the same context-assembly problem from scratch, and every catalog vendor reinvents the same data model.

OKF's bet is that the missing piece is a format, not another platform. Rather than a new service to buy and integrate, it standardizes the minimum set of conventions needed for knowledge to be portable: a directory of markdown files, each with a little YAML frontmatter, linked together with ordinary markdown links. Three reasons a business should care:

1. It attacks lock-in directly. Because a bundle is "just files," knowledge survives moving between systems, teams, and organizations. A bundle a human writes can be read by an agent; one a metadata pipeline exports can be browsed by a person; one synthesized by one vendor's LLM can be queried by another's. The format is the contract, and the tooling at each end is independently swappable. You stop betting your knowledge on any one vendor's catalog.
2. It makes knowledge a version-controlled asset ("metadata as code"). OKF bundles live in git alongside the code they describe — diffable in pull requests, reviewable, attributable, and recoverable. For data governance and platform teams, that turns "tribal knowledge" into a managed artifact with history, instead of a wiki nobody trusts is current.
3. It is cheap to adopt and hard to regret. There's no SDK to learn, no runtime to operate, no schema registry to stand up. If you can write markdown and git commit, you can produce OKF; if you can cat a file, you can consume it. That low floor means a team can pilot it in an afternoon, and even if the standard evolves, the underlying assets remain plain, readable files.

The honest caveats, because pragmatic adoption means naming them:

• It's v0.1 and explicitly a "starting point, not a finished standard." The conventions will evolve. Adopt it where portability and readability matter most; don't expect a rich, settled ecosystem of tools yet.
• "Minimally opinionated" cuts both ways. OKF requires only a type field and leaves the actual content model to you. That flexibility is the point, but without internal conventions (what types exist, what sections a body should have), two teams' bundles can be conformant yet still inconsistent. The interoperability is structural, not semantic — you still need your own style guide.
• It is a representation format, not a quality guarantee. OKF makes knowledge portable and parseable; it does nothing to make it correct or current. Stale knowledge in a tidy format is still stale. Curation discipline (and the log.md/timestamp conventions) still matter.

A useful framing for a non-technical stakeholder: "OKF is a common file format for the context our AI agents need — like PDF or CSV, but for institutional knowledge. The win isn't a new tool; it's that our knowledge stops being locked inside whatever system created it, and starts living in version control where we can manage it like code."

Concepts you'll use (2-minute primer)

OKF has remarkably few moving parts:

• Bundle — a directory of markdown files. The unit you share (as a git repo, a tarball, or a subfolder).
• Concept — one markdown file = one unit of knowledge (a table, a metric, a playbook, an API…). Its Concept ID is just its path minus .md (so tables/orders.md → tables/orders).
• Frontmatter — a small YAML block at the top. The only required field is `type`. Recommended ones: title, description, resource, tags, timestamp.
• Body — standard markdown. Conventional headings like # Schema, # Examples, # Citations have agreed meaning but aren't required.
• Links — ordinary markdown links between concepts turn the directory into a graph of relationships, richer than the folder hierarchy.
• Reserved files — index.md (a directory listing for "progressive disclosure") and log.md (a change history). These are the only filenames with special meaning.

The conformance bar (spec §9) is deliberately tiny: every non-reserved .md file must have parseable frontmatter with a non-empty type. Consumers must be permissive about everything else — unknown types, missing optional fields, even broken links.

Prerequisites

Get the starter repo. This lab refers to scripts and a sample bundle/ in the accompanying okf-lab/ repo. Unzip it and cd in before you begin.

Setup (2 minutes)

cd okf-lab
python3 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt    # just PyYAML

The repo ships with a hand-authored bundle/ (an "acme" retail sales knowledge base) and an examples/bad-bundle/ you'll use to see validation fail.

Exercise 1 — Read and author a bundle (12 min)

Goal: Internalize the format by reading a real concept, then authoring one yourself.

1a. Read a concept

Open bundle/tables/orders.md. Notice the two-part shape — frontmatter, then body:

---
type: BigQuery Table
title: Orders
description: One row per completed customer order.
resource: https://console.cloud.google.com/bigquery?p=acme&d=sales&t=orders
tags: [sales, orders, revenue]
timestamp: 2026-05-28T14:30:00Z
---

# Schema

| Column        | Type    | Description                                         |
|---------------|---------|-----------------------------------------------------|
| `order_id`    | STRING  | Globally unique order identifier.                   |
| `customer_id` | STRING  | Foreign key into [customers](/tables/customers.md). |
...

# Joins

Joined with [customers](/tables/customers.md) on `customer_id`.

Three things to notice:

• `type` is the only required field. Everything else is optional but recommended. resource is the canonical URI of the actual asset this concept describes.
• The links are the graph. That [customers](/tables/customers.md) link is a bundle-relative path (starts with /, resolved from the bundle root). It's the recommended form because it survives moving files around. The kind of relationship lives in the prose ("Joined with… on customer_id"), not in the link.
• `# Schema` and `# Joins` are conventional headings — they aid both human reading and agent retrieval, but nothing enforces them.

Now open bundle/metrics/weekly_active_users.md. It's a concept with no `resource` — because a metric is an abstract idea, not a stored asset. OKF handles tangible and intangible knowledge the same way.

1b. Author your own concept

Create bundle/tables/refunds.md describing a new table. Minimum viable concept:

---
type: BigQuery Table
title: Refunds
description: One row per refunded order.
resource: https://console.cloud.google.com/bigquery?p=acme&d=sales&t=refunds
tags: [sales, refunds]
timestamp: 2026-06-22T00:00:00Z
---

# Schema

| Column      | Type    | Description                              |
|-------------|---------|------------------------------------------|
| `refund_id` | STRING  | Unique refund identifier.                |
| `order_id`  | STRING  | The refunded [order](/tables/orders.md). |
| `amount_usd`| NUMERIC | Refund amount in USD.                    |

Refunds reverse rows in [orders](/tables/orders.md).

You just added a node and an edge to the knowledge graph — by writing a markdown file. No SDK, no migration, no API call. That's the entire authoring experience.

Business callout: the barrier to contributing knowledge is the single biggest reason internal wikis rot. OKF's barrier is "write a markdown file and commit it" — low enough that the knowledge can live next to the code, get reviewed in the same PR, and stay current because updating it is trivial.

Checkpoint: You can read and write OKF by hand. Next, prove it's correct.

Exercise 2 — Validate conformance (10 min)

Goal: Run the v0.1 conformance check, and understand the deliberately tiny bar it sets.

2a. Validate the good bundle

python validate_bundle.py bundle

Bundle: bundle
  concepts: 5   reserved files: 2 (index.md, log.md where present)

Result: CONFORMANT with OKF v0.1

(If you added refunds.md in Exercise 1, you'll see 6 concepts.)

Open validate_bundle.py and notice it enforces exactly the spec's three rules: every non-reserved .md parses as frontmatter, every one has a non-empty type, and reserved files are handled when present. Everything else — missing description, unknown types, broken links — is a warning at most, never a failure.

2b. Watch it reject a bad bundle

python validate_bundle.py examples/bad-bundle

CONFORMANCE ERRORS:
  x missing_type.md: missing or empty required field 'type'
  x no_frontmatter.md: no YAML frontmatter block

Result: NOT CONFORMANT (2 error(s))

Two distinct failures: one document has frontmatter but no type; the other has no frontmatter at all. These are the only kinds of thing that make a bundle non-conformant.

2c. Confirm the permissive rule yourself

Recall the broken link in bundle/metrics/weekly_active_users.md (it links to a dashboards/revenue.md that doesn't exist). Run the good-bundle validation again — it still passes. Broken links are not errors. The spec is explicit: a link to a missing target "may simply represent not-yet-written knowledge," and consumers MUST tolerate it.

Business callout: this permissiveness is a feature, not sloppiness. Knowledge bases are perpetually half-finished and constantly refactored, especially when agents are generating parts of them. A format that rejected a 500-document bundle because one link was stale would never survive contact with reality. OKF chooses "always usable, even when incomplete."

Checkpoint: You know precisely what conformance requires (almost nothing) and why it's intentionally minimal. Now consume a bundle as an agent would.

Exercise 3 — Consume a bundle as a graph (12 min)

Goal: Build a knowledge graph and a small query layer over the bundle — the kind of thing a consumption agent does before answering a question.

3a. Get the lay of the land

python graph.py bundle

Bundle: bundle
  concepts: 5   links: 15   broken links: 1
  by type:
     1  BigQuery Dataset
     2  BigQuery Table
     1  Metric
     1  Playbook

graph.py parsed every concept, extracted its markdown links, and built a directed graph. Because OKF is "just files + links," this took no SDK and no integration — okf_common.py is a few dozen lines.

3b. Query it

python graph.py bundle --type "BigQuery Table"     # all tables
python graph.py bundle --tag revenue               # everything tagged revenue
python graph.py bundle --neighbors tables/orders   # what links to/from orders
python graph.py bundle --broken-links              # unresolved links (tolerated)

The --neighbors view is the interesting one for agents: it shows both outbound edges (what orders references) and inbound edges (what references orders). That bidirectional view is how an agent answers "what else do I need to know to use this table?" — the exact context-assembly problem the OKF announcement describes.

3c. See progressive disclosure in action

A large bundle shouldn't dump every file into an agent's context at once. The index.md convention lets a consumer (or producer) offer a listing first, so an agent can navigate top-down. Generate one:

python generate_index.py bundle          # preview
python generate_index.py bundle --write  # overwrite bundle/index.md

It scans concepts, groups them by type, and pulls each description from frontmatter into the listing — exactly the "see what's available before opening documents" pattern from spec §6.

Business callout: "load only the few concepts that matter for this task, not the whole corpus" is the same efficiency argument behind retrieval generally. OKF's index.md + cross-links give an agent a cheap map to navigate, instead of forcing every fact into the context window. For cost and latency, that map is the difference between a bundle that scales to thousands of concepts and one that doesn't.

Checkpoint: You can validate and query OKF. The producer and consumer halves are independent — which Exercise 4 makes concrete by generating a brand-new bundle.

Exercise 4 — Produce OKF programmatically (12 min)

Goal: Write knowledge out in OKF from a data source — a miniature version of Google's reference "enrichment agent," minus the LLM.

Google ships an enrichment agent that walks a BigQuery dataset, drafts an OKF concept per table, then runs a second LLM pass to enrich each with citations and join paths. We'll do the deterministic core of that: read a schema description and emit a conformant bundle.

4a. Look at the source

Open source_schema.json — a small description of a marketing dataset with two tables (campaigns, sessions) where sessions.campaign_id references campaigns. This stands in for whatever your real source is (a catalog API, INFORMATION_SCHEMA, a dbt manifest…).

4b. Generate and validate

python produce_okf.py            # writes out_bundle/
python validate_bundle.py out_bundle

Generated 3 concepts into 'out_bundle/'.
...
Result: CONFORMANT with OKF v0.1

Open out_bundle/tables/sessions.md. The generator turned the references field in the source into a real cross-link to campaigns — so the foreign key in your schema became an edge in the knowledge graph automatically. Run the graph tool on it to confirm:

python graph.py out_bundle --neighbors tables/sessions

4c. Appreciate what just happened

You produced knowledge with a 90-line script and consumed it with a different one, and they agreed on nothing but the format. That's producer/consumer independence: the producer could just as easily be an LLM enrichment pass or a live BigQuery crawl; the consumer could be a graph viewer, a search index, or an agent. Neither needs to know how the other works.

Business callout: this is the practical payoff of "format, not platform." Your data team can generate OKF from its catalog on a schedule; your application team can build agents that consume it — with no shared SDK, no coordinated release, and no vendor sitting in the middle. The format is the integration.

Checkpoint: You've closed the loop — authored, validated, queried, and generated OKF, entirely offline.

Wrap-up

You worked both ends of the Open Knowledge Format. You can now:

• Read and author OKF concepts by hand (frontmatter + body + cross-links)
• Validate a bundle against the v0.1 conformance rules and explain why the bar is intentionally tiny
• Consume a bundle as a queryable knowledge graph, and generate an index.md for progressive disclosure
• Produce a conformant bundle programmatically from a data source

The one idea to carry forward: OKF's power comes from being a format rather than a service. It standardizes the smallest set of conventions that make knowledge portable — "just files, just markdown, just YAML frontmatter" — and deliberately leaves everything else open. The value scales with how many producers and consumers speak it, which is exactly why it's published as an open standard. For your organization, the move is low-risk: pilot it where portable, version-controlled knowledge would help most, and let the plain-files property protect you as the spec evolves.

Stretch challenges (for the labs series)

1. Build a real consumer: an HTML visualizer. Google ships a self-contained, single-file HTML visualizer that renders any bundle as an interactive graph with no backend. Write a visualize.py that emits one HTML file embedding the graph (nodes = concepts colored by type, edges = links). It's a great showcase of "consume OKF with zero integration."
2. Add an LLM enrichment pass to the producer. Extend produce_okf.py so that after drafting each concept, it calls a model (e.g. via the Lab 01 Interactions API code) to write a richer description and a # Notes section with caveats — mirroring Google's two-pass reference agent. This ties the whole series together.
3. Tag-browse synthesis. OKF intentionally has no tag-index file. Write a consumer that scans all frontmatter and synthesizes a tags/index.md grouping concepts by tag — demonstrating that producers and consumers can add views without changing the format.

Troubleshooting

References

All technical details are drawn from the announcement and the published OKF v0.1 specification:

• How the Open Knowledge Format can improve data sharing (Google Cloud Blog, June 12, 2026)
• OKF spec & reference implementations (GitHub)
• Knowledge Catalog ingest examples
• Karpathy's LLM Wiki gist (the pattern OKF formalizes)

A note on accuracy for your readers: OKF is v0.1 and explicitly expected to evolve. This lab implements the conformance rules from the published spec; before relying on it, point readers at the live spec for the current reserved filenames, fields, and conventions, and note which version you validated against. The producer here is deterministic by design — the reference implementation's LLM enrichment is one option among many, not part of the format.