Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Architecture

How the pieces fit, and why they are arranged this way.

The shape of the thing

---
config:
  layout: elk
---
flowchart LR
    n1["Source content"] --> n2(["Build-time tools"])
    n2 --> n3["Runtime"]
    n4(["art/.png"]) --> n7@{ label: "<span style=\"color:\">alchemy</span>" }
    n7 --> n8(["materials/*.ktex &amp; *.kmat_c"])
    n8 --> n9["Kerosene"]
    n5(["art/.obj"]) --> n10["forge"]
    n10 --> n11(["models/*.kmdl"])
    n11 --> n9
    n6(["maps/.kmap"]) --> n12["cleave"]
    n12 --> n13(["maps/*.kbsp"]) & n15(["`*.kprt`"])
    n13 --> n14["umbra"]
    n15 --> n14
    n14 --> n16["+vis"] & n17["radiance"] & n20["resonance"]
    n16 --> n9
    n17 --> n18["+light"]
    n20 --> n21["+sound"]
    n21 --> n16
    n18 <--> n16 & n19["chisel"]
    n19 --> n6

    n1@{ shape: rect}
    n3@{ shape: rect}
    n7@{ shape: rounded}
    n9@{ shape: rect}
    n10@{ shape: rounded}
    n12@{ shape: rounded}
    n14@{ shape: rounded}
    n16@{ shape: rect}
    n17@{ shape: rounded}
    n18@{ shape: rect}
    n19@{ shape: rounded}
    style n1 stroke:#2962FF,color:#2962FF
    style n2 stroke:#FF6D00,color:#FF6D00
    style n3 stroke:#AA00FF,color:#AA00FF
    style n4 stroke:#2962FF,color:#2962FF
    style n7 stroke:#FF6D00
    style n8 color:#D50000,stroke:#D50000
    style n9 color:#AA00FF,stroke:#AA00FF
    style n5 stroke:#2962FF,color:#2962FF
    style n10 stroke:#FF6D00
    style n11 color:#D50000,stroke:#D50000
    style n6 stroke:#2962FF,color:#2962FF
    style n12 stroke:#FF6D00
    style n13 color:#D50000,stroke:#D50000
    style n15 stroke:#D50000,color:#D50000
    style n14 stroke:#FF6D00
    style n17 stroke:#FF6D00
    style n19 stroke:#FF6D00

Everything above the line happens once, on a developer’s machine or a build server. Everything below happens sixty-four times a second on a player’s.

That division is the single most important thing about a BSP engine, and it is why the tools are not part of the engine. Visibility takes minutes to compute and microseconds to query. Lighting takes minutes to bake and nothing at all to sample. Neither belongs in the engine.

Crate dependencies

---
config:
  layout: elk
  theme: default
  look: neo
  fontFamily: '''Source Code Pro Variable'', monospace'
---
flowchart TB
    n1["Kerosene-math"] --> n2["Kerosene-kv"] & n4(["cleave"]) & n8["Kerosene-physics"] & n9["Kerosene rigid"] & n11["kerosene-entity"] & n13["Kerosene-vfs"] & n14["kerosene-asset"] & n15["kerosene-render"] & n17["Kerosene (runtime)"]
    n2 --> n3["kerosene-map"] & n5["kerosene-bsp"]
    n3 --> n5 & n4
    n4 --> n5
    n6(["umbra"]) --> n5
    n7(["radiance"]) --> n6
    n20(["resonance"]) --> n6 & n14
    n5 --> n8
    n10(["Box3d-rust"]) --> n9
    n12["kerosene-game"] --> n11
    n14 --> n13 & n15
    n15 --> n16["kerosene-engine"]
    n16 --> n18["kerosene"]
    n12 --> n18
    n18 --> n17

    n1@{ shape: rounded}
    n2@{ shape: rounded}
    n8@{ shape: rounded}
    n9@{ shape: rounded}
    n11@{ shape: rounded}
    n13@{ shape: rounded}
    n14@{ shape: rounded}
    n15@{ shape: rounded}
    n17@{ shape: rounded}
    n3@{ shape: rounded}
    n5@{ shape: rounded}
    n12@{ shape: rounded}
    n16@{ shape: rounded}

Nothing points upward. kerosene-math knows about nothing; kerosene-engine knows about everything except the game. The tools sit off to the side, depending on the format crates but never on the engine.

kerosene is the top of it all, and the product: the game crate. It re-exports the engine’s stable crates as modules and the rest under kerosene::internals, wraps the stock game as kerosene::game::Stock, and provides launch. It is the one crate a game names, its API is what Kerosene’s version number follows (Versioning), and the runtime binary is a call into it.

Underneath everything is the engine’s base content, crates/kerosene-engine/base/base.vault: the developer textures, the stock props, sounds and UI, and a plain room, kerosene_room, compiled into the engine and mounted as the last layer of the file system. A game with no content of its own still has something to show, and a game’s own file at the same path always wins.

The game seam

The engine has no game of its own. kerosene-engine defines a Game trait – classes, schema, setup, map_loaded, player_spawned, pre_tick, tick, frame, entity_request, console_request, wants_ui, ui, ui_event, save, load, can_save, player_damaged, player_died, platform_event, map_unloading and shutdown, every one with a default – and runs whatever implements it, with &mut Engine in every hook. kerosene-game is the stock implementation’s classes and knows nothing of the engine; the engine tests against it as a dev-dependency and kerosene::game::Stock is the type that joins the two.

What used to be game code the engine called by name – what a trigger does to the player, the everywhere flag on a sound – is now an engine convention (kerosene_engine::triggers, audio::SF_EVERYWHERE) that any game’s classes get by following the field names. The game is held outside the engine while a hook runs, which is what lets a hook take &mut Engine; the cost is that a hook cannot cause another hook, so a game changes maps with request_map.

kerosene-config is not on the diagram because it sits to the side of all of it: a small crate on top of kerosene-kv that reads engine.kcfg, the settings every program shares (which renderer, how big the window). The game, Chisel, and the tool windows all ask it the same question, so the renderer is chosen once in a file rather than once per program.

kerosene-rigid is the newcomer beside kerosene-physics. Player movement stays in kerosene-physics (it is Source’s gamemovement, and that is the feel); kerosene-rigid wraps box3d-rust, Erin Catto’s 3D engine ported to pure Rust, for everything the player is not – physics props, the rigid bodies that tumble and roll. It speaks Kerosene units natively: Box3D’s length-unit scale is set to inches once at startup, and no coordinate conversion happens anywhere.

In the engine, a PhysicsProps owns one PhysicsWorld: at map load every static world brush (and every detail) becomes a convex hull, moving brush entities become static bodies that follow their entity’s pose, and each prop_physics entity becomes a dynamic box (shaped from its model’s bounds, loaded through kerosene-asset). Once a tick the simulation is stepped and its poses written back to the entities. The renderer draws those props as .kmdl models at their simulated pose, and phys_debug turns the collision boxes into a wireframe overlay. The player’s movement traces against the prop boxes too (through PlayerCollision), and the use key doubles as a pick-up tool: grab a prop, carry it, press use again to launch it. prop_dynamic_spawner is the game-side spawner that creates the props in the first place.

Two edges in that picture are there for a reason worth stating. chisel depends on alchemy, because the editor builds the content tree’s textures itself rather than re-invoking the toolset for it. And everything – the editor, the compilers, the runtime – finds the content tree through kerosene_vfs::root, one function they all call, and finds the runtime and the toolset’s subcommands through kerosene_vfs::toolchain, likewise. Each of them working either out separately is not a hypothetical: they did, they disagreed, and a tool looking in the wrong directory is indistinguishable from a tool that is broken. A shared answer that explains itself is the whole of the fix.

kiln sits above the compilers rather than beside them: it is the one tool that runs the others, and it does so by re-invoking the same executable with the compiler’s name as a subcommand, so nothing about the pipeline being one program leaks into the compilers being separate stages.

The map pipeline in detail

Cleave: .kmap → .kbsp

  1. Brushes. Each .kmap solid becomes a set of interned half-space planes. Plane interning matters more than it sounds: two faces meant to be coplanar must end up sharing one plane index, or the tree splits along a hair’s width between them and the compile explodes.

  2. CSG. Every face is cut against every brush that could bury it, and the buried parts are dropped. On a real map this removes about a third of all faces before anything else runs. The subtle case is coplanar faces: two brushes flush side by side keep exactly one shared face (lower index wins, deterministically); two brushes back to back drop it entirely.

  3. The tree. Built top-down from the brushes’ own planes, choosing at each step the plane that scores best on Quake’s heuristic — coplanar faces are worth a great deal, splits cost the same amount, axial planes get a bonus. Detail brushes are held out of this entirely, which is the single biggest lever a designer has over compile time.

  4. Portals and flood fill. Portals are the polygons where two leaves touch. Flooding outward from every entity through passable portals answers three questions at once: is the map sealed, which leaves are inside it, and what is the connectivity graph the visibility compile needs.

  5. Outside removal. Leaves the flood never reached become solid. This deletes the entire outer shell of the level — the backs of every wall, the underside of the floor — and it is why a sealed map is so much cheaper than an open one.

  6. Emission. Faces are filed down the tree into the leaves that can see them, vertices are welded so adjacent faces share records (or their seams crack open under rounding), and edges are shared through the surfedge indirection.

Umbra: visibility

For every cluster, which other clusters can be seen from anywhere inside it.

Base vis is cheap and generous — two portals might see each other if each pokes out on the far side of the other, flooded transitively. The full portal flow then narrows a shrinking sight cone with separating planes: the planes touching one edge of the source and one vertex of each portal passed through. When the cone closes to nothing, everything beyond is invisible.

The result is a bit per cluster pair, run-length encoded. At runtime the renderer decompresses one row and skips every leaf not in it.

Resonance: acoustics

Every leaf is listened to from inside: a few hundred rays go out and bounce until they have nothing left, and each surface they strike gives up how much it soaks up per band, from its material. The averages — how far a ray goes between surfaces, how much it loses at each — are what Eyring’s formula for reverberation time wants, measured in the room’s actual shape rather than assumed for a box.

Leaves that touch and sound alike are then gathered into rooms by union-find over the portal graph, widest portals first, merging only while the merged average still sounds like both halves. A hall the tree cut into twenty leaves is one hall again; a doorway is a boundary because the two sides disagree. One record per room goes in the map, and a leaf-to-room table beside it.

Radiance: lighting

Every lit face carries a grid of luxels. For each, find where it sits in the world and ask every light whether it can see it, then bounce.

Two details do most of the work of making it look right. Grid points near the edge of an angled face fall inside neighbouring geometry — left alone they bake black and produce a dark rim around every surface, so they are pulled toward the face centre until they clear. And shadow rays start slightly off the surface, because starting exactly on it means the first thing every ray hits is the face it came from.

Runtime

The tick

Fixed at 64 Hz, decoupled from rendering. A 240 Hz display draws 240 smooth frames of a 64 Hz simulation, not 240 simulations. The catch-up accumulator is capped so that a stall — a breakpoint, a window drag — does not produce a burst of hundreds of ticks that looks like the world fast-forwarding.

Each tick:

  1. Categorise the player’s position: are they standing on something?
  2. Friction, then acceleration, then the move itself. That order matters: running friction after acceleration makes ground movement mushy and breaks air strafing entirely.
  3. Update triggers from the player’s box.
  4. Run the entity queue: deliver due events, run thinks, reclaim removals.

Collision

Traces work against brushes, not triangles. A brush is a handful of planes, so testing one is a handful of dot products regardless of how detailed its surface is, and the BSP narrows the search to the brushes actually along the path.

Brush entities — doors, platforms — are separate models kept out of the world tree so they can move without re-splitting it. They are traced separately and the nearest hit taken. Forgetting that is how you walk through every door in the game.

Rendering

What to draw is decided long before the GPU is involved:

  1. Find the viewer’s cluster.
  2. Take its PVS row — the clusters that can possibly be seen.
  3. Frustum-cull the leaves in those clusters, then their surfaces.
  4. Draw what is left, batched by material.

The PVS and the frustum do different jobs and neither subsumes the other: the PVS removes rooms you cannot see through any opening, the frustum removes what is behind you.

Brush entities are a second pass, and have to be. A door, a lift, anything tied to a class is compiled as its own model with its own leaves, and those leaves are not in the world’s PVS. A leaf walk therefore finds the world and nothing else — which is exactly what happened: every brush entity in every map was built into the mesh and never drawn, and the sample map’s door was simply not there. They are drawn after the world, one model at a time, culled against the frustum by their bounds rather than by the PVS. There are a handful of them, they are the things the player walks up to, and culling one wrongly costs far more than drawing it.

Each carries a displacement — where its entity has moved to — applied in the vertex shader through a uniform with a dynamic offset. That displacement comes from the same fields the collision code traces against, so what you see and what you walk into are the same thing by construction.

Lightmaps pack into one atlas per section, so a section draws in as many calls as it has materials rather than one per face.

Streamed sections

A map can be more than one machine wants resident at once. Chisel lets the designer mark a visgroup as streamed; Cleave numbers those visgroups as sections after the always-loaded world (section 0) and tags every world face and brush with its section. At runtime kerosene_engine::streaming decides, once a tick, which sections should be resident:

  • the world, always;
  • any section with a face in a cluster the player’s cluster can hear — the PAS, which is the PVS flooded one doorway further. A room therefore starts loading a doorway before it can be seen, and that doorway is the margin that hides the build;
  • any section an awake physics prop is inside, so a crate thrown into the next room does not fall through a floor that has since gone.

A section that stops being wanted lingers sv_stream_linger seconds before it is dropped, so pacing over a threshold does not thrash. With no vis data, with the player outside the world, or with sv_stream 0, everything is wanted.

What streams is the expensive part and only that: each section’s render mesh and lightmap atlas, built on a worker thread from the shared BSP and uploaded on the main thread, and its rigid-body hulls. The BSP tree, the entity list, the player’s traces and entity I/O are whole whatever is loaded — the tree is small, the traces have to work everywhere, and a logic_relay in an unloaded room still fires. Brush entities are never streamed: a door is drawn and collided with wherever it has moved to. This is sections of one map, not an open world: everything is still one .kbsp, compiled and lit as one.

r_stream_debug 1 draws each section’s bounds in the colour of its state. content/maps/two_rooms.kmap in Kerosene Demo is a small map with two streamed rooms.

Entities

Entities are a bag of named fields rather than typed structs, because the meaningful fields belong to the game, not the engine. Classes are registered handlers — the same split Source draws between its engine and its game DLL. kerosene-entity knows how to route an input; the game – kerosene-game for the stock classes, yours through the Game trait – decides what Open means.

Outputs become queued events even at zero delay, so an entity firing at itself cannot recurse into the stack, and ordering does not change when a delay is added. Handles carry a generation, so an event naming an entity that died before it fired resolves to nothing rather than to whoever took its slot.

Testing

Where a subsystem can be tested without a GPU or a window, it is:

  • Movement runs against a hand-built box world — a floor, a step, a wall — rather than a compiled map, so a failing test is a movement bug rather than a map bug.
  • Shaders are validated through naga, the same compiler wgpu uses, so a typo fails in CI rather than at pipeline creation on a machine with a display.
  • The whole pipeline is exercised by tests that build a .kmap in memory, compile it through Cleave, load the result and play it. Every crate can pass its own tests and still not add up to a level you can walk around; that suite is where the seams show.