Dark-Horse-Linux/dpm-core-ng
3
0
Fork 0
Code Issues Pull Requests Actions Activity

Route all module interaction through dispatch

Browse Source 
A module is addressed by name and command string, and nothing else.
Typed API access handed a caller a pointer into the callee's function
table, which meant compiling against that module's struct layout — a
build-time dependency between modules that the design does not permit.
Removing it also removes the manifest, the table magic constant, and
the table size field, which existed only to describe and validate
those tables.

Load validation is now three steps: reserved contract symbols resolve,
the minimum-version handshake passes, and the version and description
probes return well-formed values. The contract is four reserved
symbols, and a module's interface is the command vocabulary it
documents.

Documentation is brought in line, and artifacts are named exactly:
libdpm-core.so for the library, <dpm/core.h> for the header, the dpm
binary for the command-line tool.
This commit is contained in:
Chris Punches
2026-08-15 02:15:18 -04:00
parent 267529bee3
commit 97b39cac6c
17 changed files with 263 additions and 702 deletions
Download Patch File Download Diff File Expand all files Collapse all files

183
docs/DESIGN.md
View File

@@ -4,59 +4,55 @@
- Must be able to operate in a barren environment providing libc and libstdc++ — the standard build runs there as-is, no special variants. On a fully populated system the same binaries simply have more modules loadable.
- Capability grows in layers: each layer installs the dependencies of the next using only what already works.
- Every implementation exists exactly once, and is consumable by the CLI, by other layers, and by external programs (build systems, Dark Horse tooling) through C interfaces.
- Every implementation exists exactly once, and is consumable by the `dpm` binary, by other layers, and by external programs (build systems, Dark Horse tooling) through C interfaces.
- No module ever links, includes, or hardcodes anything belonging to another module. Every module-to-module interaction passes through libdpm-core.so.
## Architecture overview
```
dpm CLI build systems / DHL tools / other languages
\ /
v v
libdpm-core.so
(discovery, validation, routing,
version reporting, config, logging)
| |
v v
raw module pkg module ... future modules (repo, source, ...)
(one .so) (one .so)
| |
v v
backing tree sqlite3
(source of truth) (derived cache)
the dpm binary -> <dpm/core.h> -> module A -> <dpm/core.h> -> module B
a build system -> <dpm/core.h> -> module A -> <dpm/core.h> -> module B
```
`libdpm-core.so` is the single entry point for everything. Modules are shared objects that implement package functionality. All routing — library-to-module and module-to-module — passes through libdpm-core. No consumer touches `dlopen`, `dlsym`, or module discovery itself.
`<dpm/core.h>` is the declared interface; `libdpm-core.so` implements it. A caller includes the header, links `-ldpm-core`, and asks the library to discover and invoke modules by name. A module reaching a peer performs the identical two steps its own caller performed, which is why the chain above repeats rather than branching into a second mechanism.
The header names no module and carries no module-specific type. It offers discovery and interaction, and that is the entire vocabulary — which is what allows the `dpm` binary, a build system, and a module to all use it without any of them being privileged.
### One process, one library instance
The whole chain executes in the caller's address space. The `dpm` binary (or the build system) is the process; libdpm-core.so is mapped into it; every module the library loads is mapped into it as well. Calls are direct function calls — no subprocess, no serialization, no output parsing.
A module links libdpm-core.so like any other consumer, and when the module is loaded the dynamic linker binds it to the copy already mapped in the process. There is one instance, so a module drives the same context, module registry, configuration, and log targets the original caller opened.
Modules are loaded with `RTLD_LOCAL`, so a module's symbols never enter the global namespace. The only symbols a module can resolve are libdpm-core.so's. No path from one module to another exists that does not pass through the library.
## Versioning model
Compatibility is directional, and the module is the one that declares it:
- **Each module reports the minimum libdpm-core version it supports** (a reserved contract symbol). At load, the running library compares its own version against that minimum: if it is older → refuse with an explicit "libdpm-core too old for this module" report; otherwise load.
- **Consuming modules judge module versions; libdpm-core does not.** libdpm-core reports the version it saw at load and draws no conclusion from it. A module that depends on another requires it by name, reads the reported version, and decides for itself whether that version is too new or too old for what it intends to call. A load is a statement that the module is valid, never that it is compatible with a particular caller.
- **An API is identified by name and table version.** A consumer asks for the exact pair it was written against, and a module that no longer carries that pair fails the request. Retiring a table is a version change in the module that retired it, judged by every consumer that requires it.
- **Each module reports the minimum library version it supports** (a reserved contract symbol). At load, the running library compares its own version against that minimum: if it is older → refuse with an explicit "library too old for this module" report; otherwise load.
- **Consuming modules judge module versions; libdpm-core.so does not.** The library reports the version it saw at load and draws no conclusion from it. A module that depends on another requires it by name, reads the reported version, and decides for itself whether that version is suitable for the commands it intends to issue. A load is a statement that the module is valid, never that it is compatible with a particular caller.
## libdpm-core.so
Dependencies: libc, libstdc++, libdl. Never more — libdpm-core must remain loadable in the barren case forever, so package logic never leaks into it. libdpm-core routes and hosts; modules implement.
Dependencies: libc, libstdc++, libdl. Never more — the library must remain loadable in the barren case forever, so package logic never leaks into it. libdpm-core.so routes and hosts; modules implement.
libdpm-core provides:
It provides:
- **Discovery**: module path resolution, enumeration of installed module .so's.
- **Validation**: the full load-time contract enforcement described below. libdpm-core is the sole authority on what a valid module is; the contract definition lives inside the library as data. There is no SDK package — the interface is specified by this document and enforced by the library's validator.
- **Routing**:
- generic dispatch — execute a command string with arguments against a named module (what the CLI uses);
- typed access — a consumer requests a module's API at a version and receives a C function table (what modules and external programs use).
- **Validation**: the full load-time contract enforcement described below. libdpm-core.so is the sole authority on what a valid module is; the contract definition lives inside the library as data. There is no SDK package — the interface is specified by this document and enforced by the library's validator.
- **Routing**: dispatch a command string with arguments to a named module. This is the only path into module code, and it is the same path for the `dpm` binary, for a build system, and for a module calling a peer.
- **Version reporting**: require resolves a module by name, loads it, and returns a handle, or reports precisely why it can't. The loaded module's version is readable from the handle, for the caller to judge.
- **Common services**: configuration access (per-module namespaces from `/etc/dpm/conf.d/`), logging, module-path queries.
## The libdpm-core C API
## The C API
All functions are extern "C". All returned strings are owned by libdpm-core (or by the module that produced them), are valid until the context is closed, and are never freed by the caller. All functions returning int use 0 for success and nonzero error codes; details of the most recent failure are retrievable per-context.
All functions are extern "C". All returned strings are owned by the library (or by the module that produced them), are valid until the context is closed, and are never freed by the caller. All functions returning int use 0 for success and nonzero error codes; details of the most recent failure are retrievable per-context.
### Context lifecycle
**`dpm_ctx* dpm_open(const dpm_open_overrides* overrides)`**
Creates a libdpm-core context. Reads configuration from `/etc/dpm/conf.d/` (or the config directory named in overrides), resolves the module path (overrides take precedence over config, config over the built-in default), and initializes logging per configuration. Performs no module loading. Returns NULL only on allocation failure or an unreadable/invalid explicit override; a missing config directory is not an error — defaults apply. `overrides` may be NULL, and may specify: config directory, module path, target root (for chroot/image/sysroot operation), and log level. Multiple simultaneous contexts with different roots are legal.
Creates a context. Reads configuration from `/etc/dpm/conf.d/` (or the config directory named in overrides), resolves the module path (overrides take precedence over config, config over the built-in default), and initializes logging per configuration. Performs no module loading. Returns NULL only on allocation failure or an unreadable/invalid explicit override; a missing config directory is not an error — defaults apply. `overrides` may be NULL, and may specify: config directory, module path, target root (for chroot/image/sysroot operation), and log level. Multiple simultaneous contexts with different roots are legal.
**`void dpm_close(dpm_ctx* ctx)`**
Releases the context: unloads every module handle it issued, closes log targets, frees all memory owned by the context. All handles and strings obtained through the context are invalid after this call. NULL is a no-op.
@@ -67,13 +63,12 @@ Releases the context: unloads every module handle it issued, closes log targets,
Resolves the module `name` in the module path and runs the full load-time validation sequence (see Load-time enforcement) if the module is not already loaded in this context. On success returns a module handle owned by the context (repeated calls return the same handle — modules are loaded at most once per context). On failure returns NULL and records the precise reason: not found, or validation step failed with the step and detail. No version criterion is applied here; compatibility is the caller's to determine from the reported version.
**`int dpm_module_info_of(dpm_ctx* ctx, dpm_module* mod, dpm_module_info* out)`**
Fills `out` with the loaded module's name, version, description, and minimum-libdpm-core version exactly as they were read at load (string pointers valid until context close). This is how a consumer obtains the version it will judge. libdpm-core attaches no meaning to the values. Returns 0 on success, nonzero if the module cannot be reported on.
**`const void* dpm_get_api(dpm_ctx* ctx, dpm_module* mod, const char* api_name, int table_version)`**
Returns the API table `api_name` at `table_version` from a loaded module — the pointer the module exported for that table, already validated (manifest cross-check, magic, minimum size) at load. The caller casts it to the table struct type for that API and version as defined in the module's documented API. Returns NULL if the module does not provide that api/version pair; that fact is known from the manifest without further probing. The table is valid for the life of the context.
Fills `out` with the loaded module's name, version, description, and minimum-library version exactly as they were read at load (string pointers valid until context close). This is how a consumer obtains the version it will judge. The library attaches no meaning to the values. Returns 0 on success, nonzero if the module cannot be reported on.
**`int dpm_execute(dpm_ctx* ctx, dpm_module* mod, const char* command, int argc, char** argv)`**
Generic dispatch: invokes the module's `dpm_module_execute` with the context, `command`, and the argument vector. Returns the module's return value verbatim (0 = success). libdpm-core adds nothing to the call besides delivery; argument semantics beyond "argv[0] is the command" are the module's to define.
Dispatch: invokes the module's `dpm_module_execute` with the context, `command`, and the argument vector. Returns the module's return value verbatim (0 = success). The library adds nothing to the call besides delivery; argument semantics beyond "argv[0] is the command" are the module's to define.
This is the entire path into module code. A caller addresses a module by name and a capability by command string, so it holds no compile-time knowledge of the module it is calling — no headers, no struct layouts, no symbols. That is what allows a module to be developed, built, and tested with no peer present.
### Enumeration
@@ -81,7 +76,7 @@ Generic dispatch: invokes the module's `dpm_module_execute` with the context, `c
Scans the module path and returns a cursor over all *valid* modules (each candidate .so is validated on first scan; failures are logged and excluded). Returns NULL on an unreadable module path.
**`int dpm_cursor_next(dpm_cursor* cur, dpm_module_info* out)`**
Advances the cursor. Fills `out` with the next module's name, version, description, and minimum-libdpm-core version (string pointers valid until context close). Returns 0 and fills `out` while entries remain; returns nonzero at end.
Advances the cursor. Fills `out` with the next module's name, version, description, and minimum-library version (string pointers valid until context close). Returns 0 and fills `out` while entries remain; returns nonzero at end.
**`void dpm_cursor_free(dpm_cursor* cur)`**
Releases the cursor. NULL is a no-op.
@@ -89,7 +84,7 @@ Releases the cursor. NULL is a no-op.
### Services (available to modules and external consumers alike)
**`const char* dpm_core_version(void)`**
Returns the libdpm-core version as a static X.Y.Z string. Callable without a context.
Returns the library version as a static X.Y.Z string. Callable without a context.
**`const char* dpm_config_get(dpm_ctx* ctx, const char* module, const char* section, const char* key)`**
Returns the configured value for `key` in `section` of the named module's config namespace (`/etc/dpm/conf.d/`&lt;module&gt;.conf; the namespace "core", from `core.conf`, is the library's own). Returns NULL if unset. String valid until context close.
@@ -105,71 +100,65 @@ Returns a human-readable description of the most recent failure recorded on this
## Module contract
A module is one .so in the module directory. It exports, as extern "C", the following reserved symbols. Returned strings are static or module-owned, non-NULL, and valid for the lifetime of the loaded module; libdpm-core and consumers never free them.
A module is one .so in the module directory. It includes `<dpm/core.h>`, links `-ldpm-core`, and exports the following reserved symbols as extern "C". Returned strings are static or module-owned, non-NULL, and valid for the lifetime of the loaded module; the library and consumers never free them.
**`int dpm_module_execute(dpm_ctx* ctx, const char* command, int argc, char** argv)`**
The module's generic command entry point. `ctx` is the host context that dispatched the call — the module reaches every libdpm-core service (`dpm_log`, `dpm_config_get`, `dpm_module_path`, ...) through it. `command` is the subcommand name (equal to argv[0] when argc > 0); argc/argv are the remaining CLI-style arguments. NULL or empty `command` must behave as the module's help command. Returns 0 on success, nonzero on failure. This is the only entry the CLI path ever uses; it must be callable immediately after load with no other setup.
The module's command entry point, and its entire functional surface. `ctx` is the host context that dispatched the call — the module reaches every service (`dpm_log`, `dpm_config_get`, `dpm_module_path`, ...) through it, and reaches peer modules through it as well. `command` is the subcommand name (equal to argv[0] when argc > 0); argc/argv are the remaining CLI-style arguments. NULL or empty `command` must behave as the module's help command. Returns 0 on success, nonzero on failure. It must be callable immediately after load with no other setup.
**`const char* dpm_module_version(void)`**
Returns the module's own version as an X.Y.Z string. Must be constant for the life of the module. This is the value libdpm-core reports to consumers, and the value they judge compatibility against.
Returns the module's own version as an X.Y.Z string. Must be constant for the life of the module. This is the value the library reports to consumers, and the value they judge compatibility against.
**`const char* dpm_module_description(void)`**
Returns a one-line human-readable description, used in module listings.
**`const char* dpm_module_core_min(void)`**
Returns the minimum libdpm-core version (X.Y.Z) this module supports — the oldest one whose contract and services the module was written against. A libdpm-core older than that refuses to load the module, and says so.
Returns the minimum library version (X.Y.Z) this module supports — the oldest one whose contract and services the module was written against. A library older than that refuses to load the module, and says so.
**`const dpm_manifest* dpm_module_manifest(void)`**
Returns a pointer to a static manifest table declaring the module's entire functional surface: an entry count and, per entry, the API name, its table version, and the exact exported symbol that carries the table (e.g. { "raw", 1, "raw_api_v1" }). libdpm-core trusts nothing it doesn't verify: every declared symbol is resolved at load, and only manifest-declared tables are ever handed to consumers. An API absent from the manifest does not exist, even if its symbol does.
**Symbol naming**: functional exports are prefixed with the module's name (raw_\*, pkg_\*); the dpm_ prefix is reserved for the contract and for libdpm-core.so.
**API tables** (the module's functional surface): for each API version, one exported symbol (e.g. `raw_api_v1`) pointing to a plain C struct of function pointers. Every table opens with two fixed members: a magic constant (a fixed value defined by this spec, confirming the exporter agrees on table layout conventions) and the struct size in bytes (populated by the module, letting consumers accept tail-extended revisions of the same version). All parameters and returns are C types only; state passes through opaque handles; errors are int codes.
**Symbol naming**: functional exports are prefixed with the module's name (raw_\*, pkg_\*); the dpm_ prefix is reserved for the contract and libdpm-core.
**A module publishes no headers to other modules.** Its capabilities are addressed by command string through `dpm_execute`, so nothing about its internals — types, struct layouts, symbol names — is ever compiled into a caller. A module's documented command vocabulary is its interface.
## Load-time enforcement
libdpm-core is the sole authority on module validity; the contract above is enforced by its validator, not by any SDK. Validation is all-or-nothing; a module is registered only after passing every step:
libdpm-core.so is the sole authority on module validity; the contract above is enforced by its validator, not by any SDK. Validation is all-or-nothing; a module is registered only after passing every step:
1. **Resolve all reserved contract symbols.** Any missing → refuse, log the exact list, `dlclose`.
2. **Minimum-version handshake.** `dpm_module_core_min`() must be ≤ the running libdpm-core version. If the library is too old, refuse and say so — the remedy is updating libdpm-core, and the message names it. Old modules on a newer libdpm-core always pass.
2. **Minimum-version handshake.** `dpm_module_core_min`() must be ≤ the running library version. If the library is too old, refuse and say so — the remedy is updating the library, and the message names it.
3. **Probe the cheap calls.** `dpm_module_version`() and `dpm_module_description`() are invoked immediately; NULL or malformed returns → refuse.
4. **Cross-check the manifest.** Every API the module declares must actually resolve via `dlsym`. A module advertising an API it doesn't export is refused. libdpm-core validates the module's entire declared surface at load, before offering any of it.
5. **Table sanity.** Check each declared table's magic constant (catches modules built against a stale or wrong layout) and minimum size for its version.
Failures happen at install/load time, loudly and itemized. Consumers never receive a partially valid module: if a handle was handed out, the contract already validated. The residual C-ABI limit — `dlsym` cannot verify signatures — is covered in practice by the magic, size, minimum-version handshake, and probes; defeating those requires deliberate lying, which is a package-signing concern upstream of the loader.
Failures happen at install/load time, loudly and itemized. Consumers never receive a partially valid module: if a handle was handed out, the contract already validated.
## Module: raw — the file-based installer
Ships with the base system alongside libdpm-core. Depends on the baseline only; archive decompression is vendored in, and the package format is chosen to keep that small. This is what makes barren-environment operation possible: libdpm-core plus raw function with nothing else present.
Ships with the base system alongside libdpm-core.so. Depends on the baseline only; archive decompression is vendored in, and the package format is chosen to keep that small. This is what makes barren-environment operation possible: libdpm-core.so plus raw function with nothing else present.
- **Owns the backing tree** (`/var/lib/dpm/`): one directory per installed package holding manifest, metadata, and hooks. The tree is the database at this layer.
- **Operations** (exposed both as commands and in `raw_api_v1`): install a package file, remove, verify, and queries answered by walking the tree — slow but always correct, zero dependencies.
- **Owns the lock file and an append-only transaction journal** with a generation counter. Every mutation in the entire system ultimately passes through raw, so locking and journaling are implemented exactly once and inherited by every layer above.
- **Operations**, addressed as commands: install a package file, remove, verify, and queries answered by walking the tree — slow but always correct, zero dependencies.
- **Owns the lock file and a transaction journal** with a generation counter. Every mutation in the entire system ultimately passes through raw, so locking and journaling are implemented exactly once and inherited by every layer above.
## Module: pkg — the full package manager
Ships as a package, installed by raw once sqlite3 is installed. Requires raw (via libdpm-core, judging raw's reported version itself) and libsqlite3.
Ships as a package, installed by raw once sqlite3 is installed. Requires raw — by name, through libdpm-core.so, judging raw's reported version itself — and libsqlite3.
- Never touches the tree directly: every filesystem mutation is a call into raw's API table, obtained from libdpm-core, in-process — shared locking, real error propagation, no output parsing.
- Never touches the tree directly: every filesystem mutation is a dispatch to raw through the library, in-process — shared locking, real error propagation, no output parsing. pkg is built with no knowledge of raw beyond its name and its command vocabulary.
- **The sqlite database is a derived cache** under one invariant: it contains nothing that cannot be rebuilt by scanning the tree. It records the last journal generation it applied; on open, if the tree is ahead (someone used raw directly — explicitly allowed, that's the escape hatch for broken systems), it replays or rebuilds. Self-healing by construction.
- Adds what the cache enables: fast queries, dependency resolution against the installed set, multi-package transactions with rollback.
- Exposes `pkg_api_v1` for consumers that want dependency-aware operations; they transitively get raw's guarantees because there is no second code path to the tree.
- Consumers that want dependency-aware operations dispatch to pkg by name; they transitively get raw's guarantees because there is no second code path to the tree.
## The CLI
## The dpm binary
`dpm` is argument parsing and printing. It links libdpm-core, enumerates modules, and forwards subcommands through generic dispatch. Its command surface is exactly the set of loadable modules — in the barren case that's raw's commands; on a full system, everything installed. No capability logic lives in the CLI.
`dpm` is argument parsing and printing. It includes `<dpm/core.h>`, links `-ldpm-core`, enumerates modules, and forwards subcommands through dispatch. Its command surface is exactly the set of loadable modules — in the barren case that's raw's commands; on a full system, everything installed. No capability logic lives in it.
## External consumers
Build systems and Dark Horse components link `libdpm-core.so` — the same library, the same path as everything else:
Build systems and Dark Horse components link `libdpm-core.so` as an ordinary shared library dependency — `#include <dpm/core.h>`, `-ldpm-core`, the same as any other library they link:
- open a context (optionally against an alternate root),
- require the layer they need,
- require the module they need,
- read its reported version and decide whether it suits them,
- fetch its API table,
- call C functions directly.
- dispatch commands to it.
libdpm-core installs its header to the standard include path and its library to the standard lib path: a consumer writes #include &lt;`dpm/core.h`&gt;, links `-ldpm-core`, and calls package manager module functions. A consumer that opens a default context (no overrides) is operating the installed package manager itself — system configuration, system module path, system tree, system locking — exactly as if it were invoking the installed `dpm` command, because the CLI is just another caller of the same library. Overrides redirect individual paths only when a caller explicitly sets them. Whether a consumer targets raw only (image builders that just deploy trees) or pkg (dependency-aware tooling) is their choice of require(); behavior is identical to the CLI's because it is the same implementation.
The header installs to the standard include path and the library to the standard lib path. A consumer that opens a default context (no overrides) is working against system configuration, the system module path, the system tree, and system locking — the same environment the installed `dpm` binary sees, because that binary is just another caller of the same library. Overrides redirect individual paths only when a caller explicitly sets them. Whether a consumer addresses raw only (image builders that just deploy trees) or pkg (dependency-aware tooling) is their choice of require(); behavior is identical to the `dpm` binary's because it is the same implementation.
## Bootstrap chain
@@ -177,22 +166,22 @@ libdpm-core installs its header to the standard include path and its library to
minimal start: dpm + libdpm-core.so + raw module (baseline deps only)
raw installs: sqlite3 package
raw installs: dpm-pkg package (drops the pkg module .so)
now: libdpm-core discovers pkg, validates it, full management is live
now: libdpm-core.so discovers pkg, validates it, full management is live
```
Every layer is a package installed and upgraded by the layer beneath it; the package manager maintains itself with the same mechanism it offers the OS. Future modules follow the identical pattern — a repo/network module declares its requirements (pkg, a TLS library), lands as a package, and the capability appears on next discovery.
## Repository structure
Modules are developed independently from each other and independently from libdpm-core — one repository per module, plus the libdpm-core repository. Each repo owns its source, build, and tests, and produces exactly one artifact:
Modules are developed independently from each other and independently from the library — one repository per module, plus the libdpm-core.so repository. Each repo owns its source, build, and tests, and produces exactly one artifact:
- **The libdpm-core repository**: `libdpm-core.so` and the `dpm` binary. Contains no module code. Its test fixtures include deliberately broken stub modules for validating the loader, and one known-good stub — never a real package module.
- **The exception**: an info module that bundles with the `dpm` binary and libdpm-core, used to test and demonstrate full DPM system functionality where appropriate.
- **One repository per module** (raw, pkg, and every future module): produces that module's .so. Links `libdpm-core.so` — the only cross-repo build dependency in the system — and nothing else from DPM. Peer modules never appear in a module's repository, build, or test environment; peers are runtime concerns, faked at test time (contract fakes and stub modules) and real only at distribution-level integration.
- **The libdpm-core.so repository**: `libdpm-core.so` and the `dpm` binary. Contains no module code. Its test fixtures include deliberately broken stub modules for validating the loader, and one known-good stub — never a real package module.
- **The exception**: an info module that bundles with the `dpm` binary and libdpm-core.so, used to test and demonstrate full DPM system functionality where appropriate.
- **One repository per module** (raw, pkg, and every future module): produces that module's .so. Links `libdpm-core.so` — the only cross-repo build dependency in the system — and nothing else from DPM. Peer modules never appear in a module's repository, build, or test environment; peers are runtime concerns, faked at test time with stub modules and real only at distribution-level integration.
No repository can block another's development: a module builds and its full pre-integration test surface (unit, contract, module-hosting) runs with nothing present but its own checkout and an installed or vendored libdpm-core. Release coordination happens through the versioning model — each consumer judging the versions it is handed — rather than through lockstep builds.
No repository can block another's development: a module builds and its full pre-integration test surface runs with nothing present but its own checkout and an installed or vendored libdpm-core.so. Release coordination happens through the versioning model — each consumer judging the versions it is handed — rather than through lockstep builds.
### Layout of the libdpm-core repository
### Layout of the libdpm-core.so repository
```
include/dpm/ public headers — installed to the system include path; the
@@ -200,7 +189,7 @@ include/dpm/ public headers — installed to the system include path; the
consumer writes #include <dpm/core.h>
include/internal/ library-private headers — used only by src/, never installed
src/ implementations of the library
src/cli/ the dpm CLI entry point
src/cli/ the dpm binary's entry point
src/bundled-modules/info/ the bundled info module
data/ files installed as-is (core.conf)
tests/ fixture modules, the API test binary, CLI tests
@@ -211,51 +200,49 @@ Every header lives under include/: include/dpm/ is the published API surface and
## Artifacts
Terminology: **the `dpm` binary** names the command-line tool; **libdpm-core** names the library.
Terminology: **the `dpm` binary** names the command-line tool; **libdpm-core.so** names the library; **`<dpm/core.h>`** names its header.
| Artifact | Location on system |
|---|---|
| `dpm` | `/usr/bin/dpm` |
| `libdpm-core.so` | `/usr/lib/libdpm-core.so` |
| `core.h` | `/usr/include/dpm/core.h` |
| `info.so` | `/usr/lib/dpm/modules/info.so` |
| modules (`raw.so`, `pkg.so`, `repo.so`, `source.so`, ...) | `/usr/lib/dpm/modules/<name>.so` |
## Development and testing
Development works because the design has no build-time coupling between peers: nothing links against a peer module, ever. "Not all the pieces are there" is the normal, permanent condition at build time. What remains resolves into four test layers, each needing strictly less than the full system.
Development works because the design has no build-time coupling between peers: nothing links against a peer module, ever, and nothing includes a peer's headers, ever. "Not all the pieces are there" is the normal, permanent condition at build time. What remains resolves into three test layers, each needing strictly less than the full system.
### What a build requires
- A module compiles against its own declarations (written to the spec — the externs it exports, the table structs it consumes) plus **`libdpm-core.so`, the one real link dependency** — and libdpm-core is by definition the stable, always-present, baseline-only piece. Cheap to have in every dev environment, trivially vendorable as a checkout.
- Peer modules are reached at runtime through the require/get_api pair. Building pkg does not require raw to exist anywhere. The compile-time knowledge of raw is just the `raw_api_v1` struct layout, which is spec, not artifact.
A module compiles against `<dpm/core.h>` and links `libdpm-core.so` — the one real link dependency, and by definition the stable, always-present, baseline-only piece. Cheap to have in every dev environment, trivially vendorable as a checkout.
A module repo therefore builds self-contained, always.
Peers are reached at runtime by name and command string. Building pkg does not require raw to exist anywhere, and there is no compile-time knowledge of raw to acquire — not a header, not a struct, not a symbol. A module repo therefore builds self-contained, always.
### Test layers
**1. Unit tests — need nothing.** The module's implementation compiles once as an object library, linked into both the .so and a test binary. Pure logic, error paths, parsing — no libdpm-core, no peers.
**1. Unit tests — need nothing.** The module's implementation compiles once as an object library, linked into both the .so and a test binary. Pure logic, error paths, parsing — no library, no peers.
**2. Contract tests — need a struct, not a module.** Because every dependency is an API table — a plain struct of function pointers — a fake is just a struct the test fills in with functions that record calls and return canned results. pkg's logic is exercised against a fake `raw_api_v1` (verifying it calls install/remove/query correctly, handles raw's error codes, honors the journal-generation protocol) with raw nowhere on the machine. Injection is built into the architecture; no linker seams required.
**2. Module-hosting tests — need libdpm-core.so only.** A harness links the real library, points the module path at the build output plus fixtures, and has it load the just-built .so exactly as production would — full validation included, so contract violations fail here, in CI, not on a user's system. Where the module needs a peer, the fixture directory contains a **stub module**: a tiny .so exporting the reserved symbols and answering the commands the module under test issues, which the library validates and dispatches to like the real thing. The harness then drives `dpm_module_execute` end to end against fixture config and data. Because a peer is addressed by name and command, the stub is a complete substitute — there is nothing else about the real peer that the module under test could have depended on. This layer runs on a bare builder with nothing installed.
**3. Module-hosting tests — need libdpm-core only.** A harness links the real libdpm-core, points the module path at the build output plus fixtures, and has it load the just-built .so exactly as production would — full five-step validation included, so contract violations fail here, in CI, not on a user's system. Where the module needs a peer, the fixture directory contains a **stub module**: a tiny .so exporting the reserved symbols and a fake table, which libdpm-core validates and serves like the real thing. The harness then drives `dpm_module_execute` end to end against fixture config and data. This layer runs on a bare builder with nothing installed.
**4. Integration — the only layer that needs everything, and it builds itself.** Real libdpm-core plus real raw, then the actual bootstrap chain into a scratch root: `dpm_open` against an alternate root, raw installs sqlite3 and the pkg package into it, libdpm-core discovers pkg, real operations run against the throwaway tree. Because alternate roots are first-class in the API, this needs a directory, not a VM. Full-distribution CI does the same with real packages.
**3. Integration — the only layer that needs everything, and it builds itself.** Real libdpm-core.so plus real raw, then the actual bootstrap chain into a scratch root: `dpm_open` against an alternate root, raw installs sqlite3 and the pkg package into it, the library discovers pkg, real operations run against the throwaway tree. Because alternate roots are first-class in the API, this needs a directory, not a VM. Full-distribution CI does the same with real packages.
### Day-to-day workflow
- Working on **pkg**: edit, run unit + contract tests (instant, zero environment), harness run before merge. A real raw is never needed, or even possessed, until integration.
- Working on **raw**: same, except its fakes point the other way — its tests need only fixture package files and a scratch tree.
- Working on **libdpm-core**: its test fixtures are deliberately broken modules — missing symbols, wrong magic, lying manifests, a too-new minimum-version declaration — plus one known-good stub. Development never needs any real package module.
- **Debugging** is the layer-3 harness under a debugger — it is the "run the module without the system" mechanism, so no separate standalone build exists or is maintained.
- Working on **pkg**: edit, run unit tests (instant, zero environment), harness run with a raw stub before merge. A real raw is never needed, or even possessed, until integration.
- Working on **raw**: same, except its tests need only fixture package files and a scratch tree.
- Working on **libdpm-core.so**: its test fixtures are deliberately broken modules — missing symbols, a too-new minimum-version declaration, a malformed version — plus one known-good stub. Development never needs any real package module.
- **Debugging** is the layer-2 harness under a debugger — it is the "run the module without the system" mechanism, so no separate standalone build exists or is maintained.
The discipline that keeps this honest: fakes and stubs are written to the spec, and layer-3 validation plus the layer-4 bootstrap run in CI, so a fake that drifts from reality is caught by the first integration pass rather than shipped.
The discipline that keeps this honest: stubs are written to the command vocabulary the real peer documents, and layer-2 validation plus the layer-3 bootstrap run in CI, so a stub that drifts from reality is caught by the first integration pass rather than shipped.
## Development capabilities
During development the CLI must be pointable at a local libdpm-core, and that library must be configurable to local paths (module path, config dir, etc.). Two mechanisms provide this:
During development the `dpm` binary must be pointable at a locally built library, and that library must be configurable to local paths (module path, config dir, etc.). Two mechanisms provide this:
- **Pointing the CLI at a local libdpm-core** is dynamic-linker territory, needing no DPM mechanism: development builds of the CLI carry an rpath to their own build tree's lib/ directory, so the locally built binary resolves the locally built library (`LD_LIBRARY_PATH` pointed at that lib/ directory achieves the same). The system libdpm-core is never touched.
- **Pointing that library at local paths** is what the `dpm_open` overrides exist for: config directory, module path, and target root are all fields of the overrides struct, and the CLI exposes them as flags. A dev invocation:
- **Pointing the binary at a local library** is dynamic-linker territory, needing no DPM mechanism: development builds carry an rpath to their own build tree's lib/ directory, so the locally built binary resolves the locally built library (`LD_LIBRARY_PATH` pointed at that lib/ directory achieves the same). The system-installed library is never touched.
- **Pointing that library at local paths** is what the `dpm_open` overrides exist for: config directory, module path, and target root are all fields of the overrides struct, and the `dpm` binary exposes them as flags. A dev invocation:
```
./build/bin/dpm --config-dir ./tests/fixtures/conf --module-path ./build/modules raw install ./fixture.dpm
@@ -263,18 +250,20 @@ During development the CLI must be pointable at a local libdpm-core, and that li
The config-dir override matters most: once the context reads config from the local conf dir, everything configurable — log file, module path defaults, per-module settings — resolves locally, so a checkout plus its fixtures is a complete self-contained environment. Adding `--root` at a scratch directory makes even real install operations land in a throwaway tree.
**Rule**: every field of the `dpm_open` overrides struct must be exposed as a CLI flag, so anything a linked consumer can redirect, a developer at the shell can redirect too. This holds for any future override field — nothing ships reachable from code but not from the command line.
**Rule**: every field of the `dpm_open` overrides struct must be exposed as a flag on the `dpm` binary, so anything a linked consumer can redirect, a developer at the shell can redirect too. This holds for any future override field — nothing ships reachable from code but not from the command line.
## Evolution rules
- **Breaking a module API means a new table version**: `raw_api_v2` ships as its own symbol, and `raw_api_v1` is retired on raw's schedule, published in raw's own version. A consumer that requires a retired table finds out when it asks for it, by name and version, before it does any work.
- **Breaking the library's exported ABI means a new symbol version node**: the version script pins the export set, and a break ships as a new node above the existing one, so binaries already linked keep resolving the node they were built against.
- **Compatibility is decided by the consumer**: modules declare the minimum libdpm-core they support, and the library enforces that one handshake because it is the host. Everything else is reported, not enforced — libdpm-core hands a consumer the version it saw and the consumer decides whether to proceed.
- **A module's interface is its command vocabulary.** Retiring or changing the meaning of a command is a version change in the module that owns it, judged by every consumer that dispatches to it.
- **Breaking the library's exported ABI means a new symbol version generation**: the export set is what consumers link against, so a break is a deliberate, versioned event rather than an incidental one.
- **Compatibility is decided by the consumer**: modules declare the minimum library version they support, and libdpm-core.so enforces that one handshake because it is the host. Everything else is reported, not enforced — the library hands a consumer the version it saw and the consumer decides whether to proceed.
## Invariants
1. libdpm-core routes and hosts; modules implement. No package logic in the library, ever.
2. Writes flow down, never sideways: a layer mutates the system only through the layer beneath it, in-process through libdpm-core-mediated APIs.
3. Truth lives in the tree; everything above is regenerable cache or convenience.
4. Dropping down a layer by hand is always legal; layers above detect it and reconcile.
5. A module is either fully valid or not loaded — no partial states, no consumer-side defense.
1. libdpm-core.so routes and hosts; modules implement. No package logic in the library, ever.
2. `<dpm/core.h>` names no module and carries no module-specific type. It offers discovery and interaction only.
3. A module never links, includes, or hardcodes anything belonging to another module. All module-to-module interaction passes through libdpm-core.so.
4. Writes flow down, never sideways: a layer mutates the system only through the layer beneath it, in-process through library-mediated dispatch.
5. Truth lives in the tree; everything above is regenerable cache or convenience.
6. Dropping down a layer by hand is always legal; layers above detect it and reconcile.
7. A module is either fully valid or not loaded — no partial states, no consumer-side defense.