Dark-Horse-Linux/dpm-core-ng
2
0
Fork 0
Code Issues Pull Requests Actions Activity

Move version compatibility to the consumer; name libdpm-core explicitly

Browse Source 
dpm_require no longer takes a minimum version and applies no version
criterion of its own. A handle now means the module is valid, not that
it suits the caller. dpm_module_info_of is added alongside it, reporting
the name, version, description, and minimum-libdpm-core version read at
load, so a consuming module can judge a dependency's version for itself.
The one rule still enforced is the minimum-version handshake, where
libdpm-core is the host and refuses a module that demands a newer library
than the one running.

dpm_module_info_of joins the version script, so the exported surface is
now fourteen symbols under DPM_CORE_1.0.

Separately, the bare word "core" is gone from prose everywhere. It named
both the command-line tool and the library, so every use forced the
reader to guess which. Text now says "the dpm binary" or "libdpm-core".
Identifiers keep their spelling: libdpm-core, core.h, core.conf, the
"core" configuration namespace, dpm_core_version, core_min, DPM_CORE_1.0,
the dpmcore namespace, test_core, core_api.

Three user-visible strings changed with it: the load-refusal message now
reads "requires libdpm-core >= X, running libdpm-core is Y — update
libdpm-core", and the info module's description and help text name the
library. The test asserting on the refusal text was updated to match.

DESIGN.md's terminology line no longer defines "DPM Core" as the CLI,
which was the source of the ambiguity. OVERVIEW.md is restructured around
the three layers a reader meets DPM at — user, developer, filesystem —
so a code-level symbol never appears without saying whose layer it is.
MODULES.md describes the bundled info module as testing and demonstrating
full DPM system functionality rather than as a reference implementation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Punches
2026-08-10 00:38:21 -04:00
parent da35fe4758
commit c41adc2498
22 changed files with 327 additions and 243 deletions
Download Patch File Download Diff File Expand all files Collapse all files

151
docs/OVERVIEW.md
View File

@@ -2,87 +2,131 @@
## What DPM is
DPM is the package manager for Dark Horse Linux. Its core is **libdpm-core**, a shared library that discovers modules, validates them, routes calls to them, negotiates versions, and provides configuration and logging. Modules — shared objects in the module directory — implement package functionality.
DPM is the package manager for Dark Horse Linux. At its center is **libdpm-core**, a shared library that discovers modules, validates them, routes calls to them, reports their versions, and provides configuration and logging. Modules — shared objects in the module directory — implement package functionality.
The `dpm` command-line tool is one consumer of that library. Build systems, image builders, distribution tooling, and programs in any language with C FFI are equal consumers of the same library through the same interface.
**The dpm binary** is the command-line tool, and it is one consumer of that library. Build systems, image builders, distribution tooling, and programs in any language with C FFI are equal consumers of the same library through the same interface.
Core implements no package operations. It routes and hosts.
libdpm-core implements no package operations. It routes and hosts.
## The shape of the system
```
dpm CLI build systems / DHL tools / other languages
\ /
v v
libdpm-core.so
(discovery, validation, routing,
version negotiation, config, logging)
|
v
modules
(one .so each)
the dpm binary build systems / DHL tools / other languages
\ /
v v
libdpm-core.so
(discovery, validation, routing,
version reporting, config, logging)
|
v
modules
(one .so each)
```
Everything passes through core: CLI-to-module, module-to-module, program-to-module. No consumer calls `dlopen`, `dlsym`, or performs module discovery itself.
Everything passes through libdpm-core: binary-to-module, module-to-module, program-to-module. No consumer performs module discovery or dynamic loading itself.
## How it works
## Three layers to read this at
### The context
DPM is met at three different layers, and which one you are at determines what any statement below means for you.
All work happens through a context handle obtained from `dpm_open`. Opening a context reads configuration from `/etc/dpm/conf.d/`, resolves the module directory, and initializes logging. Four things can be overridden at open time — the configuration directory, the module path, the target root, and the log level — and every one is exposed as a flag on the `dpm` command, so anything a linked program can redirect, a developer at a shell can redirect too.
- **The user layer** — a person at a shell running the dpm binary. Everything here is a command and its flags.
- **The developer layer** — a program written against the C API in `<dpm/core.h>`, linking `-ldpm-core`. Everything here is a function call. Module authors live here too, on the other side of the same boundary.
- **The filesystem layer** — the directories and files DPM reads and writes: configuration, the module directory, the target root, and the installed artifacts.
The target root override is what makes chroot builds, image assembly, and sysroot management work: package operations act on the given tree instead of the running system. Multiple contexts with different roots can be open at once.
The sections that follow are labelled by layer. Where a mechanism exists at more than one layer, it is described once at each, because the same idea looks different depending on where you stand.
The context owns everything it hands out. Every string a consumer receives stays valid until `dpm_close`, and callers never free anything.
## How it works at the user layer
### Acquiring a module
The dpm binary's subcommand surface is exactly the set of loadable modules. `dpm <module> <command> [args...]` loads that module and hands the command to it; `dpm <module> help` asks the module to describe itself. There is no fixed list of operations baked into the tool, because the tool contains no capability logic — it parses arguments and prints.
`dpm_require` resolves a module by name in the module path, validates it completely, and returns a handle — or returns NULL and records exactly why. A minimum version may be stated; NULL accepts any version. Modules load at most once per context, and repeated calls return the same handle.
`dpm --list-modules` shows every module that passes validation, with its version and description. Candidates that fail are excluded from the listing and logged, so what is listed is what can actually be run.
Once a module is loaded, there are two ways to reach it:
Four flags redirect the system defaults:
- **Generic dispatch** — `dpm_execute` passes a command name and an argument vector to the module's entry point. This is the path the CLI uses, and it is why the CLI's command surface is exactly the set of loadable modules.
- **Typed access** — `dpm_get_api` returns a plain C struct of function pointers for a named API at a stated version. This is the path modules and external programs use to call functions directly.
- `-c, --config-dir PATH` — where configuration is read from
- `-m, --module-path PATH` — which directory modules are loaded from
- `-r, --root PATH` — the target root that package operations act on
- `-L, --log-level LEVEL` — FATAL, ERROR, WARN, INFO, or DEBUG
### Enumeration
These four exist because of a rule the design imposes on itself: every field a linked program can override must also be a flag, so anything reachable from code is reachable from a shell. That rule holds for any override added in the future.
`dpm_list_modules` scans the module path and yields every valid module with its name, version, description, and minimum core version. Candidates that fail validation are logged and excluded, so a listing shows what can actually be used.
## How it works at the developer layer
### The module contract
A program links libdpm-core and includes `<dpm/core.h>`. Everything it does happens through a **context**, an opaque handle obtained from `dpm_open`.
A module is one `.so` in the module directory that exports five reserved symbols: a command entry point, its own version, a one-line description, the minimum core version it supports, and a manifest. The manifest declares the module's entire functional surface — for each API, its name, table version, and the exact exported symbol carrying the table.
Opening a context reads the configuration files, resolves which directory modules will be loaded from, and initializes logging. It loads no modules. The same four things the dpm binary exposes as flags are the fields of the overrides struct passed to `dpm_open`, and passing NULL accepts the system defaults — which is what makes a default context equivalent to invoking the installed dpm binary, since that binary is just another caller doing the same thing.
The target root override is what makes chroot builds, image assembly, and sysroot management work: package operations act on the given tree instead of the running system. Multiple contexts with different roots may be open at once.
The context owns everything it hands out. Every string a caller receives stays valid until `dpm_close`, and callers never free anything.
**Acquiring a module.** `dpm_require` resolves a module by name, validates it completely, and returns a handle — or returns NULL, with `dpm_last_error` carrying the precise reason. Modules load at most once per context, and repeated calls return the same handle.
**Reading what libdpm-core saw.** `dpm_module_info_of` fills in a module's name, version, description, and minimum-libdpm-core version. Those values are reported, and no conclusion is drawn from them.
**Calling into a module.** Two paths:
- `dpm_execute` passes a command name and an argument vector to the module's entry point. This is the path the dpm binary takes.
- `dpm_get_api` returns a plain C struct of function pointers for a named API at a stated version. This is the path modules and external programs take to call functions directly.
**Enumerating.** `dpm_list_modules` yields a cursor over every valid module, which is what backs the listing the dpm binary prints.
**Services.** A module reaches libdpm-core through the context that dispatched the call: `dpm_log` to write a message, `dpm_config_get` to read a value from its own configuration namespace, `dpm_module_path` to learn where modules live, `dpm_core_version` to learn the library's version. A module needs no file handling and no logging machinery of its own.
## How it works at the filesystem layer
**Configuration** lives in `/etc/dpm/conf.d/`. Each `.conf` file in that directory is one namespace named after the file: `core.conf` holds the library's own settings, and a module named `mymodule` reads `mymodule.conf`. Files are sectioned, and a value is addressed by namespace, section, and key. The library's own file carries the log level, whether to write a log file and where, and the default module directory.
**Modules** live in `/usr/lib/dpm/modules/`. A module's name is its filename without the `.so` extension, so `info.so` is the module named `info`. Discovery is the presence of a valid file in that directory — there is no registry, and no registration step.
**Logging** goes to the console always, and to a log file when configuration enables one; the default path is `/var/log/dpm/dpm.log`.
**The target root** defaults to `/`. Pointed elsewhere, it is the tree that package operations act on, which is what turns a scratch directory into a complete target for image assembly or a chroot build.
**Installed artifacts:**
| Artifact | Location |
|---|---|
| the dpm binary | `/usr/bin/dpm` |
| `libdpm-core.so` | `/usr/lib/libdpm-core.so` |
| public header | `/usr/include/dpm/core.h` |
| modules | `/usr/lib/dpm/modules/<name>.so` |
## The module contract
*Developer layer, module-author side.*
A module is one `.so` in the module directory exporting five reserved symbols: a command entry point, its own version, a one-line description, the minimum libdpm-core version it supports, and a manifest. The manifest declares the module's entire functional surface — for each API, its name, table version, and the exact exported symbol carrying the table.
Beyond those five, a module exports one symbol per API table. A table is a C struct of function pointers opening with a magic constant and the struct's size in bytes as the module compiled it, which lets a consumer accept a tail-extended revision of the same version. Everything crossing a table is a C type; state passes through opaque handles; errors are int codes.
### Load-time enforcement
## Load-time enforcement
Core validates a module completely before offering it to anyone, in five steps:
libdpm-core validates a module completely before offering it to anyone, in five steps:
1. Every reserved contract symbol resolves.
2. The module's minimum core version is not newer than core's own.
2. The module's minimum libdpm-core version is not newer than the running library's own.
3. The version and description probes return well-formed values.
4. Every symbol the manifest declares actually resolves.
5. Every declared table carries the correct magic constant and a sane size.
A module failing any step is refused with an itemized reason and its library handle closed. Validation is all-or-nothing: if core hands out a handle, the contract already passed. Consumers never defend against partially valid modules, because they cannot receive one.
A module failing any step is refused with an itemized reason and its library handle closed. Validation is all-or-nothing: if a handle comes back, the contract already passed. Consumers never defend against partially valid modules, because they cannot receive one.
### Versioning
At the user layer this is visible as a module missing from `--list-modules` with a logged reason, or as an itemized failure when the module is named directly.
Compatibility is directional, and the module is the party that declares it.
## Versioning
Each module states the oldest core it supports. Core compares its own version against that minimum and refuses only when core is older, with a message naming the remedy. Core never rejects a module for being old, because core's contract grows append-only — a newer core supports everything an older core did.
libdpm-core enforces exactly one version rule, the one where it is the host: each module states the oldest libdpm-core it supports, and a library older than that refuses to load the module, with a message naming the remedy. A module is never rejected for being old, because the library's contract grows append-only — a newer libdpm-core supports everything an older one did.
Consumers state minimums and never maximums. Module APIs also evolve append-only: a breaking change means exporting a new table beside the old one rather than mutating the existing one. The consequence is that updating core or any module preserves every combination that previously worked, and the single possible load refusal names its own fix.
Every other version question belongs to the consumer. A module that depends on another requires it, reads the version reported back, and decides for itself whether that version is too new or too old for the calls it intends to make. libdpm-core reports what it saw and draws no conclusion from it, so a handle means the module is valid rather than that it suits any particular caller.
### Configuration and logging
Configuration lives in per-module namespaces: each module's `.conf` file under the context's configuration directory, with `core` naming core's own file. A module reads its settings through the context that dispatched the call, so a module needs no file handling of its own. Logging works the same way — a module writes through the context, and the context decides the destinations and the threshold.
Module APIs evolve append-only: a breaking change means exporting a new table beside the old one rather than mutating the existing one. A newer module therefore still carries everything an older consumer called, so updating libdpm-core or any module preserves every combination that previously worked.
## Why it works this way
### Because it must run on a barren system
The founding constraint is that DPM has to work in an environment providing nothing but libc and libstdc++, using the same binaries that run on a fully populated system. That forbids core from taking on any dependency beyond the baseline, which forbids package logic from living in core, which produces the division the whole architecture rests on: **core routes and hosts, modules implement.** Anything needing a database, compression, or TLS is a module that arrives later.
The founding constraint is that DPM has to work in an environment providing nothing but libc and libstdc++, using the same binaries that run on a fully populated system. That forbids libdpm-core from taking on any dependency beyond the baseline, which forbids package logic from living inside it, which produces the division the whole architecture rests on: **libdpm-core routes and hosts, modules implement.** Anything needing a database, compression, or TLS is a module that arrives later.
### Because capability has to grow in layers
@@ -90,47 +134,48 @@ Each layer of the system installs the dependencies of the next using only what a
### Because one implementation must serve every caller
The requirement that every implementation exist exactly once, and be consumable by the CLI, by other layers, and by external programs, is what makes libdpm-core a C ABI library rather than an application with a library carved out of it. A program that opens a default context is operating the installed package manager itself — system configuration, system module path, system tree, system locking — identically to invoking `dpm`, because `dpm` is just another caller of the same library. The CLI is argument parsing and printing, and no capability logic lives in it.
The requirement that every implementation exist exactly once, and be consumable by the dpm binary, by other layers, and by external programs, is what makes libdpm-core a C ABI library rather than an application with a library carved out of it. This is also why the user layer and the developer layer stay in step: the flags are the override fields, one for one, so a developer and a user redirect the same things by the same names.
### Because nothing may be trusted that has not been verified
Core is the sole authority on module validity, and the contract is enforced by core's validator rather than by an SDK a module author might skip, patch, or fall behind. That is why validation is all-or-nothing and why it happens at load: failures surface loudly and itemized at install time instead of halfway through an operation on a user's system. The manifest exists so core can check a module's entire declared surface before offering any of it — an API absent from the manifest does not exist, even if its symbol does.
libdpm-core is the sole authority on module validity, and the contract is enforced by its validator rather than by an SDK a module author might skip, patch, or fall behind. That is why validation is all-or-nothing and why it happens at load: failures surface loudly and itemized at install time instead of halfway through an operation on a user's system. The manifest exists so a module's entire declared surface can be checked before any of it is offered — an API absent from the manifest does not exist, even if its symbol does.
The residual limit is that `dlsym` cannot verify a function signature. The magic constant, the size field, the core-minimum handshake, and the probes cover this in practice; defeating them takes deliberate lying, which is a package-signing concern upstream of the loader.
The residual limit is that dynamic symbol lookup cannot verify a function signature. The magic constant, the size field, the minimum-version handshake, and the probes cover this in practice; defeating them takes deliberate lying, which is a package-signing concern upstream of the loader.
### Because updates must never strand a working system
Minimums everywhere, no maximums, and append-only evolution in both directions mean an update can only add satisfiable states. This is what allows modules to be released independently: coordination happens through the versioning model instead of through lockstep builds.
libdpm-core holds no maximums and no exact-match constraints, and evolution is append-only in both directions, so an update can only add satisfiable states. This is what allows modules to be released independently: coordination happens through the versioning model instead of through lockstep builds.
### Because modules are developed independently
One repository per module, plus the core repository. A module links libdpm-core and nothing else from DPM; peer modules never appear in its repository, build, or test environment. "Not all the pieces are there" is the normal, permanent condition at build time, so the architecture is arranged to make that a non-event:
One repository per module, plus the libdpm-core repository. A module links libdpm-core and nothing else from DPM; peer modules never appear in its repository, build, or test environment. "Not all the pieces are there" is the normal, permanent condition at build time, so the architecture is arranged to make that a non-event:
- A module compiles against its own declarations plus core.
- A module compiles against its own declarations plus libdpm-core.
- Every dependency it consumes is a plain struct of function pointers, so a fake is a struct the test fills in — dependency injection is inherent, with no linker seams.
- A harness that links real core and points the module path at build output plus fixture stubs exercises the real five-step validation on a bare builder.
- A harness that links the real libdpm-core and points the module path at build output plus fixture stubs exercises the real five-step validation on a bare builder.
- Only final integration needs the whole system, and because alternate roots are first-class, it needs a directory rather than a virtual machine.
The core repository's own fixtures are deliberately broken stub modules — missing symbols, bad magic, a lying manifest, a too-new core minimum, a malformed version — plus one known-good stub. Core development never requires a real package module to exist.
This repository's own fixtures are deliberately broken stub modules — missing symbols, bad magic, a lying manifest, a too-new minimum version, a malformed version — plus one known-good stub. Developing libdpm-core never requires a real package module to exist.
## What the core repository produces
## What this repository produces
- **libdpm-core.so** — the library
- **dpm** — the command-line tool
- **the dpm binary** — the command-line tool
- **info** — a module bundled with the dpm binary and libdpm-core, used to test and demonstrate full DPM system functionality where appropriate
Every other module is developed against libdpm-core and lives outside this repository.
## Invariants
1. Core routes and hosts; modules implement. No package logic in core, ever.
2. Writes flow down, never sideways: a layer mutates the system only through the layer beneath it, in-process through core-mediated APIs.
1. libdpm-core routes and hosts; modules implement. No package logic inside it, ever.
2. Writes flow down, never sideways: a layer mutates the system only through the layer beneath it, in-process through mediated APIs.
3. A module is either fully valid or not loaded — no partial states, no consumer-side defense.
4. Anything a linked program can redirect, a shell user can redirect too.
## Further reading
- **DESIGN.md** — the full design specification
- **CONSUMERS.md** — linking libdpm-core and driving it from a program
- **MODULES.md** — writing, building, testing, and installing a module
- **BUILD.md** — building, testing, and installing core itself
- **BUILD.md** — building, testing, and installing libdpm-core and the dpm binary
- **DOCUMENTATION.md** — generating the code reference