Dark-Horse-Linux/dpm-core-ng
3
0
Fork 0
Code Issues Pull Requests Actions Activity

Route all module interaction through dispatch

Browse Source 
A module is addressed by name and command string, and nothing else.
Typed API access handed a caller a pointer into the callee's function
table, which meant compiling against that module's struct layout — a
build-time dependency between modules that the design does not permit.
Removing it also removes the manifest, the table magic constant, and
the table size field, which existed only to describe and validate
those tables.

Load validation is now three steps: reserved contract symbols resolve,
the minimum-version handshake passes, and the version and description
probes return well-formed values. The contract is four reserved
symbols, and a module's interface is the command vocabulary it
documents.

Documentation is brought in line, and artifacts are named exactly:
libdpm-core.so for the library, <dpm/core.h> for the header, the dpm
binary for the command-line tool.
This commit is contained in:
Chris Punches
2026-08-15 02:15:18 -04:00
parent 267529bee3
commit 97b39cac6c
17 changed files with 263 additions and 702 deletions
Download Patch File Download Diff File Expand all files Collapse all files

116
docs/OVERVIEW.md
View File

@@ -2,28 +2,29 @@
## What DPM is
DPM is the package manager for Dark Horse Linux. At its center is **libdpm-core**, a shared library that discovers modules, validates them, routes calls to them, reports their versions, and provides configuration and logging. Modules — shared objects in the module directory — implement package functionality.
DPM is the package manager for Dark Horse Linux. At its center is **libdpm-core.so**, a shared library that discovers modules, validates them, and routes calls to them, and that provides configuration and logging to whatever it has loaded. Modules — shared objects in the module directory — implement package functionality.
**The `dpm` binary** is the command-line tool, and it is one consumer of that library. Build systems, image builders, distribution tooling, and programs in any language with C FFI are equal consumers of the same library through the same interface.
libdpm-core implements no package operations. It routes and hosts.
libdpm-core.so implements no package operations. It routes and hosts.
## The shape of the system
Everything reaches a module the same way, and a module reaching another module is the same step repeated:
```
the dpm binary build systems / DHL tools / other languages
\ /
v v
libdpm-core.so
(discovery, validation, routing,
version reporting, config, logging)
|
v
modules
(one .so each)
the dpm binary -> <dpm/core.h> -> module A -> <dpm/core.h> -> module B
a build system -> <dpm/core.h> -> module A -> <dpm/core.h> -> module B
```
Everything passes through libdpm-core: binary-to-module, module-to-module, program-to-module. No consumer performs module discovery or dynamic loading itself.
`<dpm/core.h>` is the declared interface; libdpm-core.so is what implements it. A caller includes the header, links `-ldpm-core`, and asks the library to discover and invoke modules by name.
It all happens in one address space. The `dpm` binary (or the build system) is the process; libdpm-core.so is mapped into it; every module the library loads is mapped into it as well. Calls are direct function calls with no subprocess, no serialization, and no output parsing anywhere in the chain.
Two properties make the routing rule real rather than a convention:
- **There is one instance of libdpm-core.so in the process.** A module links against the library like any other consumer, and when the module is loaded the dynamic linker binds it to the copy already mapped. A module therefore drives the same context, the same module registry, and the same configuration the original caller opened.
- **Modules cannot see each other.** They are loaded with `RTLD_LOCAL`, so a module's symbols never enter the global namespace. The only symbols a module can resolve are libdpm-core.so's, so there is physically no path from one module to another that does not pass through the library.
## Running the dpm binary
@@ -40,11 +41,11 @@ Four flags redirect the system defaults:
These four exist because of a rule the design imposes on itself: every field a linked program can override must also be a flag, so anything reachable from code is reachable from a shell. That rule holds for any override added in the future.
## Writing a program against libdpm-core
## Writing a program against the library
A program links libdpm-core and includes `<dpm/core.h>`. Everything it does happens through a **context**, an opaque handle obtained from `dpm_open`.
A program includes `<dpm/core.h>` and links `-ldpm-core`, the same as it would link any other shared library. Everything it does happens through a **context**, an opaque handle obtained from `dpm_open`.
Opening a context reads the configuration files, resolves which directory modules will be loaded from, and initializes logging. It loads no modules. The same four things the `dpm` binary exposes as flags are the fields of the overrides struct passed to `dpm_open`, and passing NULL accepts the system defaults — which is what makes a default context equivalent to invoking the installed `dpm` binary, since that binary is just another caller doing the same thing.
Opening a context reads the configuration files, resolves which directory modules will be loaded from, and initializes logging. It loads no modules. The same four things the `dpm` binary exposes as flags are the fields of the overrides struct passed to `dpm_open`, and passing NULL accepts the system defaults — which is what makes a default context equivalent to what the installed `dpm` binary sees, since that binary is just another caller doing the same thing.
The target root override is what makes chroot builds, image assembly, and sysroot management work: package operations act on the given tree instead of the running system. Multiple contexts with different roots may be open at once.
@@ -52,16 +53,33 @@ The context owns everything it hands out. Every string a caller receives stays v
**Acquiring a module.** `dpm_require` resolves a module by name, validates it completely, and returns a handle — or returns NULL, with `dpm_last_error` carrying the precise reason. Modules load at most once per context, and repeated calls return the same handle.
**Reading what libdpm-core saw.** `dpm_module_info_of` fills in a module's name, version, description, and minimum-libdpm-core version. Those values are reported, and no conclusion is drawn from them.
**Reading what the library saw.** `dpm_module_info_of` fills in a module's name, version, description, and minimum-library version. Those values are reported, and no conclusion is drawn from them.
**Calling into a module.** Two paths:
- `dpm_execute` passes a command name and an argument vector to the module's entry point. This is the path the `dpm` binary takes.
- `dpm_get_api` returns a plain C struct of function pointers for a named API at a stated version. This is the path modules and external programs take to call functions directly.
**Calling into a module.** `dpm_execute` passes a command name and an argument vector to the module's entry point and returns its result. This is the only path into module code, and it is the same one the `dpm` binary takes.
**Enumerating.** `dpm_list_modules` yields a cursor over every valid module, which is what backs the listing the `dpm` binary prints.
**Services.** A module reaches libdpm-core through the context that dispatched the call: `dpm_log` to write a message, `dpm_config_get` to read a value from its own configuration namespace, `dpm_module_path` to learn where modules live, `dpm_core_version` to learn the library's version. A module needs no file handling and no logging machinery of its own.
**Services.** A module reaches the library through the context that dispatched the call: `dpm_log` to write a message, `dpm_config_get` to read a value from its own configuration namespace, `dpm_module_path` to learn where modules live, `dpm_core_version` to learn the library's version. A module needs no file handling and no logging machinery of its own.
## How a module reaches another module
A module is a consumer of `<dpm/core.h>` exactly like the `dpm` binary is. To reach a peer it performs the identical two steps its own caller performed: ask the library for the module by name, then ask the library to invoke it.
```
int dpm_module_execute(dpm_ctx* ctx, const char* command, int argc, char** argv)
{
dpm_module* peer = dpm_require(ctx, "othermodule");
if (!peer) {
dpm_log(ctx, DPM_LOG_ERROR, dpm_last_error(ctx));
return 1;
}
return dpm_execute(ctx, peer, "somecommand", argc, argv);
}
```
The `ctx` a module needs is the one handed to it in its own entry point, so it requires nothing else to reach the library.
**A module never links, includes, or hardcodes anything belonging to another module.** No peer headers, no shared struct layouts, no peer symbols. The only build dependency a module has is libdpm-core.so, and the only knowledge it holds about a peer is the peer's name and the command it wants to run. That is what allows every module to live in its own repository and be built with no peer present anywhere on the machine.
## What DPM reads and writes on disk
@@ -86,19 +104,17 @@ The context owns everything it hands out. Every string a caller receives stays v
*What a module author implements.*
A module is one `.so` in the module directory exporting five reserved symbols: a command entry point, its own version, a one-line description, the minimum libdpm-core version it supports, and a manifest. The manifest declares the module's entire functional surface — for each API, its name, table version, and the exact exported symbol carrying the table.
A module is one `.so` in the module directory exporting four reserved symbols: a command entry point, its own version, a one-line description, and the minimum library version it supports. It includes `<dpm/core.h>` for those declarations and links `-ldpm-core`, and that is its entire build dependency.
Beyond those five, a module exports one symbol per API table. A table is a C struct of function pointers opening with a magic constant and the struct's size in bytes as the module compiled it, which lets a consumer accept a tail-extended revision of the same version. Everything crossing a table is a C type; state passes through opaque handles; errors are int codes.
The entry point receives the context that dispatched the call, the command name, and an argument vector, and returns an int. Everything a module offers the rest of the system is reachable through that one function, addressed by command name — which is what keeps a caller free of any compile-time knowledge of the module it is calling.
## Load-time enforcement
libdpm-core validates a module completely before offering it to anyone, in five steps:
libdpm-core.so validates a module completely before offering it to anyone:
1. Every reserved contract symbol resolves.
2. The module's minimum libdpm-core version is not newer than the running library's own.
2. The module's minimum library version is not newer than the running library's own.
3. The version and description probes return well-formed values.
4. Every symbol the manifest declares actually resolves.
5. Every declared table carries the correct magic constant and a sane size.
A module failing any step is refused with an itemized reason and its library handle closed. Validation is all-or-nothing: if a handle comes back, the contract already passed. Consumers never defend against partially valid modules, because they cannot receive one.
@@ -106,17 +122,15 @@ From the command line this shows up as a module missing from `--list-modules` wi
## Versioning
libdpm-core enforces exactly one version rule, the one where it is the host: each module states the oldest libdpm-core it supports, and a library older than that refuses to load the module, with a message naming the remedy.
libdpm-core.so enforces exactly one version rule, the one where it is the host: each module states the oldest library version it supports, and a library older than that refuses to load the module, with a message naming the remedy.
Every other version question belongs to the consumer. A module that depends on another requires it, reads the version reported back, and decides for itself whether that version is too new or too old for the calls it intends to make. libdpm-core reports what it saw and draws no conclusion from it, so a handle means the module is valid rather than that it suits any particular caller.
A breaking change to a module's API means a new table version under its own symbol, and the module that owns it decides when the old version is retired. Because a consumer asks for an API by name and version, a retired table surfaces as a refused request at load, before the consumer has done any work.
Every other version question belongs to the consumer. A module that depends on another requires it, reads the version reported back, and decides for itself whether that version is suitable for the commands it intends to issue. libdpm-core.so reports what it saw and draws no conclusion from it, so a handle means the module is valid rather than that it suits any particular caller.
## Why it works this way
### Because it must run on a barren system
The founding constraint is that DPM has to work in an environment providing nothing but libc and libstdc++, using the same binaries that run on a fully populated system. That forbids libdpm-core from taking on any dependency beyond the baseline, which forbids package logic from living inside it, which produces the division the whole architecture rests on: **libdpm-core routes and hosts, modules implement.** Anything needing a database, compression, or TLS is a module that arrives later.
The founding constraint is that DPM has to work in an environment providing nothing but libc and libstdc++, using the same binaries that run on a fully populated system. That forbids libdpm-core.so from taking on any dependency beyond the baseline, which forbids package logic from living inside it, which produces the division the whole architecture rests on: **libdpm-core.so routes and hosts, modules implement.** Anything needing a database, compression, or TLS is a module that arrives later.
### Because capability has to grow in layers
@@ -124,48 +138,48 @@ Each layer of the system installs the dependencies of the next using only what a
### Because one implementation must serve every caller
The requirement that every implementation exist exactly once, and be consumable by the `dpm` binary, by other layers, and by external programs, is what makes libdpm-core a C ABI library rather than an application with a library carved out of it. It is also why the command line and the C API stay in step: the flags are the override fields, one for one, so a program and a person redirect the same things by the same names.
The requirement that every implementation exist exactly once, and be consumable by the `dpm` binary, by other layers, and by external programs, is what makes libdpm-core.so a C ABI library rather than an application with a library carved out of it. It is also why the command line and the C API stay in step: the flags are the override fields, one for one, so a program and a person redirect the same things by the same names.
### Because nothing may be trusted that has not been verified
libdpm-core is the sole authority on module validity, and the contract is enforced by its validator rather than by an SDK a module author might skip, patch, or fall behind. That is why validation is all-or-nothing and why it happens at load: failures surface loudly and itemized at install time instead of halfway through an operation on a user's system. The manifest exists so a module's entire declared surface can be checked before any of it is offered — an API absent from the manifest does not exist, even if its symbol does.
The residual limit is that dynamic symbol lookup cannot verify a function signature. The magic constant, the size field, the minimum-version handshake, and the probes cover this in practice; defeating them takes deliberate lying, which is a package-signing concern upstream of the loader.
libdpm-core.so is the sole authority on module validity, and the contract is enforced by its validator rather than by an SDK a module author might skip, patch, or fall behind. That is why validation is all-or-nothing and why it happens at load: failures surface loudly and itemized at install time instead of halfway through an operation on a user's system.
### Because coordination happens at load, not at build
A consumer states the module and API version it needs and is told at load whether it is there. That is what allows modules to be released independently: agreement is reached when the pieces meet, through declared versions each consumer judges for itself, instead of through lockstep builds.
A caller states the module name and command it needs and is told at load whether the module is there. That is what allows modules to be released independently: agreement is reached when the pieces meet, through declared versions each consumer judges for itself, instead of through lockstep builds.
### Because modules are developed independently
One repository per module, plus the libdpm-core repository. A module links libdpm-core and nothing else from DPM; peer modules never appear in its repository, build, or test environment. "Not all the pieces are there" is the normal, permanent condition at build time, so the architecture is arranged to make that a non-event:
One repository per module, plus the libdpm-core.so repository. A module links libdpm-core.so and nothing else from DPM; peer modules never appear in its repository, build, or test environment. "Not all the pieces are there" is the normal, permanent condition at build time, so the architecture is arranged to make that a non-event:
- A module compiles against its own declarations plus libdpm-core.
- Every dependency it consumes is a plain struct of function pointers, so a fake is a struct the test fills in — dependency injection is inherent, with no linker seams.
- A harness that links the real libdpm-core and points the module path at build output plus fixture stubs exercises the real five-step validation on a bare builder.
- A module compiles against `<dpm/core.h>` and nothing else from DPM.
- A peer is addressed by name and command string, so nothing about a peer needs to exist at compile time.
- A harness that links the real libdpm-core.so and points the module path at build output plus fixture stubs exercises the real validation on a bare builder.
- Only final integration needs the whole system, and because alternate roots are first-class, it needs a directory rather than a virtual machine.
This repository's own fixtures are deliberately broken stub modules — missing symbols, bad magic, a lying manifest, a too-new minimum version, a malformed version — plus one known-good stub. Developing libdpm-core never requires a real package module to exist.
This repository's own fixtures are deliberately broken stub modules — missing symbols, a too-new minimum version, a malformed version — plus one known-good stub. Developing libdpm-core.so never requires a real package module to exist.
## What this repository produces
- **`libdpm-core.so`** — the library
- **the `dpm` binary** — the command-line tool
- **info** — a module bundled with the `dpm` binary and libdpm-core, used to test and demonstrate full DPM system functionality where appropriate
- **info** — a module bundled with the `dpm` binary and libdpm-core.so, used to test and demonstrate full DPM system functionality where appropriate
Every other module is developed against libdpm-core and lives outside this repository.
Every other module is developed against libdpm-core.so and lives outside this repository.
## Invariants
1. libdpm-core routes and hosts; modules implement. No package logic inside it, ever.
2. Writes flow down, never sideways: a layer mutates the system only through the layer beneath it, in-process through mediated APIs.
3. A module is either fully valid or not loaded — no partial states, no consumer-side defense.
4. Anything a linked program can redirect, a shell user can redirect too.
1. libdpm-core.so routes and hosts; modules implement. No package logic inside it, ever.
2. `<dpm/core.h>` names no module and carries no module-specific type. It offers discovery and interaction only.
3. A module never links, includes, or hardcodes anything belonging to another module. All module-to-module interaction passes through libdpm-core.so.
4. Writes flow down, never sideways: a layer mutates the system only through the layer beneath it, in-process through mediated calls.
5. A module is either fully valid or not loaded — no partial states, no consumer-side defense.
6. Anything a linked program can redirect, a shell user can redirect too.
## Further reading
- **DESIGN.md** — the full design specification
- **CONSUMERS.md** — linking libdpm-core and driving it from a program
- **CONSUMERS.md** — linking libdpm-core.so and driving it from a program
- **MODULES.md** — writing, building, testing, and installing a module
- **BUILD.md** — building, testing, and installing libdpm-core and the `dpm` binary
- **BUILD.md** — building, testing, and installing libdpm-core.so and the `dpm` binary
- **DOCUMENTATION.md** — generating the code reference