Fix clean target, correct build docs, add overview and docs guide
clean was deleting the generated build system along with the artifacts — CMakeCache.txt, the makefiles, CMakeFiles/, the CMake file API directory, and the CTest configuration. That left the build directory unconfigured after every clean, so an IDE reading its target list from the file API lost every target until the project was reloaded. clean now removes only what the build produced. Also writes CTest's scratch directory into DartConfiguration.tcl so a bare ctest and IDE test discovery use it too, not just build-driven runs. BUILD.md no longer lists libdl as a dependency; glibc 2.34 merged it into libc, so nothing links against it. The code-reference section moves to its own DOCUMENTATION.md. The fixture config logged at ERROR while the info module emits its output at INFO, so the CLI invocation BUILD.md documents exited 0 and printed nothing. The fixture now logs at INFO and the documented command works as written. OVERVIEW.md describes what DPM is, how core routes and validates modules, and why the architecture takes the shape it does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
136
docs/OVERVIEW.md
Normal file
136
docs/OVERVIEW.md
Normal file
@@ -0,0 +1,136 @@
|
||||
# DPM — An Overview
|
||||
|
||||
## What DPM is
|
||||
|
||||
DPM is the package manager for Dark Horse Linux. Its core is **libdpm-core**, a shared library that discovers modules, validates them, routes calls to them, negotiates versions, and provides configuration and logging. Modules — shared objects in the module directory — implement package functionality.
|
||||
|
||||
The `dpm` command-line tool is one consumer of that library. Build systems, image builders, distribution tooling, and programs in any language with C FFI are equal consumers of the same library through the same interface.
|
||||
|
||||
Core implements no package operations. It routes and hosts.
|
||||
|
||||
## The shape of the system
|
||||
|
||||
```
|
||||
dpm CLI build systems / DHL tools / other languages
|
||||
\ /
|
||||
v v
|
||||
libdpm-core.so
|
||||
(discovery, validation, routing,
|
||||
version negotiation, config, logging)
|
||||
|
|
||||
v
|
||||
modules
|
||||
(one .so each)
|
||||
```
|
||||
|
||||
Everything passes through core: CLI-to-module, module-to-module, program-to-module. No consumer calls `dlopen`, `dlsym`, or performs module discovery itself.
|
||||
|
||||
## How it works
|
||||
|
||||
### The context
|
||||
|
||||
All work happens through a context handle obtained from `dpm_open`. Opening a context reads configuration from `/etc/dpm/conf.d/`, resolves the module directory, and initializes logging. Four things can be overridden at open time — the configuration directory, the module path, the target root, and the log level — and every one is exposed as a flag on the `dpm` command, so anything a linked program can redirect, a developer at a shell can redirect too.
|
||||
|
||||
The target root override is what makes chroot builds, image assembly, and sysroot management work: package operations act on the given tree instead of the running system. Multiple contexts with different roots can be open at once.
|
||||
|
||||
The context owns everything it hands out. Every string a consumer receives stays valid until `dpm_close`, and callers never free anything.
|
||||
|
||||
### Acquiring a module
|
||||
|
||||
`dpm_require` resolves a module by name in the module path, validates it completely, and returns a handle — or returns NULL and records exactly why. A minimum version may be stated; NULL accepts any version. Modules load at most once per context, and repeated calls return the same handle.
|
||||
|
||||
Once a module is loaded, there are two ways to reach it:
|
||||
|
||||
- **Generic dispatch** — `dpm_execute` passes a command name and an argument vector to the module's entry point. This is the path the CLI uses, and it is why the CLI's command surface is exactly the set of loadable modules.
|
||||
- **Typed access** — `dpm_get_api` returns a plain C struct of function pointers for a named API at a stated version. This is the path modules and external programs use to call functions directly.
|
||||
|
||||
### Enumeration
|
||||
|
||||
`dpm_list_modules` scans the module path and yields every valid module with its name, version, description, and minimum core version. Candidates that fail validation are logged and excluded, so a listing shows what can actually be used.
|
||||
|
||||
### The module contract
|
||||
|
||||
A module is one `.so` in the module directory that exports five reserved symbols: a command entry point, its own version, a one-line description, the minimum core version it supports, and a manifest. The manifest declares the module's entire functional surface — for each API, its name, table version, and the exact exported symbol carrying the table.
|
||||
|
||||
Beyond those five, a module exports one symbol per API table. A table is a C struct of function pointers opening with a magic constant and the struct's size in bytes as the module compiled it, which lets a consumer accept a tail-extended revision of the same version. Everything crossing a table is a C type; state passes through opaque handles; errors are int codes.
|
||||
|
||||
### Load-time enforcement
|
||||
|
||||
Core validates a module completely before offering it to anyone, in five steps:
|
||||
|
||||
1. Every reserved contract symbol resolves.
|
||||
2. The module's minimum core version is not newer than core's own.
|
||||
3. The version and description probes return well-formed values.
|
||||
4. Every symbol the manifest declares actually resolves.
|
||||
5. Every declared table carries the correct magic constant and a sane size.
|
||||
|
||||
A module failing any step is refused with an itemized reason and its library handle closed. Validation is all-or-nothing: if core hands out a handle, the contract already passed. Consumers never defend against partially valid modules, because they cannot receive one.
|
||||
|
||||
### Versioning
|
||||
|
||||
Compatibility is directional, and the module is the party that declares it.
|
||||
|
||||
Each module states the oldest core it supports. Core compares its own version against that minimum and refuses only when core is older, with a message naming the remedy. Core never rejects a module for being old, because core's contract grows append-only — a newer core supports everything an older core did.
|
||||
|
||||
Consumers state minimums and never maximums. Module APIs also evolve append-only: a breaking change means exporting a new table beside the old one rather than mutating the existing one. The consequence is that updating core or any module preserves every combination that previously worked, and the single possible load refusal names its own fix.
|
||||
|
||||
### Configuration and logging
|
||||
|
||||
Configuration lives in per-module namespaces: each module's `.conf` file under the context's configuration directory, with `core` naming core's own file. A module reads its settings through the context that dispatched the call, so a module needs no file handling of its own. Logging works the same way — a module writes through the context, and the context decides the destinations and the threshold.
|
||||
|
||||
## Why it works this way
|
||||
|
||||
### Because it must run on a barren system
|
||||
|
||||
The founding constraint is that DPM has to work in an environment providing nothing but libc and libstdc++, using the same binaries that run on a fully populated system. That forbids core from taking on any dependency beyond the baseline, which forbids package logic from living in core, which produces the division the whole architecture rests on: **core routes and hosts, modules implement.** Anything needing a database, compression, or TLS is a module that arrives later.
|
||||
|
||||
### Because capability has to grow in layers
|
||||
|
||||
Each layer of the system installs the dependencies of the next using only what already works. That requires upper layers to reach lower ones through a stable in-process interface rather than by re-implementing them, and it requires writes to flow strictly downward — a layer mutates the system only through the layer beneath it. Shared concerns like locking are then implemented once, at the bottom, and inherited by everything above.
|
||||
|
||||
### Because one implementation must serve every caller
|
||||
|
||||
The requirement that every implementation exist exactly once, and be consumable by the CLI, by other layers, and by external programs, is what makes libdpm-core a C ABI library rather than an application with a library carved out of it. A program that opens a default context is operating the installed package manager itself — system configuration, system module path, system tree, system locking — identically to invoking `dpm`, because `dpm` is just another caller of the same library. The CLI is argument parsing and printing, and no capability logic lives in it.
|
||||
|
||||
### Because nothing may be trusted that has not been verified
|
||||
|
||||
Core is the sole authority on module validity, and the contract is enforced by core's validator rather than by an SDK a module author might skip, patch, or fall behind. That is why validation is all-or-nothing and why it happens at load: failures surface loudly and itemized at install time instead of halfway through an operation on a user's system. The manifest exists so core can check a module's entire declared surface before offering any of it — an API absent from the manifest does not exist, even if its symbol does.
|
||||
|
||||
The residual limit is that `dlsym` cannot verify a function signature. The magic constant, the size field, the core-minimum handshake, and the probes cover this in practice; defeating them takes deliberate lying, which is a package-signing concern upstream of the loader.
|
||||
|
||||
### Because updates must never strand a working system
|
||||
|
||||
Minimums everywhere, no maximums, and append-only evolution in both directions mean an update can only add satisfiable states. This is what allows modules to be released independently: coordination happens through the versioning model instead of through lockstep builds.
|
||||
|
||||
### Because modules are developed independently
|
||||
|
||||
One repository per module, plus the core repository. A module links libdpm-core and nothing else from DPM; peer modules never appear in its repository, build, or test environment. "Not all the pieces are there" is the normal, permanent condition at build time, so the architecture is arranged to make that a non-event:
|
||||
|
||||
- A module compiles against its own declarations plus core.
|
||||
- Every dependency it consumes is a plain struct of function pointers, so a fake is a struct the test fills in — dependency injection is inherent, with no linker seams.
|
||||
- A harness that links real core and points the module path at build output plus fixture stubs exercises the real five-step validation on a bare builder.
|
||||
- Only final integration needs the whole system, and because alternate roots are first-class, it needs a directory rather than a virtual machine.
|
||||
|
||||
The core repository's own fixtures are deliberately broken stub modules — missing symbols, bad magic, a lying manifest, a too-new core minimum, a malformed version — plus one known-good stub. Core development never requires a real package module to exist.
|
||||
|
||||
## What the core repository produces
|
||||
|
||||
- **libdpm-core.so** — the library
|
||||
- **dpm** — the command-line tool
|
||||
- **info** — a module bundled with the dpm binary and libdpm-core, used to test and demonstrate full DPM system functionality where appropriate
|
||||
|
||||
Every other module is developed against libdpm-core and lives outside this repository.
|
||||
|
||||
## Invariants
|
||||
|
||||
1. Core routes and hosts; modules implement. No package logic in core, ever.
|
||||
2. Writes flow down, never sideways: a layer mutates the system only through the layer beneath it, in-process through core-mediated APIs.
|
||||
3. A module is either fully valid or not loaded — no partial states, no consumer-side defense.
|
||||
|
||||
## Further reading
|
||||
|
||||
- **DESIGN.md** — the full design specification
|
||||
- **CONSUMERS.md** — linking libdpm-core and driving it from a program
|
||||
- **MODULES.md** — writing, building, testing, and installing a module
|
||||
- **BUILD.md** — building, testing, and installing core itself
|
||||
- **DOCUMENTATION.md** — generating the code reference
|
||||
Reference in New Issue
Block a user