dpm-core-ng/docs/OVERVIEW.md

# DPM — An Overview

## What DPM is

DPM is the package manager for Dark Horse Linux. At its center is **libdpm-core**, a shared library that discovers modules, validates them, routes calls to them, reports their versions, and provides configuration and logging. Modules — shared objects in the module directory — implement package functionality.

**The dpm binary** is the command-line tool, and it is one consumer of that library. Build systems, image builders, distribution tooling, and programs in any language with C FFI are equal consumers of the same library through the same interface.

libdpm-core implements no package operations. It routes and hosts.

## The shape of the system

```
 the dpm binary       build systems / DHL tools / other languages
       \                          /
        v                        v
              libdpm-core.so
     (discovery, validation, routing,
      version reporting, config, logging)
                    |
                    v
                modules
             (one .so each)
```

Everything passes through libdpm-core: binary-to-module, module-to-module, program-to-module. No consumer performs module discovery or dynamic loading itself.

## Three layers to read this at

DPM is met at three different layers, and which one you are at determines what any statement below means for you.

- **The user layer** — a person at a shell running the dpm binary. Everything here is a command and its flags.
- **The developer layer** — a program written against the C API in `<dpm/core.h>`, linking `-ldpm-core`. Everything here is a function call. Module authors live here too, on the other side of the same boundary.
- **The filesystem layer** — the directories and files DPM reads and writes: configuration, the module directory, the target root, and the installed artifacts.

The sections that follow are labelled by layer. Where a mechanism exists at more than one layer, it is described once at each, because the same idea looks different depending on where you stand.

## How it works at the user layer

The dpm binary's subcommand surface is exactly the set of loadable modules. `dpm <module> <command> [args...]` loads that module and hands the command to it; `dpm <module> help` asks the module to describe itself. There is no fixed list of operations baked into the tool, because the tool contains no capability logic — it parses arguments and prints.

`dpm --list-modules` shows every module that passes validation, with its version and description. Candidates that fail are excluded from the listing and logged, so what is listed is what can actually be run.

Four flags redirect the system defaults:

- `-c, --config-dir PATH` — where configuration is read from
- `-m, --module-path PATH` — which directory modules are loaded from
- `-r, --root PATH` — the target root that package operations act on
- `-L, --log-level LEVEL` — FATAL, ERROR, WARN, INFO, or DEBUG

These four exist because of a rule the design imposes on itself: every field a linked program can override must also be a flag, so anything reachable from code is reachable from a shell. That rule holds for any override added in the future.

## How it works at the developer layer

A program links libdpm-core and includes `<dpm/core.h>`. Everything it does happens through a **context**, an opaque handle obtained from `dpm_open`.

Opening a context reads the configuration files, resolves which directory modules will be loaded from, and initializes logging. It loads no modules. The same four things the dpm binary exposes as flags are the fields of the overrides struct passed to `dpm_open`, and passing NULL accepts the system defaults — which is what makes a default context equivalent to invoking the installed dpm binary, since that binary is just another caller doing the same thing.

The target root override is what makes chroot builds, image assembly, and sysroot management work: package operations act on the given tree instead of the running system. Multiple contexts with different roots may be open at once.

The context owns everything it hands out. Every string a caller receives stays valid until `dpm_close`, and callers never free anything.

**Acquiring a module.** `dpm_require` resolves a module by name, validates it completely, and returns a handle — or returns NULL, with `dpm_last_error` carrying the precise reason. Modules load at most once per context, and repeated calls return the same handle.

**Reading what libdpm-core saw.** `dpm_module_info_of` fills in a module's name, version, description, and minimum-libdpm-core version. Those values are reported, and no conclusion is drawn from them.

**Calling into a module.** Two paths:

- `dpm_execute` passes a command name and an argument vector to the module's entry point. This is the path the dpm binary takes.
- `dpm_get_api` returns a plain C struct of function pointers for a named API at a stated version. This is the path modules and external programs take to call functions directly.

**Enumerating.** `dpm_list_modules` yields a cursor over every valid module, which is what backs the listing the dpm binary prints.

**Services.** A module reaches libdpm-core through the context that dispatched the call: `dpm_log` to write a message, `dpm_config_get` to read a value from its own configuration namespace, `dpm_module_path` to learn where modules live, `dpm_core_version` to learn the library's version. A module needs no file handling and no logging machinery of its own.

## How it works at the filesystem layer

**Configuration** lives in `/etc/dpm/conf.d/`. Each `.conf` file in that directory is one namespace named after the file: `core.conf` holds the library's own settings, and a module named `mymodule` reads `mymodule.conf`. Files are sectioned, and a value is addressed by namespace, section, and key. The library's own file carries the log level, whether to write a log file and where, and the default module directory.

**Modules** live in `/usr/lib/dpm/modules/`. A module's name is its filename without the `.so` extension, so `info.so` is the module named `info`. Discovery is the presence of a valid file in that directory — there is no registry, and no registration step.

**Logging** goes to the console always, and to a log file when configuration enables one; the default path is `/var/log/dpm/dpm.log`.

**The target root** defaults to `/`. Pointed elsewhere, it is the tree that package operations act on, which is what turns a scratch directory into a complete target for image assembly or a chroot build.

**Installed artifacts:**

| Artifact | Location |
|---|---|
| the dpm binary | `/usr/bin/dpm` |
| `libdpm-core.so` | `/usr/lib/libdpm-core.so` |
| public header | `/usr/include/dpm/core.h` |
| modules | `/usr/lib/dpm/modules/<name>.so` |

## The module contract

*Developer layer, module-author side.*

A module is one `.so` in the module directory exporting five reserved symbols: a command entry point, its own version, a one-line description, the minimum libdpm-core version it supports, and a manifest. The manifest declares the module's entire functional surface — for each API, its name, table version, and the exact exported symbol carrying the table.

Beyond those five, a module exports one symbol per API table. A table is a C struct of function pointers opening with a magic constant and the struct's size in bytes as the module compiled it, which lets a consumer accept a tail-extended revision of the same version. Everything crossing a table is a C type; state passes through opaque handles; errors are int codes.

## Load-time enforcement

libdpm-core validates a module completely before offering it to anyone, in five steps:

1. Every reserved contract symbol resolves.
2. The module's minimum libdpm-core version is not newer than the running library's own.
3. The version and description probes return well-formed values.
4. Every symbol the manifest declares actually resolves.
5. Every declared table carries the correct magic constant and a sane size.

A module failing any step is refused with an itemized reason and its library handle closed. Validation is all-or-nothing: if a handle comes back, the contract already passed. Consumers never defend against partially valid modules, because they cannot receive one.

At the user layer this is visible as a module missing from `--list-modules` with a logged reason, or as an itemized failure when the module is named directly.

## Versioning

libdpm-core enforces exactly one version rule, the one where it is the host: each module states the oldest libdpm-core it supports, and a library older than that refuses to load the module, with a message naming the remedy. A module is never rejected for being old, because the library's contract grows append-only — a newer libdpm-core supports everything an older one did.

Every other version question belongs to the consumer. A module that depends on another requires it, reads the version reported back, and decides for itself whether that version is too new or too old for the calls it intends to make. libdpm-core reports what it saw and draws no conclusion from it, so a handle means the module is valid rather than that it suits any particular caller.

Module APIs evolve append-only: a breaking change means exporting a new table beside the old one rather than mutating the existing one. A newer module therefore still carries everything an older consumer called, so updating libdpm-core or any module preserves every combination that previously worked.

## Why it works this way

### Because it must run on a barren system

The founding constraint is that DPM has to work in an environment providing nothing but libc and libstdc++, using the same binaries that run on a fully populated system. That forbids libdpm-core from taking on any dependency beyond the baseline, which forbids package logic from living inside it, which produces the division the whole architecture rests on: **libdpm-core routes and hosts, modules implement.** Anything needing a database, compression, or TLS is a module that arrives later.

### Because capability has to grow in layers

Each layer of the system installs the dependencies of the next using only what already works. That requires upper layers to reach lower ones through a stable in-process interface rather than by re-implementing them, and it requires writes to flow strictly downward — a layer mutates the system only through the layer beneath it. Shared concerns like locking are then implemented once, at the bottom, and inherited by everything above.

### Because one implementation must serve every caller

The requirement that every implementation exist exactly once, and be consumable by the dpm binary, by other layers, and by external programs, is what makes libdpm-core a C ABI library rather than an application with a library carved out of it. This is also why the user layer and the developer layer stay in step: the flags are the override fields, one for one, so a developer and a user redirect the same things by the same names.

### Because nothing may be trusted that has not been verified

libdpm-core is the sole authority on module validity, and the contract is enforced by its validator rather than by an SDK a module author might skip, patch, or fall behind. That is why validation is all-or-nothing and why it happens at load: failures surface loudly and itemized at install time instead of halfway through an operation on a user's system. The manifest exists so a module's entire declared surface can be checked before any of it is offered — an API absent from the manifest does not exist, even if its symbol does.

The residual limit is that dynamic symbol lookup cannot verify a function signature. The magic constant, the size field, the minimum-version handshake, and the probes cover this in practice; defeating them takes deliberate lying, which is a package-signing concern upstream of the loader.

### Because updates must never strand a working system

libdpm-core holds no maximums and no exact-match constraints, and evolution is append-only in both directions, so an update can only add satisfiable states. This is what allows modules to be released independently: coordination happens through the versioning model instead of through lockstep builds.

### Because modules are developed independently

One repository per module, plus the libdpm-core repository. A module links libdpm-core and nothing else from DPM; peer modules never appear in its repository, build, or test environment. "Not all the pieces are there" is the normal, permanent condition at build time, so the architecture is arranged to make that a non-event:

- A module compiles against its own declarations plus libdpm-core.
- Every dependency it consumes is a plain struct of function pointers, so a fake is a struct the test fills in — dependency injection is inherent, with no linker seams.
- A harness that links the real libdpm-core and points the module path at build output plus fixture stubs exercises the real five-step validation on a bare builder.
- Only final integration needs the whole system, and because alternate roots are first-class, it needs a directory rather than a virtual machine.

This repository's own fixtures are deliberately broken stub modules — missing symbols, bad magic, a lying manifest, a too-new minimum version, a malformed version — plus one known-good stub. Developing libdpm-core never requires a real package module to exist.

## What this repository produces

- **libdpm-core.so** — the library
- **the dpm binary** — the command-line tool
- **info** — a module bundled with the dpm binary and libdpm-core, used to test and demonstrate full DPM system functionality where appropriate

Every other module is developed against libdpm-core and lives outside this repository.

## Invariants

1. libdpm-core routes and hosts; modules implement. No package logic inside it, ever.
2. Writes flow down, never sideways: a layer mutates the system only through the layer beneath it, in-process through mediated APIs.
3. A module is either fully valid or not loaded — no partial states, no consumer-side defense.
4. Anything a linked program can redirect, a shell user can redirect too.

## Further reading

- **DESIGN.md** — the full design specification
- **CONSUMERS.md** — linking libdpm-core and driving it from a program
- **MODULES.md** — writing, building, testing, and installing a module
- **BUILD.md** — building, testing, and installing libdpm-core and the dpm binary
- **DOCUMENTATION.md** — generating the code reference