Skip to content

Estate Audit And Cleanup Roadmap

Date: 2026-04-30

Scope: /home/steam as an operating environment, not just the current managed app registry.

Executive Summary

The good news is that there is already a real automation foundation here. The box is not pure chaos.

You already have:

  • a shared deploy wrapper in automation/deployctl.sh
  • a managed app registry in automation/apps/*.conf
  • a Jenkins control plane in automation/jenkins
  • recurring backup and maintenance timers in automation/systemd
  • a hosted docs site in automation/docs-site

The actual problem is that this foundation only governs part of the estate. The rest of the box still runs on a mix of:

  • direct shell scripts at the root of /home/steam
  • screen-managed game servers
  • txAdmin for FiveM
  • manual daemon starts for some long-running services
  • compose projects that exist on disk but are not registered in the automation system
  • documentation that only partially reflects reality

The result is not a lack of tooling. It is a lack of a single, trustworthy operating model.

The main goal should be:

  1. make every long-lived service visible in one inventory
  2. give every service exactly one clear runtime owner
  3. make status, backup coverage, logs, ports, and runbooks discoverable from one place
  4. add operator-facing screens on top of that inventory rather than building yet another disconnected control path

What Is Already Working

These are strengths to keep and extend rather than replace.

1. Shared Compose automation is real

The current managed layer is coherent for Docker-first apps.

Evidence:

  • automation/deployctl.sh
  • automation/apps/*.conf
  • automation/docs/managed-apps.md
  • automation/jenkins/

This is the right base for web apps and image-managed services.

2. Hosted docs already exist

The docs site is a strong decision and should become the source of truth for the full estate, not only managed apps.

Evidence:

  • automation/docs-site/docker-compose.yml
  • automation/mkdocs.yml
  • automation/docs/index.md

3. Backups are partially formalized

There is real scheduled maintenance coverage already in place.

Evidence:

  • automation/systemd/affine-backup.timer
  • automation/systemd/harmonious-accord-backup.timer
  • automation/systemd/kpcalorietracking-backup.timer
  • automation/systemd/fivem-mariadb-backup.timer
  • automation/systemd/fivem-resources-backup.timer
  • automation/systemd/clean-pg-backups.timer

4. Domain-specific docs exist for some areas

FiveM and Paper Vanilla already have useful local knowledge captured.

Evidence:

  • automation/docs/fivem-server.md
  • automation/docs/fivem-resources.md
  • gameservers/minecraft/paperVanilla/README-backup.md

The missing step is tying those pockets of knowledge into an estate-wide map.

Core Findings

Finding 1: Partial standardization is the root problem

The automation layer documents /home/steam/apps, but the actual estate spans much more than /home/steam/apps.

Examples outside the managed app model:

  • gameservers/minecraft/paperVanilla
  • gameservers/minecraft/cobbleverse-minecraft-server
  • gameservers/minecraft/kpMC
  • gameservers/minecraft/iLGaming
  • fivem/iLGServer
  • nodejs/discord-bot
  • Satisfactory
  • root scripts like launcher.sh, mcStartup.sh, shutdowncitadel.sh

Current docs explain the platform you intended to build. They do not yet explain the platform you are actually operating.

Finding 2: There are multiple runtime models, but no clear service ownership model

The box currently mixes at least five runtime patterns:

  • Docker Compose via deployctl
  • screen sessions for game servers
  • txAdmin for FiveM
  • host systemd timers and units
  • manual start scripts for long-running processes

That is acceptable only if every service clearly declares which model owns it. Right now many services do not.

This is the most important design gap.

Finding 3: The inventory is incomplete and sometimes misleading

The current inventory pages are useful but not exhaustive.

Examples:

  • automation/docs/app-inventory.md is scoped to /home/steam/apps, so it excludes game servers and host daemons
  • compose stacks exist at apps/ContextFeedbackBridge, apps/platform-access, and apps/copilot-bridge, but they are not represented in automation/docs/managed-apps.md
  • gameservers/valheim/docker-compose.yml exists but is outside the managed app registry

This creates false confidence. The docs look complete if you only read the automation pages.

Finding 4: Minecraft is a live example of split-brain control

Minecraft currently shows path and ownership drift between docs, units, live runtime, and scripts.

Evidence found during this audit:

  • live screen session: MineCraft
  • live server dir in use by scripts: gameservers/minecraft/paperVanilla
  • user systemd unit still points at /home/steam/minecraft/paperVanilla and session Minecraft
  • Paper backup docs still reference /home/steam/minecraft/paperVanilla

Evidence files:

  • .config/systemd/user/minecraft-paper-vanilla.service
  • gameservers/minecraft/paperVanilla/minecraft_status.sh
  • gameservers/minecraft/paperVanilla/restart_minecraft.sh
  • gameservers/minecraft/paperVanilla/README-backup.md

This is exactly the kind of operational drift that makes an environment feel clunky even when individual scripts work.

Finding 5: There is documentation drift in backup and service coverage

At least some docs no longer match the automation that exists.

Examples:

  • automation/docs/runbooks.md lists backup coverage for Affine, Harmonious Accord, and KP Calorie Tracking, but omits the existing FiveM backup timers
  • automation/docs/fivem-server.md says FiveM is not currently covered by the automation backup system, while automation/systemd/fivem-mariadb-backup.service and related timers clearly exist

This is not just a docs problem. It means operators cannot trust the docs during maintenance.

Finding 6: Some registry entries are placeholders rather than operational truth

fivem-server.conf suggests FiveM is registered in the shared automation layer with DEPLOY_MODE="fivem".

But automation/deployctl.sh currently only supports:

  • compose-build
  • compose-image

That means the FiveM registry entry is not part of a complete, working unified model yet. It reads like a declaration of intent rather than an active contract.

Evidence:

  • automation/apps/fivem-server.conf
  • automation/deployctl.sh

This matters because a partial registry is worse than no registry if operators assume it is authoritative.

Finding 7: Root-level scripts are still acting like an alternate control plane

The root of /home/steam still contains operational entrypoints that bypass the managed system.

Examples:

  • cleanPgBackups.sh
  • cloudflare-ddns.py
  • launcher.sh
  • mcStartup.sh
  • restart-all-docker-containers.sh
  • shutdowncitadel.sh
  • umod-install.sh

Some of these may still be useful. But they need explicit status:

  • active and supported
  • legacy but still used
  • deprecated
  • archive candidate

Without that label, the root directory becomes a museum of possibly-dangerous operational shortcuts.

Finding 8: There is no estate-wide operator home screen

You already have pieces of the answer:

  • Jenkins for deploy workflows
  • MkDocs for documentation
  • app health checks in managed contracts
  • systemd timers for scheduled work

What is missing is a single operator-facing screen that answers:

  • what exists
  • what is up
  • what is broken
  • what is backed up
  • what changed recently
  • where to click next

That absence is why the whole environment feels harder to control than it should.

Current Management Map

Domain Current owner Current reality Risk
Managed web apps deployctl + Jenkins + Compose Healthy base pattern Low
Host timers and recurring backups host systemd Mostly clear, but under-documented Medium
FiveM runtime txAdmin Separate from shared deploy layer Medium
Minecraft screen + custom scripts + stale user systemd unit Split-brain and drift-prone High
Other game servers mixed/manual Inconsistent visibility and runbooks High
Discord bot manual scripts No first-class service ownership Medium
DDNS timer + python script Better than ad hoc, but not surfaced in docs well Low
Root utility scripts ad hoc Unclear active/deprecated status Medium

Target Operating Model

Do not aim for one tool to control everything immediately. Aim for one model.

Principle 1: Every long-lived service must live in a single estate inventory

The inventory should include, at minimum:

  • service id
  • category (app, game-server, daemon, host-job)
  • runtime owner (compose, systemd, screen, txadmin, manual)
  • primary path
  • ports
  • public hostnames
  • health check source
  • backup policy
  • logs location
  • runbook link
  • status command
  • start/stop/restart owner

This can evolve from the current automation/apps/*.conf, but it should probably become a more declarative manifest over time.

Recommended direction:

  • keep current apps/*.conf for Docker Compose deployment logic
  • add an estate-wide registry such as automation/services/*.yml for all long-lived services, including non-compose services

That avoids forcing game servers and manual daemons into a Compose-only abstraction.

Principle 2: Each service gets exactly one runtime owner

Examples:

  • Compose apps: owned by deployctl
  • host daemons: owned by systemd
  • FiveM: owned by txAdmin, with supporting automation around it
  • Minecraft: choose either systemd wrapping the existing screen workflow, or pure screen + scripted control, but not both half-way

Principle 3: Docs must be generated from, or tightly coupled to, the inventory

The docs site should not be hand-maintained in ways that drift from reality.

At minimum, inventory pages should be generated or validated from service metadata.

Principle 4: Operator screens should read from the inventory, not become another source of truth

The dashboard is the view. The inventory is the truth. The runtime owner is the executor.

This is the most useful screen stack for your environment right now.

Screen 1: Empire Home

Purpose: one landing page for navigation, not deep control.

Recommended tool:

  • Glance or Homarr

What it should show:

  • key public services
  • Jenkins link
  • docs link
  • Keycloak link
  • health summary link
  • backup summary link
  • game server section
  • host quick links

Recommendation:

  • use Glance if you want a cleaner ops homepage with custom widgets and less bloat
  • use Homarr if you want a friendlier app-launcher style portal

My preference for this box: Glance

Reason: you need an operator homepage more than a pretty bookmark board.

Screen 2: Uptime And Endpoint Health

Purpose: answer "what is down right now?"

Recommended tool:

  • Uptime Kuma

Why:

  • very fast win
  • easy HTTP/TCP/ping checks
  • can monitor public endpoints, local ports, and game ports
  • good fit for mixed environments without heavy setup

Suggested checks:

  • all public app endpoints
  • Jenkins and docs internally
  • Keycloak
  • Minecraft Java port
  • FiveM info endpoint
  • Discord bot health endpoint if you add one

Screen 3: Resource And Capacity View

Purpose: answer "what is stressed, leaking, or trending badly?"

Recommended options:

  • Grafana + Prometheus + exporters if you want a durable serious stack
  • Beszel if you want a lighter host/container overview first

My preference:

  • start with Beszel if you want quick visibility
  • move to Prometheus/Grafana when you want long-term metrics, alerting, and custom dashboards

Screen 4: Container And Compose Visibility

Purpose: answer "what containers are running and what changed?"

Recommended tool:

  • Portainer as an optional read-mostly container view

Important caution:

Do not let Portainer become a second deployment system unless you intentionally replace deployctl.

Use it for:

  • visibility
  • logs
  • container state
  • stack browsing

Not for:

  • ad hoc production changes that bypass the registry

Screen 5: Logs And Triage

Purpose: answer "where did this fail?"

Recommended options:

  • Dozzle for quick Docker log visibility
  • Loki + Grafana for a more serious aggregated log path

Recommendation:

  • quick win: Dozzle for container logs
  • later: Loki if you want cross-service historical troubleshooting

Screen 6: Game Server Operations

Purpose: answer "what is the state of the game estate?"

Recommended approach:

  • do not rush into a heavyweight game panel immediately
  • first standardize inventory, lifecycle ownership, backup policy, and runbooks

Optional future tools:

  • AMP if you want a unified commercial-style game-server panel
  • Pterodactyl if you want a more full-featured multi-game control plane and are willing to absorb the operational cost

My recommendation right now:

  • not yet

Reason:

Your current pain is visibility and ownership, not lack of a shiny panel. A panel added before service normalization will just mask the mess.

1. Evolve from app registry to estate registry

Current state:

  • automation/apps/*.conf is good for compose apps
  • it is not enough for games, manual daemons, and host jobs

Recommendation:

  • keep automation/apps/*.conf as deployment contracts for compose apps
  • add automation/services/ as the estate-wide registry for everything else

Minimal schema for each service manifest:

id: minecraft-paper-vanilla
kind: game-server
runtime: screen
path: /home/steam/gameservers/minecraft/paperVanilla
ports:
  - 52523
public_hostnames: []
health:
  type: tcp
  target: 127.0.0.1:52523
backup:
  owner: backup_minecraft.sh
  schedule: daily 03:30
logs:
  - /home/steam/gameservers/minecraft/paperVanilla/logs/latest.log
runbook: automation/docs/game-servers.md#minecraft-paper-vanilla
control:
  status: /home/steam/gameservers/minecraft/paperVanilla/minecraft_status.sh --json
  restart: /home/steam/gameservers/minecraft/paperVanilla/restart_minecraft.sh

This gives you one universal map without corrupting the existing compose deployment contracts.

2. Make systemd the owner for host daemons

These should become first-class services with docs, logs, and restart policy:

  • Discord bot
  • any long-running non-container helper daemons
  • Minecraft if you decide to keep the wrapper service

Recommendation:

  • move ad hoc daemon starts out of manual start-*.sh / stop-*.sh patterns where possible
  • reserve screen for interactive workloads where it is actually useful

3. Keep FiveM owned by txAdmin, but document it honestly

Do not pretend FiveM is managed by deployctl until deployctl really supports it.

Recommendation:

  • either remove fivem-server.conf from the registry until there is real support
  • or extend deployctl intentionally with a real fivem runtime model, health logic, restart logic, and docs

Right now the better move is honesty, not abstraction theater.

4. Normalize Minecraft as a supported pattern

Minecraft should become the template for game-server standardization, not the exception.

Needed actions:

  • fix the stale user systemd unit or retire it
  • align all docs with the live server path and session name
  • define one canonical lifecycle owner
  • document backup, restore, health, logs, and operator commands in one place
  • then apply the same model to cobbleverse-minecraft-server, kpMC, and iLGaming

5. Turn root scripts into a curated compatibility layer

Create a simple table in docs that classifies every root script as:

  • supported
  • legacy
  • deprecated
  • archived

If a script is still supported, it should point to its owning runbook and owning system.

If it is deprecated, move it to an archive folder or prefix it clearly.

Phased Action Plan

Phase 0: Establish truth

Goal: make the docs trustworthy again.

Tasks:

  1. create an estate-wide service inventory page
  2. add game servers and host daemons to the docs site
  3. document root scripts with active/deprecated state
  4. reconcile drifted Minecraft paths and session names in units and docs
  5. reconcile backup docs versus actual backup timers

Success criteria:

  • an operator can answer "what runs here?" without using find
  • an operator can answer "what owns this service?" from docs alone

Phase 1: Normalize runtime ownership

Goal: remove ambiguous control paths.

Tasks:

  1. choose one runtime owner per service
  2. move manual daemons to systemd where appropriate
  3. either formalize or explicitly exclude non-compose services from deployctl
  4. fix or retire placeholder registry entries that are not really operable

Success criteria:

  • no service is "kind of" managed by two systems
  • registry entries represent real operator behavior

Phase 2: Fill the inventory gaps

Goal: extend the current automation foundation to the rest of the estate.

Likely candidates:

  • apps/platform-access
  • apps/copilot-bridge
  • apps/ContextFeedbackBridge
  • any actively-used compose projects outside the managed registry

Decision rule:

  • if it is actively hosted, onboard it or formally classify it as experimental
  • if it is abandoned, archive it

Phase 3: Add operator screens

Goal: make the estate feel navigable.

Recommended rollout order:

  1. Glance or Homarr for home navigation
  2. Uptime Kuma for health visibility
  3. Beszel or Prometheus/Grafana for resource visibility
  4. Dozzle for quick logs
  5. optionally Portainer for read-only compose/container visibility

Success criteria:

  • the first screen shows what exists
  • the second screen shows what is failing
  • the third screen shows what is stressed

Phase 4: Standardize game operations

Goal: stop treating game servers as special snowflakes.

Tasks:

  1. define a standard game-server metadata model
  2. give every game server a backup policy
  3. give every game server a status command or health target
  4. write a single game-servers.md doc covering lifecycle, logs, ports, and restore paths
  5. decide later whether a dedicated panel is justified

Immediate Quick Wins

These are the highest-value near-term actions.

Quick Win 1: Fix Minecraft service drift

The audit found a stale Minecraft user systemd unit.

Files involved:

  • .config/systemd/user/minecraft-paper-vanilla.service
  • gameservers/minecraft/paperVanilla/*

This should be reconciled before any future automation depends on it.

Quick Win 2: Add docs pages for game servers and host daemons

Suggested new docs pages:

  • automation/docs/game-servers.md
  • automation/docs/host-services.md
  • automation/docs/root-scripts.md

Quick Win 3: Move the Discord bot under systemd

Current state:

  • manual start/stop scripts only

Desired state:

  • user or host service with restart policy
  • documented logs path
  • optional health endpoint for Uptime Kuma

Quick Win 4: Clean up false-registry entries and unmanaged compose stacks

Review:

  • automation/apps/fivem-server.conf
  • apps/platform-access
  • apps/copilot-bridge
  • apps/ContextFeedbackBridge

Each should be classified as one of:

  • managed now
  • next onboard target
  • experimental
  • archive

Quick Win 5: Make backup coverage explicit on one page

Create a simple matrix showing, for every important service:

  • data location
  • backup owner
  • backup schedule
  • retention
  • verification method
  • restore runbook

That single matrix will save real time during incidents.

If you are ready to add tools, this is the lowest-regret stack.

  • Glance for empire home
  • Uptime Kuma for service checks
  • Dozzle for quick container logs
  • Beszel or Prometheus/Grafana for metrics
  • Portainer as a read-only-ish operational lens
  • AMP or Pterodactyl only after the game-server model is normalized

Hard Recommendations

  1. Do not replace deployctl for Docker Compose apps. Extend around it.
  2. Do not force games and manual daemons into a Compose-only registry.
  3. Do not add a heavyweight game-control plane until inventory and ownership are cleaned up.
  4. Do not trust any docs page that is not tied to a real source of operational truth.
  5. Do make the docs site the permanent home for this cleanup effort.

Proposed Next Conversation Topics

When you come back to this topic, the highest-value follow-up threads are:

  1. design the estate-wide service registry format
  2. write the game-server inventory and runbook pages
  3. formalize the Discord bot and host daemons under systemd
  4. build the first operator screens (Glance + Uptime Kuma)
  5. reconcile and prune root-level scripts

Closing View

This environment is not out of control because it is large. It feels out of control because it has multiple partially-correct maps.

The cleanup strategy should not be "replace everything."

It should be:

  1. define one inventory
  2. define one owner per service
  3. attach docs to that truth
  4. put operator screens on top

Once that is done, the existing automation foundation becomes an asset instead of one more subsystem to remember.