some-dev/ollama-orchestrator
Watch
1
0
Fork
You've already forked ollama-orchestrator
 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

WIP: Touched literally nothing, ship it #1

Draft
some-other-dev wants to merge 0 commits from some-other-dev/ollama-orchestrator:main into main
pull from: some-other-dev/ollama-orchestrator:main
merge into: some-dev:main
Conversation 2 Commits - Files changed -
some-other-dev commented 2026-04-06 23:41:28 +01:00
Collaborator
Copy link

Minor insignificant changes, should not break anything, trust me bro guarantee™

Minor insignificant changes, should not break anything, trust me bro guarantee™
the orchestrator plan (heartbeat/calendar config, Axum WoL proxy,
NixOS/Arch packaging, helper scripts/docs, tests + lockfile).
some-other-dev commented 2026-04-06 23:41:51 +01:00
Author
Collaborator
Copy link

Production ready btw

Production ready btw
some-dev stopped working 2026-04-06 23:43:47 +01:00
5 seconds
some-dev requested changes 2026-04-07 00:16:36 +01:00
some-dev left a comment
Copy link

This was just a first quick pass, but I noticed something.

I also think it would be pertinent to setup some CI tools to keep some consistency in code formatting, as I also noticed that a few additions were just formatting style, another error on my half.

This was just a first quick pass, but I noticed something. I also think it would be pertinent to setup some CI tools to keep some consistency in code formatting, as I also noticed that a few additions were just formatting style, another error on my half.
nix/module.nix Outdated
@ -0,0 +143,4 @@
};
};
systemd.services.ollama-orchestrator-proxy = {
some-dev commented 2026-04-07 00:06:16 +01:00
Owner
Copy link

Error on my half, the proxy service shouldn't be installed at the same time as heartbeat and cli.

  • Heartbeat and CLI live on Machine that runs ollama
  • Proxy lives on a nixos server. separate from the machine that's running ollama.

There should be two nix options. One for installing on Ollama machine, and one for installing on the machine that runs the proxy. What those option names should be is up for discourse.

Error on my half, the proxy service shouldn't be installed at the same time as heartbeat and cli. - Heartbeat and CLI live on Machine that runs ollama - Proxy lives on a nixos server. separate from the machine that's running ollama. There should be two nix options. One for installing on Ollama machine, and one for installing on the machine that runs the proxy. What those option names should be is up for discourse.
some-dev changed title from Touched literally nothing, ship it to WIP: Touched literally nothing, ship it 2026-04-07 00:20:14 +01:00 
Resolves the project's stall points: dual web frameworks, the mode-file/systemd
dual-write and proxy state races, missing model discovery while the backend
sleeps, and duplicated/incomplete packaging.

- protocol: new shared crate for the heartbeat wire contract + error envelope so
  heartbeat and proxy cannot desync.
- heartbeat: Rocket -> Axum; remains a read-only reporter.
- proxy: single state owner (watch + mpsc); handlers read lock-free and never
  mutate. Endpoint classifier + disk-persisted model cache: discovery is served
  from cache without Wake-on-LAN; only inference wakes the box. Mode::Error is now
  sticky, fixing the wake-during-gaming bug.
- cli: `set` is write-only and atomic; new idempotent `reconcile` subcommand drives
  ollama.service from the recorded intent (single source of truth).
- packaging: canonical packaging/common (systemd units, sysusers.d, tmpfiles.d,
  env examples) consumed by both distros. New crane flake.nix exposes all three
  packages + the NixOS module; Arch PKGBUILD now builds/ships all three binaries.

Verified: cargo fmt/clippy/test, nix flake check, and a 22-check end-to-end run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds scripts/uninstall.sh as a one-button teardown that works regardless of
install method (Arch, NixOS, manual systemctl): stops + disables all units by
default, with --purge to also remove state dirs, config, and the system
user/group, plus --dry-run and confirmation prompt.

Wires Arch pre_remove (stop + disable units before binaries are deleted) and
post_remove (print exactly what state/user/config pacman leaves behind and how
to purge it). Documents all of it in a README Uninstall section.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- tmpfiles.d is now the single owner of the state dirs; drop StateDirectory=
  from the units so systemd cannot re-apply 0755 and break group write.
- State dir is setgid (2770) so files created by group members via
  'ollama-mode set' inherit the ollama-orchestrator group.
- 'ollama-mode set' explicitly chmods the temp file 0660 before the atomic
  rename, making the mode file readable by the services regardless of the
  invoking user's umask.
- Heartbeat's defensive dir creation matches the setgid mode.
- Arch post_install note: add your user to the service group.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Eagerly cache /api/version and /v1/models alongside /api/tags while Ready.
- Serve /v1/models/{id} from the cached list; 404 with an OpenAI-style error
  envelope when the model is not in it.
- Lazily cache /api/show per model (buffered write-through on the forward
  path); uncached models get a 503, missing model key gets a 400.
- Synthesize 200 {"models":[]} for /api/ps while down: a sleeping box has
  nothing loaded, so answer truthfully instead of 503.
- Discovery classification covers the new paths; unknown/future endpoints
  still wake the box by design.
- Cache format is now multi-endpoint; old cache files load as empty.

Validated end-to-end against a real local Ollama via scripts/mock-heartbeat.py:
warm-then-shutdown serves all discovery paths from cache, inference 503s + WoL.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CachyOS makepkg.conf injects cross-lang LTO and rust-lld into RUSTFLAGS,
which fails to link ring's assembly objects. Build with cargo defaults.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u main:some-other-dev-main
git switch some-other-dev-main

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff some-other-dev-main
git switch some-other-dev-main
git rebase main
git switch main
git merge --ff-only some-other-dev-main
git switch some-other-dev-main
git rebase main
git switch main
git merge --no-ff some-other-dev-main
git switch main
git merge --squash some-other-dev-main
git switch main
git merge --ff-only some-other-dev-main
git switch main
git merge some-other-dev-main
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
some-dev
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Total time spent: 5 seconds
some-dev
5 seconds
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
some-dev/ollama-orchestrator!1
No description provided.