bcs-website/docs/gitops-deployment-strategy.md

# BlueCap Strategies GitOps Deployment Strategy (ArgoCD)

## Goal

Deploy the BlueCap Strategies Astro site to the existing k3s cluster using GitOps, without
giving Gitea Actions direct access to the homelab cluster.

The homelab already runs **ArgoCD** in k3s, so this strategy uses ArgoCD as the reconciler
rather than standing up a second GitOps controller (Flux). ArgoCD pulls from Git and the image
registry; the cluster is never exposed to the CI system.

Desired operating model:

1. Edit the site locally in this repository.
2. Commit changes.
3. Push to `main`.
4. Gitea Actions runs tests and builds a container image.
5. Gitea Actions pushes the image to Gitea Packages, tagged with the immutable `<git-sha>`.
6. Git is updated with the new desired image tag (CI commits the tag into `k8s/`).
7. ArgoCD, running inside k3s, detects the Git change.
8. ArgoCD syncs the Kubernetes manifests (Kustomize).
9. k3s pulls the new image and rolls the website deployment.
10. Traefik serves the updated site publicly.

The CI runner never receives kubeconfig or direct network access to the homelab cluster. The cluster
pulls from Git and the image registry.

> **Concrete implementation is Gitea, not GitHub/GHCR** — see [ADR-0007](adr/0007-gitea-actions-and-packages.md).
> CI = Gitea Actions (`.gitea/workflows/ci.yml`); registry = Gitea Packages
> (`10.66.15.22:3000/bmr_bluecap/bcs-website`); ArgoCD Application in `argocd/bcs-website-application.yaml`.
> Any lingering GitHub/GHCR wording below is the generic model; substitute Gitea/Gitea-Packages.

## Prerequisites & as-built registry flow

> **As built (2026-07-17) — see [ADR-0008](adr/0008-private-package-external-secrets-pull.md).**
> The earlier "k3s trusts the HTTP registry + restart" plan was dropped. Final flow avoids any k3s
> restart or per-node config.

**Registry flow (split push/pull, same private Gitea package, two access paths):**
- **CI pushes** to the **internal** `10.66.15.22:3000/bmr_bluecap/bcs-website` (HTTP, fast). The
  runner host's dockerd trusts it via `insecure-registries` (gitea Ansible role + SIGHUP
  live-reload — no restart). Docker in jobs works because act_runner's config pins jobs to
  `gitea_network` and exposes the docker socket.
- **k3s pulls** the **public HTTPS** `git.7tl-homelab.com/bmr_bluecap/bcs-website` (valid TLS) →
  **no `registries.yaml`, no node config, no k3s restart.** (`k3s_registry_mirrors` stays empty.)
- **Pull auth:** package is **private**; a `read:package` PAT lives in OpenBao (`cluster/bluecap-registry`)
  and External Secrets (`k8s/externalsecret-registry.yaml`) materializes the `gitea-registry`
  dockerconfigjson used by the Deployment's `imagePullSecrets`.
- **CI auth:** dedicated PAT `CI_TOKEN` (repo Actions secret; `write:package` + `write:repository`).

**One-time setup (all done):** repo on Gitea; runner registered (global/instance, on `gitea_network`,
correct instance URL); `CI_TOKEN` secret; OpenBao `cluster/bluecap-registry`; ArgoCD Application
applied. Deploys are now push-to-main → CI → ArgoCD, hands-off.

## Cluster Findings

Read-only `kubectl` inspection previously showed:
- k3s API reachable at `10.66.15.30:6443`.
- Ingress controller: `traefik`. Ingress class: `traefik`.
- cert-manager installed with a `letsencrypt-prod` ClusterIssuer.
- Existing namespaces for media, nextcloud, monitoring, etc.
- No `bluecap-strategies` namespace yet.
- **ArgoCD is already installed and managing other workloads in the cluster.**

Edge / DNS / TLS (research 2026-06-05):
- Edge routing uses a Pangolin/Newt path (`dav.rodgersweb.com` via Caddy).
- `bluecapstrategies.com` DNS is managed by Cloudflare.
- TLS handled by cert-manager via a Cloudflare DNS-01 token.

## Repository & Image Strategy

- **Mono-repo:** Keep site source and k8s manifests in the same repository.
- **Gitea Packages:** Use Gitea's built-in container registry (`10.66.15.22:3000`) for images.
- **Immutable tags:** Deploy via `<git-sha>` tags, never `latest`. `latest` defeats ArgoCD's
  ability to detect and record what is actually running.
- **Manifests:** Plain Kustomize under `k8s/` (already present). ArgoCD consumes
  `k8s/kustomization.yaml` natively — no Helm chart required.

## Phase 1: ArgoCD Application (Foundational GitOps)

Register the site as an ArgoCD `Application`. Two common patterns; pick one to match how the
homelab already organizes ArgoCD:

- **Direct Application** — a single `Application` CR pointing at this repo's `k8s/` path.
- **App-of-apps** — add this `Application` to the existing root/app-of-apps repo if the homelab
  uses that pattern.

Example direct Application (store in the homelab ArgoCD config repo, or apply once to bootstrap):

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: bluecap-website
  namespace: argocd
spec:
  project: default
  source:
    repoURL: http://10.66.15.22:3000/bmr_bluecap/bcs-website.git
    targetRevision: main
    path: k8s
  destination:
    server: https://kubernetes.default.svc
    namespace: bluecap-strategies
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
```

`prune: true` removes resources deleted from Git; `selfHeal: true` reverts manual drift;
`CreateNamespace=true` lets ArgoCD create `bluecap-strategies` (or keep `namespace.yaml` in
`k8s/` — both work, don't do both destructively).

## Image Update Strategy

Two options — this strategy recommends **Option A** for simplicity and auditability:

### Option A (recommended): CI commits the image tag
Gitea Actions, after pushing the image, patches the image tag in `k8s/` (via
`kustomize edit set image` or a `newTag` field) and commits back to `main`. ArgoCD auto-syncs the
new tag. Every deploy is a Git commit traceable to a source SHA — clean rollback via `git revert`.

Add to `k8s/kustomization.yaml`:

```yaml
images:
  - name: 10.66.15.22:3000/bmr_bluecap/bcs-website
    newTag: <ci-sets-this>
```

### Option B: ArgoCD Image Updater
Install ArgoCD Image Updater and annotate the Application to watch the Gitea Packages repo and write back
the tag. Removes the CI commit-back step but adds a controller and registry-credential wiring.
Only adopt if the homelab already runs Image Updater for other apps.

## Phase 2: CI Pipeline (Gitea Actions)

Workflow on push to `main`:

1. **Quality gate:** `npm ci`, `npm run typecheck`, `npm run build`.
2. **E2E validation:** `npm run test:e2e` (Playwright) where browser deps are available.
3. **Publish:** build the Docker image, push to Gitea Packages tagged `<git-sha>`.
4. **GitOps sync (Option A):** `kustomize edit set image ...=10.66.15.22:3000/bmr_bluecap/bcs-website:<git-sha>`,
   commit and push to `main`. ArgoCD reconciles.

CI never touches the cluster — it only writes to Gitea Packages and Git.

## Phase 3: Staging (Launch Readiness)

Avoid testing in production. Add a `k8s/overlays/staging/` Kustomize overlay and a second ArgoCD
Application tracking it.
- **Target:** `staging.bluecapstrategies.com`
- **Goal:** let the owner review content/insights changes before they hit the live site.

## Phase 4: Elite Authority Infrastructure (9/10 Standard)

### 1. Secret hygiene — Git-managed credentials
Avoid manual `kubectl create secret`. Use **Sealed Secrets** or **External Secrets Operator** so
Gitea Packages pull tokens and contact-form keys are encrypted in Git and the site is reproducible from
source alone. Match whichever the homelab ArgoCD stack already standardizes on.

### 2. Edge authority — Cloudflare hardening
- **WAF & protection:** enable Cloudflare WAF to block automated probes and DDoS.
- **Edge caching:** "Cache Everything" for static Astro assets so most traffic serves from
  Cloudflare's edge regardless of homelab bandwidth.
- **Cloudflare Tunnels (optional):** consider `cloudflared` to remove open router ports.

### 3. Uptime intelligence (observability) — reuse the `monylog` stack
The homelab already runs a full LGTM + Alertmanager stack on the `monylog` box (`10.66.15.21`,
Ansible role `pglta`), external to k8s. **Do not deploy Uptime Kuma.** Alloy already remote-writes
BlueCap pod/deployment metrics (kube-state-metrics) and ships nginx pod logs to Loki. To finish:
- **Add `blackbox_exporter`** to the monylog compose + a Prometheus scrape job probing the
  in-cluster Service and the public URL (uptime, HTTP status, TLS-expiry, latency).
- **Add a `web-bluecap.yml` alert rules file** (SiteDown, CertExpiringSoon <14d, High5xx,
  DeploymentDegraded, PodRestarting) mirroring the existing host-infra rules.
- **Alerting** already flows to **Pushover** (OpenBao creds) with ntfy fallback → phone; add a
  `service: bluecap` label. **Grafana** already present for dashboards. ArgoCD sync/health is an
  extra deploy-level signal. See `docs/seo-aeo-optimization.md` §6a for detail.

### 4. Digital ESG — verified sustainability
- Verify Green Web status if the homelab uses renewable energy.
- Use Astro's `Image` component to minimize data transfer.
- Document the low-energy architecture in `llms.txt` or a footer "Digital ESG" note.

### 5. Security performance (A+ rating)
- Harden `deploy/nginx.conf` toward an A+ on `securityheaders.com`: robust CSP, HSTS,
  X-Frame-Options.
- TLS 1.3 only; cert-manager keeps certs well ahead of expiry.

## Verification & Rollback

### Rollback
GitOps makes rollbacks surgical:
1. `git revert <commit-sha>`
2. `git push origin main`
3. ArgoCD reconciles the previous known-good state automatically. (Or use ArgoCD's
   History/Rollback UI to pin an earlier synced revision.)

### Automated checks
- `npm run test:e2e` (pre-deployment, in CI)
- K8s readiness/liveness probes on `/healthz` (during rollout)
- ArgoCD health status + Uptime Kuma (post-deployment)

## Open Items Before Production Ingress

- Confirm whether Pangolin/Newt terminates, passes through, or proxies TLS to Traefik, and
  whether it preserves `Host` headers for the BlueCap hostnames.
- Confirm cert-manager's Cloudflare DNS-01 token can issue for `bluecapstrategies.com`.
- Decide canonical host (apex vs `www`) and where the apex→canonical redirect lives (Cloudflare,
  Pangolin/Newt, Traefik, or nginx).