k8s: pre-issue the production TLS cert for the internal hop

Public traffic terminates TLS at the Pangolin VPS, which then proxies to the
control-plane nodes over https:443 — the pattern every other k3s-backed
Pangolin resource follows — so the cluster needs its own valid certificate for
the production hostnames, not just the staging one.

Add www.bluecapstrategies.com and bluecapstrategies.com to the tls block as
bluecap-website-tls. Safe to do before cutover, and deliberately done now:
letsencrypt-prod solves DNS-01 through Cloudflare, so issuance only needs a
temporary TXT record in the zone. No A record has to point at the cluster and
the live WordPress site is untouched. The Cloudflare token was extended to
cover the zone on 2026-10-09.

Both hostnames are listed because the canonical-host decision is still open;
holding a cert for each keeps either choice available, and the redirect is
configured separately.

Pangolin's edge certificate is HTTP-01 and cannot be pre-issued this way; it
is obtained on the first HTTPS request after DNS moves.

docs/launch-checklist.md

@@ -27,6 +27,9 @@ Status legend: `[ ]` open · `[x]` done · `[?]` needs verification · 🛑 owne
 | `npm run test:e2e` | **44 passed / 4 failed** (the 4 are stale assertions, see §2.1) |
 | CI root cause | **found + fixed in-repo 2026-09-22** — see §1.2; needs a green run to confirm |
 | Staging URL | **https://bcs.int.7tl-homelab.com** — LAN-only, valid cert (§2.6) |
+| Public traffic path | Cloudflare → Pangolin VPS 23.94.111.133 → newt tunnel → k3s :443 (§1.1) |
+| Pangolin | org `rhh-homelab`, site `7tl_lab` (newt, online), 23 resources, Pangolin 1.22.2 |
+| Contact form | **working** — Formspree, verified end to end 2026-10-09 (§1.4) |
 | Old WordPress site | untouched and serving — must stay that way until cutover |
 | Gitea host | KVM VM `gitea-1` (10.66.15.22), **Debian 12**, docker compose — *not* in k3s |
 | Gitea / act_runner | 1.27.3 / 3.4.2 — both healthy |
@@ -41,19 +44,90 @@ work is not live.
 
 Nothing below is optional. Cutover is unsafe until all of §1 is closed.
 
-### 1.1 TLS is not configured 🛑 hard blocker
+### 1.1 Cutover runbook — traffic path and TLS 🛑 hard blocker
-- [ ] Uncomment the `tls:` block and `cert-manager.io/cluster-issuer` annotation in
+Verified against the live Cloudflare zone, the Pangolin VPS and the cluster on **2026-10-09**.
-      `k8s/ingress.yaml`; set `secretName: bluecap-website-tls`.
+
-- [ ] Use the existing `letsencrypt-prod` ClusterIssuer (present in-cluster, 472d old, Ready).
+#### The actual traffic path
-- [x] **Ordering problem SOLVED.** `letsencrypt-prod` solves via **DNS-01 against Cloudflare**,
+Public traffic cannot reach k3s directly; it has to transit the Pangolin VPS.
-      not HTTP-01, so a cert can be issued *before* any DNS points at the cluster — no insecure
+
-      window. Proven end to end on 2026-09-23: the staging host issued a valid LE cert in ~75s
+```
-      with no public reachability (§2.6).
+visitor → Cloudflare DNS (bluecapstrategies.com)
-- [ ] 🛑 **Verify the one remaining dependency:** `bluecapstrategies.com` must be in the
+        → Pangolin VPS 23.94.111.133  (Traefik v3.7.13 + CrowdSec, terminates TLS)
-      Cloudflare account that cert-manager's API token can reach. The domain is at Bluehost
+        → gerbil/newt tunnel  → site `7tl_lab` (org `rhh-homelab`)
-      today, so this is probably **not** yet true. It is the single fact separating a clean
+        → k3s traefik 10.66.15.30/.31/.32 :443  (second TLS hop)
-      zero-downtime cutover from one with a gap — check it early, not on launch day.
+        → bluecap-website service
-- [ ] Verify `https://` serves a valid chain and HTTP→HTTPS redirects before announcing.
+```
+
+**Two certificates are needed, from two different systems:**
+
+| Hop | Issued by | Challenge | Can pre-issue? |
+|-----|-----------|-----------|----------------|
+| edge (Pangolin Traefik) | its own `letsencrypt` resolver | **HTTP-01** on entrypoint `web` | **No** |
+| internal (k3s ingress) | cert-manager `letsencrypt-prod` | **DNS-01** via Cloudflare | **Yes** |
+
+#### Corrections to earlier assumptions in this doc
+- There is **one** Cloudflare token, not two. Pangolin's Traefik has **no** Cloudflare
+  credentials — it uses HTTP-01. Nothing to configure on the VPS for DNS.
+- The "ordering problem SOLVED" claim was **half right**. cert-manager does use DNS-01 and can
+  pre-issue (proven on the staging host, §2.6). But **Pangolin cannot** — its
+  `certificatesResolvers.letsencrypt.acme.httpChallenge.entryPoint: web` means the edge cert is
+  only obtainable once DNS already points at the VPS. A short window at cutover is unavoidable
+  unless the edge resolver is converted to DNS-01, which would change renewal behaviour for all
+  23 existing resources and is not worth it to save seconds.
+- Earlier advice said "use DNS-only, matching your existing pattern." The BlueCap zone is in fact
+  **already fully proxied** (orange cloud) while the homelab zones are DNS-only. It is a real
+  decision — see below.
+
+#### Prerequisites (all invisible to the public; the old site keeps serving)
+- [x] Cloudflare token now covers the zone — `7tl-homelab.com`, **`bluecapstrategies.com`**,
+      `rodgersweb.com` (done 2026-10-09). Token is DNS-edit-only, which is correct least privilege.
+- [ ] **k3s ingress:** add `www.bluecapstrategies.com` + `bluecapstrategies.com` to the `tls:`
+      block in `k8s/ingress.yaml` (issuer annotation is already present and correct —
+      `letsencrypt-prod`). cert-manager pre-issues via DNS-01, so do this **before** cutover and
+      confirm the Certificate goes Ready. Needed because Pangolin proxies to **https:443**.
+- [ ] **Pangolin:** add `bluecapstrategies.com` as a domain under org **`rhh-homelab`**
+      (`orgDomains`), `certResolver=letsencrypt`.
+- [ ] **Pangolin:** create the resource(s) with **three targets**, matching the established
+      pattern for every k3s-backed resource (Authentik, paperless, immich, dav all do this):
+      `10.66.15.30`, `10.66.15.31`, `10.66.15.32` — method **https**, port **443**.
+      Set the host header so k3s traefik routes on the real hostname.
+- [ ] 🛑 **Check Cloudflare's SSL/TLS mode in the dashboard** (the API token cannot read zone
+      settings). If it is **Flexible**, Cloudflare talks HTTP to Pangolin, Pangolin redirects to
+      HTTPS, and you get a redirect loop. Must be **Full (strict)** if the record stays proxied.
+- [ ] **Test before any DNS change:** `curl --resolve bluecapstrategies.com:443:23.94.111.133`
+      and confirm the full path serves the site.
+
+#### Proxied vs DNS-only — decide before cutover 🛑
+The zone is currently **proxied**: apex `A → 50.87.136.52 PROXIED`, `www CNAME → apex PROXIED`.
+
+- **DNS-only (recommended for cutover):** the exact path all 23 working resources use. CrowdSec
+  sees real client IPs as currently configured; HTTP-01 goes direct. Cost: no Cloudflare CDN/DDoS,
+  and the VPS IP becomes visible.
+- **Stay proxied:** keeps CDN/DDoS and hides the origin, but CrowdSec will see Cloudflare IPs
+  unless it is configured to trust `CF-Connecting-IP` — bans would land on the wrong addresses or
+  on Cloudflare itself. Also requires Full (strict) per above.
+
+Revisit proxying after launch as a deliberate change with the CrowdSec real-IP config done
+properly — not as part of cutover.
+
+#### Cutover
+- [ ] Point apex + `www` at **23.94.111.133** in Cloudflare (replacing `50.87.136.52`).
+- [ ] **Immediately** `curl https://bluecapstrategies.com` yourself to trigger Pangolin's HTTP-01
+      issuance, before a real visitor hits an unissued cert.
+- [ ] Verify a valid chain on both hostnames and that HTTP→HTTPS redirects.
+- [ ] Apply the canonical-host decision (§1.6): redirect the non-canonical name.
+
+**Rollback:** set the A record back to `50.87.136.52`. All records are `ttl=1` (Cloudflare "Auto",
+~300s for DNS-only), so propagation and rollback are both ~5 minutes — not hours.
+
+#### Mail is not affected
+MX (`eagle.mxlogin.com` / `eagle-relay.mxlogin.com`) and `mail.bluecapstrategies.com` are **not
+proxied** and are untouched by the cutover. MXroute keeps serving mail throughout. See
+`mxroute-mail-setup` notes.
+
+- [ ] Minor cleanup (post-launch): `autoconfig` and `autodiscover` still point at Bluehost
+      (`50.87.136.52`) while mail is on MXroute — stale records that can hand mail clients the
+      wrong settings.
 
 ### 1.2 CI/CD is stalled — no deploy since 2026-07-20 🛑 hard blocker
 **ROOT CAUSE FOUND AND FIXED IN-REPO 2026-09-22.** Not a runner problem — the runner was online

k8s/ingress.yaml

@@ -25,10 +25,26 @@ metadata:
     cert-manager.io/cluster-issuer: letsencrypt-prod
 spec:
   tls:
-    # Staging only, for now. Production hosts join this list at cutover (see header).
     - hosts:
         - bcs.int.7tl-homelab.com
       secretName: bcs-int-tls
+    # Production hostnames. This is the INTERNAL hop only: public traffic terminates TLS at the
+    # Pangolin VPS, which then proxies to these nodes over https:443 (the pattern every other
+    # k3s-backed Pangolin resource uses), so the cluster needs its own valid cert for these names.
+    #
+    # Safe to issue before cutover and deliberately done so: letsencrypt-prod solves DNS-01 via
+    # Cloudflare, which only needs a temporary TXT record in the zone — no A record has to point
+    # here, and the live WordPress site is unaffected. Pangolin's own edge cert is HTTP-01 and
+    # CANNOT be pre-issued; it is obtained on the first request after DNS moves. See
+    # docs/launch-checklist.md section 1.1.
+    #
+    # Both names are listed because the canonical-host decision (section 1.6) is still open;
+    # holding a cert for each keeps either choice available. The redirect is configured
+    # separately and does not depend on this.
+    - hosts:
+        - www.bluecapstrategies.com
+        - bluecapstrategies.com
+      secretName: bluecap-website-tls
   rules:
     - host: bcs.int.7tl-homelab.com
       http: