# Launch Checklist — DNS Cutover to the Astro Rebuild **Single source of truth** for everything standing between today and pointing `bluecapstrategies.com` at the homelab k3s deployment. Verified against the working tree, the cluster, and ArgoCD on **2026-09-18**. Supersedes the task lists that were scattered across `planning/parity-checklist.md` (git-ignored, content-only) and `docs/seo-aeo-optimization.md` §2/§7 (SEO-only, stale) — those now point here. Status legend: `[ ]` open · `[x]` done · `[?]` needs verification · 🛑 owner decision --- ## 0. As-built state (2026-09-18) | Thing | State | |-------|-------| | ArgoCD app `bcs-website` | Synced / Healthy | | Pods | 2/2 Running, ns `bluecap-strategies` | | Deployed image | `git.7tl-homelab.com/bmr_bluecap/bcs-website:fc64909f3df1` (built **2026-07-20**) | | `k8s/kustomization.yaml` newTag | `fc64909f3df1` — same, so ArgoCD is correct; CI never bumped it | | HEAD | `9f1cbca` (2026-08-03), pushed to `origin/main`, **never deployed** | | Ingress | `bluecap-website` (traefik), hosts apex + `www`, **port 80 only, no TLS** | | `npm run build` | green, 21 pages, 0 errors / 0 warnings | | `npm run typecheck` | green, 28 files, 0/0/0 | | `npm run seo:lint` | green, 21 pages clean | | `npm run test:e2e` | **44 passed / 4 failed** (the 4 are stale assertions, see §2.1) | | CI root cause | **found + fixed in-repo 2026-09-22** — see §1.2; needs a green run to confirm | | Staging URL | **https://bcs.int.7tl-homelab.com** — LAN-only, valid cert (§2.6) | | Public traffic path | Cloudflare → Pangolin VPS 23.94.111.133 → newt tunnel → k3s :443 (§1.1) | | Pangolin | org `rhh-homelab`, site `7tl_lab` (newt, online), 23 resources, Pangolin 1.22.2 | | Contact form | **working** — Formspree, verified end to end 2026-10-09 (§1.4) | | Old WordPress site | untouched and serving — must stay that way until cutover | | Gitea host | KVM VM `gitea-1` (10.66.15.22), **Debian 12**, docker compose — *not* in k3s | | Gitea / act_runner | 1.27.3 / 3.4.2 — both healthy | | CI job image | **`node:22-bookworm`** (Debian 12) as of 2026-09-22 — was EOL bullseye; see §1.2 | **The headline:** the public-facing deployment is ~7 weeks behind `main`. The Aug 3 exact-copy work is not live. --- ## 1. Cutover blockers Nothing below is optional. Cutover is unsafe until all of §1 is closed. ### 1.1 Cutover runbook — traffic path and TLS 🛑 hard blocker Verified against the live Cloudflare zone, the Pangolin VPS and the cluster on **2026-10-09**. #### The actual traffic path Public traffic cannot reach k3s directly; it has to transit the Pangolin VPS. ``` visitor → Cloudflare DNS (bluecapstrategies.com) → Pangolin VPS 23.94.111.133 (Traefik v3.7.13 + CrowdSec, terminates TLS) → gerbil/newt tunnel → site `7tl_lab` (org `rhh-homelab`) → k3s traefik 10.66.15.30/.31/.32 :443 (second TLS hop) → bluecap-website service ``` **Two certificates are needed, from two different systems:** | Hop | Issued by | Challenge | Can pre-issue? | |-----|-----------|-----------|----------------| | edge (Pangolin Traefik) | its own `letsencrypt` resolver | **HTTP-01** on entrypoint `web` | **No** | | internal (k3s ingress) | cert-manager `letsencrypt-prod` | **DNS-01** via Cloudflare | **Yes** | #### Corrections to earlier assumptions in this doc - There is **one** Cloudflare token, not two. Pangolin's Traefik has **no** Cloudflare credentials — it uses HTTP-01. Nothing to configure on the VPS for DNS. - The "ordering problem SOLVED" claim was **half right**. cert-manager does use DNS-01 and can pre-issue (proven on the staging host, §2.6). But **Pangolin cannot** — its `certificatesResolvers.letsencrypt.acme.httpChallenge.entryPoint: web` means the edge cert is only obtainable once DNS already points at the VPS. A short window at cutover is unavoidable unless the edge resolver is converted to DNS-01, which would change renewal behaviour for all 23 existing resources and is not worth it to save seconds. - Earlier advice said "use DNS-only, matching your existing pattern." The BlueCap zone is in fact **already fully proxied** (orange cloud) while the homelab zones are DNS-only. It is a real decision — see below. #### Prerequisites (all invisible to the public; the old site keeps serving) - [x] Cloudflare token now covers the zone — `7tl-homelab.com`, **`bluecapstrategies.com`**, `rodgersweb.com` (done 2026-10-09). Token is DNS-edit-only, which is correct least privilege. - [ ] **k3s ingress:** add `www.bluecapstrategies.com` + `bluecapstrategies.com` to the `tls:` block in `k8s/ingress.yaml` (issuer annotation is already present and correct — `letsencrypt-prod`). cert-manager pre-issues via DNS-01, so do this **before** cutover and confirm the Certificate goes Ready. Needed because Pangolin proxies to **https:443**. - [ ] **Pangolin:** add `bluecapstrategies.com` as a domain under org **`rhh-homelab`** (`orgDomains`), `certResolver=letsencrypt`. - [ ] **Pangolin:** create the resource(s) with **three targets**, matching the established pattern for every k3s-backed resource (Authentik, paperless, immich, dav all do this): `10.66.15.30`, `10.66.15.31`, `10.66.15.32` — method **https**, port **443**. Set the host header so k3s traefik routes on the real hostname. - [ ] 🛑 **Check Cloudflare's SSL/TLS mode in the dashboard** (the API token cannot read zone settings). If it is **Flexible**, Cloudflare talks HTTP to Pangolin, Pangolin redirects to HTTPS, and you get a redirect loop. Must be **Full (strict)** if the record stays proxied. - [ ] **Test before any DNS change:** `curl --resolve bluecapstrategies.com:443:23.94.111.133` and confirm the full path serves the site. #### Proxied vs DNS-only — decide before cutover 🛑 The zone is currently **proxied**: apex `A → 50.87.136.52 PROXIED`, `www CNAME → apex PROXIED`. - **DNS-only (recommended for cutover):** the exact path all 23 working resources use. CrowdSec sees real client IPs as currently configured; HTTP-01 goes direct. Cost: no Cloudflare CDN/DDoS, and the VPS IP becomes visible. - **Stay proxied:** keeps CDN/DDoS and hides the origin, but CrowdSec will see Cloudflare IPs unless it is configured to trust `CF-Connecting-IP` — bans would land on the wrong addresses or on Cloudflare itself. Also requires Full (strict) per above. Revisit proxying after launch as a deliberate change with the CrowdSec real-IP config done properly — not as part of cutover. #### Cutover - [ ] Point apex + `www` at **23.94.111.133** in Cloudflare (replacing `50.87.136.52`). - [ ] **Immediately** `curl https://bluecapstrategies.com` yourself to trigger Pangolin's HTTP-01 issuance, before a real visitor hits an unissued cert. - [ ] Verify a valid chain on both hostnames and that HTTP→HTTPS redirects. - [ ] Apply the canonical-host decision (§1.6): redirect the non-canonical name. **Rollback:** set the A record back to `50.87.136.52`. All records are `ttl=1` (Cloudflare "Auto", ~300s for DNS-only), so propagation and rollback are both ~5 minutes — not hours. #### Mail is not affected MX (`eagle.mxlogin.com` / `eagle-relay.mxlogin.com`) and `mail.bluecapstrategies.com` are **not proxied** and are untouched by the cutover. MXroute keeps serving mail throughout. See `mxroute-mail-setup` notes. - [ ] Minor cleanup (post-launch): `autoconfig` and `autodiscover` still point at Bluehost (`50.87.136.52`) while mail is on MXroute — stale records that can hand mail clients the wrong settings. ### 1.2 CI/CD is stalled — no deploy since 2026-07-20 🛑 hard blocker **ROOT CAUSE FOUND AND FIXED IN-REPO 2026-09-22.** Not a runner problem — the runner was online and the run *did* execute. Evidence (Gitea 1.27.3 on `gitea-1`, sqlite `action_run` for repo_id 3): - Run #6, commit `9f1cbca1b148`, created 2026-08-03 13:46:53, **status 2 = failure**, ran 13m (13:47:02 → 14:00:11) on runner 59. - No `9f1cbca1b148` entry in `package_version` → it died **at or before** the image push. - Job logs are past retention (`/data/gitea/actions_log` no longer holds run 205), so the cause was reproduced directly instead. **The cause:** the `Ensure docker + kustomize are available` step ran `apt-get install -y docker.io`. The job container image is **`node:18-bullseye`** (Debian 11), and bullseye went EOL — `debian-security` no longer serves those package versions. `apt-get update` still succeeds, so this fails only at install time, with 404s on `docker.io`, `containerd`, `systemd`, `libcap2`. Reproduced verbatim in a `node:18-bullseye` container on the runner host. That timing matches exactly: last green run 2026-07-20, first failure 2026-08-03. **The fix (applied to `.gitea/workflows/ci.yml`):** fetch the docker CLI and kustomize as pinned **static binaries** (`download.docker.com` / GitHub releases) instead of distro packages. Verified in the real job environment: the static client installs, reports `server=28.1.1` through the mounted socket, and `docker ps` succeeds. kustomize tarball and `git` are fine. Everything else on the runner path checked out and needs no change: - [x] act_runner container up, `v3.4.2`, registered and polling (the daily 07:30 "connection refused" entries are the Gitea restart window, harmless). - [x] Runner scope **already fixed** — runner id 59 is `owner_id=0, repo_id=0` (instance-scoped), so the old user-scoped problem is resolved. 58 stale duplicate runner rows remain; cosmetic, prune with the role's `gitea_runner_prune` tag when convenient. - [x] Runner host dockerd trusts `10.66.15.22:3000` as insecure — confirmed in `docker info`. - [x] `config.yaml` pins jobs to `gitea_network` and passes the docker socket — correct. - [x] `CI_TOKEN` repo secret present (repo_id 3). - [x] `paths-ignore` did not swallow the push — the commit touched `src/**`. - [x] `docker build` of the current tree succeeds locally, so the Dockerfile and lockfile are fine. Remaining: - [ ] **Re-trigger CI and confirm a green run** — the fix is verified at the step level but the full pipeline has not run end-to-end since July. Requires a push touching a non-ignored path. - [ ] Confirm the tag-bump commit lands and ArgoCD syncs the new image. - [ ] Do not cut DNS onto a stale image — confirm the deployed tag matches HEAD. - [x] **CI job image upgraded 2026-09-22.** `gitea_runner_labels` in `~/Homelab/ansible/roles/gitea/defaults/main.yml` changed `node:18-bullseye` → **`node:22-bookworm`**; playbook run succeeded (ok=40 changed=8 failed=0). Runner **60** registered and online, `owner_id=0` (instance-scoped), label confirmed in `/data/.runner`. The job image now matches the host VM and k3s nodes (Debian 12) and the Node 22 that workflows install anyway. - Gotcha for next time: `--tags gitea_runner_prune` restricts the run to *only* that tag, so it cannot be combined with `-e gitea_runner_force_reregister=true`. Two separate runs. - Gotcha 2: `gitea_admin` had `must_change_password=1` (set during the 1.27.3 container recreate on 2026-09-14), which made Gitea reject **all** admin API calls with 403 — not 401, so credentials were fine. Cleared 2026-09-22 with `docker exec -u git gitea gitea admin user must-change-password --unset gitea_admin` (password unchanged; vault still valid). The admin API works again. - [ ] Optional cleanup: prune the stale duplicate runner rows (59 of them now). Unblocked by the admin-API fix above. `ansible-playbook -i inventory/hosts.ini gitea-deploy.yml --tags gitea_runner_prune`. Purely cosmetic — deletes only runners the API reports offline. **Note on the two fixes:** CI now has *two* independent reasons to work — the workflow no longer uses apt at all, and the job image is no longer EOL (bookworm still serves `docker.io` fine). The static-binary approach remains the more durable of the two; keep it regardless. ### 1.3 Developer-facing copy on the contact page ✅ FIXED 2026-09-23 Two notes written for developers, not visitors, were being rendered: - the form note explaining `PUBLIC_CONTACT_FORM_ENDPOINT`, and - a "Discovery note" about the WordPress page holding placeholder contact values. Replaced the first with reader-facing copy and dropped the second (the deviation it described is already recorded in `docs/content-deviations.md`). Commit `4e354d2`. ### 1.4 Contact form endpoint — plumbing DONE 2026-09-23, needs your Formspree ID 🛑 **Correction to the original entry:** it said to set `PUBLIC_CONTACT_FORM_ENDPOINT` in `k8s/configmap.yaml`. That would never have worked. - The Deployment has **no `env`/`envFrom`** referencing that ConfigMap — it is dead config. (`PUBLIC_SITE_URL` is in the same state; it only appears to work because the default baked into `astro.config.mjs` is already correct.) - Even wired up it could not work: `output: "static"` means Astro **inlines** `PUBLIC_*` at image build time, and the pod only runs nginx over pre-built HTML. There is no runtime knob. **Done (commit `af4cc4a`):** - [x] `Dockerfile` takes `ARG PUBLIC_CONTACT_FORM_ENDPOINT` / `PUBLIC_SITE_URL` → `ENV` before build. - [x] CI passes `--build-arg` from a `CONTACT_FORM_ENDPOINT` Actions secret. - [x] ConfigMap rewritten as documentation, with a warning not to add `envFrom`. - [x] Submit posts in the background via `fetch()` and redirects to our own `/thank-you/` instead of navigating to the provider's page. - [x] Failure path: stays on the page, re-enables the button, shows the phone/email fallback. - [x] Honeypot `_gotcha` (Formspree treats it as spam), positioned off-screen rather than `display:none` so bots that skip hidden fields still fill it. - [x] e2e covers the real fetch path (mocked endpoint, asserts the payload) and the failure case. **Still needs you:** - [ ] 🛑 Create the Formspree form for **info@bluecapstrategies.com** and confirm the address. - [ ] 🛑 Add the endpoint as a Gitea repo Actions secret named **`CONTACT_FORM_ENDPOINT`** (Settings → Actions → Secrets on `bmr_bluecap/bcs-website`), value `https://formspree.io/f/`. - [ ] Push any commit touching `src/**` so CI rebuilds with the secret baked in. - [ ] Verify on the staging URL: submit a real inquiry, confirm it arrives, confirm the browser lands on `/thank-you/`. - [ ] Confirm the destination inbox is actually monitored. A working form delivering into an unwatched inbox is its own kind of failure. **Degradation if the secret is missing:** the build still succeeds and the form falls back to opening a `mailto:` draft. Survivable, but it fails silently for webmail users with no OS mail client — so treat a missing secret as a launch blocker, not a warning. ### 1.5 Draft insight was being published and indexed ✅ FIXED 2026-09-23 `src/pages/insights/[slug].astro` called `getCollection("insights")` unfiltered, so the `draft: true` placeholder built to `/insights/example-draft/` **and** appeared in `sitemap-0.xml`. The schema declared the flag and `rss.xml.ts` already honoured it — only the page route ignored it. Filtered in `getStaticPaths`, matching the RSS pattern. Build is 21 → 20 pages and the URL is gone from the sitemap. Commit `4e354d2`. - [ ] 🛑 **Still open (owner):** whether `/insights/` should ship at all with zero real articles. It is currently in the primary nav. See also §2.5. ### 1.6 Canonical host is undecided 🛑 owner The ingress answers on **both** apex and `www`; `astro.config.mjs`, `robots.txt`, and the ConfigMap all hardcode `https://www.bluecapstrategies.com`. Serving identical content on two hosts with no redirect splits SEO signal from day one. - [ ] Owner picks apex or `www`. - [ ] Configure a 301 from the loser to the winner (traefik middleware or edge). - [ ] Reconcile `PUBLIC_SITE_URL`, canonical tags, `robots.txt` Sitemap line, and the ingress. --- ## 2. Should-fix before cutover ### 2.5 `/insights/` index still renders developer-facing copy Found while fixing §1.5. `src/pages/insights/index.astro` shows visitors two cards written for a developer, not a reader: - *"Publishing path"* — "Future posts can be added as Markdown or MDX in `src/content/insights` with typed frontmatter for title, description, publish date…" - *"RSS support"* — "The feed is ready for future article imports even though no public posts are published yet." The hero also announces the section as a "Future publishing home", i.e. it tells visitors the section is empty. Same defect class as §1.3, but left alone deliberately: what the page *should* say depends on the owner's call in §1.5 on whether `/insights/` ships at all. - [ ] If `/insights/` ships: delete both note-cards and rewrite the hero as reader-facing copy. - [ ] If it does not: drop it from the primary nav and exclude the route from the sitemap. ### 2.6 Staging URL ✅ LIVE 2026-09-23 **https://bcs.int.7tl-homelab.com** — review the rebuild in a browser, valid cert, no warning. - LAN-only: `*.int.7tl-homelab.com` resolves to the internal traefik LB (10.66.15.30). Not reachable from the internet; the real domain is untouched and WordPress still serves it. - Cert issued by Let's Encrypt via cert-manager DNS-01 in ~75s, `CN=bcs.int.7tl-homelab.com`, valid to 2026-12-21, auto-renewing. - All 9 public routes verified 200. - Commit `171c8c9`. Note it also corrected the issuer name in `k8s/ingress.yaml`: the commented placeholder said `letsencrypt-production`, but the ClusterIssuer is **`letsencrypt-prod`** — uncommenting it as written at cutover would have silently failed to issue. - ArgoCD may take a few minutes to poll a push; force it with `kubectl annotate application bcs-website -n argocd argocd.argoproj.io/refresh=hard --overwrite`. ### 2.1 e2e suite is red — stale assertions, not a regression 4 of 46 tests fail. Both failures assert **invented copy that the exact-copy work correctly removed**: - `tests/site.spec.ts:67` waits for a link named `Explore services`; the homepage CTA is now `All services` (matches live WP). - `tests/site.spec.ts:101` waits for a heading matching `/Focused expertise/`, which no longer exists on `/focus-areas/`. Navigation itself works. The tests are wrong, not the site. - [ ] Update the two tests to assert current copy. - [ ] Get back to 46/46 so the suite is trustworthy at cutover. ### 2.2 Repo hygiene Commit `9f1cbca` committed ~50 `.playwright-mcp/` screenshots and YAML snapshots plus a 2.8 MB `home-full.png` at the repo root. - [ ] Add `.playwright-mcp/` to `.gitignore`; remove the tracked artifacts and `home-full.png`. ### 2.3 Uptime monitoring is not deployed Per ADR-0004 the plan is to reuse the homelab `monylog` stack (not Uptime Kuma). Pod metrics and nginx logs already flow via Alloy; the external-availability piece was never added. - [?] Verify current state on monylog-1 — the items below are from notes, not re-verified today. - [ ] Add `blackbox_exporter` + `web-bluecap.yml` alert rules; re-run the monylog ansible role to reload Prometheus. - [ ] Confirm an external HTTPS check fires to Pushover on a simulated outage. **Do this before cutover** — the first hours on new infrastructure are when you most need paging. ### 2.4 Analytics is a no-op `src/components/Analytics.astro` is wired into `BaseLayout` but gated on `PUBLIC_` env vars that are absent from `k8s/configmap.yaml`, so nothing is collected. - [ ] Decide whether to launch with analytics or add it after (ADR-0005: PostHog + Umami trial). - [ ] **PostHog requires a privacy/consent notice before production use** — that gates it, not the code. - [ ] Set the env vars and confirm pageviews plus a consultation/CTA event. --- ## 3. Owner decisions (Brad) These are yours, not mine. Several block content that is otherwise ready. - [ ] 🛑 **Canonical host** — apex vs `www` (also §1.6). - [ ] 🛑 **Contact details** — confirm `804-332-6618` / `info@bluecapstrategies.com` are the production values. (The live WP contact page carries placeholders `123-456-8912` / `info@gmail.com`, deliberately **not** reproduced; see `docs/content-deviations.md`.) - [ ] 🛑 **AI training-bot policy** — `GPTBot` and `ClaudeBot` are currently **Allow: /** in `public/robots.txt`, with a ready-to-uncomment `Disallow: /`. Search and answer crawlers stay allowed either way. Decide before the site is publicly crawlable. - [ ] 🛑 **Brad Rodgers bio + credentials + portrait** — gates `Person` schema, the About buildout, and principal-led E-E-A-T/GEO positioning. Core asset, still missing. - [ ] 🛑 **Real insights articles** — no fabricated or unreviewed AI content. Until one exists, `/insights/` is an empty section in the nav (see §1.5). - [ ] 🛑 **Social profiles** — the live site only has per-page share links and empty `#` hrefs. Provide real profiles or omit the footer block. - [ ] 🛑 **`/test/` page** — the live site has a staging duplicate of the homepage. Default: omit. - [ ] 🛑 **Proof modules / representative engagements** — no fabricated client work or dollar figures. Provide real ones or ship without. --- ## 4. Post-cutover (not blocking) - [ ] Google Search Console + Bing Webmaster verified; `sitemap-index.xml` submitted. - [ ] Per-page 1200×630 OG images. Every page currently shares one fallback; `BaseLayout.astro:49` emits a single `og:image`. - [ ] `robots` meta wired to a `noindex` frontmatter flag. `/thank-you/` is excluded from the sitemap but emits **no** `noindex` meta tag. - [ ] `article:published_time` / `article:modified_time` OG tags on insights posts (absent today). - [ ] `lastmod` in the sitemap from content dates. - [ ] Lighthouse ≥ 95 on SEO / Perf / Best-Practices / Accessibility; watch CLS from unsized images. - [ ] Trailing-slash audit — `trailingSlash: "always"`; confirm canonicals and internal links agree. - [ ] Decommission plan for the WordPress/Bluehost instance (keep it live and reachable through cutover + a rollback window; see the "keep old site live" constraint). --- ## 5. Already done — do not redo Recorded because the older checklists still list several of these as open. - [x] Full copy parity: home, about, services, focus areas, all 8 service detail pages, all 4 focus-area detail pages — verbatim live copy in Markdown content collections (ADR-0002). - [x] Real imagery imported to `public/assets/live-site/` (+ `/icons`); placeholders deleted. - [x] Zod `seoSchema` in `src/content.config.ts` — build **fails** on title/description/OG violations. - [x] `scripts/seo-lint.mjs` + `npm run seo:lint` — single-h1, heading order, canonical, alt text, JSON-LD parse. Green on 21 pages. - [x] `BreadcrumbList` schema (`src/utils/schema.ts`) applied across detail pages — the SEO doc still lists this as to-do; it is done. - [x] Per-agent `public/robots.txt`; `@astrojs/sitemap` excluding `/thank-you/`. - [x] Deviations logged in `docs/content-deviations.md`. - [x] GitOps: Gitea Actions CI, Gitea Packages registry, ArgoCD Application, External-Secrets registry pull auth over public HTTPS (ADR-0007, ADR-0008). Proven end-to-end in July — currently stalled, see §1.2. - [x] Monitoring approach decided: reuse `monylog`, not Uptime Kuma (ADR-0004). The SEO doc's "Uptime Kuma" line is superseded.