bcs-website/docs/launch-checklist.md

Launch Checklist — DNS Cutover to the Astro Rebuild

Single source of truth for everything standing between today and pointing bluecapstrategies.com at the homelab k3s deployment.

Verified against the working tree, the cluster, and ArgoCD on 2026-09-18. Supersedes the task lists that were scattered across planning/parity-checklist.md (git-ignored, content-only) and docs/seo-aeo-optimization.md §2/§7 (SEO-only, stale) — those now point here.

Status legend: [ ] open · [x] done · [?] needs verification · 🛑 owner decision

---

0. As-built state (2026-09-18)

Thing	State
ArgoCD app bcs-website	Synced / Healthy
Pods	2/2 Running, ns bluecap-strategies
Deployed image	git.7tl-homelab.com/bmr_bluecap/bcs-website:fc64909f3df1 (built 2026-07-20)
k8s/kustomization.yaml newTag	fc64909f3df1 — same, so ArgoCD is correct; CI never bumped it
HEAD	9f1cbca (2026-08-03), pushed to origin/main, never deployed
Ingress	bluecap-website (traefik), hosts apex + www, port 80 only, no TLS
npm run build	green, 21 pages, 0 errors / 0 warnings
npm run typecheck	green, 28 files, 0/0/0
npm run seo:lint	green, 21 pages clean
npm run test:e2e	44 passed / 4 failed (the 4 are stale assertions, see §2.1)
CI root cause	found + fixed in-repo 2026-09-22 — see §1.2; needs a green run to confirm
Staging URL	https://bcs.int.7tl-homelab.com — LAN-only, valid cert (§2.6)
Old WordPress site	untouched and serving — must stay that way until cutover
Gitea host	KVM VM gitea-1 (10.66.15.22), Debian 12, docker compose — not in k3s
Gitea / act_runner	1.27.3 / 3.4.2 — both healthy
CI job image	node:22-bookworm (Debian 12) as of 2026-09-22 — was EOL bullseye; see §1.2

The headline: the public-facing deployment is ~7 weeks behind main. The Aug 3 exact-copy work is not live.

---

1. Cutover blockers

Nothing below is optional. Cutover is unsafe until all of §1 is closed.

1.1 TLS is not configured 🛑 hard blocker

• Uncomment the tls: block and cert-manager.io/cluster-issuer annotation in k8s/ingress.yaml; set secretName: bluecap-website-tls.
• Use the existing letsencrypt-prod ClusterIssuer (present in-cluster, 472d old, Ready).
• Ordering problem SOLVED. letsencrypt-prod solves via DNS-01 against Cloudflare, not HTTP-01, so a cert can be issued before any DNS points at the cluster — no insecure window. Proven end to end on 2026-09-23: the staging host issued a valid LE cert in ~75s with no public reachability (§2.6).
• 🛑 Verify the one remaining dependency: bluecapstrategies.com must be in the Cloudflare account that cert-manager's API token can reach. The domain is at Bluehost today, so this is probably not yet true. It is the single fact separating a clean zero-downtime cutover from one with a gap — check it early, not on launch day.
• Verify https:// serves a valid chain and HTTP→HTTPS redirects before announcing.

1.2 CI/CD is stalled — no deploy since 2026-07-20 🛑 hard blocker

ROOT CAUSE FOUND AND FIXED IN-REPO 2026-09-22. Not a runner problem — the runner was online and the run did execute.

Evidence (Gitea 1.27.3 on gitea-1, sqlite action_run for repo_id 3):

• Run #6, commit 9f1cbca1b148, created 2026-08-03 13:46:53, status 2 = failure, ran 13m (13:47:02 → 14:00:11) on runner 59.
• No 9f1cbca1b148 entry in package_version → it died at or before the image push.
• Job logs are past retention (/data/gitea/actions_log no longer holds run 205), so the cause was reproduced directly instead.

The cause: the Ensure docker + kustomize are available step ran apt-get install -y docker.io. The job container image is node:18-bullseye (Debian 11), and bullseye went EOL — debian-security no longer serves those package versions. apt-get update still succeeds, so this fails only at install time, with 404s on docker.io, containerd, systemd, libcap2. Reproduced verbatim in a node:18-bullseye container on the runner host. That timing matches exactly: last green run 2026-07-20, first failure 2026-08-03.

The fix (applied to .gitea/workflows/ci.yml): fetch the docker CLI and kustomize as pinned static binaries (download.docker.com / GitHub releases) instead of distro packages. Verified in the real job environment: the static client installs, reports server=28.1.1 through the mounted socket, and docker ps succeeds. kustomize tarball and git are fine.

Everything else on the runner path checked out and needs no change:

• act_runner container up, v3.4.2, registered and polling (the daily 07:30 "connection refused" entries are the Gitea restart window, harmless).
• Runner scope already fixed — runner id 59 is owner_id=0, repo_id=0 (instance-scoped), so the old user-scoped problem is resolved. 58 stale duplicate runner rows remain; cosmetic, prune with the role's gitea_runner_prune tag when convenient.
• Runner host dockerd trusts 10.66.15.22:3000 as insecure — confirmed in docker info.
• config.yaml pins jobs to gitea_network and passes the docker socket — correct.
• CI_TOKEN repo secret present (repo_id 3).
• paths-ignore did not swallow the push — the commit touched src/**.
• docker build of the current tree succeeds locally, so the Dockerfile and lockfile are fine.

Remaining:

• Re-trigger CI and confirm a green run — the fix is verified at the step level but the full pipeline has not run end-to-end since July. Requires a push touching a non-ignored path.
• Confirm the tag-bump commit lands and ArgoCD syncs the new image.
• Do not cut DNS onto a stale image — confirm the deployed tag matches HEAD.
• CI job image upgraded 2026-09-22. gitea_runner_labels in ~/Homelab/ansible/roles/gitea/defaults/main.yml changed node:18-bullseye → node:22-bookworm; playbook run succeeded (ok=40 changed=8 failed=0). Runner 60 registered and online, owner_id=0 (instance-scoped), label confirmed in /data/.runner. The job image now matches the host VM and k3s nodes (Debian 12) and the Node 22 that workflows install anyway. - Gotcha for next time: --tags gitea_runner_prune restricts the run to only that tag, so it cannot be combined with -e gitea_runner_force_reregister=true. Two separate runs. - Gotcha 2: gitea_admin had must_change_password=1 (set during the 1.27.3 container recreate on 2026-09-14), which made Gitea reject all admin API calls with 403 — not 401, so credentials were fine. Cleared 2026-09-22 with docker exec -u git gitea gitea admin user must-change-password --unset gitea_admin (password unchanged; vault still valid). The admin API works again.
• Optional cleanup: prune the stale duplicate runner rows (59 of them now). Unblocked by the admin-API fix above. ansible-playbook -i inventory/hosts.ini gitea-deploy.yml --tags gitea_runner_prune. Purely cosmetic — deletes only runners the API reports offline.

Note on the two fixes: CI now has two independent reasons to work — the workflow no longer uses apt at all, and the job image is no longer EOL (bookworm still serves docker.io fine). The static-binary approach remains the more durable of the two; keep it regardless.

1.3 Developer-facing copy on the contact page ✅ FIXED 2026-09-23

Two notes written for developers, not visitors, were being rendered:

• the form note explaining PUBLIC_CONTACT_FORM_ENDPOINT, and
• a "Discovery note" about the WordPress page holding placeholder contact values.

Replaced the first with reader-facing copy and dropped the second (the deviation it described is already recorded in docs/content-deviations.md). Commit 4e354d2.

1.4 Contact form endpoint — plumbing DONE 2026-09-23, needs your Formspree ID 🛑

Correction to the original entry: it said to set PUBLIC_CONTACT_FORM_ENDPOINT in k8s/configmap.yaml. That would never have worked.

• The Deployment has no env/envFrom referencing that ConfigMap — it is dead config. (PUBLIC_SITE_URL is in the same state; it only appears to work because the default baked into astro.config.mjs is already correct.)
• Even wired up it could not work: output: "static" means Astro inlines PUBLIC_* at image build time, and the pod only runs nginx over pre-built HTML. There is no runtime knob.

Done (commit af4cc4a):

• Dockerfile takes ARG PUBLIC_CONTACT_FORM_ENDPOINT / PUBLIC_SITE_URL → ENV before build.
• CI passes --build-arg from a CONTACT_FORM_ENDPOINT Actions secret.
• ConfigMap rewritten as documentation, with a warning not to add envFrom.
• Submit posts in the background via fetch() and redirects to our own /thank-you/ instead of navigating to the provider's page.
• Failure path: stays on the page, re-enables the button, shows the phone/email fallback.
• Honeypot _gotcha (Formspree treats it as spam), positioned off-screen rather than display:none so bots that skip hidden fields still fill it.
• e2e covers the real fetch path (mocked endpoint, asserts the payload) and the failure case.

Still needs you:

• 🛑 Create the Formspree form for info@bluecapstrategies.com and confirm the address.
• 🛑 Add the endpoint as a Gitea repo Actions secret named CONTACT_FORM_ENDPOINT (Settings → Actions → Secrets on bmr_bluecap/bcs-website), value https://formspree.io/f/<your-id>.
• Push any commit touching src/** so CI rebuilds with the secret baked in.
• Verify on the staging URL: submit a real inquiry, confirm it arrives, confirm the browser lands on /thank-you/.
• Confirm the destination inbox is actually monitored. A working form delivering into an unwatched inbox is its own kind of failure.

Degradation if the secret is missing: the build still succeeds and the form falls back to opening a mailto: draft. Survivable, but it fails silently for webmail users with no OS mail client — so treat a missing secret as a launch blocker, not a warning.

1.5 Draft insight was being published and indexed ✅ FIXED 2026-09-23

src/pages/insights/[slug].astro called getCollection("insights") unfiltered, so the draft: true placeholder built to /insights/example-draft/ and appeared in sitemap-0.xml. The schema declared the flag and rss.xml.ts already honoured it — only the page route ignored it. Filtered in getStaticPaths, matching the RSS pattern. Build is 21 → 20 pages and the URL is gone from the sitemap. Commit 4e354d2.

• 🛑 Still open (owner): whether /insights/ should ship at all with zero real articles. It is currently in the primary nav. See also §2.5.

1.6 Canonical host is undecided 🛑 owner

The ingress answers on both apex and www; astro.config.mjs, robots.txt, and the ConfigMap all hardcode https://www.bluecapstrategies.com. Serving identical content on two hosts with no redirect splits SEO signal from day one.

• Owner picks apex or www.
• Configure a 301 from the loser to the winner (traefik middleware or edge).
• Reconcile PUBLIC_SITE_URL, canonical tags, robots.txt Sitemap line, and the ingress.

---

2. Should-fix before cutover

2.5 /insights/ index still renders developer-facing copy

Found while fixing §1.5. src/pages/insights/index.astro shows visitors two cards written for a developer, not a reader:

• "Publishing path" — "Future posts can be added as Markdown or MDX in src/content/insights with typed frontmatter for title, description, publish date…"
• "RSS support" — "The feed is ready for future article imports even though no public posts are published yet."

The hero also announces the section as a "Future publishing home", i.e. it tells visitors the section is empty. Same defect class as §1.3, but left alone deliberately: what the page should say depends on the owner's call in §1.5 on whether /insights/ ships at all.

• If /insights/ ships: delete both note-cards and rewrite the hero as reader-facing copy.
• If it does not: drop it from the primary nav and exclude the route from the sitemap.

2.6 Staging URL ✅ LIVE 2026-09-23

https://bcs.int.7tl-homelab.com — review the rebuild in a browser, valid cert, no warning.

• LAN-only: *.int.7tl-homelab.com resolves to the internal traefik LB (10.66.15.30). Not reachable from the internet; the real domain is untouched and WordPress still serves it.
• Cert issued by Let's Encrypt via cert-manager DNS-01 in ~75s, CN=bcs.int.7tl-homelab.com, valid to 2026-12-21, auto-renewing.
• All 9 public routes verified 200.
• Commit 171c8c9. Note it also corrected the issuer name in k8s/ingress.yaml: the commented placeholder said letsencrypt-production, but the ClusterIssuer is letsencrypt-prod — uncommenting it as written at cutover would have silently failed to issue.
• ArgoCD may take a few minutes to poll a push; force it with kubectl annotate application bcs-website -n argocd argocd.argoproj.io/refresh=hard --overwrite.

2.1 e2e suite is red — stale assertions, not a regression

4 of 46 tests fail. Both failures assert invented copy that the exact-copy work correctly removed:

• tests/site.spec.ts:67 waits for a link named Explore services; the homepage CTA is now All services (matches live WP).
• tests/site.spec.ts:101 waits for a heading matching /Focused expertise/, which no longer exists on /focus-areas/.

Navigation itself works. The tests are wrong, not the site.

• Update the two tests to assert current copy.
• Get back to 46/46 so the suite is trustworthy at cutover.

2.2 Repo hygiene

Commit 9f1cbca committed ~50 .playwright-mcp/ screenshots and YAML snapshots plus a 2.8 MB home-full.png at the repo root.

• Add .playwright-mcp/ to .gitignore; remove the tracked artifacts and home-full.png.

2.3 Uptime monitoring is not deployed

Per ADR-0004 the plan is to reuse the homelab monylog stack (not Uptime Kuma). Pod metrics and nginx logs already flow via Alloy; the external-availability piece was never added.

• [?] Verify current state on monylog-1 — the items below are from notes, not re-verified today.
• Add blackbox_exporter + web-bluecap.yml alert rules; re-run the monylog ansible role to reload Prometheus.
• Confirm an external HTTPS check fires to Pushover on a simulated outage. Do this before cutover — the first hours on new infrastructure are when you most need paging.

2.4 Analytics is a no-op

src/components/Analytics.astro is wired into BaseLayout but gated on PUBLIC_ env vars that are absent from k8s/configmap.yaml, so nothing is collected.

• Decide whether to launch with analytics or add it after (ADR-0005: PostHog + Umami trial).
• PostHog requires a privacy/consent notice before production use — that gates it, not the code.
• Set the env vars and confirm pageviews plus a consultation/CTA event.

---

3. Owner decisions (Brad)

These are yours, not mine. Several block content that is otherwise ready.

• 🛑 Canonical host — apex vs www (also §1.6).
• 🛑 Contact details — confirm 804-332-6618 / info@bluecapstrategies.com are the production values. (The live WP contact page carries placeholders 123-456-8912 / info@gmail.com, deliberately not reproduced; see docs/content-deviations.md.)
• 🛑 AI training-bot policy — GPTBot and ClaudeBot are currently Allow: / in public/robots.txt, with a ready-to-uncomment Disallow: /. Search and answer crawlers stay allowed either way. Decide before the site is publicly crawlable.
• 🛑 Brad Rodgers bio + credentials + portrait — gates Person schema, the About buildout, and principal-led E-E-A-T/GEO positioning. Core asset, still missing.
• 🛑 Real insights articles — no fabricated or unreviewed AI content. Until one exists, /insights/ is an empty section in the nav (see §1.5).
• 🛑 Social profiles — the live site only has per-page share links and empty # hrefs. Provide real profiles or omit the footer block.
• 🛑 /test/ page — the live site has a staging duplicate of the homepage. Default: omit.
• 🛑 Proof modules / representative engagements — no fabricated client work or dollar figures. Provide real ones or ship without.

---

4. Post-cutover (not blocking)

• Google Search Console + Bing Webmaster verified; sitemap-index.xml submitted.
• Per-page 1200×630 OG images. Every page currently shares one fallback; BaseLayout.astro:49 emits a single og:image.
• robots meta wired to a noindex frontmatter flag. /thank-you/ is excluded from the sitemap but emits no noindex meta tag.
• article:published_time / article:modified_time OG tags on insights posts (absent today).
• lastmod in the sitemap from content dates.
• Lighthouse ≥ 95 on SEO / Perf / Best-Practices / Accessibility; watch CLS from unsized images.
• Trailing-slash audit — trailingSlash: "always"; confirm canonicals and internal links agree.
• Decommission plan for the WordPress/Bluehost instance (keep it live and reachable through cutover + a rollback window; see the "keep old site live" constraint).

---

5. Already done — do not redo

Recorded because the older checklists still list several of these as open.

• Full copy parity: home, about, services, focus areas, all 8 service detail pages, all 4 focus-area detail pages — verbatim live copy in Markdown content collections (ADR-0002).
• Real imagery imported to public/assets/live-site/ (+ /icons); placeholders deleted.
• Zod seoSchema in src/content.config.ts — build fails on title/description/OG violations.
• scripts/seo-lint.mjs + npm run seo:lint — single-h1, heading order, canonical, alt text, JSON-LD parse. Green on 21 pages.
• BreadcrumbList schema (src/utils/schema.ts) applied across detail pages — the SEO doc still lists this as to-do; it is done.
• Per-agent public/robots.txt; @astrojs/sitemap excluding /thank-you/.
• Deviations logged in docs/content-deviations.md.
• GitOps: Gitea Actions CI, Gitea Packages registry, ArgoCD Application, External-Secrets registry pull auth over public HTTPS (ADR-0007, ADR-0008). Proven end-to-end in July — currently stalled, see §1.2.
• Monitoring approach decided: reuse monylog, not Uptime Kuma (ADR-0004). The SEO doc's "Uptime Kuma" line is superseded.