0 Fork 0

Files

T

bmr_bluecap 002845c880 chore: untrack Playwright MCP captures and root screenshot; trim docker build context

Also records the gitea runner durable fix in the launch checklist.

2026-10-23 16:58:13 -04:00

12 KiB

Raw Blame History

Launch Checklist — Post-Cutover Status

www.bluecapstrategies.com went live on the Astro rebuild on 2026-10-23. This file is now the single source of truth for what is running, how it is wired, and what is left. The pre-launch runbook it replaced is in git history (git log -- docs/launch-checklist.md, before 2026-10-23).

Status legend: [ ] open · [x] done · [?] needs verification · 🛑 owner decision

0. As-built state (2026-10-23)

Thing	State
Public site	https://www.bluecapstrategies.com — canonical. Apex 301s to www, path preserved
Old WordPress / Bluehost	Gone. Access lost and it was returning 404 before cutover; nothing to roll back to
DNS	Cloudflare, DNS-only (grey cloud) for apex, `www`, `stats` → `23.94.111.133`
Edge	Pangolin VPS 23.94.111.133 (Traefik + CrowdSec), org `rhh-homelab`, site `7tl_lab`
Pangolin resources	`Bluecap-www`, `Bluecap-base`, `stats` — 3 https targets `10.66.15.30/.31/.32:443` each
Edge certs	Let's Encrypt HTTP-01 via Pangolin's `letsencrypt` resolver, issued 2026-10-23
Cluster certs	cert-manager `letsencrypt-prod` DNS-01: `bluecap-website-tls` (www + apex), `bcs-int-tls`
ArgoCD	`bcs-website` and `umami` — Synced / Healthy
Deployed image	tracks `main` via CI tag-bump (`ci: deploy <sha>` commits)
Staging	https://bcs.int.7tl-homelab.com — LAN only
Contact form	Formspree, verified end to end 2026-10-09
Analytics	Umami (self-hosted, `stats.bluecapstrategies.com`) + GA4 `G-Z273QL1F0N`, both verified recording 2026-10-23
Monitoring	monylog blackbox probes on www + apex → Alertmanager → Pushover; UptimeRobot (external) → Pushover
Search	Google Search Console domain property verified; sitemap submitted and fetched by Googlebot
Mail	MXroute. SPF pass, DKIM pass (selector `x`), DMARC `p=none` — mail-tester 9.5/10
`npm run build` / `typecheck` / `seo:lint`	green, 21 pages
`npm run test:e2e`	48/48 green

Traffic path

visitor → Cloudflare DNS (DNS-only) → Pangolin VPS 23.94.111.133 (TLS #1, CrowdSec)
        → gerbil/newt tunnel → k3s traefik 10.66.15.30/.31/.32 :443 (TLS #2)
        → bluecap-website (nginx, 2 replicas)        ns bluecap-strategies
        → umami (+ umami-postgres)                   ns umami

1. Open — do soon

1.1 External uptime check ✅ 2026-10-23

monylog runs inside the homelab, so a homelab power or internet outage takes the site and its monitor down together, and nothing pages. Needs a vantage point outside the homelab.

UptimeRobot (free tier, 5-min interval) probes https://www.bluecapstrategies.com/.
Alerts go to Pushover via a separate email alert contact (the @pomail.net gateway address), not the account email, so password resets etc. stay in the normal inbox.
Reading alerts: UptimeRobot and monylog both firing = site/VPS problem; UptimeRobot alone (monylog silent) usually means the homelab itself is down.

1.2 Homelab intercepts port-53 DNS

Found 2026-10-23. Queries from the homelab — including k3s nodes — to any DNS server, even Cloudflare's authoritative NS, are answered by a local recursive resolver (dig @arely.ns.cloudflare.com returns rd ra with no aa). A name looked up before its record exists gets NXDOMAIN cached for 30 minutes (the zone's SOA negative TTL).

This, not Cloudflare, explains the 4-hour cert-manager DNS-01 stall of 2026-10-09 and the stats NXDOMAIN at launch. Consequences:

Pointing cert-manager at 1.1.1.1/8.8.8.8 (the fix previously suggested here) will not help while port 53 is intercepted.
To see true public DNS, query from the VPS or via DoH: curl -H 'accept: application/dns-json' 'https://cloudflare-dns.com/dns-query?name=<name>&type=A'
Create DNS records before anything in the homelab (cert-manager included) looks the name up.
🛑 Owner decision: leave as is, or exempt the k3s nodes from the redirect. The redirect rule is not in ~/Homelab/mikrotik/*.rsc; find it on the router (or Pi-hole/AdGuard, if any).
umami-stats-tls (internal hop for stats) stalled on the stale NXDOMAIN at launch and issued by itself once the cache expired (~18:35 UTC, 2026-10-23) — the expected pattern.

1.3 Gitea runner loses Docker after a dockerd restart

dockerd on gitea-1 runs with live-restore=true. When it restarts, containers keep running but act_runner's bind mount of /var/run/docker.sock points at the dead socket, so every CI job fails in its first second with Cannot connect to the Docker daemon. Happened 2026-10-21 → discovered 2026-10-23.

Workaround: sudo docker restart act_runner on gitea-1.
Durable fix in the gitea ansible role (2026-10-23): a docker.service drop-in (/etc/systemd/system/docker.service.d/restart-act-runner.conf, template docker-restart-act-runner.conf.j2) whose ExecStartPost restarts act_runner whenever dockerd starts. Live-restore stays on, so Gitea itself keeps running through dockerd restarts.
Apply it: run gitea-deploy.yml (daemon-reload only, nothing restarts), then confirm with systemctl show docker -p DropInPaths on gitea-1. Optional proof, when no CI job is running: sudo systemctl restart docker and check docker inspect -f '{{.State.StartedAt}}' act_runner moved forward.

1.4 `/insights/` ✅ 2026-10-23

Now a reader-facing "Coming soon" page (hero + contact CTA); the developer-facing cards are gone. It stays in the nav. When the first real article lands, restore a listing on this page.

1.5 Privacy policy ✅ 2026-10-23

/privacy/ reviewed and accepted by the owner. No consent banner (owner's call). Anything added to src/components/Analytics.astro must be disclosed on /privacy/.

2. Should fix

2.1 e2e suite ✅ green 2026-10-23 — 48/48

The stale assertions (invented copy removed by the exact-copy work) now check the real headings and use the header Services link; the homepage body has no direct /services/ link.

2.2 Repo hygiene ✅ 2026-10-23

Commit 9f1cbca had committed 51 .playwright-mcp/ screenshots/snapshots plus a 2.8 MB home-full.png at the repo root. Now untracked (files kept on disk) and ignored: .gitignore has .playwright-mcp/ and /*.png; .dockerignore also drops them plus planning/ and tests/ from the build context. History was not rewritten (.git is 17 MB — not worth a force-push).

2.3 DMARC tightening

Tighten DMARC from p=none to p=quarantine after a few weeks of clean reports to info@bluecapstrategies.com.

3. Owner decisions (Brad)

Canonical host — www (2026-10-23). Apex 301s via Traefik middleware in k8s/ingress.yaml.
Analytics — Umami + GA4 side by side (2026-10-23).
🛑 Contact details — confirm 804-332-6618 / info@bluecapstrategies.com are final.
AI training-bot policy — training crawlers blocked (2026-10-23): GPTBot, ClaudeBot, Google-Extended, CCBot, Applebot-Extended, meta-externalagent, Bytespider. Search and AI answer/search crawlers (Googlebot, Bingbot, OAI-SearchBot, Claude-SearchBot, …) stay allowed.
🛑 Brad Rodgers bio + credentials + portrait — gates Person schema, the About buildout, and principal-led E-E-A-T/GEO positioning.
🛑 Real insights articles — no fabricated or unreviewed AI content. /insights/ is "Coming soon" until the first one exists (§1.4).
🛑 Social profiles — provide real profiles or omit the footer block.
🛑 Proof modules / representative engagements — no fabricated client work.

4. Post-launch improvements (not blocking)

Bing Webmaster Tools — import from Search Console (covers Bing, DuckDuckGo, ChatGPT search).
Check Search Console in ~1 week: sitemap "Success", pages indexed, no 5xx in Coverage.
Per-page 1200×630 OG images; BaseLayout.astro emits one shared fallback og:image.
robots meta wired to a noindex frontmatter flag. /thank-you/ is excluded from the sitemap but emits no noindex meta tag.
article:published_time / article:modified_time OG tags on insights posts.
lastmod in the sitemap from content dates.
Lighthouse ≥ 95 on SEO / Perf / Best-Practices / Accessibility.
Umami/GA custom event for contact-form submits (today the /thank-you/ pageview is the conversion signal).
Optional: move bluecapstrategies.com into Pangolin's config.yml domains: so it shows "Config Managed" like the other three (needs a Pangolin restart; purely cosmetic).
Deferred: VPS static failover for homelab outages (see memory note / earlier discussion).

5. Launch-day record (2026-10-23)

What happened, so it does not have to be rediscovered.

Pangolin certs need a real config change to retry. HTTP-01 failed while www was still proxied (Let's Encrypt reached Cloudflare's IPv6, got 404) and hit the 5-failures/hour limit. Clicking Save with no changes does nothing; toggling Enable Resource off/on (saving between) re-adds the router and triggers issuance.
Pangolin health checks probe the target IP with no hostname, which k3s traefik answers with 404. Set the health check's Host header and TLS server name to the resource's hostname, path / (or /api/heartbeat for Umami), expect 200.
nginx 404 loop — error_page pointed at /404/index.html but Astro emits 404.html, so every unknown URL returned 500. Fixed in d1c1726; unknown URLs now return a real 404.
CI tag-bump push rejected when main moved during a build (or on a re-run of an older commit). The step now rebases onto origin/main and retries (93415c6). Do not "re-run" old runs to deploy — push instead.
Cloudflare cleanup — removed all Bluehost/cPanel leftovers (autoconfig/autodiscover, cpanel/whm/webdisk/ftp, cal/carddav SRV, localhost), replaced Bluehost's default._domainkey with MXroute's x._domainkey, added _dmarc.
Monitoring safety — the monylog playbook only renders Pushover routing when VAULT_TOKEN is set; without it Alertmanager silently falls back to ntfy. Always run it with the token.
Umami ships with admin / umami. It was brought up LAN-only (umami.int.7tl-homelab.com), the password changed, and only then exposed via Pangolin. Manifests live in homelab-gitops (apps/umami.yaml, analytics/umami/); credentials in OpenBao cluster/umami.

6. Already done — do not redo

Full copy parity with the WordPress site, verbatim in content collections (ADR-0002).
Real imagery in public/assets/live-site/; placeholders deleted.
Zod seoSchema; scripts/seo-lint.mjs (npm run seo:lint).
BreadcrumbList schema across detail pages.
Per-agent public/robots.txt; @astrojs/sitemap excluding /thank-you/.
Draft insights excluded from routes and sitemap.
Contact form: build-time Formspree endpoint, background submit, honeypot, failure fallback.
GitOps: Gitea Actions CI → Gitea Packages → tag-bump commit → ArgoCD (ADR-0007, ADR-0008).
CI job image node:22-bookworm; docker CLI + kustomize as pinned static binaries.
Production TLS in-cluster (DNS-01) and at the edge (HTTP-01).
Canonical www + apex 301; real 404s.
Monitoring: blackbox probes + WebsiteDown / WebsiteCertExpiringSoon / WebsiteSlow / BlackboxExporterDown rules in the monylog role (ADR-0004: monylog, not Uptime Kuma).
Analytics: Umami + GA4, production-host only; /privacy/ page.
Search Console verified; sitemap submitted.
Mail authentication: SPF, DKIM, DMARC passing.

12 KiB Raw Blame History Unescape Escape