Compare commits

...
2 Commits
Author SHA1 Message Date
bmr_bluecap d1c1726796 fix(nginx): return a real 404 instead of a 500 redirect loop
CI / ci (push) Failing after 1m50s
The 404 fallback pointed at /404/index.html, but Astro emits 404.html,
so every unknown URL looped internally and nginx answered 500. The
try_files fallback also served the error page with a 200 status.
Fall through to =404 and point error_page at /404.html.
2026-10-23 11:36:00 -04:00
bmr_bluecap e85edbfe34 docs: production cert issued; record the DNS-01 stall and resolver hardening
Marks the k3s half of section 1.1 done: bluecap-website-tls is Ready for both
production hostnames, valid to 2027-01-07, and the cluster serves it. Verified
by resolving straight to a control-plane node, so the public site stayed on
Cloudflare/Bluehost throughout.

Also records the Pangolin target table for step 2, and adds section 2.7 for
cert-manager validating DNS-01 propagation through CoreDNS rather than public
resolvers. That is not confirmed as the cause of the four-hour stall -
Cloudflare's own authoritative nameservers were returning NXDOMAIN for records
its API reported as present - but it is standard hardening worth doing before
renewals matter.
2026-10-09 15:19:12 -04:00
2 changed files with 41 additions and 6 deletions

No files matched your search

+2 -2
View File
@@ -14,10 +14,10 @@ server {
} }
location / { location / {
try_files $uri $uri/ /404/index.html; try_files $uri $uri/ =404;
} }
error_page 404 /404/index.html; error_page 404 /404.html;
location ~* \.(?:css|js|jpg|jpeg|png|webp|svg|ico|woff2)$ { location ~* \.(?:css|js|jpg|jpeg|png|webp|svg|ico|woff2)$ {
try_files $uri =404; try_files $uri =404;
+39 -4
View File
@@ -81,10 +81,20 @@ visitor → Cloudflare DNS (bluecapstrategies.com)
#### Prerequisites (all invisible to the public; the old site keeps serving) #### Prerequisites (all invisible to the public; the old site keeps serving)
- [x] Cloudflare token now covers the zone — `7tl-homelab.com`, **`bluecapstrategies.com`**, - [x] Cloudflare token now covers the zone — `7tl-homelab.com`, **`bluecapstrategies.com`**,
`rodgersweb.com` (done 2026-10-09). Token is DNS-edit-only, which is correct least privilege. `rodgersweb.com` (done 2026-10-09). Token is DNS-edit-only, which is correct least privilege.
- [ ] **k3s ingress:** add `www.bluecapstrategies.com` + `bluecapstrategies.com` to the `tls:` - [x] **k3s ingress — DONE 2026-10-09 (commit `6f80282`).** Both production hostnames added to
block in `k8s/ingress.yaml` (issuer annotation is already present and correct — the `tls:` block as `bluecap-website-tls`. Certificate **Ready**, issued by Let's Encrypt
`letsencrypt-prod`). cert-manager pre-issues via DNS-01, so do this **before** cutover and via DNS-01: `CN=www.bluecapstrategies.com`, SANs `bluecapstrategies.com` +
confirm the Certificate goes Ready. Needed because Pangolin proxies to **https:443**. `www.bluecapstrategies.com`, valid to **2027-01-07**, auto-renewing. Verified the cluster
serves it for both names (`--resolve` straight to 10.66.15.30 → 200, correct cert, rebuild
content). The public site is untouched and still on Cloudflare/Bluehost throughout.
Both names are on one cert, so the canonical-host decision stays open.
- Issuance stalled ~4h: the TXT records were present in Cloudflare's API immediately but
Cloudflare's own authoritative nameservers returned NXDOMAIN for them for a long while.
It cleared on its own. If this recurs, cert-manager's DNS-01 self-check resolver is the
thing to look at — it is currently CoreDNS (`nameservers=["10.43.0.10:53"]`), and the
usual hardening is to point it at public resolvers instead:
`--dns01-recursive-nameservers-only=true --dns01-recursive-nameservers=1.1.1.1:53,8.8.8.8:53`.
Worth doing before cutover day so renewals do not inherit this fragility (§2.7).
- [ ] **Pangolin:** add `bluecapstrategies.com` as a domain under org **`rhh-homelab`** - [ ] **Pangolin:** add `bluecapstrategies.com` as a domain under org **`rhh-homelab`**
(`orgDomains`), `certResolver=letsencrypt`. (`orgDomains`), `certResolver=letsencrypt`.
- [ ] **Pangolin:** create the resource(s) with **three targets**, matching the established - [ ] **Pangolin:** create the resource(s) with **three targets**, matching the established
@@ -97,6 +107,19 @@ visitor → Cloudflare DNS (bluecapstrategies.com)
- [ ] **Test before any DNS change:** `curl --resolve bluecapstrategies.com:443:23.94.111.133` - [ ] **Test before any DNS change:** `curl --resolve bluecapstrategies.com:443:23.94.111.133`
and confirm the full path serves the site. and confirm the full path serves the site.
#### Pangolin resource targets (for step 2)
Three targets, matching every other k3s-backed resource:
| Target | Method | Port |
|--------|--------|------|
| `10.66.15.30` | https | 443 |
| `10.66.15.31` | https | 443 |
| `10.66.15.32` | https | 443 |
`k3-control-1/2/3`. Three gives node-level failover. Set the host header to the real hostname so
k3s traefik routes on it. The cluster now holds a valid cert for both production names, so the
https hop verifies properly.
#### Proxied vs DNS-only — decide before cutover 🛑 #### Proxied vs DNS-only — decide before cutover 🛑
The zone is currently **proxied**: apex `A → 50.87.136.52 PROXIED`, `www CNAME → apex PROXIED`. The zone is currently **proxied**: apex `A → 50.87.136.52 PROXIED`, `www CNAME → apex PROXIED`.
@@ -270,6 +293,18 @@ say depends on the owner's call in §1.5 on whether `/insights/` ships at all.
- [ ] If `/insights/` ships: delete both note-cards and rewrite the hero as reader-facing copy. - [ ] If `/insights/` ships: delete both note-cards and rewrite the hero as reader-facing copy.
- [ ] If it does not: drop it from the primary nav and exclude the route from the sitemap. - [ ] If it does not: drop it from the primary nav and exclude the route from the sitemap.
### 2.7 cert-manager DNS-01 self-check uses CoreDNS
`cert-manager` logs `configured acme dns01 nameservers nameservers=["10.43.0.10:53"]`, i.e. it
validates DNS-01 propagation through the cluster's own CoreDNS rather than public resolvers. That
makes issuance sensitive to internal caching — including negative caching of the `_acme-challenge`
name it is waiting on.
- [ ] Point it at public resolvers before cutover:
`--dns01-recursive-nameservers-only=true --dns01-recursive-nameservers=1.1.1.1:53,8.8.8.8:53`.
Not confirmed as the cause of the 4h stall on 2026-10-09 (Cloudflare's authoritative NS were
themselves returning NXDOMAIN for records its API reported as present), but it is the
standard hardening and removes one variable from certificate renewal.
### 2.6 Staging URL ✅ LIVE 2026-09-23 ### 2.6 Staging URL ✅ LIVE 2026-09-23
**https://bcs.int.7tl-homelab.com** — review the rebuild in a browser, valid cert, no warning. **https://bcs.int.7tl-homelab.com** — review the rebuild in a browser, valid cert, no warning.