docs: production cert issued; record the DNS-01 stall and resolver hardening

Marks the k3s half of section 1.1 done: bluecap-website-tls is Ready for both
production hostnames, valid to 2027-01-07, and the cluster serves it. Verified
by resolving straight to a control-plane node, so the public site stayed on
Cloudflare/Bluehost throughout.

Also records the Pangolin target table for step 2, and adds section 2.7 for
cert-manager validating DNS-01 propagation through CoreDNS rather than public
resolvers. That is not confirmed as the cause of the four-hour stall -
Cloudflare's own authoritative nameservers were returning NXDOMAIN for records
its API reported as present - but it is standard hardening worth doing before
renewals matter.
This commit is contained in:
bmr_bluecap committed 2026-10-09 15:19:12 -04:00
1 parent 6f802826df
commit e85edbfe34
1 file changed
+39 -4
+39 -4
View File
@@ -81,10 +81,20 @@ visitor → Cloudflare DNS (bluecapstrategies.com)
#### Prerequisites (all invisible to the public; the old site keeps serving)
- [x] Cloudflare token now covers the zone — `7tl-homelab.com`, **`bluecapstrategies.com`**,
`rodgersweb.com` (done 2026-10-09). Token is DNS-edit-only, which is correct least privilege.
- [ ] **k3s ingress:** add `www.bluecapstrategies.com` + `bluecapstrategies.com` to the `tls:`
block in `k8s/ingress.yaml` (issuer annotation is already present and correct —
`letsencrypt-prod`). cert-manager pre-issues via DNS-01, so do this **before** cutover and
confirm the Certificate goes Ready. Needed because Pangolin proxies to **https:443**.
- [x] **k3s ingress — DONE 2026-10-09 (commit `6f80282`).** Both production hostnames added to
the `tls:` block as `bluecap-website-tls`. Certificate **Ready**, issued by Let's Encrypt
via DNS-01: `CN=www.bluecapstrategies.com`, SANs `bluecapstrategies.com` +
`www.bluecapstrategies.com`, valid to **2027-01-07**, auto-renewing. Verified the cluster
serves it for both names (`--resolve` straight to 10.66.15.30 → 200, correct cert, rebuild
content). The public site is untouched and still on Cloudflare/Bluehost throughout.
Both names are on one cert, so the canonical-host decision stays open.
- Issuance stalled ~4h: the TXT records were present in Cloudflare's API immediately but
Cloudflare's own authoritative nameservers returned NXDOMAIN for them for a long while.
It cleared on its own. If this recurs, cert-manager's DNS-01 self-check resolver is the
thing to look at — it is currently CoreDNS (`nameservers=["10.43.0.10:53"]`), and the
usual hardening is to point it at public resolvers instead:
`--dns01-recursive-nameservers-only=true --dns01-recursive-nameservers=1.1.1.1:53,8.8.8.8:53`.
Worth doing before cutover day so renewals do not inherit this fragility (§2.7).
- [ ] **Pangolin:** add `bluecapstrategies.com` as a domain under org **`rhh-homelab`**
(`orgDomains`), `certResolver=letsencrypt`.
- [ ] **Pangolin:** create the resource(s) with **three targets**, matching the established
@@ -97,6 +107,19 @@ visitor → Cloudflare DNS (bluecapstrategies.com)
- [ ] **Test before any DNS change:** `curl --resolve bluecapstrategies.com:443:23.94.111.133`
and confirm the full path serves the site.
#### Pangolin resource targets (for step 2)
Three targets, matching every other k3s-backed resource:
| Target | Method | Port |
|--------|--------|------|
| `10.66.15.30` | https | 443 |
| `10.66.15.31` | https | 443 |
| `10.66.15.32` | https | 443 |
`k3-control-1/2/3`. Three gives node-level failover. Set the host header to the real hostname so
k3s traefik routes on it. The cluster now holds a valid cert for both production names, so the
https hop verifies properly.
#### Proxied vs DNS-only — decide before cutover 🛑
The zone is currently **proxied**: apex `A → 50.87.136.52 PROXIED`, `www CNAME → apex PROXIED`.
@@ -270,6 +293,18 @@ say depends on the owner's call in §1.5 on whether `/insights/` ships at all.
- [ ] If `/insights/` ships: delete both note-cards and rewrite the hero as reader-facing copy.
- [ ] If it does not: drop it from the primary nav and exclude the route from the sitemap.
### 2.7 cert-manager DNS-01 self-check uses CoreDNS
`cert-manager` logs `configured acme dns01 nameservers nameservers=["10.43.0.10:53"]`, i.e. it
validates DNS-01 propagation through the cluster's own CoreDNS rather than public resolvers. That
makes issuance sensitive to internal caching — including negative caching of the `_acme-challenge`
name it is waiting on.
- [ ] Point it at public resolvers before cutover:
`--dns01-recursive-nameservers-only=true --dns01-recursive-nameservers=1.1.1.1:53,8.8.8.8:53`.
Not confirmed as the cause of the 4h stall on 2026-10-09 (Cloudflare's authoritative NS were
themselves returning NXDOMAIN for records its API reported as present), but it is the
standard hardening and removes one variable from certificate renewal.
### 2.6 Staging URL ✅ LIVE 2026-09-23
**https://bcs.int.7tl-homelab.com** — review the rebuild in a browser, valid cert, no warning.