insights: coming-soon page; block AI training crawlers; fix stale e2e

- /insights/ is a reader-facing Coming soon page; developer cards removed
- robots.txt: Disallow GPTBot, ClaudeBot, Google-Extended, CCBot,
  Applebot-Extended, meta-externalagent, Bytespider. Search and AI
  answer crawlers stay allowed. Corrects the old Google-Extended
  comment: it does not affect Search or AI Overviews.
- e2e: assert the real headings and the header Services link (48/48)

public/robots.txt

@@ -1,8 +1,6 @@
 # BlueCap Strategies robots.txt — see docs/seo-aeo-optimization.md §5
 # Policy: allow classic search + AI answer/search crawlers for maximum discoverability and
-# answer-engine citation. Training-crawler policy (GPTBot, ClaudeBot, Google-Extended) is an
-# owner decision — currently ALLOWED for maximum reach. To opt out of model training, uncomment
-# the "Disallow: /" lines in the training-crawler block below.
+# answer-engine citation. Model-TRAINING crawlers (GPTBot, ClaudeBot, Google-Extended, …) are BLOCKED (owner decision, 2026-10-23).
 
 # --- Classic search engines ---
 User-agent: Googlebot
@@ -27,19 +25,29 @@ Allow: /
 User-agent: PerplexityBot
 Allow: /
 
-# Google-Extended controls Gemini + Google AI Overviews (this is also a training signal for
-# Google — treated as answer-visibility here; move to the training block to opt out).
-User-agent: Google-Extended
-Allow: /
-
-# --- AI model-training crawlers (owner decision — allowed by default) ---
+# --- AI model-training crawlers — blocked (owner decision, 2026-10-23) ---
 User-agent: GPTBot
-Allow: /
-# Disallow: /
+Disallow: /
 
 User-agent: ClaudeBot
-Allow: /
-# Disallow: /
+Disallow: /
+
+User-agent: CCBot
+Disallow: /
+
+User-agent: Applebot-Extended
+Disallow: /
+
+User-agent: meta-externalagent
+Disallow: /
+
+User-agent: Bytespider
+Disallow: /
+
+# Google-Extended is Google's opt-out for Gemini training/grounding only. Google Search and AI
+# Overviews are governed by Googlebot (allowed above), so blocking this does not affect search.
+User-agent: Google-Extended
+Disallow: /
 
 # --- Everything else ---
 User-agent: *