GitGuardian emailed us at 19:08 UTC with a chilling subject line: “Cloudflare tunnel token exposed in EF-Map.” Within minutes we confirmed that a Cloudflared credential for the EVE Frontier assemblies tunnel had slipped into history. This post documents the full incident response—rotation, cleanup, and the guardrails we strengthened so future agents have a playbook.
Our infrastructure philosophy blends developer velocity with strict containment: fast preview deploys, private Postgres access via Cloudflared, and front-end automation described in Context7 MCP Integration. The leak challenged that balance. By sharing the exact steps here, we reinforce our culture of transparency and help other EVE Frontier projects learn from the experience.
Detection and Triage
The alert highlighted tools/assembly-api/.env committed to a feature branch. Because the repository had seen significant rebases during Solar System Phase 6, we first confirmed the scope:
- Ran
git log -- tools/assembly-api/.envto ensure no other references remained. - Cross-referenced Solar System View: A Three-Day Journey worklogs to see where the tunnel integration branched from.
- Validated that the token’s Cloudflared tunnel ID matched the production assemblies hostname.
Once scoped, we paused all other commits and followed the incident playbook.
Rotating the Tunnel Token
Token rotation started immediately to invalidate the leaked secret. The Windows script we wrote during the tunnel rollout proved invaluable:
tools\win\get_cloudflared_token.ps1 generated a fresh credential and wrote it to the gitignored tools/assembly-api/.env. Because the script locates cloudflared.exe automatically, operators didn’t need manual CLI steps.
We restarted the tunnel with tools\win\start_assembly_api.ps1 and polled https://assemblies.ef-map.com/health. The route returned {"status":"healthy"}, confirming the token swap succeeded before we touched git history.
At this stage, live traffic was secure again but the leaked token still existed in commits. The next priority was history scrubbing.
Scrubbing Git History Safely
Instead of a full repo rewrite we surgically removed the file:
- Stashed local changes from the Solar System label work to avoid conflicts.
- Installed
git-filter-repoviapy -3.11 -m pip install git-filter-repo(documented in the incident runbook now appended to the decision log). - Executed
git filter-repo --invert-paths --path tools/assembly-api/.env. - Restored the stash, re-added staged documentation updates, and force-pushed
mainwith the new history.
Team coordination was the final step—everyone rebased against the rewritten main. Because the change set was limited to one file, conflicts were minimal.
We added tools/assembly-api/.env* to .gitignore before running git filter-repo. This prevented future slips and ensured the history rewrite didn’t immediately reintroduce the problem during restaging.
Updating Guardrails and Documentation
The incident spurred a documentation sweep:
- Decision Log: Logged the rotation steps, risk classification, and follow-ups right next to the assembly tunnel entry.
- LLM Troubleshooting Guide: Cross-linked token regeneration guidance in the Assembly API section, reinforcing how Cloudflared operates when ingress rules are missing.
- Blog Guide Alignment: Ensured the Bandwidth Optimization and Context7 articles already mirrored our zero-trust narrative.
We also reinstated a lightweight manual check: before every wrangler pages deploy --branch main, run git status --ignored | findstr ".env" to confirm no credential files slipped through.
Lessons for Future Incidents
This wasn’t just about one token. It highlighted patterns that will keep EVE Frontier infrastructure resilient:
- Automation Saves Minutes: The PowerShell token script eliminated guesswork, reducing the rotation window to seconds.
- Decision Logs Matter: Our curated log meant agents onboarding after the rewrite can trace the incident timeline without digging through Slack.
- Preview Deploy Discipline: We redeployed a production build immediately after the scrub to ensure the main branch still bundled the latest Solar System fixes.
Security is an ongoing practice. Publishing this retrospective keeps us accountable and helps the wider EVE Frontier ecosystem prepare for similar discoveries.
Related Posts
- Reducing Cloud Costs by 93%: A Cloudflare KV Optimization Story – Another look at balancing Cloudflare infrastructure with security and cost controls.
- Context7 MCP Integration: Accelerating AI Documentation Retrieval by 20x – How we keep documentation synchronized for fast incident response.
- Vibe Coding: Building a 124,000-Line Project in 3 Months Without Writing Code – The methodology that guided today’s playbook-driven remediation.
- Dual Database Pipeline: Preparing for EVE Frontier Universe Updates – Shows how we protect data pipelines that depend on secure credentials.