So this article has been sitting on the blog since November 2025 and most of what it said stopped being true on the 5th of August 2026. It described the two data pipelines by the names of the external tools that ran them, and both of those tools have since been retired. It also opened by telling you how big the universe was and got it wrong by four hundred, because the real figure is 24,026 solar systems. That overcount turns out to have a proper explanation rather than being a typo, so instead of quietly deleting the article I have rewritten it around what the pipelines actually are now, and where the bad figure came from.
Quick context if you are landing here fresh. EF-Map is a free interactive 3D map of EVE Frontier, and none of the map data is made up. All of it is extracted out of the game client on my own machine into two SQLite databases that the site then ships. One of them draws and routes the map. The other one is what you get when you click into a single system. They come from different client files, they are built by different scripts, and they go wrong in different ways, which is the entire reason there are two of them to talk about.
The two databases
The map database is the small one, about 5 MB, and it is the first thing the site fetches because nothing can be drawn without it. It holds 24,026 solar systems with their coordinates, 274 regions, 2,163 constellations and 7,072 stargate rows. Those rows are stored directionally and every single one has a matching return trip, so what you see on the map is 3,536 bidirectional links. That sounds like a healthy network until you notice that only 3,033 systems touch it at all. The other 20,993 have no gate whatsoever, and you reach those by pointing your ship at them and jumping, which is why route planning here is a geometry problem rather than a walk over a graph somebody handed you.
The solar system database is the big one, around 67 MB, and it does not ship with the site at all. It sits in Cloudflare R2 and gets fetched on demand, because there is no sense making everybody download that to look at one system. Planets, moons, asteroid belts, star statistics and Lagrange points all live in there. I went through what is actually in that one in more detail a while back, so this article is mostly about how both of them get built.
The map pipeline is four Python stages now
It runs as four scripts in this order. First tools/game-data-extractor/build_stellar_from_starmapcache.py, which reads starmapcache.pickle and the two localization pickles straight out of the client and writes the intermediate JSON. Then process_labels.py assembles the label file, create_map_data.py writes the actual SQLite, and then two override passes, apply_solarsystemname_to_map_db.py and apply_region_names_to_map_db.py, correct the names. All four are Python 3.12 and all four live in this repo. That last part is new as of August, and it is the bit that matters.
The stage that reads the client used to be an external Node tool I did not write and did not control. Replacing it looked like a big job right up until I found out that starmapcache.pickle is a plain protocol-2 dictionary. No CCP loaders, no compiled bytecode, nothing clever. Stock pickle.load opens it. The replacement script is about 300 lines and the only genuinely fiddly part was porting the name resolution logic faithfully rather than reinventing it.
Where the wrong number came from
Here is the part I did not expect. The old figure was not a slip of the keyboard, it was a real extraction, just an extremely old one. The stellar_*.json input files sitting at the repo root were tracked in git and dated 2025-12-10, which makes them Cycle 5 artefacts. Anybody rebuilding the database from the tracked inputs, agent or human, would have quietly produced a Cycle 5 map and never known. Regenerating them dropped 400 phantom systems, 10 phantom regions and 50 phantom constellations, which is exactly the gap between the number in the old article and the number today.
I think that is the more useful lesson than anything about tooling. The pipeline was not broken. Every script ran, every script exited zero, and the output was a perfectly valid database of a universe that no longer existed. Stale inputs do not announce themselves, and a row count that looks plausible is the worst kind of wrong, because a figure four hundred out reads as perfectly reasonable unless you happen to know the real one.
Proving the rewrite changed nothing
Swapping out a stage that feeds the live map is the sort of thing that deserves more than a shrug and a deploy. So the new script has a --verify-against mode that diffs its output against a previous extraction, and all seven files came back byte identical to the old tool's. Then the whole chain ran end to end and the resulting database was compared table by table against the previously shipped one. Constellations, labels, nebula types, region labels, regions, stations and systems were identical row for row. Stargates matched as a 7,072 edge set and differed only in the ordering of an autoincrement id, which is not data.
One trap nearly ruined that. The pickle stores coordinates as float64, and Node prints the shortest form that round-trips while Python expands the exact binary value, so the same coordinate reads as -10456139802864780000 in one language and -10456139802864779264 in the other. Without deliberately matching the old formatting, 24,024 of the 24,026 system records look changed when absolutely nothing has changed. If I had trusted that first diff I would have concluded the rewrite was broken and gone back to the Node tool.
The names are not where you would look for them
Most system names in EVE Frontier are codes, things like A4T-SL7 or A.062.E25, and only about a hundred systems carry a pronounceable name like Zhurb. The awkward bit is that the authoritative display name is not in the star map file at all. It lives in a completely different client resource, and that was found out after cutover, by which point 7,376 of the 24,026 names on the live map were wrong. That is what the fourth stage exists to fix, and there is a matching pass for regions that overrides 56 of the 274. Both are pure override steps, bolted on after the fact, and I have left them that way rather than folding them in, because they document the mistake.
While on the subject of things that are not in the file. The map database has columns for security class, security status and star class, and all three are NULL for all 24,026 rows. That is not data loss, those fields do not exist in the game data. The old tool read them anyway and got empty structures back every time, which is a nice reminder that a column full of nulls tells you nothing about whether the extraction worked.
The other pipeline needs Python 3.12 exactly
The solar system side is one command, tools/game-data-extractor/extract_solarsystemcontent.py, and it is fussier. It pulls an 80 MB binary out of the client's content-addressed storage, decodes it into roughly 389 MB of JSON, then builds that down into the 67 MB database that goes up to R2. The version requirement is not a preference. CCP ships compiled Python bytecode inside the client and it is compiled for 3.12, so any other interpreter simply cannot load it. If you get the version wrong the failure is immediate and loud, which is honestly the nicest kind of dependency to have.
Knowing what to run in the first place
Both pipelines are only half the problem. The other half is deciding whether a patch even needs them. That used to mean reading the patch notes and guessing, which missed things, including several entire new data files that were only discovered weeks after they appeared. So there is a third tool now, tools/client-diff/, and it is the cheapest thing in the whole setup. Every build ships a manifest of about 48,000 lines with an MD5 per resource, so diffing two builds is a set difference between two text files. No decoding, no game launch, any Python. It reports which resources changed and which product cares. The one rule is snapshot before the patch, because the launcher overwrites the manifest in place and a build you did not capture can never be diffed again.
Things still wrong on purpose
Two loose ends I would rather write down than pretend away. filter_map_data.py is dead code, it reads and writes a JSON file that nothing in the chain produces, and it sat in the documentation as a live stage for months. And process_labels.py misreads a three-key wrapper as the region label map, which leaves exactly three bogus region label entries in the database. I reproduced that behaviour deliberately during the rewrite, because fixing it changes what gets rendered on the map, and that is a product decision rather than something to slip in under cover of a migration.
So the short version is that both databases now come out of scripts I control, the one stage that used to depend on somebody else's tool does not any more, and the whole thing was proven equivalent rather than assumed. If you spot a system, a region or a gate on the map that does not match what you see in the client, that is worth telling me about, because it means one of these stages has drifted again, and I would rather hear it from a player than find it nine months later in an old blog post.
Related Posts
What EF-Map Knows About Every EVE Frontier Solar System is a plain English tour of the bigger database, the one with the planets, L-points and frost lines in it.
A* vs Dijkstra: When Your Heuristic Has to Be Zero is what the map database gets used for, and why 20,993 gateless systems make routing a geometry problem.
Solar System View: A Three-Day Journey from Concept to Production is the feature the 67 MB database was extracted for in the first place.
A year of EF-Map, read back from its own decision log covers a lot more of this sort of thing, where the record turned out not to match what I remembered.