Measuring the repo before splitting it into packages

I’m splitting the Blobber repo into smaller pieces, and the first phase is done. I measured before moving anything, and the measurement changed the plan. The game’s seven obvious packages all call each other, so the game gets split on paper first, and the Python tools and the assets get their own repos now.

Why split the repo at all?

The repo has grown complex enough that I want smaller repos the agents can work in with more autonomy and less context. What I want at the end is a set of libraries that can be built on their own, shared, tested, cached and reused without rebuilds. That should speed up tests, builds and iteration, cut how much context an agent needs, and give each piece its own work list.

I picked a hybrid on the Review Desk. The tools and the assets become their own repos, the game becomes packages inside one repo, the assets are used at a pinned version, and each package gets a work list. I also asked for the measuring to come first.

How does the dependency tool work?

tools/dep_map.py reads the whole repository without starting Godot. On the GDScript side it builds a map of every class_name to its file, then counts uses of those names after stripping comments and string contents. It also follows preload() and load() of res:// paths, extends, the autoloads in project.godot, and every other res:// string. A path built at runtime, like res://x/%s.tscn, becomes an edge to the folder before the %s. A reference into assets/ becomes an edge to the bucket for its folder, so 18,294 asset files don’t each get a line.

On the Python side it follows imports, scripts that name or run other scripts, and every game path a script names. Each path is marked as read or written by its line. Those paths are what a Python repo would have to keep agreeing on with the game.

A list of rules assigns each file to a package, and the first match wins. Tests, films and tool scripts go to the package most of their dependencies are in. Then the tool counts distinct file pairs between packages, finds the cycles with Tarjan’s algorithm, and searches every ordering of the packages for the one with the fewest calls going against it. The test, tools/dep_map_test.py, runs the tool on a made-up tree with a known graph and a known cycle.

What did it find?

The repo has 22,147 tracked files. 1,036 are GDScript with 250,979 lines, and 339 are Python.

The seven candidate game packages are core, actors, dungeon, world, content, ui and shell. They form one cycle. Every pair of them calls the other both ways, with 1,265 cross-package file pairs, and 309 of those go against the best possible layer order. Actors and ui, for example, reach into each other 88 and 61 times, and dungeon and world 54 and 55 times.

Only 59% of the tests’ own dependency edges stay inside their package. That’s 3,012 of 5,116. A package’s tests alone would miss the rest.

The foundations are clear, though. Audio is used from 96 places and uses 2. Input is used from 70 and uses 5. The leaves are the other way round: the builder is used from nowhere and uses 73 places.

Why not just move the folders into addons?

That was my first idea, and the numbers killed it. Moving 1,036 GDScript files into addons/ would rewrite thousands of res:// paths, scenes and saves, and the packages still couldn’t be built alone, since each needs the others.

So the game is split logically first. A manifest maps paths to packages, and a checker holds a ratchet: today’s backward edges are allowed and no new ones are. Each package gets its own tests, cache, docs and work list. A package moves on disk only once it has no backward edges, and the editor goes first.

I also re-ran the assignment with five proposed moves applied:

1. Core stays the foundation, the save and feedback code moves to shell, and the event log moves into core as the log bus.
2. The ui kit stays a leaf, and the hud, menus, party bar and creation screens become their own “screens” package above actors.
3. Dungeon and world merge into one “levels” package.
4. The builder and editor addon are cut out as “editor”.
5. Content merges into actors, because the tables are bound to the classes they fill.

With those, cross-package pairs fall from 1,265 to 1,128 and backward pairs from 309 to 160, which is 14%. What’s left is mostly actors reaching into levels, 91 pairs. That needs a small levels API that actors would have injected, and it is filed for later.

Which parts can be split now?

The Python tools and the assets have real boundaries. The art kit is 651 files and 59,231 lines of Python, and it needs tools/workspace.py and the world generator’s noise functions. The dungeon generator is 52 files and 14,481 lines, and its contract is the dungeon part of the content JSON plus the dungeon test fixtures. The assets are 18,294 files, with 181.3 MB of models alone. Only 36 product files name an asset path directly, so the contract there is the folder layout.

What comes next?

The plan has nine more phases, done one at a time:

1. The package manifest and the ratchet checker.
2. Tests assigned to packages, with a way to run one package’s tests.
3. A test cache keyed on the package’s files, its dependencies, the Godot version and the test’s own files.
4. The dungeon generator in its own repo, pinned by commit.
5. The art kit in its own repo.
6. The assets in their own repo, pinned by hash.
7. Docs and work lists per package.
8. The editor moved out for real, with a tool to rewrite the res:// paths.
9. A full suite run and a second report to compare with this one.

Measuring was the phase before these. I’ll add the rest to the same log entry as they land.