Finally add a second option for Chunk MapAccessor implementation besides the
MapAccessor. The RLEAccessor is basically a MapAccessor that will compress
your drawing with Run Length Encoding (RLE) in the on-disk format in the ZIP
file.
This slashes the file sizes of most levels:
* Shapeshifter: 21.8 MB -> 8.1 MB
* Jungle: 10.4 MB -> 4.1 MB
* Zoo: 2.8 MB -> 1.3 MB
Implementation details:
* The RLE binary format for Chunks is a stream of Uvarint pairs storing the
palette index number and the number of pixels to repeat it (along the Y,X
axis of the chunk).
* Null colors are represented by a Uvarint that decodes to 0xFFFF
or 65535 in decimal.
* Gameplay logic currently limits maps to 256 colors.
* The default for newly created chunks in-game will be RLE by default.
* Its in-memory representation is still a MapAccessor (a map of absolute
world coordinates to palette index).
* The game can still open and play legacy MapAccessor maps.
* On save in the editor, the game will upgrade/convert MapAccessor chunks over
to RLEAccessors, improving on your level's file size with a simple re-save.
Current Bugs
* On every re-save to RLE, one pixel is lost in the bottom-right corner of
each chunk. Each subsequent re-save loses one more pixel to the left, so what
starts as a single pixel per chunk slowly evolves into a horizontal line.
* Some pixels smear vertically as well.
* Off-by-negative-one errors when some chunks Iter() their pixels but compute
a relative coordinate of (-1,0)! Some mismatch between the stored world coords
of a pixel inside the chunk vs. the chunk's assigned coordinate by the Chunker:
certain combinations of chunk coord/abs coord.
To Do
* The `doodad touch` command should re-save existing levels to upgrade them.
Especially to further optimize memory for large levels, Levels and
Doodads can now read and write to a ZIP file format on disk with
chunks in external files within the zip.
Existing doodads and levels can still load as normal, and will be
converted into ZIP files on the next save:
* The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz
file, now becomes the cache of "hot chunks" loaded from ZIP. If there is
a ZIP file, chunks not accessed recently are flushed from the ChunkMap
to save on memory.
* During save, the ChunkMap is flushed to ZIP along with any non-loaded
chunks from a previous zipfile. So legacy levels "just work" when
saving, and levels loaded FROM Zip will manage their ChunkMap hot
memory more carefully.
Memory savings observed on "Azulian Tag - Forest.level":
* Before: 1716 MB was loaded from the old level format into RAM along
with a slow load screen.
* After: only 243 MB memory was used by the game and it loaded with
a VERY FAST load screen.
Updates to the F3 Debug Overlay:
* "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the
viewport (having bitmaps and textures loaded) vs. chunks outside which
have their textures freed (but data kept), and the number of chunks
currently hot cached in the ChunkMap.
The `doodad` tool has new commands to "touch" your existing levels
and doodads, to upgrade them to the new format (or you can simply
open and re-save them in-game):
doodad edit-level --touch ./example.level
doodad edit-doodad --touch ./example.doodad
The output from that and `doodad show` should say "File format: zipfile"
in the headers section.
To do:
* File attachments should also go in as ZIP files, e.g. wallpapers
* Discovered a bug where if you hit the Undo key to erase pixels and an
entire chunk became empty by it, the chunk would have rendering errors
and show as a solid black square instead of the level wallpaper
showing through.
* Chunks that have no pixels in them are culled from the chunker
immediately when you call a Delete() operation.
* The level file saver also calls a maintenance function to prune all
empty chunks upon saving the file. So existing levels with broken
chunks need only be re-saved to fix them.
* Touching "fire" pixels in a level will pop up the End Level alert box
saying you've died by fire and can restart the level.
* Update level.WriteFile() to prune broken links between actors before
save. So when a linked actor is deleted, the leftover link data is
cleaned up.
* Slight optimization in Canvas.drawStrokes: if either end of the stroke
is not within view of the screen, don't show the stroke.
Noah