Levels can now be converted to RLE encoded chunk accessors and be re-saved
continuously without any loss of information.
Off-by-one errors resolved:
* The rle.NewGrid() was adding a +1 everywhere making the 2D grids have 129
elements to a side for a 128 chunk size.
* In rle.Decompress() the cursor value and translation to X,Y coordinates is
fixed to avoid a pixel going missing at the end of the first row (128,0)
* The abs.X-- hack in UnmarshalBinary is no longer needed to prevent the
chunks from scooting a pixel to the right on every save.
Doodad tool updates:
* Remove unused CLI flags in `doodad resave` (actors, chunks, script,
attachment, verbose) and add a `--output` flag to save to a different file
name to the original.
* Update `doodad show` to allow debugging of RLE compressed chunks:
* CLI flag `--chunk=1,2` to specify a single chunk coordinate to debug
* CLI flag `--visualize-rle` will Visualize() RLE compressed chunks in
their 2D grid form in your terminal window (VERY noisy for large
levels! Use the --chunk option to narrow to one chunk).
Bug fixes and misc changes:
* Chunk.Usage() to return a better percentage of chunk utilization.
* Chunker.ChunkFromZipfile() was split out into two functions:
* RawChunkFromZipfile retrieves the raw bytes of the chunk as well as the
file extension discovered (.bin or .json) so the caller can interpret
the bytes correctly.
* ChunkFromZipfile calls the former function and then depending on file
extension, unmarshals from binary or json.
* The Raw function enables the `doodad show` command to debug and visualize
the raw contents of the RLE compressed chunks.
* Updated the Visualize() function for the RLE encoder: instead of converting
palette indexes to hex (0-F) which would begin causing problems for palette
indexes above 16 (as they would use two+ characters), indexes are mapped to
a wider range of symbols (0-9A-Z) and roll over if you have more than 36
colors on your level. This at least keeps the Visualize() grid an easy to
read 128x128 characters in your terminal.
Finally add a second option for Chunk MapAccessor implementation besides the
MapAccessor. The RLEAccessor is basically a MapAccessor that will compress
your drawing with Run Length Encoding (RLE) in the on-disk format in the ZIP
file.
This slashes the file sizes of most levels:
* Shapeshifter: 21.8 MB -> 8.1 MB
* Jungle: 10.4 MB -> 4.1 MB
* Zoo: 2.8 MB -> 1.3 MB
Implementation details:
* The RLE binary format for Chunks is a stream of Uvarint pairs storing the
palette index number and the number of pixels to repeat it (along the Y,X
axis of the chunk).
* Null colors are represented by a Uvarint that decodes to 0xFFFF
or 65535 in decimal.
* Gameplay logic currently limits maps to 256 colors.
* The default for newly created chunks in-game will be RLE by default.
* Its in-memory representation is still a MapAccessor (a map of absolute
world coordinates to palette index).
* The game can still open and play legacy MapAccessor maps.
* On save in the editor, the game will upgrade/convert MapAccessor chunks over
to RLEAccessors, improving on your level's file size with a simple re-save.
Current Bugs
* On every re-save to RLE, one pixel is lost in the bottom-right corner of
each chunk. Each subsequent re-save loses one more pixel to the left, so what
starts as a single pixel per chunk slowly evolves into a horizontal line.
* Some pixels smear vertically as well.
* Off-by-negative-one errors when some chunks Iter() their pixels but compute
a relative coordinate of (-1,0)! Some mismatch between the stored world coords
of a pixel inside the chunk vs. the chunk's assigned coordinate by the Chunker:
certain combinations of chunk coord/abs coord.
To Do
* The `doodad touch` command should re-save existing levels to upgrade them.
Convert the Chunker size to a uint8 so chunk sizes are limited to 255px. This
means that inside of a chunk, uint8's can track the relative pixel coordinates
and result in a great memory savings since all of these uint8's are currently
64-bits wide apiece.
WIP on rectangular shaped doodads:
* You can create such a doodad in the editor and draw it normally.
* It doesn't draw the right size when dragged into your level however:
- In uix.Actor.Size() it gets a rect of the doodad's square Chunker size,
instead of getting the proper doodad.Size rect.
- If you give it the doodad.Size rect, it draws the Canvas size correctly
instead of a square - the full drawing appears and in gameplay its hitbox
(assuming the same large rectangle size) works correctly in-game.
- But, the doodad has scrolling issues when it gets to the top or left edge
of the screen! This old gnarly bug has come back. For some reason square
canvas doodads draw correctly but rectangular ones have the drawing scroll
just a bit - how far it scrolls is proportional to how big the doodad is,
with the Start Flag only scrolling a few pixels before it stops.
* New built-in wallpaper: "Dotted paper (dark)" is a dark-themed wallpaper.
* New built-in palette: "Neon Bright" with bright colors for dark levels.
* New cheat: "warp whistle" to automatically win the level.
* In case the user has a VERY LARGE screen resolution bigger than the full
bounds of a Bounded level, the Play Scene will cap the size and center
the level canvas onto the window. This is preferable to being able to see
beyond the level's boundaries and hitting an invisible wall in-game.
* Make the titlescreen Lazy Scroll work on unbounded levels. It can't bounce
off scroll boundaries but it will reverse course if it reaches the level's
furthest limits.
* Bugfix: characters' white eyes were transparent in-game. Multiple culprits
from the `doodad convert` tool defaulting the chroma key to white, to the
SDL2 textures considering white to be transparent. For the latter, the game
offsets the color by -1 blue.
* Editor: Auto-save on a background goroutine so you don't randomly freeze
the editor up during.
* Fix actor linking issues when you drag and re-place a linked doodad: the
level was too eagerly calling PruneLinks() whenever a doodad was 'destroyed'
(such as the one just picked up) breaking half of the link connection.
* Chunk unloader: do not unload a chunk that has been modified (Set or Delete
called on), keep them in memory until the next ZIP file save to flush them
out to disk.
* Link Tool: if you clicked an actor and don't want to connect a link, click
the first actor again to de-select it.
Updates to the `doodad` tool:
* `doodad edit-level --resize <int>` can re-chunk a level to use a different
chunk size than the default 128. Large chunk sizes 512+ lead to performance
problems.
Especially to further optimize memory for large levels, Levels and
Doodads can now read and write to a ZIP file format on disk with
chunks in external files within the zip.
Existing doodads and levels can still load as normal, and will be
converted into ZIP files on the next save:
* The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz
file, now becomes the cache of "hot chunks" loaded from ZIP. If there is
a ZIP file, chunks not accessed recently are flushed from the ChunkMap
to save on memory.
* During save, the ChunkMap is flushed to ZIP along with any non-loaded
chunks from a previous zipfile. So legacy levels "just work" when
saving, and levels loaded FROM Zip will manage their ChunkMap hot
memory more carefully.
Memory savings observed on "Azulian Tag - Forest.level":
* Before: 1716 MB was loaded from the old level format into RAM along
with a slow load screen.
* After: only 243 MB memory was used by the game and it loaded with
a VERY FAST load screen.
Updates to the F3 Debug Overlay:
* "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the
viewport (having bitmaps and textures loaded) vs. chunks outside which
have their textures freed (but data kept), and the number of chunks
currently hot cached in the ChunkMap.
The `doodad` tool has new commands to "touch" your existing levels
and doodads, to upgrade them to the new format (or you can simply
open and re-save them in-game):
doodad edit-level --touch ./example.level
doodad edit-doodad --touch ./example.doodad
The output from that and `doodad show` should say "File format: zipfile"
in the headers section.
To do:
* File attachments should also go in as ZIP files, e.g. wallpapers
Instead of the loadscreen eager-loading ALL level chunks to Go Images, only
load the chunks within the "LoadingViewport" - which is the on-screen
Viewport plus a margin of chunks off the screen edges.
During gameplay, every few ticks, reevaluate which chunks are inside or
outside the LoadingViewport; for chunks outside, free their SDL2 textures
and free their cached bitmaps to keep overall memory usage down. The
AzulianTag-Forest level now stays under 200 Textures at any given time
and the loadscreen goes faster as it doesn't have to load every chunk's
images up front.
The LoadUnloadChunk feature can be turned on/off with feature flags. If
disabled the old behavior is restored: loadscreen loads all images and
the LoadUnloadChunks function is not run.
Other changes:
* loadscreen: do not free textures in the Hide() function as this runs on
a different goroutine and may break. The 4 wallpaper textures are OK
to keep in memory anyway, the loadscreen is reused often!
* Free more leaked textures: on the Inventory frame and when an actor
calls Self.Destroy()
* Stop leaking goroutines in the PubSub feature of the doodad script
engine; scripting.Supervisor.Teardown() sends a stop signal to all
scripts to clean up neatly. Canvas.Destroy() tears down its scripting
supervisor automatically.
* Added to the F3 Debug Overlay is a "Texture:" label that counts the number
of textures currently loaded by the (SDL2) render engine.
* Added Teardown() functions to Level, Doodad and the Chunker they both use
to free up SDL2 textures for all their cached graphics.
* The Canvas.Destroy() function now cleans up all textures that the Canvas
is responsible for: calling the Teardown() of the Level or Doodad, calling
Destroy() on all level actors, and cleaning up Wallpaper textures.
* The Destroy() method of the game's various Scenes will properly Destroy()
their canvases to clean up when transitioning to another scene. The
MainScene, MenuScene, EditorScene and PlayScene.
* Fix the sprites package to actually cache the ui.Image widgets. The game
has very few sprites so no need to free them just yet.
Some tricky places that were leaking textures have been cleaned up:
* Canvas.InstallActors() destroys the canvases of existing actors before it
reinitializes the list and installs the replacements.
* The DraggableActor when the user is dragging an actor around their level
cleans up the blueprint masked drag/drop actor before nulling it out.
Misc changes:
* The player character cheats during Play Mode will immediately swap out the
player character on the current level.
* Properly call the Close() function instead of Hide() to dismiss popup
windows. The Close() function itself calls Hide() but also triggers
WindowClose event handlers. The Doodad Dropper subscribes to its close
event to free textures for all its doodad canvases.
* Clean up unused msgpack code for levels and doodads
* Fix the cosmetic bug where actors in your level would display wrongly
when scrolling off the top/left edges of the screen: they used to
anchor at their own 0,0 coordinate and crop their width/height leading
to a 'scrolling' effect that didn't happen on the right/bottom edges.
New feature: link a Start Flag to another doodad in your level
and you will play as that doodad instead of Boy. All Creatures
are designed to be playable. Playing as "other" doodads leads
to interesting effects, like not being able to activate buttons,
switches, or warp doors and not having an inventory to pick up
keys. The Anvil is fun: it can destroy other mobile doodads by
jumping on them.
If the actor does not specify that it has gravity, the gameplay
starts in antigravity mode. This will be the vast majority of
non-mobile doodads and the Bird.
Other changes:
* The Blue and Red Azulians now share a doodad script.
* The Azulians AI is still to walk back and forth, pickup keys and
press buttons. The Blue Azulian walks slower than the red one.
* The Blue Azulian is no longer hidden from the doodads list.
* Actor UUID values in levels are now V1 UUIDs (time-ordered).
This will help to reliably resolve conflicts in draw order
of overlapping doodads (newest added to level wins).
* Link Tool: clicking on a pair of already-linked doodads will
now unlink them, so you don't have to delete one to delete
the link.
* Actor Tool: deleting an actor immediately calls PruneLinks()
to clean up any links that the deleted doodad might have.
* pkg/loadscreen implements a global Loading Screen for loading heavy
levels for playing or editing.
* All chunks in a level are pre-rendered to bitmap before gameplay
begins, which reduces stutter as chunks were being lazily rendered on
first appearance before.
* The loading screen can be played with in the developer console:
$ loadscreen.Show()
$ loadscreen.Hide()
Along with ShowWithProgress(), SetProgress(float64) and IsActive()
* Chunker: separate the concerns between Bitmaps an (SDL2) Textures.
* Chunker.Prerender() converts a chunk to a bitmap (a Go image.Image)
and caches it, only re-rendering if marked as dirty.
* Chunker.Texture() will use the pre-cached bitmap if available to
immediately produce the SDL2 texture.
Other miscellaneous changes:
* Added to the Colored Pencil palette: Sandstone
* Added "perlin noise" brush pattern
Note: this commit introduces instability and crashes:
* New `asyncSetup()` functions run on a goroutine, but SDL2 texture
calls must run on the main thread.
* Chunker avoids this by caching bitmaps, not textures.
* Wallpaper though is unstable, sometimes works, sometimes has graphical
glitches, sometimes crashes the game.
* Wallpaper.Load() and the *Texture() functions are where it crashes.
Palette swatches gain a new property: Pattern.
Patterns are grayscale textures that the swatch color will sample
against when drawing pixels to the level, by taking the world coordinate
modulo a value inside the texture.
A few algorithms were tried (Screen, Overlay), this branch lands on one
that tries to cast the color from grayscale which comes out rather dark;
to get a patterned color to look black while still seeing the pattern,
the color needs to be as bright as #777 to get the effect.
* Start the program window maximized with the `-w maximized` CLI option.
* Move the Doodad Palette off the right-side dock of the Editor Scene and
into its own pop-up window: the DoodadDropper.
* Shrink the width of the Color Palette panel and show only the colors in
the buttons. The name of the swatch is available in the mouse-over tooltip.
* Added an "Edit" button to the Color Palette. It opens a Palette Editor
window where you can rename, change colors and attributes of existing colors
OR insert new colors into your palette. (Deleting colors not yet supported).
* level.Chunker gets a Redraw method: invalidates all cached textures of all
chunks forcing the level to redraw itself, possibly with an updated palette.
* New doodads: Switches.
* They come in four varieties: wall switch (background element, with
"ON/OFF" text) and three side-profile switches for the floor, left
or right walls.
* On collision with the player, they flip their state from "OFF" to
"ON" or vice versa. If the player walks away and then collides
again, the switch flips again.
* Can be used to open/close Electric Doors when turned on/off. Their
default state is "off"
* If a switch receives a power signal from another linked switch, it
sets its own state to match. So, two "on/off" switches that are
connected to a door AND to each other will both flip on/off when one
of them flips.
* Update the Level Collision logic to support Decoration, Fire and Water
pixel collisions.
* Previously, ALL pixels in the level were acting as though solid.
* Non-solid pixels don't count for collision detection, but their
attributes (fire and water) are collected and returned.
* Updated the MenuScene to support loading a map file in Play Mode
instead of Edit Mode. Updated the title screen menu to add a button
for playing levels instead of editing them.
* Wrote some documentation.
Since SDL2 is using in-memory bitmaps the same as Canvas engine, the
function names of the render.Engine interface have been cleaned up:
* NewTexture(filename, image) -> StoreTexture(name, image)
Create a new cached texture with a given name.
* NewBitmap(filename) -> LoadTexture(name)
Recall a stored texture with a given name.
* level.Chunk.ToBitmap uses simpler names for the textures instead of
userdir.CacheFilename file-like paths.
* Refactor texture caching in render.Engine:
* New interface method: NewTexture(filename string, image.Image)
* WASM immediately encodes the image to PNG and generates a JavaScript
`Image()` object to load it with a data URI and keep it in memory.
* SDL2 saves the bitmap to disk as it did before.
* WASM: deprecate the sessionStorage for holding image data. Session
storage methods panic if called. The image data is directly kept in
Go memory as a js.Value holding an Image().
* Shared Memory workaround: the level.Chunk.ToBitmap() function is where
chunk textures get cached, but it had no access to the render.Engine
used in the game. The `pkg/shmem` package holds global pointers to
common structures like the CurrentRenderEngine as a work-around.
* Also shmem.Flash() so Doodle can make its d.Flash() function
globally available, any sub-package can now flash text to the screen
regardless of source code location.
* JavaScript API for Doodads now has a global Flash() function
available.
* WASM: Handle window resize so Doodle can recompute its dimensions
instead of scaling/shrinking the view.
* Add RGBA color blending support in WASM build.
* Initial texture caching API for Canvas renderer engine. The WASM build
writes the chunk caches as a "data:image/png" base64 URL on the
browser's sessionStorage, for access to copy into the Canvas.
* Separated the ClickEvent from the MouseEvent (motion) in the WASM
event queue system, to allow clicking and dragging.
* Added the EscapeKey handler, which will abruptly terminate the WASM
application, same as it kills the window in the desktop build.
* Optimization fix: I discovered that if the user clicks and holds over
a single pixel when drawing a level, repeated Set() operations were
firing meaning multiple cache invalidations. Not noticeable on PC but
on WebAssembly it crippled the browser. Now if the cursor isn't moving
it doesn't do anything.
* Refactor the event system in the WASM render engine to serialize the
async JavaScript events into a channel, so that queued events are read
off serially in the main loop similar to SDL. This fixes keyboard
input issues, altho if you type really fast some input keys get lost.
* Add a Red Azulian as a test for mobile enemies.
* Its A.I. has it walk back and forth, changing directions when it
comes up against an obstacle for a few moments.
* It plays walking animations and can trigger collision events with
other Doodads, such as the Electric Door and Trapdoor.
* Move Gravity responsibility to the doodad scripts themselves.
* Call `Self.SetGravity(true)` to opt the Doodad in to gravity.
* The canvas.Loop() adds gravity to any doodad that has it enabled.
* Add some encoding/decoding functions for binary msgpack format for
levels and doodads. Currently it writes msgpack files that can be
decoded and printed by Python (mp2json.py) but it can't re-read from
the binary format. For now, levels will continue to write in JSON
format.
* Add filesystem abstraction functions to the balance/ package to search
multiple paths to find Levels and Doodads, to make way for
system-level doodads.
