doodle/pkg/level/fmt_readwrite.go

128 lines
3.0 KiB
Go
Raw Permalink Normal View History

package level
import (
"fmt"
"io/ioutil"
(Experimental) Run Length Encoding for Levels Finally add a second option for Chunk MapAccessor implementation besides the MapAccessor. The RLEAccessor is basically a MapAccessor that will compress your drawing with Run Length Encoding (RLE) in the on-disk format in the ZIP file. This slashes the file sizes of most levels: * Shapeshifter: 21.8 MB -> 8.1 MB * Jungle: 10.4 MB -> 4.1 MB * Zoo: 2.8 MB -> 1.3 MB Implementation details: * The RLE binary format for Chunks is a stream of Uvarint pairs storing the palette index number and the number of pixels to repeat it (along the Y,X axis of the chunk). * Null colors are represented by a Uvarint that decodes to 0xFFFF or 65535 in decimal. * Gameplay logic currently limits maps to 256 colors. * The default for newly created chunks in-game will be RLE by default. * Its in-memory representation is still a MapAccessor (a map of absolute world coordinates to palette index). * The game can still open and play legacy MapAccessor maps. * On save in the editor, the game will upgrade/convert MapAccessor chunks over to RLEAccessors, improving on your level's file size with a simple re-save. Current Bugs * On every re-save to RLE, one pixel is lost in the bottom-right corner of each chunk. Each subsequent re-save loses one more pixel to the left, so what starts as a single pixel per chunk slowly evolves into a horizontal line. * Some pixels smear vertically as well. * Off-by-negative-one errors when some chunks Iter() their pixels but compute a relative coordinate of (-1,0)! Some mismatch between the stored world coords of a pixel inside the chunk vs. the chunk's assigned coordinate by the Chunker: certain combinations of chunk coord/abs coord. To Do * The `doodad touch` command should re-save existing levels to upgrade them.
2024-05-24 06:02:01 +00:00
"os"
Bindata: Embedding Doodads and Levels (for WASM) * Use `go-bindata` to embed built-in doodads and levels directly into the Doodle binary. `make bindata` produces the bindata source file. * Add `FromJSON()` method to Levels and Doodads to load objects from JSON strings in memory (for bindata built-ins or WASM ajax requests) * Update file loading functions to check the embedded bindata files. * pkg/config.go#EditFile: * Supports editing a level from bindata (TODO: remove this support) * If the "assets/levels/%(simple-name.level)" exists in bindata, edits that drawing. * No such support for editing built-in doodads. * WASM has no filesystem access to edit files except built-in levels (yet) * pkg/doodads#ListDoodads: * Prepends built-in doodads from bindata to the returned list. * WASM: no filesystem access so gets only the built-ins. * pkg/doodads#LoadFile: * Checks built-in bindata store first for doodad files. * WASM: tries an HTTP request if not found in bindata but can go no further if not found (no filesystem access) * pkg/filesystem#FindFile: * This function finds a level/doodad by checking all the places. * If the level or doodad exists in bindata built-in, always returns its system path like "assets/doodads/test.doodad" * WASM: always returns the built-in candidate path even if not found in bindata so that ajax GET can be attempted. * pkg/level#ListSystemLevels: * New function that lists the system level files, similar to the equivalent doodads function. * Prepends the bindata built-in level files. * WASM: only returns the built-ins (no filesystem support) * Desktop: also lists and returns the assets/levels/ directory. * pkg/level#LoadFile: * Like the doodads.LoadFile, tries from built-in bindata first, then ajax request (WASM) before accessing the filesystem (desktop) * Menu Scene: TODO, list the built-in levels in the Load Level menu. This feature will soon go away when WASM gets its own storage for user levels (localStorage instead of filesystem)
2019-06-27 22:07:34 +00:00
"runtime"
"strings"
2022-09-24 22:17:25 +00:00
"git.kirsle.net/SketchyMaze/doodle/assets"
"git.kirsle.net/SketchyMaze/doodle/pkg/branding"
"git.kirsle.net/SketchyMaze/doodle/pkg/enum"
"git.kirsle.net/SketchyMaze/doodle/pkg/filesystem"
"git.kirsle.net/SketchyMaze/doodle/pkg/log"
"git.kirsle.net/SketchyMaze/doodle/pkg/userdir"
"git.kirsle.net/SketchyMaze/doodle/pkg/wasm"
)
Bindata: Embedding Doodads and Levels (for WASM) * Use `go-bindata` to embed built-in doodads and levels directly into the Doodle binary. `make bindata` produces the bindata source file. * Add `FromJSON()` method to Levels and Doodads to load objects from JSON strings in memory (for bindata built-ins or WASM ajax requests) * Update file loading functions to check the embedded bindata files. * pkg/config.go#EditFile: * Supports editing a level from bindata (TODO: remove this support) * If the "assets/levels/%(simple-name.level)" exists in bindata, edits that drawing. * No such support for editing built-in doodads. * WASM has no filesystem access to edit files except built-in levels (yet) * pkg/doodads#ListDoodads: * Prepends built-in doodads from bindata to the returned list. * WASM: no filesystem access so gets only the built-ins. * pkg/doodads#LoadFile: * Checks built-in bindata store first for doodad files. * WASM: tries an HTTP request if not found in bindata but can go no further if not found (no filesystem access) * pkg/filesystem#FindFile: * This function finds a level/doodad by checking all the places. * If the level or doodad exists in bindata built-in, always returns its system path like "assets/doodads/test.doodad" * WASM: always returns the built-in candidate path even if not found in bindata so that ajax GET can be attempted. * pkg/level#ListSystemLevels: * New function that lists the system level files, similar to the equivalent doodads function. * Prepends the bindata built-in level files. * WASM: only returns the built-ins (no filesystem support) * Desktop: also lists and returns the assets/levels/ directory. * pkg/level#LoadFile: * Like the doodads.LoadFile, tries from built-in bindata first, then ajax request (WASM) before accessing the filesystem (desktop) * Menu Scene: TODO, list the built-in levels in the Load Level menu. This feature will soon go away when WASM gets its own storage for user levels (localStorage instead of filesystem)
2019-06-27 22:07:34 +00:00
// ListSystemLevels returns a list of built-in levels.
func ListSystemLevels() ([]string, error) {
var names = []string{}
// Add the levels embedded inside the binary.
if levels, err := assets.AssetDir("assets/levels"); err == nil {
Bindata: Embedding Doodads and Levels (for WASM) * Use `go-bindata` to embed built-in doodads and levels directly into the Doodle binary. `make bindata` produces the bindata source file. * Add `FromJSON()` method to Levels and Doodads to load objects from JSON strings in memory (for bindata built-ins or WASM ajax requests) * Update file loading functions to check the embedded bindata files. * pkg/config.go#EditFile: * Supports editing a level from bindata (TODO: remove this support) * If the "assets/levels/%(simple-name.level)" exists in bindata, edits that drawing. * No such support for editing built-in doodads. * WASM has no filesystem access to edit files except built-in levels (yet) * pkg/doodads#ListDoodads: * Prepends built-in doodads from bindata to the returned list. * WASM: no filesystem access so gets only the built-ins. * pkg/doodads#LoadFile: * Checks built-in bindata store first for doodad files. * WASM: tries an HTTP request if not found in bindata but can go no further if not found (no filesystem access) * pkg/filesystem#FindFile: * This function finds a level/doodad by checking all the places. * If the level or doodad exists in bindata built-in, always returns its system path like "assets/doodads/test.doodad" * WASM: always returns the built-in candidate path even if not found in bindata so that ajax GET can be attempted. * pkg/level#ListSystemLevels: * New function that lists the system level files, similar to the equivalent doodads function. * Prepends the bindata built-in level files. * WASM: only returns the built-ins (no filesystem support) * Desktop: also lists and returns the assets/levels/ directory. * pkg/level#LoadFile: * Like the doodads.LoadFile, tries from built-in bindata first, then ajax request (WASM) before accessing the filesystem (desktop) * Menu Scene: TODO, list the built-in levels in the Load Level menu. This feature will soon go away when WASM gets its own storage for user levels (localStorage instead of filesystem)
2019-06-27 22:07:34 +00:00
names = append(names, levels...)
}
// WASM
if runtime.GOOS == "js" {
// Return just the embedded ones, no filesystem access.
return names, nil
}
// Read filesystem for system levels.
files, err := ioutil.ReadDir(filesystem.SystemLevelsPath)
for _, file := range files {
name := file.Name()
if strings.HasSuffix(strings.ToLower(name), enum.DoodadExt) {
names = append(names, name)
}
}
return names, err
}
// LoadFile reads a level file from disk, checking a few locations.
func LoadFile(filename string) (*Level, error) {
if !strings.HasSuffix(filename, enum.LevelExt) {
filename += enum.LevelExt
}
// Search the system and user paths for this level.
filename, err := filesystem.FindFile(filename)
if err != nil {
return nil, err
}
Bindata: Embedding Doodads and Levels (for WASM) * Use `go-bindata` to embed built-in doodads and levels directly into the Doodle binary. `make bindata` produces the bindata source file. * Add `FromJSON()` method to Levels and Doodads to load objects from JSON strings in memory (for bindata built-ins or WASM ajax requests) * Update file loading functions to check the embedded bindata files. * pkg/config.go#EditFile: * Supports editing a level from bindata (TODO: remove this support) * If the "assets/levels/%(simple-name.level)" exists in bindata, edits that drawing. * No such support for editing built-in doodads. * WASM has no filesystem access to edit files except built-in levels (yet) * pkg/doodads#ListDoodads: * Prepends built-in doodads from bindata to the returned list. * WASM: no filesystem access so gets only the built-ins. * pkg/doodads#LoadFile: * Checks built-in bindata store first for doodad files. * WASM: tries an HTTP request if not found in bindata but can go no further if not found (no filesystem access) * pkg/filesystem#FindFile: * This function finds a level/doodad by checking all the places. * If the level or doodad exists in bindata built-in, always returns its system path like "assets/doodads/test.doodad" * WASM: always returns the built-in candidate path even if not found in bindata so that ajax GET can be attempted. * pkg/level#ListSystemLevels: * New function that lists the system level files, similar to the equivalent doodads function. * Prepends the bindata built-in level files. * WASM: only returns the built-ins (no filesystem support) * Desktop: also lists and returns the assets/levels/ directory. * pkg/level#LoadFile: * Like the doodads.LoadFile, tries from built-in bindata first, then ajax request (WASM) before accessing the filesystem (desktop) * Menu Scene: TODO, list the built-in levels in the Load Level menu. This feature will soon go away when WASM gets its own storage for user levels (localStorage instead of filesystem)
2019-06-27 22:07:34 +00:00
// Do we have the file in bindata?
if jsonData, err := assets.Asset(filename); err == nil {
log.Debug("Level %s: loaded from embedded bindata", filename)
Bindata: Embedding Doodads and Levels (for WASM) * Use `go-bindata` to embed built-in doodads and levels directly into the Doodle binary. `make bindata` produces the bindata source file. * Add `FromJSON()` method to Levels and Doodads to load objects from JSON strings in memory (for bindata built-ins or WASM ajax requests) * Update file loading functions to check the embedded bindata files. * pkg/config.go#EditFile: * Supports editing a level from bindata (TODO: remove this support) * If the "assets/levels/%(simple-name.level)" exists in bindata, edits that drawing. * No such support for editing built-in doodads. * WASM has no filesystem access to edit files except built-in levels (yet) * pkg/doodads#ListDoodads: * Prepends built-in doodads from bindata to the returned list. * WASM: no filesystem access so gets only the built-ins. * pkg/doodads#LoadFile: * Checks built-in bindata store first for doodad files. * WASM: tries an HTTP request if not found in bindata but can go no further if not found (no filesystem access) * pkg/filesystem#FindFile: * This function finds a level/doodad by checking all the places. * If the level or doodad exists in bindata built-in, always returns its system path like "assets/doodads/test.doodad" * WASM: always returns the built-in candidate path even if not found in bindata so that ajax GET can be attempted. * pkg/level#ListSystemLevels: * New function that lists the system level files, similar to the equivalent doodads function. * Prepends the bindata built-in level files. * WASM: only returns the built-ins (no filesystem support) * Desktop: also lists and returns the assets/levels/ directory. * pkg/level#LoadFile: * Like the doodads.LoadFile, tries from built-in bindata first, then ajax request (WASM) before accessing the filesystem (desktop) * Menu Scene: TODO, list the built-in levels in the Load Level menu. This feature will soon go away when WASM gets its own storage for user levels (localStorage instead of filesystem)
2019-06-27 22:07:34 +00:00
return FromJSON(filename, jsonData)
}
// WASM: try the file from localStorage or HTTP ajax request.
Bindata: Embedding Doodads and Levels (for WASM) * Use `go-bindata` to embed built-in doodads and levels directly into the Doodle binary. `make bindata` produces the bindata source file. * Add `FromJSON()` method to Levels and Doodads to load objects from JSON strings in memory (for bindata built-ins or WASM ajax requests) * Update file loading functions to check the embedded bindata files. * pkg/config.go#EditFile: * Supports editing a level from bindata (TODO: remove this support) * If the "assets/levels/%(simple-name.level)" exists in bindata, edits that drawing. * No such support for editing built-in doodads. * WASM has no filesystem access to edit files except built-in levels (yet) * pkg/doodads#ListDoodads: * Prepends built-in doodads from bindata to the returned list. * WASM: no filesystem access so gets only the built-ins. * pkg/doodads#LoadFile: * Checks built-in bindata store first for doodad files. * WASM: tries an HTTP request if not found in bindata but can go no further if not found (no filesystem access) * pkg/filesystem#FindFile: * This function finds a level/doodad by checking all the places. * If the level or doodad exists in bindata built-in, always returns its system path like "assets/doodads/test.doodad" * WASM: always returns the built-in candidate path even if not found in bindata so that ajax GET can be attempted. * pkg/level#ListSystemLevels: * New function that lists the system level files, similar to the equivalent doodads function. * Prepends the bindata built-in level files. * WASM: only returns the built-ins (no filesystem support) * Desktop: also lists and returns the assets/levels/ directory. * pkg/level#LoadFile: * Like the doodads.LoadFile, tries from built-in bindata first, then ajax request (WASM) before accessing the filesystem (desktop) * Menu Scene: TODO, list the built-in levels in the Load Level menu. This feature will soon go away when WASM gets its own storage for user levels (localStorage instead of filesystem)
2019-06-27 22:07:34 +00:00
if runtime.GOOS == "js" {
if result, ok := wasm.GetSession(filename); ok {
log.Info("recall level data from localStorage")
return FromJSON(filename, []byte(result))
}
// Ajax request.
Bindata: Embedding Doodads and Levels (for WASM) * Use `go-bindata` to embed built-in doodads and levels directly into the Doodle binary. `make bindata` produces the bindata source file. * Add `FromJSON()` method to Levels and Doodads to load objects from JSON strings in memory (for bindata built-ins or WASM ajax requests) * Update file loading functions to check the embedded bindata files. * pkg/config.go#EditFile: * Supports editing a level from bindata (TODO: remove this support) * If the "assets/levels/%(simple-name.level)" exists in bindata, edits that drawing. * No such support for editing built-in doodads. * WASM has no filesystem access to edit files except built-in levels (yet) * pkg/doodads#ListDoodads: * Prepends built-in doodads from bindata to the returned list. * WASM: no filesystem access so gets only the built-ins. * pkg/doodads#LoadFile: * Checks built-in bindata store first for doodad files. * WASM: tries an HTTP request if not found in bindata but can go no further if not found (no filesystem access) * pkg/filesystem#FindFile: * This function finds a level/doodad by checking all the places. * If the level or doodad exists in bindata built-in, always returns its system path like "assets/doodads/test.doodad" * WASM: always returns the built-in candidate path even if not found in bindata so that ajax GET can be attempted. * pkg/level#ListSystemLevels: * New function that lists the system level files, similar to the equivalent doodads function. * Prepends the bindata built-in level files. * WASM: only returns the built-ins (no filesystem support) * Desktop: also lists and returns the assets/levels/ directory. * pkg/level#LoadFile: * Like the doodads.LoadFile, tries from built-in bindata first, then ajax request (WASM) before accessing the filesystem (desktop) * Menu Scene: TODO, list the built-in levels in the Load Level menu. This feature will soon go away when WASM gets its own storage for user levels (localStorage instead of filesystem)
2019-06-27 22:07:34 +00:00
jsonData, err := wasm.HTTPGet(filename)
if err != nil {
return nil, err
}
return FromJSON(filename, jsonData)
}
// Load as JSON.
if level, err := LoadJSON(filename); err == nil {
return level, nil
} else {
log.Warn(err.Error())
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
return nil, err
}
}
// WriteFile saves a level to disk in the user's config directory.
func (m *Level) WriteFile(filename string) error {
if !strings.HasSuffix(filename, enum.LevelExt) {
filename += enum.LevelExt
}
// Set the version information.
m.Version = 1
m.GameVersion = branding.Version
// Maintenance functions, clean up cruft before save.
(Experimental) Run Length Encoding for Levels Finally add a second option for Chunk MapAccessor implementation besides the MapAccessor. The RLEAccessor is basically a MapAccessor that will compress your drawing with Run Length Encoding (RLE) in the on-disk format in the ZIP file. This slashes the file sizes of most levels: * Shapeshifter: 21.8 MB -> 8.1 MB * Jungle: 10.4 MB -> 4.1 MB * Zoo: 2.8 MB -> 1.3 MB Implementation details: * The RLE binary format for Chunks is a stream of Uvarint pairs storing the palette index number and the number of pixels to repeat it (along the Y,X axis of the chunk). * Null colors are represented by a Uvarint that decodes to 0xFFFF or 65535 in decimal. * Gameplay logic currently limits maps to 256 colors. * The default for newly created chunks in-game will be RLE by default. * Its in-memory representation is still a MapAccessor (a map of absolute world coordinates to palette index). * The game can still open and play legacy MapAccessor maps. * On save in the editor, the game will upgrade/convert MapAccessor chunks over to RLEAccessors, improving on your level's file size with a simple re-save. Current Bugs * On every re-save to RLE, one pixel is lost in the bottom-right corner of each chunk. Each subsequent re-save loses one more pixel to the left, so what starts as a single pixel per chunk slowly evolves into a horizontal line. * Some pixels smear vertically as well. * Off-by-negative-one errors when some chunks Iter() their pixels but compute a relative coordinate of (-1,0)! Some mismatch between the stored world coords of a pixel inside the chunk vs. the chunk's assigned coordinate by the Chunker: certain combinations of chunk coord/abs coord. To Do * The `doodad touch` command should re-save existing levels to upgrade them.
2024-05-24 06:02:01 +00:00
if err := m.Vacuum(); err != nil {
log.Error("Vacuum level %s: %s", filename, err)
}
bin, err := m.ToJSON()
if err != nil {
return err
}
// Save it to their profile directory.
filename = userdir.LevelPath(filename)
log.Info("Write Level: %s", filename)
// WASM: place in localStorage.
if runtime.GOOS == "js" {
log.Info("wasm: write %s to localStorage", filename)
wasm.SetSession(filename, string(bin))
return nil
}
// Desktop: write to disk.
(Experimental) Run Length Encoding for Levels Finally add a second option for Chunk MapAccessor implementation besides the MapAccessor. The RLEAccessor is basically a MapAccessor that will compress your drawing with Run Length Encoding (RLE) in the on-disk format in the ZIP file. This slashes the file sizes of most levels: * Shapeshifter: 21.8 MB -> 8.1 MB * Jungle: 10.4 MB -> 4.1 MB * Zoo: 2.8 MB -> 1.3 MB Implementation details: * The RLE binary format for Chunks is a stream of Uvarint pairs storing the palette index number and the number of pixels to repeat it (along the Y,X axis of the chunk). * Null colors are represented by a Uvarint that decodes to 0xFFFF or 65535 in decimal. * Gameplay logic currently limits maps to 256 colors. * The default for newly created chunks in-game will be RLE by default. * Its in-memory representation is still a MapAccessor (a map of absolute world coordinates to palette index). * The game can still open and play legacy MapAccessor maps. * On save in the editor, the game will upgrade/convert MapAccessor chunks over to RLEAccessors, improving on your level's file size with a simple re-save. Current Bugs * On every re-save to RLE, one pixel is lost in the bottom-right corner of each chunk. Each subsequent re-save loses one more pixel to the left, so what starts as a single pixel per chunk slowly evolves into a horizontal line. * Some pixels smear vertically as well. * Off-by-negative-one errors when some chunks Iter() their pixels but compute a relative coordinate of (-1,0)! Some mismatch between the stored world coords of a pixel inside the chunk vs. the chunk's assigned coordinate by the Chunker: certain combinations of chunk coord/abs coord. To Do * The `doodad touch` command should re-save existing levels to upgrade them.
2024-05-24 06:02:01 +00:00
err = os.WriteFile(filename, bin, 0644)
if err != nil {
return fmt.Errorf("level.WriteFile: %s", err)
}
return nil
}