doodle/pkg/level/fmt_json.go

286 lines
7.2 KiB
Go
Raw Permalink Normal View History

package level
import (
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
"archive/zip"
"bytes"
"compress/gzip"
"encoding/json"
"fmt"
"io/ioutil"
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
"net/http"
2022-09-24 22:17:25 +00:00
"git.kirsle.net/SketchyMaze/doodle/pkg/balance"
Update savegame format, Allow out-of-bounds camera Updates the savegame.json file format: * Levels now have a UUID value assigned at first save. * The savegame.json will now track level completion/score based on UUID, making it robust to filename changes in either levels or levelpacks. * The savegame file is auto-migrated on startup - for any levels not found or have no UUID, no change is made, it's backwards compatible. * Level Properties window adds an "Advanced" tab to show/re-roll UUID. New JavaScript API for doodad scripts: * `Actors.CameraFollowPlayer()` tells the camera to return focus to the player character. Useful for "cutscene" doodads that freeze the player, call `Self.CameraFollowMe()` and do a thing before unfreezing and sending the camera back to the player. (Or it will follow them at their next directional input control). * `Self.MoveBy(Point(x, y int))` to move the current actor a bit. New option for the `doodad` command-line tool: * `doodad resave <.level or .doodad>` will load and re-save a drawing, to migrate it to the newest file format versions. Small tweaks: * On bounded levels, allow the camera to still follow the player if the player finds themselves WELL far out of bounds (40 pixels margin). So on bounded levels you can create "interior rooms" out-of-bounds to Warp Door into. * New wallpaper: "Atmosphere" has a black starscape pattern that fades into a solid blue atmosphere. * Camera strictly follows the player the first 20 ticks, not 60 of level start * If player is frozen, directional inputs do not take the camera focus back.
2023-03-08 05:55:10 +00:00
"git.kirsle.net/SketchyMaze/doodle/pkg/branding"
2022-09-24 22:17:25 +00:00
"git.kirsle.net/SketchyMaze/doodle/pkg/log"
"git.kirsle.net/SketchyMaze/doodle/pkg/usercfg"
Update savegame format, Allow out-of-bounds camera Updates the savegame.json file format: * Levels now have a UUID value assigned at first save. * The savegame.json will now track level completion/score based on UUID, making it robust to filename changes in either levels or levelpacks. * The savegame file is auto-migrated on startup - for any levels not found or have no UUID, no change is made, it's backwards compatible. * Level Properties window adds an "Advanced" tab to show/re-roll UUID. New JavaScript API for doodad scripts: * `Actors.CameraFollowPlayer()` tells the camera to return focus to the player character. Useful for "cutscene" doodads that freeze the player, call `Self.CameraFollowMe()` and do a thing before unfreezing and sending the camera back to the player. (Or it will follow them at their next directional input control). * `Self.MoveBy(Point(x, y int))` to move the current actor a bit. New option for the `doodad` command-line tool: * `doodad resave <.level or .doodad>` will load and re-save a drawing, to migrate it to the newest file format versions. Small tweaks: * On bounded levels, allow the camera to still follow the player if the player finds themselves WELL far out of bounds (40 pixels margin). So on bounded levels you can create "interior rooms" out-of-bounds to Warp Door into. * New wallpaper: "Atmosphere" has a black starscape pattern that fades into a solid blue atmosphere. * Camera strictly follows the player the first 20 ticks, not 60 of level start * If player is frozen, directional inputs do not take the camera focus back.
2023-03-08 05:55:10 +00:00
"github.com/google/uuid"
)
Update savegame format, Allow out-of-bounds camera Updates the savegame.json file format: * Levels now have a UUID value assigned at first save. * The savegame.json will now track level completion/score based on UUID, making it robust to filename changes in either levels or levelpacks. * The savegame file is auto-migrated on startup - for any levels not found or have no UUID, no change is made, it's backwards compatible. * Level Properties window adds an "Advanced" tab to show/re-roll UUID. New JavaScript API for doodad scripts: * `Actors.CameraFollowPlayer()` tells the camera to return focus to the player character. Useful for "cutscene" doodads that freeze the player, call `Self.CameraFollowMe()` and do a thing before unfreezing and sending the camera back to the player. (Or it will follow them at their next directional input control). * `Self.MoveBy(Point(x, y int))` to move the current actor a bit. New option for the `doodad` command-line tool: * `doodad resave <.level or .doodad>` will load and re-save a drawing, to migrate it to the newest file format versions. Small tweaks: * On bounded levels, allow the camera to still follow the player if the player finds themselves WELL far out of bounds (40 pixels margin). So on bounded levels you can create "interior rooms" out-of-bounds to Warp Door into. * New wallpaper: "Atmosphere" has a black starscape pattern that fades into a solid blue atmosphere. * Camera strictly follows the player the first 20 ticks, not 60 of level start * If player is frozen, directional inputs do not take the camera focus back.
2023-03-08 05:55:10 +00:00
/*
FromJSON loads a level from "JSON string" (gzip supported).
This is the primary "load level from file on disk" method. It can read
levels of all historical file formats the game supported:
- If the data begins with a `{` it is parsed in the legacy (v1) JSON format.
- If the level begins with a Gzip header (hex `1F8B) it is taken to be
a gzip compressed (v2) level JSON file.
- If the file is identified by `net/http#DetectContentType()` to be an
application/zip file (v3) it is loaded from zipfile format.
*/
Bindata: Embedding Doodads and Levels (for WASM) * Use `go-bindata` to embed built-in doodads and levels directly into the Doodle binary. `make bindata` produces the bindata source file. * Add `FromJSON()` method to Levels and Doodads to load objects from JSON strings in memory (for bindata built-ins or WASM ajax requests) * Update file loading functions to check the embedded bindata files. * pkg/config.go#EditFile: * Supports editing a level from bindata (TODO: remove this support) * If the "assets/levels/%(simple-name.level)" exists in bindata, edits that drawing. * No such support for editing built-in doodads. * WASM has no filesystem access to edit files except built-in levels (yet) * pkg/doodads#ListDoodads: * Prepends built-in doodads from bindata to the returned list. * WASM: no filesystem access so gets only the built-ins. * pkg/doodads#LoadFile: * Checks built-in bindata store first for doodad files. * WASM: tries an HTTP request if not found in bindata but can go no further if not found (no filesystem access) * pkg/filesystem#FindFile: * This function finds a level/doodad by checking all the places. * If the level or doodad exists in bindata built-in, always returns its system path like "assets/doodads/test.doodad" * WASM: always returns the built-in candidate path even if not found in bindata so that ajax GET can be attempted. * pkg/level#ListSystemLevels: * New function that lists the system level files, similar to the equivalent doodads function. * Prepends the bindata built-in level files. * WASM: only returns the built-ins (no filesystem support) * Desktop: also lists and returns the assets/levels/ directory. * pkg/level#LoadFile: * Like the doodads.LoadFile, tries from built-in bindata first, then ajax request (WASM) before accessing the filesystem (desktop) * Menu Scene: TODO, list the built-in levels in the Load Level menu. This feature will soon go away when WASM gets its own storage for user levels (localStorage instead of filesystem)
2019-06-27 22:07:34 +00:00
func FromJSON(filename string, data []byte) (*Level, error) {
var m = New()
// Inspect if this file is JSON or gzip compressed.
if len(data) > 0 && data[0] == '{' {
// Looks standard JSON.
err := json.Unmarshal(data, m)
if err != nil {
return nil, err
}
} else if len(data) > 1 && data[0] == 0x1f && data[1] == 0x8b {
// Gzip compressed. `1F8B` is gzip magic number.
log.Debug("Decompress level %s", filename)
if gzmap, err := FromGzip(data); err != nil {
return nil, err
} else {
m = gzmap
}
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
} else if http.DetectContentType(data) == "application/zip" {
if zipmap, err := FromZipfile(data); err != nil {
return nil, err
} else {
m = zipmap
}
} else {
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
return nil, fmt.Errorf("invalid file format")
}
Bindata: Embedding Doodads and Levels (for WASM) * Use `go-bindata` to embed built-in doodads and levels directly into the Doodle binary. `make bindata` produces the bindata source file. * Add `FromJSON()` method to Levels and Doodads to load objects from JSON strings in memory (for bindata built-ins or WASM ajax requests) * Update file loading functions to check the embedded bindata files. * pkg/config.go#EditFile: * Supports editing a level from bindata (TODO: remove this support) * If the "assets/levels/%(simple-name.level)" exists in bindata, edits that drawing. * No such support for editing built-in doodads. * WASM has no filesystem access to edit files except built-in levels (yet) * pkg/doodads#ListDoodads: * Prepends built-in doodads from bindata to the returned list. * WASM: no filesystem access so gets only the built-ins. * pkg/doodads#LoadFile: * Checks built-in bindata store first for doodad files. * WASM: tries an HTTP request if not found in bindata but can go no further if not found (no filesystem access) * pkg/filesystem#FindFile: * This function finds a level/doodad by checking all the places. * If the level or doodad exists in bindata built-in, always returns its system path like "assets/doodads/test.doodad" * WASM: always returns the built-in candidate path even if not found in bindata so that ajax GET can be attempted. * pkg/level#ListSystemLevels: * New function that lists the system level files, similar to the equivalent doodads function. * Prepends the bindata built-in level files. * WASM: only returns the built-ins (no filesystem support) * Desktop: also lists and returns the assets/levels/ directory. * pkg/level#LoadFile: * Like the doodads.LoadFile, tries from built-in bindata first, then ajax request (WASM) before accessing the filesystem (desktop) * Menu Scene: TODO, list the built-in levels in the Load Level menu. This feature will soon go away when WASM gets its own storage for user levels (localStorage instead of filesystem)
2019-06-27 22:07:34 +00:00
// Fill in defaults.
if m.Wallpaper == "" {
m.Wallpaper = DefaultWallpaper
}
// Inflate the chunk metadata to map the pixels to their palette indexes.
m.Inflate()
Bindata: Embedding Doodads and Levels (for WASM) * Use `go-bindata` to embed built-in doodads and levels directly into the Doodle binary. `make bindata` produces the bindata source file. * Add `FromJSON()` method to Levels and Doodads to load objects from JSON strings in memory (for bindata built-ins or WASM ajax requests) * Update file loading functions to check the embedded bindata files. * pkg/config.go#EditFile: * Supports editing a level from bindata (TODO: remove this support) * If the "assets/levels/%(simple-name.level)" exists in bindata, edits that drawing. * No such support for editing built-in doodads. * WASM has no filesystem access to edit files except built-in levels (yet) * pkg/doodads#ListDoodads: * Prepends built-in doodads from bindata to the returned list. * WASM: no filesystem access so gets only the built-ins. * pkg/doodads#LoadFile: * Checks built-in bindata store first for doodad files. * WASM: tries an HTTP request if not found in bindata but can go no further if not found (no filesystem access) * pkg/filesystem#FindFile: * This function finds a level/doodad by checking all the places. * If the level or doodad exists in bindata built-in, always returns its system path like "assets/doodads/test.doodad" * WASM: always returns the built-in candidate path even if not found in bindata so that ajax GET can be attempted. * pkg/level#ListSystemLevels: * New function that lists the system level files, similar to the equivalent doodads function. * Prepends the bindata built-in level files. * WASM: only returns the built-ins (no filesystem support) * Desktop: also lists and returns the assets/levels/ directory. * pkg/level#LoadFile: * Like the doodads.LoadFile, tries from built-in bindata first, then ajax request (WASM) before accessing the filesystem (desktop) * Menu Scene: TODO, list the built-in levels in the Load Level menu. This feature will soon go away when WASM gets its own storage for user levels (localStorage instead of filesystem)
2019-06-27 22:07:34 +00:00
return m, nil
Bindata: Embedding Doodads and Levels (for WASM) * Use `go-bindata` to embed built-in doodads and levels directly into the Doodle binary. `make bindata` produces the bindata source file. * Add `FromJSON()` method to Levels and Doodads to load objects from JSON strings in memory (for bindata built-ins or WASM ajax requests) * Update file loading functions to check the embedded bindata files. * pkg/config.go#EditFile: * Supports editing a level from bindata (TODO: remove this support) * If the "assets/levels/%(simple-name.level)" exists in bindata, edits that drawing. * No such support for editing built-in doodads. * WASM has no filesystem access to edit files except built-in levels (yet) * pkg/doodads#ListDoodads: * Prepends built-in doodads from bindata to the returned list. * WASM: no filesystem access so gets only the built-ins. * pkg/doodads#LoadFile: * Checks built-in bindata store first for doodad files. * WASM: tries an HTTP request if not found in bindata but can go no further if not found (no filesystem access) * pkg/filesystem#FindFile: * This function finds a level/doodad by checking all the places. * If the level or doodad exists in bindata built-in, always returns its system path like "assets/doodads/test.doodad" * WASM: always returns the built-in candidate path even if not found in bindata so that ajax GET can be attempted. * pkg/level#ListSystemLevels: * New function that lists the system level files, similar to the equivalent doodads function. * Prepends the bindata built-in level files. * WASM: only returns the built-ins (no filesystem support) * Desktop: also lists and returns the assets/levels/ directory. * pkg/level#LoadFile: * Like the doodads.LoadFile, tries from built-in bindata first, then ajax request (WASM) before accessing the filesystem (desktop) * Menu Scene: TODO, list the built-in levels in the Load Level menu. This feature will soon go away when WASM gets its own storage for user levels (localStorage instead of filesystem)
2019-06-27 22:07:34 +00:00
}
Update savegame format, Allow out-of-bounds camera Updates the savegame.json file format: * Levels now have a UUID value assigned at first save. * The savegame.json will now track level completion/score based on UUID, making it robust to filename changes in either levels or levelpacks. * The savegame file is auto-migrated on startup - for any levels not found or have no UUID, no change is made, it's backwards compatible. * Level Properties window adds an "Advanced" tab to show/re-roll UUID. New JavaScript API for doodad scripts: * `Actors.CameraFollowPlayer()` tells the camera to return focus to the player character. Useful for "cutscene" doodads that freeze the player, call `Self.CameraFollowMe()` and do a thing before unfreezing and sending the camera back to the player. (Or it will follow them at their next directional input control). * `Self.MoveBy(Point(x, y int))` to move the current actor a bit. New option for the `doodad` command-line tool: * `doodad resave <.level or .doodad>` will load and re-save a drawing, to migrate it to the newest file format versions. Small tweaks: * On bounded levels, allow the camera to still follow the player if the player finds themselves WELL far out of bounds (40 pixels margin). So on bounded levels you can create "interior rooms" out-of-bounds to Warp Door into. * New wallpaper: "Atmosphere" has a black starscape pattern that fades into a solid blue atmosphere. * Camera strictly follows the player the first 20 ticks, not 60 of level start * If player is frozen, directional inputs do not take the camera focus back.
2023-03-08 05:55:10 +00:00
/*
ToJSON serializes the level as JSON (gzip supported).
This is the primary "write level to disk" function and can output in a
vairety of historical formats controlled by pkg/balance#DrawingFormat:
- balance.FormatJSON (the default): writes the level as an original-style
single JSON document that contains all chunk data directly. These levels
take a long time to load from disk for any non-trivial level design. (v1)
- balance.FormatGzip: writes as a gzip compressed JSON file (v2)
- balance.FormatZipfile: creates a zip file where most of the level JSON
is stored as "index.json" and chunks and attached doodads are separate
members of the zipfile.
Notice about gzip: if the pkg/balance.CompressLevels boolean is true, this
function will apply gzip compression before returning the byte string.
This gzip-compressed level can be read back by any functions that say
"gzip supported" in their descriptions.
*/
func (m *Level) ToJSON() ([]byte, error) {
// Gzip compressing?
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
if balance.DrawingFormat == balance.FormatGZip {
return m.ToGzip()
}
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
// Zipfile?
if balance.DrawingFormat == balance.FormatZipfile {
return m.ToZipfile()
}
return m.AsJSON()
}
// AsJSON returns it just as JSON without any fancy gzip/zip magic.
func (m *Level) AsJSON() ([]byte, error) {
Update savegame format, Allow out-of-bounds camera Updates the savegame.json file format: * Levels now have a UUID value assigned at first save. * The savegame.json will now track level completion/score based on UUID, making it robust to filename changes in either levels or levelpacks. * The savegame file is auto-migrated on startup - for any levels not found or have no UUID, no change is made, it's backwards compatible. * Level Properties window adds an "Advanced" tab to show/re-roll UUID. New JavaScript API for doodad scripts: * `Actors.CameraFollowPlayer()` tells the camera to return focus to the player character. Useful for "cutscene" doodads that freeze the player, call `Self.CameraFollowMe()` and do a thing before unfreezing and sending the camera back to the player. (Or it will follow them at their next directional input control). * `Self.MoveBy(Point(x, y int))` to move the current actor a bit. New option for the `doodad` command-line tool: * `doodad resave <.level or .doodad>` will load and re-save a drawing, to migrate it to the newest file format versions. Small tweaks: * On bounded levels, allow the camera to still follow the player if the player finds themselves WELL far out of bounds (40 pixels margin). So on bounded levels you can create "interior rooms" out-of-bounds to Warp Door into. * New wallpaper: "Atmosphere" has a black starscape pattern that fades into a solid blue atmosphere. * Camera strictly follows the player the first 20 ticks, not 60 of level start * If player is frozen, directional inputs do not take the camera focus back.
2023-03-08 05:55:10 +00:00
// Always write the game version and ensure levels have a UUID set.
m.GameVersion = branding.Version
if m.UUID == "" {
m.UUID = uuid.New().String()
log.Info("Note: assigned new level UUID %s", m.UUID)
}
out := bytes.NewBuffer([]byte{})
encoder := json.NewEncoder(out)
if usercfg.Current.JSONIndent {
encoder.SetIndent("", "\t")
}
err := encoder.Encode(m)
return out.Bytes(), err
}
// ToGzip serializes the level as gzip compressed JSON.
func (m *Level) ToGzip() ([]byte, error) {
var (
handle = bytes.NewBuffer([]byte{})
zipper = gzip.NewWriter(handle)
encoder = json.NewEncoder(zipper)
)
if err := encoder.Encode(m); err != nil {
return nil, err
}
err := zipper.Close()
return handle.Bytes(), err
}
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
// ToZipfile serializes the level as a ZIP archive and also migrates
// data loaded from an older save into the new zip format.
func (m *Level) ToZipfile() ([]byte, error) {
// If we do not have a Zipfile yet, migrate legacy data into one.
// if m.Zipfile == nil {
fh := bytes.NewBuffer([]byte{})
zipper := zip.NewWriter(fh)
defer zipper.Close()
// Migrate any legacy Chunker data into external files in the zip.
if err := m.Chunker.MigrateZipfile(zipper); err != nil {
return nil, fmt.Errorf("MigrateZipfile: %s", err)
}
// Migrate attached files to ZIP.
if err := m.Files.MigrateZipfile(zipper); err != nil {
return nil, fmt.Errorf("FileSystem.MigrateZipfile: %s", err)
}
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
// Write the header json.
{
header, err := m.AsJSON()
if err != nil {
return nil, err
}
writer, err := zipper.Create("level.json")
if err != nil {
return nil, fmt.Errorf("zipping index.js: %s", err)
}
if n, err := writer.Write(header); err != nil {
return nil, err
} else {
2023-12-02 20:33:14 +00:00
log.Debug("Written level.json to zipfile: %d bytes", n)
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
}
}
zipper.Close()
// Refresh our Zipfile reader from the zipper we just wrote.
bin := fh.Bytes()
if err := m.ReloadZipfile(bin); err != nil {
log.Error("ReloadZipfile: %s", err)
}
return fh.Bytes(), nil
}
// FromGzip deserializes a gzip compressed level JSON.
func FromGzip(data []byte) (*Level, error) {
// This function works, do not touch.
var (
level = New()
buf = bytes.NewBuffer(data)
reader *gzip.Reader
decoder *json.Decoder
)
reader, err := gzip.NewReader(buf)
if err != nil {
return nil, err
}
decoder = json.NewDecoder(reader)
decoder.Decode(level)
Wallpapers and Bounded Levels Implement the Wallpaper system into the levels and the concept of Bounded and Unbounded levels. The first wallpaper image is notepad.png which looks like standard ruled notebook paper. On bounded levels, the top/left edges of the page look as you would expect and the blue lines tile indefinitely in the positive directions. On unbounded levels, you only get the repeating blue lines but not the edge pieces. A wallpaper is just a rectangular image file. The image is divided into four equal quadrants to be the Corner, Top, Left and Repeat textures for the wallpaper. The Repeat texture is ALWAYS used and fills all the empty space behind the drawing. (Doodads draw with blank canvases as before because only levels have wallpapers!) Levels have four options of a "Page Type": - Unbounded (default, infinite space) - NoNegativeSpace (has a top left edge but can grow infinitely) - Bounded (has a top left edge and bounded size) - Bordered (bounded with bordered texture; NOT IMPLEMENTED!) The scrollable viewport of a Canvas will respect the wallpaper and page type settings of a Level loaded into it. That is, if the level has a top left edge (not Unbounded) you can NOT scroll to see negative coordinates below (0,0) -- and if the level has a max dimension set, you can't scroll to see pixels outside those dimensions. The Canvas property NoLimitScroll=true will override the scroll locking and let you see outside the bounds, for debugging. - Default map settings for New Level are now: - Page Type: NoNegativeSpace - Wallpaper: notepad.png (default) - MaxWidth: 2550 (8.5" * 300 ppi) - MaxHeight: 3300 ( 11" * 300 ppi)
2018-10-28 05:22:13 +00:00
return level, nil
}
Implement Chunk System for Pixel Data Starts the implementation of the chunk-based pixel storage system for levels and drawings. Previously the levels had a Pixels structure which was just an array of X,Y and palette index triplets. The new chunk system divides the map up into square chunks, and lets each chunk manage its own memory layout. The "MapAccessor" layout is implemented first which is a map of X,Y coordinates to their Swatches (pointer to an index of the palette). When serialized the MapAccessor maps the "X,Y": "index" similarly to the old Pixels array. The object hierarchy for the chunk system is like: * Chunker: the manager of the chunks who keeps track of the ChunkSize and a map of "chunk coordinates" to the chunk in charge of it. * Chunk: a part of the drawing ChunkSize length square. A chunk has a Type (of how it stores its data, 0 being a map[Point]Swatch and 1 being a [][]Swatch 2D array), and the chunk has an Accessor which implements the underlying type. * Accessor: an interface for a Chunk to provide access to its pixels. * MapAccessor: a "sparse map" of coordinates to their Swatches. * GridAccessor: TBD, will be a "dense" 2D grid of Swatches. The JSON files are loaded in two passes: 1. The chunks only load their swatch indexes from disk. 2. With the palette also loaded, the chunks are "inflated" and linked to their swatch pointers. Misc changes: * The `level.Canvas` UI widget switches from the old Grid data type to being able to directly use a `level.Chunker` * The Chunker is a shared data type between the on-disk level format and the actual renderer (level.Canvas), so saving the level is easy because you can just pull the Chunker out from the canvas. * ChunkSize is stored inside the level file and the default value is at balance/numbers.go: 1000
2018-09-23 22:20:45 +00:00
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
// FromZipfile reads a level in zipfile format.
func FromZipfile(data []byte) (*Level, error) {
var (
level = New()
err = level.populateFromZipfile(data)
)
return level, err
}
// ReloadZipfile re-reads the level's zipfile after a write.
func (m *Level) ReloadZipfile(data []byte) error {
return m.populateFromZipfile(data)
}
// Common function between FromZipfile and ReloadZipFile.
func (m *Level) populateFromZipfile(data []byte) error {
var (
buf = bytes.NewReader(data)
zf *zip.Reader
decoder *json.Decoder
)
zf, err := zip.NewReader(buf, buf.Size())
if err != nil {
return err
}
// Read the level.json.
file, err := zf.Open("level.json")
if err != nil {
return err
}
decoder = json.NewDecoder(file)
err = decoder.Decode(m)
// Keep the zipfile reader handy.
m.Zipfile = zf
m.Chunker.Zipfile = zf
m.Files.Zipfile = zf
// Re-inflate the level: ensures Actor instances get their IDs
// and everything is reloaded after saving the level.
m.Inflate()
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
return err
}
// LoadJSON loads a map from JSON file (gzip supported).
func LoadJSON(filename string) (*Level, error) {
data, err := ioutil.ReadFile(filename)
if err != nil {
return nil, err
}
return FromJSON(filename, data)
}
// WriteJSON writes a level to JSON on disk.
func (m *Level) WriteJSON(filename string) error {
json, err := m.ToJSON()
if err != nil {
return fmt.Errorf("Level.WriteJSON: JSON encode error: %s", err)
}
err = ioutil.WriteFile(filename, json, 0755)
if err != nil {
return fmt.Errorf("Level.WriteJSON: WriteFile error: %s", err)
}
return nil
}
Zipfiles as File Format for Levels and Doodads Especially to further optimize memory for large levels, Levels and Doodads can now read and write to a ZIP file format on disk with chunks in external files within the zip. Existing doodads and levels can still load as normal, and will be converted into ZIP files on the next save: * The Chunker.ChunkMap which used to hold ALL chunks in the main json/gz file, now becomes the cache of "hot chunks" loaded from ZIP. If there is a ZIP file, chunks not accessed recently are flushed from the ChunkMap to save on memory. * During save, the ChunkMap is flushed to ZIP along with any non-loaded chunks from a previous zipfile. So legacy levels "just work" when saving, and levels loaded FROM Zip will manage their ChunkMap hot memory more carefully. Memory savings observed on "Azulian Tag - Forest.level": * Before: 1716 MB was loaded from the old level format into RAM along with a slow load screen. * After: only 243 MB memory was used by the game and it loaded with a VERY FAST load screen. Updates to the F3 Debug Overlay: * "Chunks: 20 in 45 out 20 cached" shows the count of chunks inside the viewport (having bitmaps and textures loaded) vs. chunks outside which have their textures freed (but data kept), and the number of chunks currently hot cached in the ChunkMap. The `doodad` tool has new commands to "touch" your existing levels and doodads, to upgrade them to the new format (or you can simply open and re-save them in-game): doodad edit-level --touch ./example.level doodad edit-doodad --touch ./example.doodad The output from that and `doodad show` should say "File format: zipfile" in the headers section. To do: * File attachments should also go in as ZIP files, e.g. wallpapers
2022-04-30 03:34:59 +00:00
// Loop may be called each loop to allow the level to maintain its
// memory usage, e.g., for chunks not requested recently from a zipfile
// level to free those from RAM.
func (m *Level) Loop() error {
m.Chunker.FreeCaches()
return nil
}