Hey all. It’s been half a year since I’ve written anything. Last time I wrote about lowering emulator fixed overheads. Since then I have started a new project. I didn’t want to get into Godot because I like writing my own game engines, however I have a friend who is just too good at pixel art to make yet another unpublishable game. Not that my engines suck, it’s just that the effort going into modern game engines is just unbelievable. As an example here is a single PR to Godot with 118k changes. Since I can’t clone myself, I’d rather just get into an engine. But how to do drudgery? It turns out you can stomach some of it by creating a problem and then presenting it as the solution.
I present.
The Godot Sandbox. It’s a safe way to run untrusted code in Godot. It’s backed by my RISC-V emulator library libriscv, and it turned out to be a good choice: It supports every platform Godot can be exported to, as it only depends on the C++ standard library. The GIF above is from the playable demo project running in a browser.
It’s something I have become proud of, I would say. It’s gotten way bigger than I ever intended it to be. Every step of the way there’s a (new) problem, and my community keeps letting me know what I need to do to make it more accessible. My daily driver is the Linux-terminal, practically. For someone who lives and breathes Windows (or Wintendo, as I jokingly call it), it’s not easy to figure out how exactly you install a RISC-V toolchain and use that to build a program. I get that, but I am still struggling with the fact that I want Godot Sandbox to be the best, and to not compromise too much on my vision. For example, in order to build a complex project, you really do want a CMake build system for C++. So, I think that’s always going to be a thing — how else are you going to add external dependencies of your own? But, it is still possible to build simple programs that just uses the sandbox API.
EDIT: I discovered that Zig can build full Linux programs on any platform, and we’re experimenting with that now. Windows. macOS.
The Sandbox API
The sandbox exposes an API to sandboxed programs that let them use Godot in the same way that GDScript does, as well as godot-cpp. Many of those joining our discord seem surprised when we tell them that you don’t need to install SDL2. Just create and manipulate Godot nodes directly, as if in GDScript, except with that extra oomph.
#include "api.hpp"
static int coins = 0;
PUBLIC Variant reset_game()
{
coins = 0;
return Nil;
}
static void add_coin(const Node &player)
{
coins++;
// In our demo project we can access the coin label from the player
// using a node path: Player -> get_parent() -> Texts -> CoinLabel
Label coinlabel = player.get_node<Label>("../Texts/CoinLabel");
coinlabel.set_text("You have collected " + std::to_string(coins) + ((coins == 1) ? " coin" : " coins"));
}
PUBLIC Variant _on_body_entered(CharacterBody2D body)
{
// This function is called when a body enters the coin
// Most likely it's the player, but we still check!
if (body.get_name() != "Player")
return Nil;
Node coin = get_node();
if (coin.get_meta("secret", false))
{
print("Was a secret!");
Node2D scene = load<PackedScene>("res://scenes/mod/mod.tscn").instantiate();
get_node("../..").add_child(scene);
scene.set_name("mod");
scene.set_position(Vector2(250, 180));
}
coin.queue_free(); // Remove the current coin!
add_coin(body);
return Nil;
}
PUBLIC Variant _ready()
{
if (is_editor_hint())
{
get_node().set_process_input(false);
}
return Nil; //
}
PUBLIC Variant _process(double delta)
{
if (is_editor_hint())
{
AnimatedSprite2D sprite("AnimatedSprite2D");
sprite.play("idle");
sprite.set_speed_scale(1.0f);
}
return Nil;
}
PUBLIC Variant _input(InputEvent event)
{
if (event.is_action_pressed("jump"))
{
get_node<Node2D>().set_modulate(0xFF6060FF);
}
else if (event.is_action_released("jump"))
{
get_node<Node2D>().set_modulate(0xFFFFFFFF);
}
return Nil;
}Above is an example used in the demo project. As you can see, one of the coins hold a secret. One of the things I still haven’t been able to solve fully is return values. For now, we must return a Variant, always. But, it works alright.
You might have noticed that certain Godot classes are being used directly in the sandbox program, like InputEvent. That is the second part of the API which I call the run-time generated API. Even GD extensions that you load at run-time can be exposed in that API, as it is generated at run-time. As an example, you can instantiate sandboxes too:
Sandbox s = ClassDB::instantiate<Sandbox>("Sandbox");
print(s.get_general_registers()); // Print CPU register state
s.queue_free();libriscv
A RISC-V emulator written by me. You might have seen it or the scripting with C++ series. Honestly, the more I’ve been doing it, the more I enjoy what I’m doing. Do not think that this means I think that you should be using libriscv to sandbox your engine, or something. I think that you should consider using a systems language in a sandbox, as a viable scripting solution. The RISC-V emulator can easily be 5x faster than GDScript in interpreter mode. Up to 50x faster with binary translation.
If you have an interpreter and it’s easy to run CoreMark on it, just throw it my way and I will run it on the same hardware as shown. I’ll be happy to add it.
Godot Sandbox
Godot supports plugins, simply called GDExtensions.
In the screenshot I have a Sandbox node that I’ve added on my mobile phones Godot editor, and loaded it with a RISC-V ELF program. It’s compiled from the C++ shown in the editor, and to the right you can see some properties shared between all Sandbox nodes. Max references, max memory, execution timeout, max (heap) allocations etc. Resource consumption can be constrained (or loosened) depending on needs. If this all sounds interesting, have a look at the docs.
We built a re-implementation of a 2023 SIGGRAPH paper called Robust Skin Weights Transfer via Weight Inpainting, and we discovered that it (our implementation) would quickly run afoul of the heap allocation limit. Digging more into it, we found that the function point_mesh_squared_distance from libigl, created 100k allocations for a sphere mesh against a bone mesh we had. That’s right: 100 000 heap allocations, 20 000 of them concurrently. It’s called only twice. How can one make that run fast? I still don’t know the answer to that. But anyway, it’s been fun trying to build large complex programs and sandboxing them in Godot. If interested you can find a cache of sandbox programs here. They’re built automatically by CI and uploaded to a draft release. Introspection is key, and remote debugging still hasn’t been implemented, but we’ve so far only had one emulation bug. A story for another time.
Story time
So, instead of the emulation bug we had in Godot Sandbox, that only appeared for a single program, and I still have no idea how that bug has been undiscovered for so long, I want to tell you about the time that… I was embedding LuaJit (yes, it has a RISC-V port now! Great work!), and I generated a binary translation for it. A binary translation is C99 code that you can compile anywhere. If you embed it somewhere, anywhere, in your project — voila, now the sandboxed program will perform 5–50x faster.
I was testing it in my browser. In Godot you can just press a button and the game will be exported to the browser and opened in a browser tab, just like that. It really feels like 2024 software. Good job, guys. And once I enter the game, I have a print statement that tells me if the binary translation is enabled or not.
As you can see, it was loaded. Except if I jumped off a cliff and died, it would sometimes reload and loop forever. Only sometimes, though. The execution timeout wasn’t working, and it was somehow never exiting some kind of loop. Uh oh. I was debugging this for hours, until I saw that it was sometimes stuck in binary translation, and sometimes stuck outside. I enabled tracing, and I saw that it just kept going in and out, in and out. How is that even possible? The binary translation has anti-footgun malware: Upon entering, a jump table ensures that you either belong to one of the jump entries or you go right back to the interpreter.
f_10440_jumptbl:;
switch (pc) {
case 0x10440: goto f_10440_10440;
...
default:
cpu->pc = pc; return (ReturnValues){counter, max_counter};
}I saw that PC (the current execution address) didn’t belong to any binary translated block. So, it would go to the default switch case, and return back. The only way this was possible was if the execute segment was buggy. I ended up tracing it all the way back to the source: When it’s made. The execute segment is allocated and filled out during startup, with one exception: When binary translation is enabled, it gets to pre-fill the segment. So, you can perhaps imagine what happens if I forget to zero it before binary translation writes here and there: Occasionally, an index with random garbage in it will look like a binary translation entry.
The removal of a std::memset was done as an optimization, of course. The source of all our troubles, really.
I removed the zeroing in an effort to make it load faster. And yeah, it loads faster, I guess, but I would like my several hours of debugging back! Lesson learned. Don’t cheap out on the zeroing.
There are several other stories, like CRC’s being different because you can’t just iterate maps on different platforms and expect the same order. I ended up sorting a vector and checksumming that. Oh yeah, still haven’t figured out why static variables aren’t initialized right in WASM yet. That one will be interesting to figure out. It would be nice if it was a PEBCAK because I could just fix it and move on, but the code looks dead simple, except it uses static inline variables. Too new?
In the end, though, Godot Sandbox now works on quite a few platforms with binary translations intact, and so you can expect it to perform well everywhere: Mobile, console, VR, desktop and web.
This didn’t really turn into the introduction I would have liked it to be, so not an introduction to Godot Sandbox. I’m just finished up my work on being able to use zig c++ to compile Sandbox programs. It uses libc++ which took a bit of effort to support, but it’s passing unit tests and runs the demo project programs now. I’m very happy with it: Zig is easy to install and acts like a complete RISC-V cross compiler, and it also produces smaller binaries, by at least 50% compared to the RISC-V GNU toolchain. Even smaller with LTO. I don’t like that enabling LTO disabled section alignments in the executables, though. But, it works, so good job Zig guys.
-gonzo