Live-debugging Nim in a Game Engine

Using remote GDB support to live-debug programs

Virtual machines can be black boxes, especially if debugging them is a chore. And again, even more so if you can’t debug them to begin with. I have been using a RISC-V emulator as a way to support scripting in a game I’m making for a long while now, and debugging got on my mind lately when a commenter on one of my blog posts told me had implemented remote GDB already. I never got to look at his implementation, and I assume he was too busy to create a PR, but it also made me very curious about this RSP protocol and if I could implement this myself. It seemed like a killer feature.

GDB RSP protocol

The protocol is fairly straight-forward. It’s text-based, has a very tiny minimal implementation and has simple error-checking and retransmission so that you can use it with serial port too.

Perhaps the most interesting part about it is that it makes no assumptions about the state of the target its debugging at almost any point, which can reveal even the most insidious bugs, such as those where even the machine or emulator itself is wrong cough. GDB will after almost every command retrieve the full register file. Another thing worth mentioning is that GDB already knows how to move up and down the callstack of the target it’s debugging, so you won’t have to implement anything other than simple query and modification commands.

Language support

GDB is probably the most widely supported debugger in existence. It has support for something called pretty printers, which allow languages to specify how every symbol should be printed. Some languages enable this functionality using scripts, however there is another option.

Nim bakes the pretty printing into the debug info in the executable by itself, provided you pass --debugger:native to the nim invocation. Building programs this way allow you to see Nim programs as opposed to the transpiled C code when you are debugging, and it works in any GDB.

Ease of use

One of the big worries when implementing debugging like this is that it will be too cumbersome to support, and perhaps in some ways it is. You have to build the program with debugging enabled, and you have to support debugging in your game engine on demand, ideally without rebuilding it.

So, I did that and the reason that was possible was because you only have to disable binary translation to be able to fully debug any program in the RISC-V machine. It has to be disabled because binary translation takes a chunk of code (called a code block) and turns that into a single native function which you can only debug by hooking your game engine into GDB. Possible, of course, but cumbersome. Worth mentioning is that binary translation is a run-time feature that can be disabled, so let’s just turn it off using an environment variable and suddenly we can debug game script at run-time.

Still, how does that work, exactly?

Implementation

One way to make it work would be to make every call into the games script open up a TCP listener socket, wait for you to connect with GDB, and then go from there. Instead, I chose to make this feature a system call in the VM. That is, to start listening for a remote connection from GDB, simply put a “breakpoint” somewhere in your games script, and eventually, if that line is executed it will trigger the system call that waits for GDB to connect.

That just makes it so much nicer. It means you don’t need a special build of the game engine to debug any of the games’ script.

var j = %* {
    "name": "Hello",
    "email": "World",
    "books": ["Foundation"]
}
proc hello_nim() =
    print "Before debugging\n"
    remote_breakpoint()
    print "Hello Nim World!\n" & j.pretty() & "\n"

We can get into GDB just after the remote_breakpoint() function ends by using the target remote command:

Reading symbols from hello_nim...
(gdb) target remote localhost:2159
Remote debugging using localhost:2159
hello_nim () at .../micronim/src/hello.nim:14
14     print "Hello Nim World!\n" & j.pretty() & "\n"

Once you detach or close the session the emulator will just continue where you stopped at, continuing execution as if nothing happened. Also, if debugging isn’t enabled a line is logged and the breakpoint is skipped over. Everyone loves reading kilometers of log.

One idea of mine is to fork-execve a GDB instance when hitting a breakpoint. The reason that would work is because GDB doesn’t immediately give up when it can’t connect somewhere. It will keep trying until you interrupt it. So, start GDB just before blocking on the listener. Of course this mode would probably only work for me, but that’s the idea. Instant GDB terminal connected and ready to go. If it’s silly and it works, is it silly?

-gonzo