edit to add: The very helpful author of RenderDoc told me on Twitter that the command line capture utility is meant as an internal thing, and not for general use. So don’t complain if this breaks in the future. Also, he said that my “--opt-ref-all-resources” is probably superfluous in most cases and will generally bloat the capture for no benefit.
One of the things I really like about Vulkan vs. OpenGL is that Vulkan is “offscreen by default” while OpenGL depends on the windowing system. With Vulkan, you can just allocate some space for an image, and draw/compute to that image. You can pick which specific device you are using, or do it once for every device. If you want to display the image, blit it to the swapchain. With OpenGL, you tend to need to open a Window with certain options, initialize OpenGL and create a Context using that Window handle, and use one of several extensions for offscreen rendering. (Probably FBO, maybe PBO. It’s an archaeological expedition through revisions of the spec and outdated documentation to be sure exactly what to do.)
Consequently, it is muuuuuch easier to make a “little” Vulkan application (that admittedly depends on a not-so-little engine/library with a bunch of boiler plate and convenience code) that does one thing off screen and exits without needing to pop up anything in the GUI as a part of the process.
This naturally raises the question… How are you sure that little utility actually does what you told it to?
RenderDoc
RenderDoc is a super cool utility. You can use it to launch your Vulkan (or OpenGL, or Direct3D…) application, and when you push a button, it will intercept the steps used to render a frame presented to the active window. Here’s an obligatory screenshot to give you a sense of what I am talking about. On the left, you can see that a bunch of vkCmdDrawIndexed calls are used to render the image. On the top, you can see a timeline of when everything happened. In the middle, you can see the rendered frame, etc.
This is super cool for a video game. You can play test it until something looks funky, then capture the render for a frame that shows the bug. If you are working on general purpose engine/library code, or something like a unit test, it may not be so convenient. I mentioned, “when you push a button, it will intercept the steps used to render a frame presented to the active window.” This presents two problems for an automated test:
- One is that you don’t want to push a button.
- The other is that you don’t have a window.
If you just try and launch something that invokes a compute shader and exits, nothing will get captured. You’ll get no output. No error messages. You won’t even be able to tell if the application worked or failed before executing for some reason. You’ll need a few extra steps to capture this sort of offscreen automated test and get the information you want.
renderdoccmd
It’s barely mentioned in the documentation, but RenderDoc comes with a command line utility for doing the capturing, called renderdoccmd. The basic syntax for launching a program for capture is pretty straightforward.
$ renderdoccmd capture ./my_vulkan_test
Conceptually this is exactly what we want in an automated CI test script, rather than a GUI application. But this won’t actually work in your build/test script out of the box, for a couple of reasons. We’ll crank through a few options for the capture utility, then talk about some small modifications you need to make to your program in the next section. But, at very least you can now see stdout of your program when invoked through renderdoccmd, so it’s clear that the program is running and doing what you want it to.
Suppose you made a shell script that wants to run three tests, one after another:
renderdoccmd capture ./my_vulkan_test
renderdoccmd capture ./my_vulkan_images_test
renderdoccmd capture ./my_vulkan_shaders_test
By default renderdoccmd will instantly return as soon as it spawns the child process, leaving those tests running in parallel. In some cases, that will be what you want, but in other cases your benchmarks will get confused because they expect to be run independently without a bunch of other tests hogging the GPU, and your memory allocation tests will get confused because they expect to have all of V-RAM to allocate by themselves, etc. Here’s a more complete invocation:
$ renderdoccmd capture -w --opt-ref-all-resources ./my_vulkan_test
This gets closer to what we want in a test script. “-w” Means that renderdoccmd will wait for the child process to finish. The other flag to get “all resources” means that RenderDoc won’t just be trying to capture the resources used in rendering a frame, which is nice because a small test case may not really be rendering a frame in the first place.
Unfortunately, this still won’t actually generate the “.rdc” capture file with the information we want. We still need to make some modifications to the test to hook directly into the RenderDoc API when not using a normal swapchain presentation pattern.
RenderDoc API
The RenderDoc API is a little odd. You don’t have to link to an extra library at build time (you do need one header) because it just gets injected into your process at runtime by the launcher. I guess that’s one way to solve the C++ package management problem.
The docs start out by saying, “This API is not necessary to use RenderDoc by default.” As noted, an offscreen unit test isn’t really the default use case for RenderDoc, so don’t read that and gloss over the rest like I did when I was first getting this working. Sigh.
On Windows, you need to GetProcAddress, on Linux you need to dlopen. You get function pointers at runtime. This will generally be familiar to anybody who did OpenGL >1.1 on Windows. That API documentation page has an example. Go look at it.
So, what do you actually need? Not much. After you do the init,
rdoc_api->StartFrameCapture(NULL, NULL);and
rdoc_api->EndFrameCapture(NULL, NULL);They are both important. There’s no implicit StartFrameCapture when the init happens. And critically, there’s no implicit EndFrameCapture when the process exist. You might think it’s obvious that the frame isn’t going to get any extra steps after the process has ended, but you need to explicitly call the function anyway. And significantly, if you have a bug that causes the process to crash before that EndFrameCapture call happens, you won’t get a capture file with the state of things up to that bug. Repeat the Start/End for as many “frames” as you have different tests, if you want them to be in separate capture files, regardless of whether they are literally frames.
Verify something like if(rdoc_api!=nullptr) before you try and execute those. If you run your program outside of RenderDoc, the API won’t be available, so you’ll want to skip over trying to execute functions from a dangling pointer.
Check out the documentation for optional stuff like setting options, setting the name of the capture output file, and even adding comments to the capture file, so it’s easy to remember what was actually being tested.
Since my mind was rotted in the 1990’s by pro- Object Oriented Programming extremists, I wrapped the functionality into a C++ class that works as a scoped recorder. The constructor starts the frame capture. The destructor ends the frame capture. So, I can just do
{
RenderDoc_Capture cap;
run_my_test();
run_phase_two();
whatever_else();
}
And I don’t have to worry about keeping track of ending the frame capture because it will just happen whenever that object goes out of scope. With a simple API like that, you just have to be careful not to try and capture in nested scopes.
Bringing It All Together
Alright, so once the application is modified to explicitly use the RenderDoc API to start and stop fram captures, and you run the test from the command line with whatever flags are appropriate, you get a few megabytes of .rdc file that you can open up in the RenderDoc GUI application to inspect and complain about.
Many thanks to folks on the Vulkan Discord server who helped me nibble through the several issues I ran into trying to understand what was going wrong with my attempt to use RenderDoc when I was getting no output and no error messages and I had no idea why. Hopefully spelling this all out in one place will be helpful if somebody else runs into the same problem.
Will Rosecrans Blog 