Debugging Your Hardware

There are a lot of things that can go wrong when building custom hardware. In the software world, tooling builds in debugging support automatically until you explicitly turn it all off. We don’t have that luxury in hardware so we must do it ourselves.

You have already seen testbenches, which are an important tool for testing the functional correctness of your code. However they can only test things at the software abstraction level. They can’t determine issues that come from your use of directives, or from the interfacing between the ARM CPUs and the custom hardware.

Nothing Works At All!

First, check your ARM software is OK and that you can see the output of printf calls. Then, check that you can talk to the IP core by calling one of the setter APIs like xtoplevel_set_arg1 or similar.

xtoplevel_set_arg1(test);
printf("Do you see this output?\n");

If you see that output then the ARM core is able to talk to the IP core, and so the AXI Slave connection is correctly connected. You can further verify this by calling “set” and then checking that the corresponding “get” gives you the same value back. If not, then something more fundamental is wrong.

Regenerate Sources

First, try right clicking on your system and selecting Regenerate BSP Sources, then clean and rebuild your project. This builds the driver code and ensures the software interfaces are up to date.

Check Bitfile

Are you programming the FPGA with the correct bitfile? When running your application, select Run Configurations:

and in the resulting dialog select the run configuration you’ve been using. (If at this point you realise you’ve accidentally created millions of them you can clear them out using this dialog!) Under “Target Setup → Bitstream File” you will see the bitfile that the run configuration programs your FPGA with. This will probably be set to _ide/bitstream/design_1_wrapper.bit which usually is what you exported from Vivado. However this link can become broken so click the Browse button and navigate to the bitstream you made. This will be in your Vivado project under projectname.runs/impl_1/design_1_wrapper.bit. Check the modify date/time of that file so you know that it was recently created.

Still nothing?

If you still see nothing, then you will have to take your project back to a known working version (or a new project) and go from there step by step.

Debug Interfaces

Once you have the slave interface connected and communicating, we can use it to help more debugging. Let’s imagine the following custom hardware IP core:

uint32 toplevel(uint32 *ram) {
  #pragma HLS INTERFACE m_axi port=ram offset=slave bundle=MAXI
  #pragma HLS INTERFACE s_axilite port=return bundle=AXILiteS
  
  ram[2] = ram[1] + ram[0];
  return 0;
}

This hardware simply adds two number in main memory together and stores the result in a third. If we run this and nothing happens, how do we know what is wrong? There are (at least) the following problems that we need to isolate in order to debug this:

Your code
Some problem in the tooling - a corrupted project etc.
You forgot to rebuild and reexport the hardware
The AXI Slave interface (IP core to CPU)
The AXI Master interface (IP core to main memory)
and probably a few other things

We can make the situation better by building in debug interfaces. Look at the following small modification.

uint32 toplevel(uint32 *ram, int version) {
  #pragma HLS INTERFACE m_axi port=ram offset=slave bundle=MAXI
  #pragma HLS INTERFACE s_axilite port=return bundle=AXILiteS
  #pragma HLS INTERFACE s_axilite port=version bundle=AXILiteS
  
  version = 42;
  ram[2] = ram[1] + ram[0];
  return 0;
}

We’ve added a version parameter and bundled it into the AXI Slave interface. Now from the Arm CPU we can use the generated API call (which will be called XToplevel_Get_version()) to read this, and if we get 42 then we know that our projects are set up correctly and that we have wired up the slave interface. We can also at any point change this value, rebuild everything, and reassure ourselves that we are running the latest version of everything.

Debug Modes

We can go further and add full debugging modes into our hardware.

uint32 toplevel(uint32 *ram, int version, int mode, int arg1, int arg2) {
  #pragma HLS INTERFACE m_axi port=ram offset=slave bundle=MAXI
  #pragma HLS INTERFACE s_axilite port=return bundle=AXILiteS
  #pragma HLS INTERFACE s_axilite port=version bundle=AXILiteS
  #pragma HLS INTERFACE s_axilite port=mode bundle=AXILiteS
  #pragma HLS INTERFACE s_axilite port=arg1 bundle=AXILiteS
  #pragma HLS INTERFACE s_axilite port=arg2 bundle=AXILiteS
  
  version = 42;
  
  switch(mode) {
    //Do our normal thing
    case 0:
      ram[2] = ram[1] + ram[0];
      return 0;
  
    //Simple AXI Slave test    
    case 1:
      return arg1 + arg2;
    
    //AXI Master read tests
    case 2:
      return ram[0];
    case 3:
      return ram[arg1];
      
    //AXI Master write tests
    case 4:
      ram[arg1] = arg2;
      return 1;    
  }
}

Here, we have added alternate execution modes that let us test different bits of functionality, thereby narrowing down the problem. By setting mode and the debug parameters arg1 and arg2 we can peek and poke main memory and thereby observe the effects from the CPU. If mode 1 is working, but mode 2, 3 and 4 aren’t, then maybe we wired up the slave interface correctly but not the master interface.

You can carry this concept forward into your actual hardware design. Instead of treating it as a single monolithic block, alternate execution modes and debugging registers can help a lot with figuring out why something isn’t doing what you expect.