Tracking Traps on the MODM7AE70 – Part 1

Microchip Traps

Have you ever found yourself writing an application that, at runtime, reboots seemingly at random? Maybe your application prints trap output—diagnostic messages that indicate system faults—to a serial port. It happens to the best of us.

In this article, we will explain what to do with the trap output, and the workflow commonly used by the engineers here at NetBurner when tracking down the source of a trap. We will use practical examples run using our MODM7AE70 system on a module (SOM) powered by the Microchip SAM E70.

The NetBurner MODM7AE70 and SBE70LC products use the Microchip SAME70 and SAMD20 processors to provide high performing and reliable ARM® embedded IoT capabilities for secure automation and industrial IoT applications.

For a very low price point, this ARM® Cortex-M7 embedded system-on-module (SOM) solves the problem of securely network-enabling devices with 10/100 Ethernet, including those requiring SSL/TLS, auto certificate creation, industrial protocols, digital, analog, I2C, SPI, CAN, and much more. The products can also be used as an edge node microcontroller, edge processor or IoT gateway. Microchip® ARM® processors, combined with NetBurner’s acclaimed reliability and ease-of-use, provide a solution you can depend on.
Netburner ARM Cortex M7 embedded Development Kit for IoT product development and industrial automation.

Or, learn more about NetBurner IoT.

What Is A Trap?

A trap is an exception in the application that is caused by the execution of an illegal instruction. Some examples of an illegal instruction could be an attempt to read/write to an invalid memory address, or performing an arithmetic calculation that divides by zero. If an application attempts to execute an illegal instruction, it will reboot.

Some Good News

Luckily, NetBurner supports a feature called Smart Traps (link to PC tools docs) that prints trap information to the default serial port when a trap occurs. This information provides details on the call stacks of all your tasks, including the one that was running while the trap occurred. In most cases, this information will point you directly to the line of code that caused the trap. Smart Traps are provided for most NetBurner platforms, but in this article, we will be dealing specifically with the MODM7AE70.

There is no performance hit in including this feature in your application. You may want to avoid using this feature if you’re using your serial ports to interface with peripherals. Otherwise, we recommend using this feature, particularly during development.

No performance hit embedded
Smart Traps won't slow you down!

Smart Traps In Action

Let’s take a look at how to use Smart Traps and dive into an example. Smart Traps can be enabled by including the smarttrap.h header file and by calling the function EnableSmartTraps() at the beginning of UserMain().

The following example demonstrates how to enable Smart Traps. It also shows a divide-by-zero trap.

Make sure your NetBurner dev kit connected to the local network via ethernet cable, your computer installed with the NNDK tools, and the module connected to your computer via USB for both power and serial support.

We’ll start with a clean project for this tutorial.

  1. Open NBEclipse IDE. Click File > New > Project.
  2. The New Project panel will open. Choose the “NetBurner Project” wizard > Next.
  3. The New NetBurner C/C++ Project panel will now display. Add a project name. Project Type should be “NetBurner C++ Executable Project” and Toolchains should be “NetBurner Toolchain” (both are selected by default). Click Next.
  4. In the Select Configurations panel accept the default options and Next.
  5. In NetBurner Project Options make sure you have the correct Target Platform selected. In this case the MODM7AE70. The IP address can be found in the NBEclipse “NBFind” panel or by using https://discover.netburner.com.  Leave the other fields empty and as shown in the figure below.
  6. In the NetBurner Project Content panel accept the default settings as shown in the figure below. Press Finish. This will initial the build process which may take several minutes.

NetBurner Project Content wizard, showing the Wizard initialization option and Standard Initialization checked.

You are now ready to copy and paste our code example.

  1. In the NBEclipse Project Explorer open the project’s src directory > open the main.cpp file.
  2. Delete all of the code from the default main.cpp and replace it with the code below.
  3. Save the file.
				
					#include <predef.h>
#include <stdio.h>
#include <nbrtos.h>
#include <init.h>
#include <smarttrap.h>

const char * AppName="SmartTrap";

/* Can cause a divide-by-zero trap if the divisor is equal to zero */
uint32_t doDivide(uint32_t divisor)
{
    return (TimeTick/divisor);  // TimeTick represents the Ticks since the system has booted
}

/* This function is safe from a divide-by-zero trap since it checks for a zero value before dividing */
uint32_t doSafeDivide(uint32_t divisor)
{
    if (divisor == 0)
    {
        return 0;
    }
    else
    {
        return (TimeTick/divisor);  // TimeTick represents the Ticks since the system has booted
    }
}


void UserMain(void *pd)
{
    init();                     // Initialize network stack
    WaitForActiveNetwork();
    EnableSmartTraps();         // Allows the system to print trap info to serial

    uint32_t myDivisor = 0;  // Variable storing the divisor for division operations

    iprintf("Application: %s\r\n", AppName);  // Print application name to serial

    while (1)
    {
        iprintf("Press any key to proceed.\r\n");
        getchar();

        iprintf("doSafeDivide() returned: %lu\r\n", (unsigned long)doSafeDivide(myDivisor) );

        iprintf("Press any key to proceed.\r\n");
        getchar();

        iprintf("doDivide() returned    : %lu\r\n", (unsigned long)doDivide(myDivisor) );  // Trap occurs here
    }
}
				
			

Notice that the example has two functions that perform divide calculations. Those functions are doDivide() and doSafeDivide(). The doSafeDivide() function does not trap since it checks for a divisor equal to zero before performing the division. In contrast, the doDivide() function does not perform the check. When the application calls doDivide(), it will result in the following trap output.

After following the project set up and code implementation steps you are ready to run the project and generate the SmartTrap report. Your module needs to be connected to your development computer via USB for MTTTY serial readout to work.

  1. Click on the Run icon found in the top toolbar or from the top menu Run > Run. If Eclipse asks about Run Configuration, select NetBurner Application. The Eclipse Console should indicate that the application has been sent to your NetBurner device with a Program Success message.
  2. From the top menu select NetBurner > Mttty. This will open a serial interface to interact with and view the SmartTrap features. Set the COM Port to the option that makes sense (if only one device is connected to your computer then there will only be one option. You may disconnect your NetBurner’s USB cable and restart Mttty to see if there is a difference.) Set the Baud Rate to 115200. Press Connect. Press return/enter key when a cursor appears.
  3. Follow the prompts given within the white display area. There can be a few seconds of latency between prompts so be patient. The SmartTrap report will be produced within this window (see section below on how to read it). You will need to scroll up in the MTTTY window to read it. NOTE: you can adjust the font size of your MTTTY utility readouts to make it more readable.
smart trap output screenshot
It's not pretty... but it can be helpful.

There are three pieces of information that should be looked at (outlined in red in the figure above) – those are:

  • The ”Faulted PC” address (found in the “Trap information” section)
  • “The current running task” (found in the “RTOS information” section)
  • The call stack for that task (in the “Task information” section)

The task running while the trap occurred is Main#32 (that is a task priority of 0x32 in hex or 50 in decimal). Note that the Faulted PC value is also in the Main#32 call stack. Using this information, we can usually find which line of code caused the trap. This is done by using the addr2line tool.

The addr2line tool (link to docs) converts the address of an instruction from your application’s .elf file into a line of code in your application’s source files. The .elf file is automatically generated during the build process, along with the application binary. It is VERY important that you use the .elf file that corresponds to the application that is running on your module. If you make even a small incremental change, it can change your .elf file drastically. Realizing the.elf file you’re drilling into doesn’t match what’s running on your module is only funny the first dozen times or so.

To run the addr2line program for your MODM7AE70 (or other ARM based modules) follow these steps (for users of our older Coldfire platforms follow directions at the bottom of this section) :

  1. After successfully running the MTTTY steps, open a Terminal in NBEclipse IDE.1
  2. In the terminal, change directory to the project directory.
  3. Enter the following command replacing “YourProjectElfFile.elf” with your actual ELF file name within your Release subdirectory. Also update with your hex value for the Fault PC value as we found in the previous MTTTY steps (replace 70019D92 with your value) … 
				
					C:/nburn/gcc/bin/arm-unknown-eabi-addr2line.exe -ifCe Release/YourProjectElfFile.elf 70019D92
				
			

The output in the terminal will look similar to below, where the relevant main.cpp line numbers are called out.

				
					$ C:/nburn/gcc/bin/arm-unknown-eabi-addr2line.exe -ifCe Release/SmartTrapsEx.elf 70023536
doDivide(unsigned long)
C:\Users\myuser\nbworkspace\SmartTrapsEx\Release/../src/main.cpp:12
UserMain(void*)
C:\Users\myuser\nbworkspace\SmartTrapsEx\Release/../src/main.cpp:49
				
			

And of course, looking at line 12 of main.cpp in the doDivide function, we see that it’s very possible to divide by zero on that line, especially when called from line 49.

Note: If you’re developing on OS X or Linux, you can also use the addr2line command-line tool that’s available within the OS. The following snippet shows example usage by inputting the Faulted PC address from the trap above. The addr2line flags used in this example are some that we like using, but you can find more flags with their descriptions by using the command, arm-eabi-addr2line --help (note that for ColdFire based platforms, this command is m68k-elf-addr2line).

				
					> arm-eabi-addr2line -Cife obj/release/SmartTrap.elf 70019D92

doDivide(unsigned long)
/SmartTrap/src/main.cpp:12
UserMain(void*)
/SmartTrap/src/main.cpp:49
				
			

Something to note from the trap output is that the Faulted PC address is also in the Main#32 call stack. It is not uncommon for the call stack to have several addresses. In which case, it might be helpful to input all of those addresses in the addr2line tool. This allows you to get the full picture of what is going on, and can provide more insight as to what went wrong between function calls.

Note: For our Coldfire based systems (not ARM), you can run the above process with the Winaddr2line tool instead simply by doing the following.

  1. Top menu > NetBurner > Select addr2line
  2. In Project ELF field browse for the correct .elf file for the project.
  3. Enter the hex code form the Fault PC line in the MTTTY step.
  4. Press Decode and read the result

Wrapping Up

This divide-by-zero trap example was a simple one to help explain the tools and techniques, but Smart Traps can be helpful in solving complex bugs. Some bugs we’ve solved with Smart Traps include race conditions in variables that are shared across tasks/interrupts, deadlocks in critical sections, and much more.

Now that you know the trap hunting workflow used by the engineers here at NetBurner, I challenge you to fearlessly develop complex applications!

In part two of this article, we will explain a bit more about the trap output, such as register information. We will also explore an example of a faulted PC address that will have us searching through disassembly. Stay tuned!

As always, we love feedback. If you have any comments or thoughts, please feel free to drop them in the comments section below. Alternatively, you can mail us directly at [email protected].

Share this post

Subscribe to our Newsletter

Get monthly updates from our Learn Blog with the latest in IoT and Embedded technology news, trends, tutorial and best practices. Or just opt in for product change notifications.

2 thoughts on “Tracking Traps on the MODM7AE70 – Part 1

  1. This functionality is one of the things I like the most about programming netburner devices … but until today I didn’t know how to use it effectively, this post could save someone’s life.

    Any estimated date for the second part?

    PD:
    by the way, the following phrase is repeated in the post…

    Luckily, NetBurner supports a feature called Smart Traps that prints trap information to the default serial port when a trap occurs.

    1. Great catch, Jordan, thank you, and it’s great to hear that you’ve found the article useful! I’ll reach out to the author and see when he plans on writing the second part. Also, thank you for pointing out our editing mistakes. We’ve fixed that, now. =)

Leave a Reply