Post

Your Fault Handler Is in RAM. Is Everything It Calls?

TL;DR

A fault handler that has to survive a flash fault must run from RAM, and so must everything it calls. Marking the handler with a section attribute only moves the handler. The compiler adds calls you didn’t write (memset for = {0}, memcpy for a struct copy), helpers that don’t get inlined stay in flash, and the linker routes the calls through veneers that sit in RAM and jump straight back to flash. None of that shows up in the source.

So I check the linked image instead. A 200-line Python script disassembles the ELF, walks the call graph from named root functions, and fails the build when a path leaves RAM. It runs after every link, in every configuration, on both boards. The script, a README and a small example are free to use under the Apache License 2.0: check_ramfunc.

Edit: this post has mostly been made with Claude. Despite it being AI generated, the script at the bottom of this post has been in use in my own project for several months now and has caught issues a couple of times already. I hope it’s still useful to you.

The Setting

The firmware is the test fixture from the ML-DSA post, now on two boards: an STM32U5A5 and an STM32C5A3, both Cortex-M33 parts with two flash banks and ECC on the flash.

Three situations make flash unsafe to execute from:

  • Flash is being erased or programmed. On these parts a bank that is being written can’t be read. Code running from it stalls until the operation ends, which is milliseconds for an erase.
  • Flash raised the fault. An ECC error that can’t be corrected raises an NMI. If the handler then fetches its next instruction from the same damaged area, the fault repeats.
  • The handler has to leave evidence. When something goes wrong, the most valuable thing the firmware can do is write a fault record that survives the reset. That write must not depend on the part of the chip that might be broken.

So the ECC recovery path, the part of the fault handlers that writes the fault record, the log output’s DMA interrupt and the RTOS tick hook all run from RAM. The interesting question is how to know they really do.

A Section Attribute Isn’t Enough

The usual approach is an attribute and a linker script entry:

1
#define RAM_FUNC __attribute__((section(".ram_text"), noinline))

with .ram_text placed in RAM and copied there at startup. That moves the function it’s attached to, and nothing else. Here is a small example, the same one that ships with the script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
struct FaultRecord { uint32_t cfsr, hfsr, pc, lr, regs[16]; };
static struct FaultRecord saved;

void print_fault_report(const struct FaultRecord *record) { (void)record; for (;;) {} }
uint32_t read_cfsr(void) { return *(volatile uint32_t *)0xE000ED28u; }

RAM_FUNC void save_fault_record(const struct FaultRecord *record) { saved = *record; }

RAM_FUNC void HardFault_Handler(void) {
    struct FaultRecord record = {0};
    record.cfsr = read_cfsr();
    save_fault_record(&record);
    print_fault_report(&record);
}

In the source, HardFault_Handler calls three functions, and one of them is marked. Built with Arm GNU Toolchain 15.3 at -O0 for a Cortex-M33, the binary tells a different story:

1
2
3
4
5
6
$ check_ramfunc.py --root '^HardFault_Handler$' example.elf
error: example.elf: a chain that must run from RAM reaches flash:
  HardFault_Handler -> memset at 0x0800003c is in flash
  HardFault_Handler -> read_cfsr at 0x0800001c is in flash
  HardFault_Handler -> print_fault_report at 0x08000010 is in flash
  HardFault_Handler -> save_fault_record -> memcpy at 0x0800004c is in flash

Four calls into flash, two of which aren’t in the source at all:

flowchart LR
    hf["HardFault_Handler (RAM)"] -->|"= {0}"| v1["__memset_veneer (RAM)"] --> ms["memset (flash)"]
    hf --> v2["__read_cfsr_veneer (RAM)"] --> rc["read_cfsr (flash)"]
    hf --> sf["save_fault_record (RAM)"] -->|"saved = *record"| v3["__memcpy_veneer (RAM)"] --> mc["memcpy (flash)"]
    hf --> v4["__print_fault_report_veneer (RAM)"] --> pr["print_fault_report (flash)"]

Each of the three sources of trouble shows up here:

  • Calls the compiler adds. = {0} became a call to memset, and the struct copy became a call to memcpy. In my own firmware it is worse by design: I build with -ftrivial-auto-var-init=zero, which clears every local array, and those clears are memset calls the source never mentions. 64-bit division adds __aeabi_* helpers the same way.
  • Helpers that aren’t inlined. At -O0, read_cfsr stays a call. With optimisation and LTO it is often inlined and the problem disappears, until a change makes the compiler decide otherwise.
  • Veneers. RAM starts at 0x20000000 and flash at 0x08000000, further apart than a bl instruction can reach. The linker inserts a small veneer in RAM for each such call, which loads the target address and jumps. The veneer is in RAM; the code it jumps to is not. A check that only looks at where the direct call target lives would pass all four calls.

Reading the Linked Image

The check works on the ELF after linking, because that is the only place where all of this is visible:

  1. Disassemble the ELF with objdump -d -C.
  2. For each function, record its direct calls and tail calls (bl, b), its indirect calls (blx/bx to a register), and whether it is a veneer: an ldr pc, ... followed by a literal address.
  3. Starting from the root functions, walk the call graph breadth first. A call through a veneer is followed to the function the veneer jumps to.
  4. Report every reached function whose address is outside RAM, with the call chain that reached it. The walk stops there: the callees of a flash function are not the RAM path’s concern, and one report per chain is enough.

The roots are named explicitly, as regular expressions on the demangled symbol names. They aren’t taken from the section, because .ram_text also holds code that is in RAM for speed and may call flash freely. A root that matches no symbol is an error: a renamed function would otherwise make the check pass while checking nothing.

The fix for the example is three lines: mark read_cfsr, and move memcpy and memset into .ram_text in the linker script. My firmware does the latter for its own memset and memcpy, together with the RTOS kernel.

The Part That May Stay in Flash

Rooted at the fault handlers without exceptions, the check fails, and it is right to. After the fault record is written, the handler prints a report: a backtrace, the fault status registers, formatted with fmt, sent through the log output. That code is large, and moving it into RAM would cost far more RAM than the record is worth. It also doesn’t need to survive a flash fault: by the time it runs, the record is written and recovery has been given up.

A check that reports the design itself gets ignored, which is worse than having no check. So the fault handlers are two halves with different rules, and the check has a list of exemptions (--allow) for the reporting half. Two properties keep the list honest:

  • An exemption only suppresses the report. The walk stops at any function in flash either way, so an exemption can never hide a function that is in RAM, or anything a RAM function calls through it.
  • An unused exemption isn’t an error. Inlining differs between -O0 and -Os, and between gcc and clang, so an exemption needed in one build is often unused in another. A stale exemption only makes the check stricter.

The list is in the build files, next to the roots, so it is something someone can argue with rather than a silent gap.

Things That Went Wrong

  • Truncated names can match the wrong exemption. Stopping at the first > when reading a symbol name from objdump’s output is the obvious parse, and it is wrong for C++. A demangled template carries its own angle brackets, so Storage<Schema<...>> becomes Storage<Schema<, which can then match patterns it should not. The name has to run to the last > on the line.
  • clang’s names weren’t demangled. objdump’s demangler gives up on what clang produces for constrained template parameters, so those symbols arrived mangled, and exemptions written for the demangled names silently didn’t match. The check now takes a second demangler, llvm-cxxfilt, for the names objdump leaves mangled.
  • Interrupt handlers in RAM that weren’t. The UART and USB interrupt handlers were placed in RAM so they wouldn’t wait on flash. Both reached driver code in flash through veneers on every interrupt anyway. The RAM placement bought nothing and cost a veneer on every call into the driver, so they moved back to flash. The tick hook went the other way: it now runs entirely from RAM, and the check holds it there.

What It Finds in Real Firmware

The numbers below are for the full fixture firmware on both boards, for the fault-record path, which has six roots (one per fault type, the default handler and the NMI). They come from the check itself, run on existing builds; every build uses LTO.

Board, compiler, buildReachableIn RAMAllowed in flashRuntime helpers in RAMCalls through a veneer
U5, gcc 15.3, -O0311813memset15
U5, clang 22, -O0321913memset, __aeabi_memclr415
U5, gcc 15.3, -Os301812memset47
U5, clang 22, -Os331716none27
C5, gcc 15.3, -Os301812memset47
C5, clang 22, -Os331716none27

“Calls through a veneer” counts call instructions in the RAM functions on the path that go through a veneer, which here means into the exempted reporting code. A few things stand out:

  • The set of functions changes with every build. Between 30 and 33 functions are reachable, and 12 to 16 of them are exempted. A check run in one configuration says little about another, which is why it runs after every link.
  • clang adds a helper of its own at -O0. It clears memory through __aeabi_memclr4, which gcc doesn’t use here. A list of “functions that must be in RAM” written by hand would have missed it.
  • Optimisation multiplies the crossings. gcc’s -Os build has three times as many calls through a veneer as its -O0 build. Most likely, functions that are inlined into the RAM code bring their calls into the reporting code with them.

The other three paths are smaller and stable across all six builds: the ECC recovery chain is 7 functions, all in RAM; the log output’s DMA interrupt is 2 functions with 1 exemption; the tick hook is 1 function.

It Has Been Done Before, Differently

The Linux kernel has a close relative: objtool’s noinstr validation, in the kernel since 2020. Functions marked noinstr live in their own section, and objtool checks the compiled objects for any call that leaves that section, warning call to foo() leaves .noinstr.text section. Like my check, it reads machine code, so it catches calls the compiler adds; its instrumentation_begin() and instrumentation_end() markers play the role of my exemptions.

The differences come from the target. The kernel checks a section name; on a microcontroller the question is an address range, because the linker script decides what ends up in RAM. My check starts from named roots and walks the graph instead of checking every function in a section, because RAM also holds code that may call flash. And it has to see through veneers, which the kernel’s x86 build doesn’t have.

Limits

  • Calls through function pointers aren’t followed, because the target isn’t in the disassembly. --indirect lists them on the walked paths, so they can at least be reviewed by hand.
  • It over-reports. The walk finds every function a path could reach through direct calls. A call behind a condition that never holds is still reported, and has to be exempted or restructured.
  • It reads Arm Thumb disassembly. Other architectures need other patterns for calls and veneers.
  • The RAM range is fixed in the script: the Cortex-M SRAM region. For ITCM or a part with SRAM elsewhere, change two constants. A general version would read the memory regions from the ELF.

Other Uses

The same rule fits more than fault handlers. Each gets its own roots and exemptions:

  • routines that erase or program flash, on parts where flash can’t be read while it’s written;
  • code that runs while execute-in-place from external flash is turned off;
  • interrupt handlers that must never wait for flash;
  • handlers that must be in ITCM or another tightly coupled memory.

A related idea, which I don’t run yet: numbers from the linked image that may not grow compared with the previous build, such as RAM and flash per region, the size of .data, the number of veneers, or the worst-case stack depth. I compare sizes by hand today.

Get the Script

The script, its README with the examples above, and the example sources are on this site, under the Apache License 2.0:

The published version checks one set of roots per run and needs only Python and arm-none-eabi-objdump. Run it from your build system after every link; the README has a CMake example.

Lessons

  • Check the artefact, not the intent. An attribute states where code should go. Only the linked image says where it went, and what else came along.
  • The compiler is a co-author. Hardening flags, struct copies and zero-initialisation all add calls. Any rule about “what this code calls” has to be checked after the compiler has had its say.
  • Make exemptions explicit. A check that reports the design gets switched off. A named, reviewable list of exceptions keeps it on.
  • Run it in every configuration. Depending on the build, the same fault path reached 30 to 33 functions and needed 12 to 16 exemptions.
This post is licensed under CC BY 4.0 by the author.