The method presented here, can tell you a maximum stack usage that the system had from the last reset or power on.
It will NOT protect your system from stack overflow but can still be useful for knowing what was your system's maximum stack consumption after running for some time.
The idea is simple: before launching the program, write a byte pattern to the stack area, an then, from time to time, during runtime, check how much of the stack area is still untouched, having the original pattern still there.
When the program runs, functions are called, parameters are passed, local function variables are used and these all take up stack size and will overwrite the original pattern from the stack area. Looking at the stack area at a moment in time, we will see how much of the stack area contains overwritten patterns and how much of it still contains the original patterns and we can calculate a percent value of it's maximum usage. However, we do not know in what point in time in the past this maximum usage was achieved.
I implemented this on a STM32F051 micro-controller in a programmable 100W bench load on a IAR tool-chain. For preparing the pattern (0xABCDABCD) into the stack area, I modified a startup file provided by IAR which you can find attached at the bottom, here is the code snippet which does this:
Stack pattern initialization
Reset_Handler_stack_pattern
LDR R0, =__ICFEDIT_size_cstack__ ;get configured stack size
LDR R1, =0x08000000 ;here is where SP is stored
LDR R1, [R1] ;get SP in R1
SUBS R1, R1, #0x4 ;
SUBS R1, R1, R0 ;get stack lowest address
LDR R2, =0xABCDABCD
MOV R3, SP
SUBS R3, R3, #4
stack_pattern_loop:
CMP R3, R1 ;current stack address in R3, lowest allowed stack address in R1
BEQ stack_pattern_done ;if equal the above addresses, jump to finish, else continue to
STR R2, [R3] ;copy pattern in stack area
SUBS R3, R3, #0x4
B stack_pattern_loop
stack_pattern_done:
After this, the startup file launches the main() function. After the SW enters main(), you can see that the stack area will be already touched.
The stack grows from a high address to a low address, so the patterns at the top will be the first that will be touched.
Calling the function void StackUsage_CalcMaxUsage() simply starts from the bottom of the stack and searches up for the patterns that are still there: 0xABCDABCD and based on how many patterns are still there it will calculate a percent usage of the stack.
Function void StackUsage_Init() initializes the addresses for the bottom and the top of the stack. For the bottom address calculation, a linker symbol is used which contains the configured stack size from the IAR environment: __ICFEDIT_size_cstack__
To avoid a stack overflow, when developing, you can choose a bigger stack first, and in the end, after the development is complete you can check the maximum stack usage for the system after running for some time and with as many different working scenarios as possible. After you got this maximum stack usage, you can leave some safety margin to this and finally configure the stack size for your system. Of course, this will not guarantee that the system will never overflow the stack.
You may also implement a warning level for the usage of the stack, which will lie somewhere between the normal stack consumption and the bottom of the stack, and have a safety handler for this, but this will still not guarantee you that the stack will not overflow, even if you check the stack level very often.
To be safe on stack overflow handling I think that hardware should be our friend here, it would have been nice if STM32051's architecture would have generated a TRAP if a certain memory address was written (bottom of the stack) and in this TRAP handler you would bring the system to a known state instead of potential catastrophic consequences in the case of a stack overflow, but unfortunately STM32F051 does not offer this hardware help.