You need to disable inlining. __builtin_frame_address can give surprising addresses when inline enabled. The code itself is likely not be wrong, but you won't confined to the lexical scope as if you see in the code (the defer can be triggered when the parent function returned).
Disclaimer: I haven't looked at the code too closely.