The entire point of an secure enclave is to completely enclose all the hardware and software needed to generate encryption keys in a single lump of silicon.
This means that all of its processing requirements (it's a complete co-processor) are on chip, it's RAM is on chip (not shared with it the main CPU, and probably has ECC), and it uses secure boot to cryptographically verify that it's firmware has not been tampered with before it starts executing. Additionally it may even be possible to update it bootloader in the future to prevent further updates without a passcode.
The end result means that attacking a secure element is very difficult. There are few, if any, exposure points that would allow you to fiddle with its internal state, and any attempts too should result in the secure element wiping stored keys, making further attacks a moot point.
