The problem is that Terraform is stateful. Terraform will revert the Lambda code back to the state defined in Terraform when Terraform is applied after the code was updated outside of Terraform.
There is a way to mitigate this a by making Terraform ignore changes to the actual code of the Lambda.