But would removing the justification really mater given the entrenched power structure that has grown dependant on a certain way of veiwing other humans?

The cannibalism is not the primary issue, but is a clear indicator that the inspectors have been lied to and slavery is present, which was previously clearly established as one of the major disqualifying cultural traits.