This is just silly. Everywhere had forced labour, but didn't manage to build what the west did. The African slavers selling their fellow continent-dwellers didn't somehow manage to pick all the people who could build the most advanced things in the world at the time.
Oxford University was founded in 1096, long before what you're describing. This is very strong evidence that the UK has a thousand years of excellent investment in education, which much better explains all the advantages that built its empire, the good bits and bad. Its advances are in part due to the Roman colonisation, which allowed Britain to rediscover things that much more advanced civilisation had discovered 1000 years prior to that founding, and then push on to far greater heights.
There are entire countries that still wouldn't have universities today if left to their own devices. But they would still have slaves, because western powers wouldn't have ended this practice, either through Christianity or, if that didn't work, by force.