The point is, it wasn't immoral of them to "take the work of others" to begin with. This is how culture works: mythology, classic stories like the Arthurian cycle, fables, lots of classic literature etc. came to be by just taking the stories and characters people liked and doing whatever the next artist wanted. (Today's equivalents are all owned by Disney, for example.) This is also how science and philosophy functioned for ages.
The worldview holding the modern copyright system to be the moral reality is the elaborate rationalization, ingrained into people by the interested parties. You can research the history of copyright law if you want.
This is a separate issue from caring about compensating and nurturing the artists, which is often what people who are mindful about this stuff do. Just not necessarily inside the "traditional" framework.