It sounds like you might have hit a blocker by trying to move to a part of the town that isn't on the map. I've forbidden those actions for now, to make sure users don't go too far off the beaten path.
You should be able to attack the townsfolk, though, (although you may need to insist). I've just added a formal option for you to attack the merchant. There's a 50/50 chance that it succeeds at each round. After that, you can escape down the alleyway and the AI will occasionally begin to speak like a pirate.
This is just one proof-of-concept for one possible domain for controllable voice AI – which I think shows potential. I appreciate that you disagree, and that’s fine!
But I can absolutely see a world where people play AI video games because they want the unbounded complexity and subtlety of a TTRPG with a human-level DM, powered by AI. I hope to have the opportunity to convince you with another play through!