We should all delete "I'm sorry you feel that way" from our mental phrasebooks. It is a phrase that always falls flat, comes across as potentially sarcastic, and is the textbook definition of a non-apology apology.
If you want to clarify something, go for it. If you don't want to apologize, don't feel obligated to do so. But please, for your own benefit, steer clear of this no-good phrase.
