https://plato.stanford.edu/entries/peirce-semiotics/I'm leaning on the fact that theories are significations of events, but significations are only possible when relative to an interpretive scheme that allows those events to be connected to the theory.
A deep-math version of this would be picking whether or not you want to work with the axiom of choice or not when using groups to represent vectors (the groups and vectors aren't important here, but they play the role of theories and objects). Deciding whether or not you accept the axiom of choice can a have major impact on what theorems you are willing to accept as justifiable, but the decision is not purely aesthetical, because it's possible to have technical reasons for using the axiom of choice.
Resolving whether or not the axiom of choice for the purposes of your practice can be done but it won't usually be within the scope of the system. Often you need to introduce further information from outside the system to justify use of the axiom. And so you are building on further postulates, making more choices at the exclusion of others.
A more science-y example: It can be seen as a restatement of the Duhem-Quine thesis: you always need postulates to relate a hypothesis to its observation. These postulates might correspond to truth or might not but you are almost working with a set of assumptions with varying levels of validation when doing any scientific work.
https://en.wikipedia.org/wiki/Duhem%E2%80%93Quine_thesis
All of this drives out from philosophical school of American Pragmatism, which includes philosophers like Charles Peirce and Charles Morris who pioneered the semiotic theory. To a lesser extent Carnap might fit in this category since he bought the semantics/syntactics/pragmatics distinction that Peirce and Morris created, then set up classical logic as we know it (including model theory), although he's usually considered a positivist.