It's not
that difficult to live without a credit card. Debit cards don't have that same collections concept and can almost always be charged through credit card networks netting the same outward experience.
As for not building a credit score, well, it sounds like you're not really interested in taking out loans in terms that others wish to offer them, so what good is a credit score? The people who care about credit scores are largely the same who will only do contracts with collections classes and be unwilling to adjust contracts. If you're working with a lender who is willing to change contracts like that they're probably also going to be fine evaluating your non-normal financial arrangements.
