Codds relational database model adds the further constraint that nested tables are not allowed (first normal form), instead representing relationships through foreign keys.
Codds motivation for disallowing nested tables is that it makes query languages much simpler. He develops relational algebra which is the foundation behind SQL, which is why SQL does not allow nested tables.
Document databases does not follow first normal form and allows nested structures, so they cannot be queried with relational algebra, since it doesnt have a way to “drill down” into nested structures.
It is unclear to me what “mathematical principles” remain if you remove the notion of relations from the relational model.
