So few people appreciate why Bittorrent (Mainline) and DNS have been world-conquering successes while everything else (cough IPFS cough) struggles. Be stateless, be sessionless -- send only packets, allocate no state. Nothing else in the computing world scales the way the Mainline bootstrap nodes and DNS root servers do. Until recently the DNS root servers were just a handful of decent workstations at universities; Mainline's bootstrap nodes still are. None of the heavyweight cloudtech can match that kind of efficiency.
You should publish a spec for Pkarr -- the layer below did-dht -- first. Right now Pkarr is a software program/library, not a specification. I think this will help you simplify and articulate your work more clearly to people who aren't immersed in it. I think it will also be extremely useful to people who don't need the incredible complexity of w3c DIDs.
The choice to sign an entire DNS packet seems very strange and probably hasn't been through through properly.
Why use DNS packets? Presumably because you want to leverage the existing infrastrucure of recursive DNS resolvers. However these resolvers do not preserve packets!. If I send a query to my recursive resolver, and it makes a query to the authoritative server, it can (and almost always does) modify the resulting packet from the authoritative before returning a reply to me. The upshot here is: if you're signing packets, almost all recursive resolvers will destroy your signatures. This is why DNSSEC signs individual resource records instead of packets. I think that's what you want to be doing: sign an RR, not a packet. If you absolutely need to sign multiple RRs, you'll need to specify a canonical way to assemble the RRs (i.e. sort them).
I really think you want to sign an RR (which might include the hash of other RRs), not an entire DNS packet.
Lastly, please take this issue more seriously: https://github.com/TBD54566975/did-dht-method/issues/80#issu... the only response given was that "the DHT-DID [spec] uses Pkarr [a piece of software]" which makes no sense... specs depend on specs, not implementations. Then the issue derailed (as unthreaded discussions always do; github is why we can't have nice things) into some side tangent about KRPC and CBOR instead of answering the issue's question "why DNS?". Pretty sure the answer is "to leverage existing recursive resolver infrastructure" but you should state that clearly -- and then think through the consequences.
First I very much share your opinion of Mainline and IPFS. Mainline is a miracle.
As for the spec, I personally prefer working code first and at least won't allow for design by committee to turn a good thing to awful mess.
That being said, Pkarr is so simple that did-dht doesn't really rely on software, in fact they have their own Go implemntation.
I agree with you that Pkarr is better on its own without w3c complexity.
And I agree with you that stable simple ideas deserve spec, but the spec is simple so far (put DNS packet in BEP44 without salt), still, I owe everyone to write that down.
> The choice to sign an entire DNS packet seems very strange and probably hasn't been through through properly.
I did think about the encoding choice thoroughly, in fact the earliest version was just a JSON inside BEP44.
The reason why Pkarr signs the entire Packet, is because that is the most efficient way to pack the 1000 bytes limit in BEP44.
The reason we use DNS Packet is because I couldn't come up with any smart encoding that offers any non-marginal advantage vs using an old, tried and tested spec that would work great with plenty of encoder and parsers, and really fits the use case.
I understand that you think that leveraging DNS infrastructure doesn't make sense because they won't pass the signature, but so would be the case even if you sign individual RRs, in fact even if you try to use DNSSec, recursive resolvers have no way to verify these DNSSEC signatures because they don't understand that Pkarr TLD is the key and it doesn't need a certificate.
Using Pkarr through DNS infrastructure, is not trustless, but it is still good to have, 90% of DNS is not trustlees (doesn't use DNSSEC) and it is great to have.
If you want to use Pkarr in trustless way, then either use Mainline directly, or use the relays spec https://github.com/Nuhvi/pkarr/blob/main/design/relays.md
But if by a strike of luck we managed to convince 1.1.1.1 or 8.8.8.8 to resolve Pkarr TLDs, then sticking to DNS packet encoding will prove to be really useful, even if their answers are not trustless, at the very least, we are not forcing them to integrate with bespoke encoding for no reason at all.
If anyone thinks that this encoding is bad for their use case, they are welcome to use BEP0044 directly, but they won't benefit from the network of relays I and others will deploy for lowering latency with variant degrees of trustlessness, for that we need to agree on an encoding, and DNS Packet encoding is the choice that makes most sense, if only for the track record.
Finally, if you still have more questions, or arguments, please bring them to https://pkarr.org
