I’d be all for a user notification that says “fnord.com wants to access 192.168.0.10 on tcp/443, which seems to be a web server on your home/work network. Are you sure you want to allow this?” I’d want to see this for each new access request, such that port scanning would not be a use case that was supported.
Sure, have an about:config toggle to shut this off, with appropriate warnings.