Re: ai_flags in calls to getaddrinfo

[Robin Tarsiger <rtt@dasyatidae.com>]

So, as I come back to this, I think I tunnel-visioned on the AI_V4MAPPED
behavior at first since that was what you changed, but I now think that's a
red herring, and the potential problem in Emacs is the _last_ part of this:

>>> On one of the systems on which I work, getaddrinfo called with
>>> AF_INET6 fails with EAI_NODATA, for some reason.  That causes
>>> network-lookup-address-info to fail when passed 'ipv6' as the last
>>> argument.

Some non-zero return values from getaddrinfo, including EAI_NODATA,
don't actually indicate resolution failures. From glibc getaddrinfo(3),
there's also EAI_NONAME and EAI_ADDRFAMILY. I think the real point
of difficulty is that network_lookup_address_info_1 always turns these
into messageable warnings. To be clear, is that the behavior you are
seeing on the Windows 10 machine, or is there some different
failure occurring?

I notice that EAI_NODATA isn't mentioned in the Windows docs[1],
but if it's actually getting returned, I'd guess it has the same
semantics as in glibc: "got a valid result indicating no addresses
found". The exact distinction between "answer is: no data" error
codes is more nonportable, but that won't matter if you treat them
identically.

[1] https://docs.microsoft.com/en-us/windows/win32/api/ws2tcpip/nf-ws2tcpip-getaddrinfo#return-value

From an elisp standpoint, what do you want to have actually happen:

 1. If a program requests a name that doesn't exist at all?

 2. If a program requests a name that exists and has addresses of
    some families (such as IPv4 addresses), but the program has
    specifically requested another family (such as IPv6)?

 3. If a program requests a name that exists but has no addresses
    at all? This is entirely possible in the DNS; dasyatidae.com
    is even currently such a name, having only MX records and no
    address records.

Based on the docstring, I would imagine a silent nil for all three.
In particular, the docstring implies that out-of-family addresses
are _not_ returned when FAMILY is not nil. But I don't know how
elisp programs currently use network-lookup-address-info.

From another perspective, when would a program pass a non-nil
FAMILY value in the first place? If a program just wants any
address, that's already handled by passing nil for FAMILY. If
what you want is to treat it more like a preference rather than
a requirement, you'll probably have to do something more complicated.
If FAMILY expresses a requirement, then nil would be correct
if nothing matches the criteria.

(There's a potential question of whether AI_ADDRCONFIG is
relevant if you want to be DWIM-y, but let's leave that for later.)

I think that's the key here in terms of what can be done in Emacs.
More comments below on other things.

>> Does it have any IPv6 addresses configured, or no? What is
>> its resolver configuration like?
> 
> No idea.  If you tell me how to find out, I will try.  This system is
> behind firewalls up the kazoo, so it could be something essential for
> IPv6 DNS is blocked or intentionally disabled.

If you run 'ipconfig/all' from a command line (which, if you are not
familiar, you can launch by running 'cmd' from the 'Run' dialog,
which in turn can be accessed through the Windows+R shortcut), it
should display a bunch of information about active network connections,
including local IP addresses and DNS resolver addresses.

>> AI_V4MAPPED is meant for the case where an application expects only
>> IPv6-formatted addresses in a list, but a host with only IPv4 addresses
>> should have those addresses included in a mangled form. It is an
>> address format compatibility hack. (The compatibility hack extends
>> to other parts of the network stack, so that attempting to connect to
>> such an address actually results in IPv4 packets on the wire, etc.)
>
> Yes, I understand that much.  But it could be better than a total
> failure, at least in some use cases, no?  That's why this flag exists,
> right?

Not really. It's more a convenience for C programs that for whatever
reason don't want to or can't dispatch on the address family all the
time, potentially do multiple getaddrinfo calls, etc. In almost all cases,
handling v4 and v6 addresses as separate is saner, or at least I think so,
and it looks like we're in that straightforward dual-format world already
from elisp's perspective.

To put it differently: from my perspective, the main use for AI_V4MAPPED
is when there is no IPv4-handling code path at all, and all IPv4 addresses
are routed through the "IPv6" path as compatibility-mapped, which can
save some complexity and hassle elsewhere. If you code against "true"
multi-address-family, then it's just a faucet for bugs (say, trying
to filter out some IPv4 netblocks but then letting them through anyway
when they come through mapped into pseudo-IPv6 addresses). I'm not sure
all C network programmers would agree with me.

>> On a GNU system, I think this will do nothing; it's not clear to me
>> what AI_V4MAPPED would even mean when ai_family = AF_UNSPEC,
>> because AF_UNSPEC means IPv4 addresses can be returned directly.
> 
> According to the GNU/Linux getaddrinfo man page:
> 
>        If hints.ai_flags specifies the AI_V4MAPPED flag, and
>        hints.ai_family was specified as AF_INET6, and no matching IPv6
>        addresses could be found, then return IPv4-mapped IPv6 addresses
>        in the list pointed to by res.  If both AI_V4MAPPED and AI_ALL
>        are specified in hints.ai_flags, then return both IPv6 and
>        IPv4-mapped IPv6 addresses in the list pointed to by res.  AI_ALL
>        is ignored if AI_V4MAPPED is not also specified.
> 
> So it does sound like these flags do have effect on GNU/Linux.

If ai_family is AF_INET6, it does. I don't think it has an effect if
ai_family is AF_UNSPEC. I am not 100% sure, but a quick test with
resolving some names from a GNU/Linux machine (v4-only, v6-only,
dual-stack) with AF_UNSPEC didn't return anything different with
AI_V4MAPPED or AI_V4MAPPED | AI_ALL compared to none of those.

>> What about "localhost"?
> Works as expected, without EAI_NODATA.

I interpret "as expected" to mean that when you ask for localhost
in AF_INET6, you get ::1 back. Is that right?

> On the
> system in question, using the patched code and nil as FAMILY in the
> network-lookup-address-info yields the expected result, including,
> surprisingly, what looks like a valid IPv6 address, even though
> specifying 'ipv6 for FAMILY does indeed return a compatibility-hacked
> IPv4 address.

What happens if you pass nil for FAMILY but without the AI_V4MAPPED
patch, on that system? Have you observed AI_V4MAPPED producing better
results in any circumstance when you do not set FAMILY to ipv6?

I conjecture that your upstream resolver is blocking AAAA queries, and
that AF_UNSPEC is going through a slightly different path, maybe even
issuing an ANY query which is actually responded to with records from
both address families, which are then passed through because response
filtering and query filtering are different. If this is close to true,
I'm not sure there's a sane way for an application to handle
inconsistent resolver behavior of that stripe.

> If this is controversial, maybe having it as opt-in behavior,
> controlled by some new optional argument, would be useful?

If it turns out to be something Windows-specific, you could of course
conditionalize on that.

If it's network-specific, and there's an actual need to handle it,
then maybe a global "workaround for this particular bad behavior"
toggle, but that could be starting down a rabbit hole, because there's
a lot of possible network-specific bogus behaviors.

It would separately be sane to add a hint-list argument or something
to network-lookup-address-info and derive ai_flags from that; if it
is determined that AI_V4MAPPED is actually desirable sometimes, then
that would be a good way to go about it, essentially passing the
getaddrinfo behavior up one level.

But let's see what's actually going on first.

> I found about this issue because 2 tests in test/src/process-tests.el
> failed, both of them called network-lookup-address-info with second
> arg 'ipv6'.

You mean lookup-family-specification and lookup-google, I imagine?
Hmm. If I get a chance, I will see what Emacs does with the forms
from those tests on one of my Windows machines and report back.

-RTT