Neon and reverse address lookup

Neon and reverse address lookup

[Andreas Enge]

Hi,

when trying to package the neon library:
   http://www.webdav.org/neon/ ,
the tests fail inside the chroot with
   addr_reverse (reverse lookup for 127.0.0.1 failed)

Should I disable the corresponding test?

Andreas

Re: Neon and reverse address lookup

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 563 bytes --]

On Sat, Jan 18, 2014 at 01:48:47PM +0100, Andreas Enge wrote:
     Hi,
     
     when trying to package the neon library:
        http://www.webdav.org/neon/ ,
     the tests fail inside the chroot with
        addr_reverse (reverse lookup for 127.0.0.1 failed)

I thought that the lo interface should be available inside the chroot.
So 127.0.0.1 should resolve ok.
     

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Neon and reverse address lookup

[Mark H Weaver]

John Darrington <john@darrington.wattle.id.au> writes:

> On Sat, Jan 18, 2014 at 01:48:47PM +0100, Andreas Enge wrote:
>      Hi,
>      
>      when trying to package the neon library:
>         http://www.webdav.org/neon/ ,
>      the tests fail inside the chroot with
>         addr_reverse (reverse lookup for 127.0.0.1 failed)
>
> I thought that the lo interface should be available inside the chroot.
> So 127.0.0.1 should resolve ok.

127.0.0.1 works in the chroot, but DNS name lookups (either forward or
reverse) do not.

      Mark

Re: Neon and reverse address lookup

[Mark H Weaver]

Andreas Enge <andreas@enge.fr> writes:

> when trying to package the neon library:
>    http://www.webdav.org/neon/ ,
> the tests fail inside the chroot with
>    addr_reverse (reverse lookup for 127.0.0.1 failed)
>
> Should I disable the corresponding test?

I think we should include /etc/hosts in the chroot with an entry for
localhost.  We might also need /etc/nsswitch.conf; I'm not sure.  I see
no reason not to do this, and I suspect it would ultimately take less
time and effort than disabling tests.

When I suggested this on IRC, Ludovic seemed open to the idea.

   Thoughts?
      Mark

Re: Neon and reverse address lookup

[Andreas Enge]

On Sat, Jan 18, 2014 at 09:41:14AM -0500, Mark H Weaver wrote:
> I think we should include /etc/hosts in the chroot with an entry for
> localhost.  We might also need /etc/nsswitch.conf; I'm not sure.  I see
> no reason not to do this, and I suspect it would ultimately take less
> time and effort than disabling tests.
> 
> When I suggested this on IRC, Ludovic seemed open to the idea.
> 
>    Thoughts?

If it can be implemented easily, this sounds like a good idea. I think we
already had another package with the same problem.

Andreas

Re: Neon and reverse address lookup

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]

On Sat, Jan 18, 2014 at 05:45:08PM +0100, Andreas Enge wrote:
     On Sat, Jan 18, 2014 at 09:41:14AM -0500, Mark H Weaver wrote:
     > I think we should include /etc/hosts in the chroot with an entry for
     > localhost.  We might also need /etc/nsswitch.conf; I'm not sure.  I see
     > no reason not to do this, and I suspect it would ultimately take less
     > time and effort than disabling tests.
     > 
     > When I suggested this on IRC, Ludovic seemed open to the idea.
     > 
     >    Thoughts?
     
     If it can be implemented easily, this sounds like a good idea. I think we
     already had another package with the same problem.
     
Trying to package gnumeric comes up with a similar problem.  Its tests rely
on dbus.  Which it seems needs a file /etc/machine-id  Can we add that one
too?

J'
     

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Neon and reverse address lookup

[Ludovic Courtès]

John Darrington <john@darrington.wattle.id.au> skribis:

> On Sat, Jan 18, 2014 at 05:45:08PM +0100, Andreas Enge wrote:
>      On Sat, Jan 18, 2014 at 09:41:14AM -0500, Mark H Weaver wrote:
>      > I think we should include /etc/hosts in the chroot with an entry for
>      > localhost.  We might also need /etc/nsswitch.conf; I'm not sure.  I see
>      > no reason not to do this, and I suspect it would ultimately take less
>      > time and effort than disabling tests.
>      > 
>      > When I suggested this on IRC, Ludovic seemed open to the idea.
>      > 
>      >    Thoughts?
>      
>      If it can be implemented easily, this sounds like a good idea. I think we
>      already had another package with the same problem.
>      
> Trying to package gnumeric comes up with a similar problem.  Its tests rely
> on dbus.  Which it seems needs a file /etc/machine-id

What exactly needs it, dbus-daemon?  (May machine runs dbus-daemon 1.4
and I don’t have that file.)

Ludo’.

Re: Neon and reverse address lookup

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> Andreas Enge <andreas@enge.fr> writes:
>
>> when trying to package the neon library:
>>    http://www.webdav.org/neon/ ,
>> the tests fail inside the chroot with
>>    addr_reverse (reverse lookup for 127.0.0.1 failed)
>>
>> Should I disable the corresponding test?
>
> I think we should include /etc/hosts in the chroot with an entry for
> localhost.  We might also need /etc/nsswitch.conf; I'm not sure.  I see
> no reason not to do this, and I suspect it would ultimately take less
> time and effort than disabling tests.

Actually, I just discovered that it’s been the case since before Guix
was even started.  :-)

nix/libstore/build.cc reads this:

      /* Create /etc/hosts with localhost entry. */
      writeFile(chrootRootDir + "/etc/hosts", "127.0.0.1 localhost\n");

So lookups of ‘localhost’ should just work.

To make sure this is the case, try this:

--8<---------------cut here---------------start------------->8---
#!/path/to/bin/guile -s
!#

(use-modules (guix) (guix monads))

(define (with-/etc/hosts?)
  (define b
    '(and (file-exists? "/etc/hosts")
          (getaddrinfo "localhost")
          (call-with-output-file (assoc-ref %outputs "out")
            (const #t))))

  (mlet* %store-monad ((d (derivation-expression "etc-hosts" b)))
    (built-derivations (list d))))

(let* ((s (open-connection)))
  (run-with-store s (with-/etc/hosts?)))
--8<---------------cut here---------------end--------------->8---

I don’t think nsswitch.conf will be necessary.

Thanks,
Ludo’.

Re: Neon and reverse address lookup

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> when trying to package the neon library:
>    http://www.webdav.org/neon/ ,
> the tests fail inside the chroot with
>    addr_reverse (reverse lookup for 127.0.0.1 failed)

I just tried this and it fails with “Host name lookup failure”:

--8<---------------cut here---------------start------------->8---
#!/.../bin/guile -s
!#

(use-modules (guix) (guix monads))

(define (with-/etc/hosts?)
  (define b
    '(and (file-exists? "/etc/hosts")
          (pk 'gethostbyaddr
              (hostent:name (gethostbyaddr
                             (inet-pton AF_INET
                                        "127.0.0.1"))))
          (call-with-output-file (assoc-ref %outputs "out")
            (const #t))))

  (mlet* %store-monad ((d (derivation-expression "etc-hosts" b)))
    (built-derivations (list d))))

(let* ((s (open-connection)))
  (run-with-store s (with-/etc/hosts?)))
--8<---------------cut here---------------end--------------->8---

So /etc/hosts is not enough for reverse lookups to work.

With nscd turned off, I tried this:

  guile -c '(pk (gethostbyaddr (inet-pton AF_INET "127.0.0.1")))'

If nsswitch.conf is removed, then that resolves to “localhost” (instead
of the actual host name.)

If, in addition, resolv.conf is removed, then that fails as seen above.

With nsswitch.conf alone, it resolves correctly (Mark was right.)
And this works with only “hosts: files” in nsswitch.conf.

It’s surprising that there’s this discrepancy between how NSS works for
getaddrinfo and for gethostbyaddr.  At any rate, we may be able to solve
this at the libc level.

Thanks,
Ludo’.

Re: Neon and reverse address lookup

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 1285 bytes --]

I'm not sure if I'm reading your message as you intended, but for
the record, 127.0.0.1 should ALWAYS resolve to "localhost" - without
the domain name, and NOT to the canonical hostname.

If it does not, then a lot of things break.

J'


On Sat, Jan 18, 2014 at 10:32:31PM +0100, Ludovic Court??s wrote:
     
     So /etc/hosts is not enough for reverse lookups to work.
     
     With nscd turned off, I tried this:
     
       guile -c '(pk (gethostbyaddr (inet-pton AF_INET "127.0.0.1")))'
     
     If nsswitch.conf is removed, then that resolves to ???localhost??? (instead
     of the actual host name.)
     
     If, in addition, resolv.conf is removed, then that fails as seen above.
     
     With nsswitch.conf alone, it resolves correctly (Mark was right.)
     And this works with only ???hosts: files??? in nsswitch.conf.
     
     It???s surprising that there???s this discrepancy between how NSS works for
     getaddrinfo and for gethostbyaddr.  At any rate, we may be able to solve
     this at the libc level.
     
     Thanks,
     Ludo???.
     

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Neon and reverse address lookup

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> So /etc/hosts is not enough for reverse lookups to work.
>
> With nscd turned off, I tried this:
>
>   guile -c '(pk (gethostbyaddr (inet-pton AF_INET "127.0.0.1")))'
>
> If nsswitch.conf is removed, then that resolves to “localhost” (instead
> of the actual host name.)

As John said, it should resolve to "localhost".  It certainly should
_not_ resolve to the actual host name.

> If, in addition, resolv.conf is removed, then that fails as seen above.
>
> With nsswitch.conf alone, it resolves correctly (Mark was right.)
> And this works with only “hosts: files” in nsswitch.conf.

When you say "it resolves correctly", which name does it resolve to?

> It’s surprising that there’s this discrepancy between how NSS works for
> getaddrinfo and for gethostbyaddr.  At any rate, we may be able to solve
> this at the libc level.

Solve what?  What are you thinking of changing in libc?

     Mark

Re: Neon and reverse address lookup

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> So /etc/hosts is not enough for reverse lookups to work.
>>
>> With nscd turned off, I tried this:
>>
>>   guile -c '(pk (gethostbyaddr (inet-pton AF_INET "127.0.0.1")))'
>>
>> If nsswitch.conf is removed, then that resolves to “localhost” (instead
>> of the actual host name.)
>
> As John said, it should resolve to "localhost".  It certainly should
> _not_ resolve to the actual host name.

Right, but my /etc/hosts has “localhost” plus another name for 127.0.0.1.

>> If, in addition, resolv.conf is removed, then that fails as seen above.
>>
>> With nsswitch.conf alone, it resolves correctly (Mark was right.)
>> And this works with only “hosts: files” in nsswitch.conf.
>
> When you say "it resolves correctly", which name does it resolve to?

Whatever /etc/hosts suggests.

>> It’s surprising that there’s this discrepancy between how NSS works for
>> getaddrinfo and for gethostbyaddr.  At any rate, we may be able to solve
>> this at the libc level.
>
> Solve what?  What are you thinking of changing in libc?

The issue is that the default config for the ‘hosts’ database when
resolv.conf is missing is (see nss/hosts-lookup.c):

  dns [!UNAVAIL=return] files

When doing a host name lookup, ‘dns’ is unavailable, so we go on with
‘files’ and /etc/hosts is honored as expected.

However, when doing a reverse lookup, the DNS resolver
(__libc_res_nquery) returns ‘notfound’, and so ‘gethostbyaddr’ returns
immediately.

Changing the ‘hosts’ config to “dns files” solves this particular.

I’ll ask for advice on libc-help.

Thanks,
Ludo’.

Re: Neon and reverse address lookup

[Andreas Enge]

On Sun, Jan 19, 2014 at 04:40:43PM +0100, Ludovic Courtès wrote:
> The issue is that the default config for the ‘hosts’ database when
> resolv.conf is missing is (see nss/hosts-lookup.c):
>   dns [!UNAVAIL=return] files
> When doing a host name lookup, ‘dns’ is unavailable, so we go on with
> ‘files’ and /etc/hosts is honored as expected.
> However, when doing a reverse lookup, the DNS resolver
> (__libc_res_nquery) returns ‘notfound’, and so ‘gethostbyaddr’ returns
> immediately.
> Changing the ‘hosts’ config to “dns files” solves this particular.
> I’ll ask for advice on libc-help.

Did anything come out of this? How should I proceed for the neon package?

Andreas

Re: Neon and reverse address lookup

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> On Sun, Jan 19, 2014 at 04:40:43PM +0100, Ludovic Courtès wrote:
>> The issue is that the default config for the ‘hosts’ database when
>> resolv.conf is missing is (see nss/hosts-lookup.c):
>>   dns [!UNAVAIL=return] files
>> When doing a host name lookup, ‘dns’ is unavailable, so we go on with
>> ‘files’ and /etc/hosts is honored as expected.
>> However, when doing a reverse lookup, the DNS resolver
>> (__libc_res_nquery) returns ‘notfound’, and so ‘gethostbyaddr’ returns
>> immediately.
>> Changing the ‘hosts’ config to “dns files” solves this particular.
>> I’ll ask for advice on libc-help.
>
> Did anything come out of this?

Well, this has been reported:

  http://sourceware.org/ml/libc-alpha/2014-01/msg00351.html
  https://sourceware.org/bugzilla/show_bug.cgi?id=16475

I was hoping it would be fixed in time for 2.19, but it seems it won’t
happen.

> How should I proceed for the neon package?

If it’s not too difficult to skip just those tests that require reverse
lookup, that’s the best thing.  Otherwise just #:tests? #f.  (Either
way, with a comment pointing to the report above.)

Thanks,
Ludo’.