* Extra files in build container
@ 2017-06-15 7:29 Gregor Giesen
2017-06-16 7:58 ` Ludovic Courtès
0 siblings, 1 reply; 8+ messages in thread
From: Gregor Giesen @ 2017-06-15 7:29 UTC (permalink / raw
To: help-guix
Hello Guix,
I'm trying to pack unbound, however its unittests require /etc/protocols
and /etc/services. Now, I have found them in the net-base package
which I included in native-inputs, but that is not sufficient for
glibc's getprotoent and getservent to find them. Can I symlink the
net-base ones to the build container's /etc? Adding an extra phase
before 'check does not allow me to write /etc in the container.
Cheers,
Gregor
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Extra files in build container
2017-06-15 7:29 Extra files in build container Gregor Giesen
@ 2017-06-16 7:58 ` Ludovic Courtès
2017-06-17 6:58 ` Gregor Giesen
0 siblings, 1 reply; 8+ messages in thread
From: Ludovic Courtès @ 2017-06-16 7:58 UTC (permalink / raw
To: Gregor Giesen; +Cc: help-guix
Hi Gregor,
Gregor Giesen <giesen@zaehlwerk.net> skribis:
> I'm trying to pack unbound, however its unittests require /etc/protocols
> and /etc/services. Now, I have found them in the net-base package
> which I included in native-inputs, but that is not sufficient for
> glibc's getprotoent and getservent to find them. Can I symlink the
> net-base ones to the build container's /etc? Adding an extra phase
> before 'check does not allow me to write /etc in the container.
Indeed, /etc is read-only in the build container, and it contains only
the bare minimum:
https://www.gnu.org/software/guix/manual/html_node/Build-Environment-Setup.html
The code that populates it can be found at:
https://git.savannah.gnu.org/cgit/guix.git/tree/nix/libstore/build.cc#n1852
In cases like the one you describe, we usually end up modifying tests to
use the numerical values for services and protocols rather than their
names.
It would probably make sense to allow derivations to populate /etc
though.
HTH,
Ludo’.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Extra files in build container
2017-06-16 7:58 ` Ludovic Courtès
@ 2017-06-17 6:58 ` Gregor Giesen
2017-06-19 11:26 ` Ludovic Courtès
0 siblings, 1 reply; 8+ messages in thread
From: Gregor Giesen @ 2017-06-17 6:58 UTC (permalink / raw
To: help-guix
Dear Ludo’,
thanks for the help!
On Fri, Jun 16, 2017 at 09:58:55AM +0200, Ludovic Courtès wrote:
> In cases like the one you describe, we usually end up modifying tests to
> use the numerical values for services and protocols rather than their
> names.
Unfortunately, this turns out to be quite cumbersome since in my case
(unittests for unbound) there is a lot of test data to be modified and
in many cases not only plain text but also encrypted records (DNSSEC
tests). On the other hand the values to be looked up are mostly “udp”
and “tcp” in /etc/protocols and “domain” in /etc/services, so I decided
that using a preload library for these few glibc calls just in case of
the unittest should do the trick rather than no checks at all. However,
it is an ugly hack and bloats the package definition.
> It would probably make sense to allow derivations to populate /etc
> though.
That might help!
Many thanks to all contributers for this very nice peace of software!
Best,
Gregor
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Extra files in build container
2017-06-17 6:58 ` Gregor Giesen
@ 2017-06-19 11:26 ` Ludovic Courtès
2017-06-19 11:44 ` Gregor Giesen
0 siblings, 1 reply; 8+ messages in thread
From: Ludovic Courtès @ 2017-06-19 11:26 UTC (permalink / raw
To: Gregor Giesen; +Cc: help-guix
Hi Gregor,
Gregor Giesen <giesen@zaehlwerk.net> skribis:
> On Fri, Jun 16, 2017 at 09:58:55AM +0200, Ludovic Courtès wrote:
>> In cases like the one you describe, we usually end up modifying tests to
>> use the numerical values for services and protocols rather than their
>> names.
> Unfortunately, this turns out to be quite cumbersome since in my case
> (unittests for unbound) there is a lot of test data to be modified and
> in many cases not only plain text but also encrypted records (DNSSEC
> tests). On the other hand the values to be looked up are mostly “udp”
> and “tcp” in /etc/protocols and “domain” in /etc/services, so I decided
> that using a preload library for these few glibc calls just in case of
> the unittest should do the trick rather than no checks at all.
I think it would be easier to just use ‘substitute*’ to replace all the
occurrences of “tcp”, etc., wouldn’t it?
> However, it is an ugly hack and bloats the package definition.
I agree, but it’s hard to improve on it without compromising
reproducibility.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Extra files in build container
2017-06-19 11:26 ` Ludovic Courtès
@ 2017-06-19 11:44 ` Gregor Giesen
2017-06-19 14:49 ` Ludovic Courtès
0 siblings, 1 reply; 8+ messages in thread
From: Gregor Giesen @ 2017-06-19 11:44 UTC (permalink / raw
To: help-guix
Dear Ludo’,
On Mon, Jun 19, 2017 at 13:26AM +0200 Ludovic Courtès wrote:
>> On Fri, Jun 16, 2017 at 09:58:55AM +0200, Ludovic Courtès wrote:
>>> In cases like the one you describe, we usually end up modifying tests to
>>> use the numerical values for services and protocols rather than their
>>> names.
>> Unfortunately, this turns out to be quite cumbersome since in my case
>> (unittests for unbound) there is a lot of test data to be modified and
>> in many cases not only plain text but also encrypted records (DNSSEC
>> tests). On the other hand the values to be looked up are mostly “udp”
>> and “tcp” in /etc/protocols and “domain” in /etc/services, so I decided
>> that using a preload library for these few glibc calls just in case of
>> the unittest should do the trick rather than no checks at all.
>
> I think it would be easier to just use ‘substitute*’ to replace all the
> occurrences of “tcp”, etc., wouldn’t it?
alas many occurences of "tcp", "udp", etc. are hidden in
encrypted/hashed records, so simple plain text substitutions would break
the signatures.
>> However, it is an ugly hack and bloats the package definition.
> I agree, but it’s hard to improve on it without compromising
> reproducibility.
It's a good thing that the hack only affects the tests rather than any
installation files.
I have submitted the package in a patch yesterday (27419). I wonder
whether one might want to put the source code of the preload library in
an extra file rather than the package definition?
Cheers,
Gregor
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Extra files in build container
2017-06-19 11:44 ` Gregor Giesen
@ 2017-06-19 14:49 ` Ludovic Courtès
2017-06-19 16:48 ` Gregor Giesen
0 siblings, 1 reply; 8+ messages in thread
From: Ludovic Courtès @ 2017-06-19 14:49 UTC (permalink / raw
To: Gregor Giesen; +Cc: help-guix
Hello,
Gregor Giesen <giesen@zaehlwerk.net> skribis:
> On Mon, Jun 19, 2017 at 13:26AM +0200 Ludovic Courtès wrote:
>>> On Fri, Jun 16, 2017 at 09:58:55AM +0200, Ludovic Courtès wrote:
>>>> In cases like the one you describe, we usually end up modifying tests to
>>>> use the numerical values for services and protocols rather than their
>>>> names.
>>> Unfortunately, this turns out to be quite cumbersome since in my case
>>> (unittests for unbound) there is a lot of test data to be modified and
>>> in many cases not only plain text but also encrypted records (DNSSEC
>>> tests). On the other hand the values to be looked up are mostly “udp”
>>> and “tcp” in /etc/protocols and “domain” in /etc/services, so I decided
>>> that using a preload library for these few glibc calls just in case of
>>> the unittest should do the trick rather than no checks at all.
>>
>> I think it would be easier to just use ‘substitute*’ to replace all the
>> occurrences of “tcp”, etc., wouldn’t it?
> alas many occurences of "tcp", "udp", etc. are hidden in
> encrypted/hashed records, so simple plain text substitutions would
> break the signatures.
Yes, I can imagine. But maybe (maybe not!), by selecting the right set
of files to modify, and by adjusting the regexp (let’s say “\<tcp\>” or
“"tcp"”) to match only what matters, it could work.
If that’s really not workable, then the LD_PRELOAD approach is fine.
>>> However, it is an ugly hack and bloats the package definition.
>> I agree, but it’s hard to improve on it without compromising
>> reproducibility.
> It's a good thing that the hack only affects the tests rather than any
> installation files.
>
> I have submitted the package in a patch yesterday (27419). I wonder
> whether one might want to put the source code of the preload library
> in an extra file rather than the package definition?
Since it’s small I think it’s OK as inline code.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Extra files in build container
2017-06-19 14:49 ` Ludovic Courtès
@ 2017-06-19 16:48 ` Gregor Giesen
2017-06-19 21:21 ` Ludovic Courtès
0 siblings, 1 reply; 8+ messages in thread
From: Gregor Giesen @ 2017-06-19 16:48 UTC (permalink / raw
To: Ludovic Courtès; +Cc: help-guix
Dear Ludo’,
On Mon, Jun 19, 2017 at 04:49:43PM +0200, Ludovic Courtès wrote:
> >> I think it would be easier to just use ‘substitute*’ to replace all the
> >> occurrences of “tcp”, etc., wouldn’t it?
> > alas many occurences of "tcp", "udp", etc. are hidden in
> > encrypted/hashed records, so simple plain text substitutions would
> > break the signatures.
>
> Yes, I can imagine. But maybe (maybe not!), by selecting the right set
> of files to modify, and by adjusting the regexp (let’s say “\<tcp\>” or
> “"tcp"”) to match only what matters, it could work.
here is a sample record pair:
wks02.types-signed.wb.sidnlabs.nl. 60 IN WKS 10.0.0.1 udp 0 1 2 domain
wks02.types-signed.wb.sidnlabs.nl. 60 IN RRSIG WKS 8 5 60 20140201000000 20130930114324 62298 types-signed.wb.sidnlabs.nl. P+p1gcQbO4I/sBQUZktrz4Q1osWIFIFGlK8o+SzIXMjxmdNy36vjwW6Sfy97CycbLRFIQ2grPv/cPaXtoMb+usHCoDtl5sSvLTJFmg9hpQ+xm12GvunZvAYAGx9fZic+QvLahSg+cjqX1M0oR9B68gcx+duMdLzawlUcIqX8gmA=
So changing the first line requires to recreating the signature record
in the second line which I find much more difficult then the LD_PRELOAD.
> If that’s really not workable, then the LD_PRELOAD approach is fine.
> Since it’s small I think it’s OK as inline code.
Thanks, I just submitted a revised patch including your remarks and
some additional comments as explanation.
Best,
Gregor
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Extra files in build container
2017-06-19 16:48 ` Gregor Giesen
@ 2017-06-19 21:21 ` Ludovic Courtès
0 siblings, 0 replies; 8+ messages in thread
From: Ludovic Courtès @ 2017-06-19 21:21 UTC (permalink / raw
To: Gregor Giesen; +Cc: help-guix
Hi Gregor,
Gregor Giesen <giesen@zaehlwerk.net> skribis:
> On Mon, Jun 19, 2017 at 04:49:43PM +0200, Ludovic Courtès wrote:
>> >> I think it would be easier to just use ‘substitute*’ to replace all the
>> >> occurrences of “tcp”, etc., wouldn’t it?
>> > alas many occurences of "tcp", "udp", etc. are hidden in
>> > encrypted/hashed records, so simple plain text substitutions would
>> > break the signatures.
>>
>> Yes, I can imagine. But maybe (maybe not!), by selecting the right set
>> of files to modify, and by adjusting the regexp (let’s say “\<tcp\>” or
>> “"tcp"”) to match only what matters, it could work.
> here is a sample record pair:
>
> wks02.types-signed.wb.sidnlabs.nl. 60 IN WKS 10.0.0.1 udp 0 1 2 domain
> wks02.types-signed.wb.sidnlabs.nl. 60 IN RRSIG WKS 8 5 60 20140201000000 20130930114324 62298 types-signed.wb.sidnlabs.nl. P+p1gcQbO4I/sBQUZktrz4Q1osWIFIFGlK8o+SzIXMjxmdNy36vjwW6Sfy97CycbLRFIQ2grPv/cPaXtoMb+usHCoDtl5sSvLTJFmg9hpQ+xm12GvunZvAYAGx9fZic+QvLahSg+cjqX1M0oR9B68gcx+duMdLzawlUcIqX8gmA=
>
> So changing the first line requires to recreating the signature record
> in the second line which I find much more difficult then the LD_PRELOAD.
Indeed, I hadn’t realized that test data included such things. All
right then!
Ludo’.
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2017-06-19 21:21 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-06-15 7:29 Extra files in build container Gregor Giesen
2017-06-16 7:58 ` Ludovic Courtès
2017-06-17 6:58 ` Gregor Giesen
2017-06-19 11:26 ` Ludovic Courtès
2017-06-19 11:44 ` Gregor Giesen
2017-06-19 14:49 ` Ludovic Courtès
2017-06-19 16:48 ` Gregor Giesen
2017-06-19 21:21 ` Ludovic Courtès
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).