‘core-updates’ is back!

‘core-updates’ is back!

[Ludovic Courtès]

Hello Guix!

Yup, I just created a new ‘core-updates’ branch by pushing
<https://bugs.gnu.org/27849>.  Enjoy!

Let’s freeze in one month, say Oct. 1st?

Ludo’.

Re: ‘core-updates’ is back!

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 753 bytes --]

On Tue, Aug 29, 2017 at 12:01:40AM +0200, Ludovic Courtès wrote:
> Hello Guix!
> 
> Yup, I just created a new ‘core-updates’ branch by pushing
> <https://bugs.gnu.org/27849>.  Enjoy!
> 
> Let’s freeze in one month, say Oct. 1st?
> 
> Ludo’.
> 

Lets also have a reminder about a week before to make sure we get things
like tzdata.

I'm currently test building texinfo@6.4. I tried a blind update of
glibc, binutils, texinfo and grep but it failed during building a gcc,
so now i'm breaking it into smaller chunks.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: ‘core-updates’ is back!

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1192 bytes --]

On Tue, Aug 29, 2017 at 11:44:53AM +0300, Efraim Flashner wrote:
> On Tue, Aug 29, 2017 at 12:01:40AM +0200, Ludovic Courtès wrote:
> > Hello Guix!
> > 
> > Yup, I just created a new ‘core-updates’ branch by pushing
> > <https://bugs.gnu.org/27849>.  Enjoy!
> > 
> > Let’s freeze in one month, say Oct. 1st?
> > 
> > Ludo’.
> > 
> 
> Lets also have a reminder about a week before to make sure we get things
> like tzdata.
> 
> I'm currently test building texinfo@6.4. I tried a blind update of
> glibc, binutils, texinfo and grep but it failed during building a gcc,
> so now i'm breaking it into smaller chunks.
> 

It looks like binutils@2.29.1 is coming mid-September¹ so I'm not
touching binutils for now. I'm not sure why updating glibc is causing
gcc to fail to build.

Also, in gnu/packages/commencement, should we change package/inherit
back to 'package (inherit...'?


¹ https://sourceware.org/ml/binutils/2017-08/msg00176.html


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: ‘core-updates’ is back!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1337 bytes --]

Efraim Flashner <efraim@flashner.co.il> writes:

> On Tue, Aug 29, 2017 at 11:44:53AM +0300, Efraim Flashner wrote:
>> On Tue, Aug 29, 2017 at 12:01:40AM +0200, Ludovic Courtès wrote:
>> > Hello Guix!
>> > 
>> > Yup, I just created a new ‘core-updates’ branch by pushing
>> > <https://bugs.gnu.org/27849>.  Enjoy!
>> > 
>> > Let’s freeze in one month, say Oct. 1st?
>> > 
>> > Ludo’.
>> > 
>> 
>> Lets also have a reminder about a week before to make sure we get things
>> like tzdata.
>> 
>> I'm currently test building texinfo@6.4. I tried a blind update of
>> glibc, binutils, texinfo and grep but it failed during building a gcc,
>> so now i'm breaking it into smaller chunks.
>> 
>
> It looks like binutils@2.29.1 is coming mid-September¹ so I'm not
> touching binutils for now. I'm not sure why updating glibc is causing
> gcc to fail to build.

Can you post the glibc update patch?

Probably unrelated, but we also have to change the '--enable-kernel'
configure flag to say '3.2.0' since that is the minimum supported by
glibc 2.26.

Also, one month, yay!  Hopefully a shorter schedule will reduce the
amount of build issues during each round.

Since we're on the topic, I would like to switch to GCC 6 or 7 soon...
Are we agile enough to use the very latest GCC by default yet? :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: ‘core-updates’ is back!

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2094 bytes --]

On Wed, Aug 30, 2017 at 09:10:42PM +0200, Marius Bakke wrote:
> Efraim Flashner <efraim@flashner.co.il> writes:
> 
> > On Tue, Aug 29, 2017 at 11:44:53AM +0300, Efraim Flashner wrote:
> >> On Tue, Aug 29, 2017 at 12:01:40AM +0200, Ludovic Courtès wrote:
> >> > Hello Guix!
> >> > 
> >> > Yup, I just created a new ‘core-updates’ branch by pushing
> >> > <https://bugs.gnu.org/27849>.  Enjoy!
> >> > 
> >> > Let’s freeze in one month, say Oct. 1st?
> >> > 
> >> > Ludo’.
> >> > 
> >> 
> >> Lets also have a reminder about a week before to make sure we get things
> >> like tzdata.
> >> 
> >> I'm currently test building texinfo@6.4. I tried a blind update of
> >> glibc, binutils, texinfo and grep but it failed during building a gcc,
> >> so now i'm breaking it into smaller chunks.
> >> 
> >
> > It looks like binutils@2.29.1 is coming mid-September¹ so I'm not
> > touching binutils for now. I'm not sure why updating glibc is causing
> > gcc to fail to build.
> 
> Can you post the glibc update patch?

I didn't actually keep a patch of the update. 2.25->2.26, the 3 CVE
patches and the memchr-overflow and vectorized-strcspn get dropped. in
gnu/packages/commencement.scm I changed the two 'package/inherit' to
'package (inherit '.

> 
> Probably unrelated, but we also have to change the '--enable-kernel'
> configure flag to say '3.2.0' since that is the minimum supported by
> glibc 2.26.

I didn't change this in my patch.

> 
> Also, one month, yay!  Hopefully a shorter schedule will reduce the
> amount of build issues during each round.
> 
> Since we're on the topic, I would like to switch to GCC 6 or 7 soon...
> Are we agile enough to use the very latest GCC by default yet? :-)

That would be nice to at least move to GCC 6. My aarch64 board is
currently idle, I can see how well it works on my machine.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: ‘core-updates’ is back!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 530 bytes --]

Efraim Flashner <efraim@flashner.co.il> writes:

> On Wed, Aug 30, 2017 at 09:10:42PM +0200, Marius Bakke wrote:
>> Since we're on the topic, I would like to switch to GCC 6 or 7 soon...
>> Are we agile enough to use the very latest GCC by default yet? :-)
>
> That would be nice to at least move to GCC 6. My aarch64 board is
> currently idle, I can see how well it works on my machine.

The main issue with GCC 6 is that we need to port the SOURCE_DATE_EPOCH
patches again.  But we might want to do that even if switching to 7.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: ‘core-updates’ is back!

[Ludovic Courtès]

Marius Bakke <mbakke@fastmail.com> skribis:

> Efraim Flashner <efraim@flashner.co.il> writes:
>
>> On Wed, Aug 30, 2017 at 09:10:42PM +0200, Marius Bakke wrote:
>>> Since we're on the topic, I would like to switch to GCC 6 or 7 soon...
>>> Are we agile enough to use the very latest GCC by default yet? :-)
>>
>> That would be nice to at least move to GCC 6. My aarch64 board is
>> currently idle, I can see how well it works on my machine.
>
> The main issue with GCC 6 is that we need to port the SOURCE_DATE_EPOCH
> patches again.  But we might want to do that even if switching to 7.

I think we should upgrade.  My preference would be GCC 6, which I think
may trigger fewer build failures than GCC 7, but maybe GCC 7 would be
fine.

Are you sure the SOURCE_DATE_EPOCH thing isn’t already in GCC 6?

Thanks,
Ludo’.

Re: ‘core-updates’ is back!

[Marius Bakke]

[-- Attachment #1.1: Type: text/plain, Size: 1290 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Marius Bakke <mbakke@fastmail.com> skribis:
>
>> Efraim Flashner <efraim@flashner.co.il> writes:
>>
>>> On Wed, Aug 30, 2017 at 09:10:42PM +0200, Marius Bakke wrote:
>>>> Since we're on the topic, I would like to switch to GCC 6 or 7 soon...
>>>> Are we agile enough to use the very latest GCC by default yet? :-)
>>>
>>> That would be nice to at least move to GCC 6. My aarch64 board is
>>> currently idle, I can see how well it works on my machine.
>>
>> The main issue with GCC 6 is that we need to port the SOURCE_DATE_EPOCH
>> patches again.  But we might want to do that even if switching to 7.
>
> I think we should upgrade.  My preference would be GCC 6, which I think
> may trigger fewer build failures than GCC 7, but maybe GCC 7 would be
> fine.
>
> Are you sure the SOURCE_DATE_EPOCH thing isn’t already in GCC 6?

I just checked out the gcc-6_4_0-release tag and ran `git grep
SOURCE_DATE_EPOCH`.  No results :/

However I tried cherry-picking the two commits and there was only one
trivial conflict in gcc/c-family/c-common.h (apart from ChangeLog
updates, which were omitted).  Patch attached and building!

I also prefer going through GCC 6 before 7.  Let's go for it, assuming
this patch works.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-gnu-gcc-6-Respect-SOURCE_DATE_EPOCH-in-__DATE__-and-.patch --]
[-- Type: text/x-patch, Size: 22877 bytes --]

From 329e016e35c51426ccf30aa2335bfd3ef12678a6 Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke@fastmail.com>
Date: Thu, 31 Aug 2017 18:13:08 +0200
Subject: [PATCH] gnu: gcc@6: Respect SOURCE_DATE_EPOCH in __DATE__ and
 __TIME__ macros.

* gnu/packages/patches/gcc-6-source-date-epoch-1.patch,
gnu/packages/patches/gcc-6-source-date-epoch-2.patch: New files.
* gnu/local.mk (dist_patch_DATA): Register them.
* gnu/packages/gcc.scm (gcc-6)[source]: Use them.
---
 gnu/local.mk                                       |   2 +
 gnu/packages/gcc.scm                               |   2 +
 .../patches/gcc-6-source-date-epoch-1.patch        | 187 +++++++++++
 .../patches/gcc-6-source-date-epoch-2.patch        | 346 +++++++++++++++++++++
 4 files changed, 537 insertions(+)
 create mode 100644 gnu/packages/patches/gcc-6-source-date-epoch-1.patch
 create mode 100644 gnu/packages/patches/gcc-6-source-date-epoch-2.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 175434bcd..0bc64179f 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -628,6 +628,8 @@ dist_patch_DATA =						\
   %D%/packages/patches/gcc-5-source-date-epoch-2.patch		\
   %D%/packages/patches/gcc-6-arm-none-eabi-multilib.patch	\
   %D%/packages/patches/gcc-6-cross-environment-variables.patch	\
+  %D%/packages/patches/gcc-6-source-date-epoch-1.patch		\
+  %D%/packages/patches/gcc-6-source-date-epoch-2.patch		\
   %D%/packages/patches/gcr-disable-failing-tests.patch		\
   %D%/packages/patches/gcr-fix-collection-tests-to-work-with-gpg-21.patch	\
   %D%/packages/patches/gdk-pixbuf-list-dir.patch		\
diff --git a/gnu/packages/gcc.scm b/gnu/packages/gcc.scm
index da4d66ce3..e8efb0e42 100644
--- a/gnu/packages/gcc.scm
+++ b/gnu/packages/gcc.scm
@@ -402,6 +402,8 @@ Go.  It also includes runtime support libraries for these languages.")
                (base32
                 "1m0lr7938lw5d773dkvwld90hjlcq2282517d1gwvrfzmwgg42w5"))
               (patches (search-patches "gcc-strmov-store-file-names.patch"
+                                       "gcc-6-source-date-epoch-1.patch"
+                                       "gcc-6-source-date-epoch-2.patch"
                                        "gcc-5.0-libvtv-runpath.patch"))))))
 (define-public gcc-7
   (package
diff --git a/gnu/packages/patches/gcc-6-source-date-epoch-1.patch b/gnu/packages/patches/gcc-6-source-date-epoch-1.patch
new file mode 100644
index 000000000..26f62bc9f
--- /dev/null
+++ b/gnu/packages/patches/gcc-6-source-date-epoch-1.patch
@@ -0,0 +1,187 @@
+Make GCC respect SOURCE_DATE_EPOCH in __DATE__ and __TIME__ macros.
+
+Cherry-picked from upstream commit:
+
+https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=e3e8c48c4a494d9da741c1c8ea6c4c0b7c4ff934
+
+diff --git a/gcc/c-family/c-common.c b/gcc/c-family/c-common.c
+index 6cf8c610b4e..b5daea65ba7 100644
+--- a/gcc/c-family/c-common.c
++++ b/gcc/c-family/c-common.c
+@@ -12750,4 +12750,37 @@ valid_array_size_p (location_t loc, tree type, tree name)
+   return true;
+ }
+ 
++/* Read SOURCE_DATE_EPOCH from environment to have a deterministic
++   timestamp to replace embedded current dates to get reproducible
++   results.  Returns -1 if SOURCE_DATE_EPOCH is not defined.  */
++time_t
++get_source_date_epoch ()
++{
++  char *source_date_epoch;
++  long long epoch;
++  char *endptr;
++
++  source_date_epoch = getenv ("SOURCE_DATE_EPOCH");
++  if (!source_date_epoch)
++    return (time_t) -1;
++
++  errno = 0;
++  epoch = strtoll (source_date_epoch, &endptr, 10);
++  if ((errno == ERANGE && (epoch == LLONG_MAX || epoch == LLONG_MIN))
++      || (errno != 0 && epoch == 0))
++    fatal_error (UNKNOWN_LOCATION, "environment variable $SOURCE_DATE_EPOCH: "
++		 "strtoll: %s\n", xstrerror(errno));
++  if (endptr == source_date_epoch)
++    fatal_error (UNKNOWN_LOCATION, "environment variable $SOURCE_DATE_EPOCH: "
++		 "no digits were found: %s\n", endptr);
++  if (*endptr != '\0')
++    fatal_error (UNKNOWN_LOCATION, "environment variable $SOURCE_DATE_EPOCH: "
++		 "trailing garbage: %s\n", endptr);
++  if (epoch < 0)
++    fatal_error (UNKNOWN_LOCATION, "environment variable $SOURCE_DATE_EPOCH: "
++		 "value must be nonnegative: %lld \n", epoch);
++
++  return (time_t) epoch;
++}
++
+ #include "gt-c-family-c-common.h"
+diff --git a/gcc/c-family/c-common.h b/gcc/c-family/c-common.h
+index dd74d0dd62e..c6e0ed12b55 100644
+--- a/gcc/c-family/c-common.h
++++ b/gcc/c-family/c-common.h
+@@ -1467,4 +1467,9 @@ extern bool reject_gcc_builtin (const_tree, location_t = UNKNOWN_LOCATION);
+ extern void warn_duplicated_cond_add_or_warn (location_t, tree, vec<tree> **);
+ extern bool valid_array_size_p (location_t, tree, tree);
+ 
++/* Read SOURCE_DATE_EPOCH from environment to have a deterministic
++   timestamp to replace embedded current dates to get reproducible
++   results.  Returns -1 if SOURCE_DATE_EPOCH is not defined.  */
++extern time_t get_source_date_epoch (void);
++
+ #endif /* ! GCC_C_COMMON_H */
+diff --git a/gcc/c-family/c-lex.c b/gcc/c-family/c-lex.c
+index 96da4fc974e..bf1db6c0252 100644
+--- a/gcc/c-family/c-lex.c
++++ b/gcc/c-family/c-lex.c
+@@ -385,6 +385,9 @@ c_lex_with_flags (tree *value, location_t *loc, unsigned char *cpp_flags,
+   enum cpp_ttype type;
+   unsigned char add_flags = 0;
+   enum overflow_type overflow = OT_NONE;
++  time_t source_date_epoch = get_source_date_epoch ();
++
++  cpp_init_source_date_epoch (parse_in, source_date_epoch);
+ 
+   timevar_push (TV_CPP);
+  retry:
+diff --git a/gcc/doc/cppenv.texi b/gcc/doc/cppenv.texi
+index 22c8cb37624..e958e93e97e 100644
+--- a/gcc/doc/cppenv.texi
++++ b/gcc/doc/cppenv.texi
+@@ -79,4 +79,21 @@ main input file is omitted.
+ @ifclear cppmanual
+ @xref{Preprocessor Options}.
+ @end ifclear
++
++@item SOURCE_DATE_EPOCH
++
++If this variable is set, its value specifies a UNIX timestamp to be
++used in replacement of the current date and time in the @code{__DATE__}
++and @code{__TIME__} macros, so that the embedded timestamps become
++reproducible.
++
++The value of @env{SOURCE_DATE_EPOCH} must be a UNIX timestamp,
++defined as the number of seconds (excluding leap seconds) since
++01 Jan 1970 00:00:00 represented in ASCII, identical to the output of
++@samp{@command{date +%s}}.
++
++The value should be a known timestamp such as the last modification
++time of the source or package and it should be set by the build
++process.
++
+ @end vtable
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 35b0375c09c..4998b3a8ab8 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -784,6 +784,9 @@ extern void cpp_init_special_builtins (cpp_reader *);
+ /* Set up built-ins like __FILE__.  */
+ extern void cpp_init_builtins (cpp_reader *, int);
+ 
++/* Initialize the source_date_epoch value.  */
++extern void cpp_init_source_date_epoch (cpp_reader *, time_t);
++
+ /* This is called after options have been parsed, and partially
+    processed.  */
+ extern void cpp_post_options (cpp_reader *);
+diff --git a/libcpp/init.c b/libcpp/init.c
+index 4343075ba85..f5ff85b3bae 100644
+--- a/libcpp/init.c
++++ b/libcpp/init.c
+@@ -533,8 +533,15 @@ cpp_init_builtins (cpp_reader *pfile, int hosted)
+     _cpp_define_builtin (pfile, "__OBJC__ 1");
+ }
+ 
++/* Initialize the source_date_epoch value.  */
++void
++cpp_init_source_date_epoch (cpp_reader *pfile, time_t source_date_epoch)
++{
++  pfile->source_date_epoch = source_date_epoch; 
++}
++
+ /* Sanity-checks are dependent on command-line options, so it is
+-   called as a subroutine of cpp_read_main_file ().  */
++   called as a subroutine of cpp_read_main_file.  */
+ #if CHECKING_P
+ static void sanity_checks (cpp_reader *);
+ static void sanity_checks (cpp_reader *pfile)
+diff --git a/libcpp/internal.h b/libcpp/internal.h
+index 9ce870738cc..e3eb26b1f27 100644
+--- a/libcpp/internal.h
++++ b/libcpp/internal.h
+@@ -502,6 +502,10 @@ struct cpp_reader
+   const unsigned char *date;
+   const unsigned char *time;
+ 
++  /* Externally set timestamp to replace current date and time useful for
++     reproducibility.  */
++  time_t source_date_epoch;
++
+   /* EOF token, and a token forcing paste avoidance.  */
+   cpp_token avoid_paste;
+   cpp_token eof;
+diff --git a/libcpp/macro.c b/libcpp/macro.c
+index c2515534504..c2a83764660 100644
+--- a/libcpp/macro.c
++++ b/libcpp/macro.c
+@@ -357,13 +357,20 @@ _cpp_builtin_macro_text (cpp_reader *pfile, cpp_hashnode *node,
+ 	  time_t tt;
+ 	  struct tm *tb = NULL;
+ 
+-	  /* (time_t) -1 is a legitimate value for "number of seconds
+-	     since the Epoch", so we have to do a little dance to
+-	     distinguish that from a genuine error.  */
+-	  errno = 0;
+-	  tt = time(NULL);
+-	  if (tt != (time_t)-1 || errno == 0)
+-	    tb = localtime (&tt);
++	  /* Set a reproducible timestamp for __DATE__ and __TIME__ macro
++	     usage if SOURCE_DATE_EPOCH is defined.  */
++	  if (pfile->source_date_epoch != (time_t) -1)
++	     tb = gmtime (&pfile->source_date_epoch);
++	  else
++	    {
++	      /* (time_t) -1 is a legitimate value for "number of seconds
++		 since the Epoch", so we have to do a little dance to
++		 distinguish that from a genuine error.  */
++	      errno = 0;
++	      tt = time (NULL);
++	      if (tt != (time_t)-1 || errno == 0)
++		tb = localtime (&tt);
++	    }
+ 
+ 	  if (tb)
+ 	    {
+-- 
+2.14.1
+
diff --git a/gnu/packages/patches/gcc-6-source-date-epoch-2.patch b/gnu/packages/patches/gcc-6-source-date-epoch-2.patch
new file mode 100644
index 000000000..cd5b09867
--- /dev/null
+++ b/gnu/packages/patches/gcc-6-source-date-epoch-2.patch
@@ -0,0 +1,346 @@
+Cherry-picked from upstream commit:
+
+https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=dfa5c0d3f3e23e4fdb14857a42de376d9ff8601c
+
+diff --git a/gcc/c-family/c-common.c b/gcc/c-family/c-common.c
+index b5daea65ba7..a4539da15ce 100644
+--- a/gcc/c-family/c-common.c
++++ b/gcc/c-family/c-common.c
+@@ -12753,8 +12753,9 @@ valid_array_size_p (location_t loc, tree type, tree name)
+ /* Read SOURCE_DATE_EPOCH from environment to have a deterministic
+    timestamp to replace embedded current dates to get reproducible
+    results.  Returns -1 if SOURCE_DATE_EPOCH is not defined.  */
++
+ time_t
+-get_source_date_epoch ()
++cb_get_source_date_epoch (cpp_reader *pfile ATTRIBUTE_UNUSED)
+ {
+   char *source_date_epoch;
+   long long epoch;
+@@ -12766,19 +12767,14 @@ get_source_date_epoch ()
+ 
+   errno = 0;
+   epoch = strtoll (source_date_epoch, &endptr, 10);
+-  if ((errno == ERANGE && (epoch == LLONG_MAX || epoch == LLONG_MIN))
+-      || (errno != 0 && epoch == 0))
+-    fatal_error (UNKNOWN_LOCATION, "environment variable $SOURCE_DATE_EPOCH: "
+-		 "strtoll: %s\n", xstrerror(errno));
+-  if (endptr == source_date_epoch)
+-    fatal_error (UNKNOWN_LOCATION, "environment variable $SOURCE_DATE_EPOCH: "
+-		 "no digits were found: %s\n", endptr);
+-  if (*endptr != '\0')
+-    fatal_error (UNKNOWN_LOCATION, "environment variable $SOURCE_DATE_EPOCH: "
+-		 "trailing garbage: %s\n", endptr);
+-  if (epoch < 0)
+-    fatal_error (UNKNOWN_LOCATION, "environment variable $SOURCE_DATE_EPOCH: "
+-		 "value must be nonnegative: %lld \n", epoch);
++  if (errno != 0 || endptr == source_date_epoch || *endptr != '\0'
++      || epoch < 0 || epoch > MAX_SOURCE_DATE_EPOCH)
++    {
++      error_at (input_location, "environment variable SOURCE_DATE_EPOCH must "
++	        "expand to a non-negative integer less than or equal to %wd",
++		MAX_SOURCE_DATE_EPOCH);
++      return (time_t) -1;
++    }
+ 
+   return (time_t) epoch;
+ }
+diff --git a/gcc/c-family/c-common.h b/gcc/c-family/c-common.h
+index c6e0ed12b55..44699f716e0 100644
+--- a/gcc/c-family/c-common.h
++++ b/gcc/c-family/c-common.h
+@@ -1084,6 +1084,16 @@ extern vec<tree, va_gc> *make_tree_vector_copy (const vec<tree, va_gc> *);
+    c_register_builtin_type.  */
+ extern GTY(()) tree registered_builtin_types;
+ 
++/* Read SOURCE_DATE_EPOCH from environment to have a deterministic
++   timestamp to replace embedded current dates to get reproducible
++   results.  Returns -1 if SOURCE_DATE_EPOCH is not defined.  */
++extern time_t cb_get_source_date_epoch (cpp_reader *pfile);
++
++/* The value (as a unix timestamp) corresponds to date
++   "Dec 31 9999 23:59:59 UTC", which is the latest date that __DATE__ and
++   __TIME__ can store.  */
++#define MAX_SOURCE_DATE_EPOCH HOST_WIDE_INT_C (253402300799)
++
+ /* In c-gimplify.c  */
+ extern void c_genericize (tree);
+ extern int c_gimplify_expr (tree *, gimple_seq *, gimple_seq *);
+@@ -1467,9 +1477,4 @@ extern bool reject_gcc_builtin (const_tree, location_t = UNKNOWN_LOCATION);
+ extern void warn_duplicated_cond_add_or_warn (location_t, tree, vec<tree> **);
+ extern bool valid_array_size_p (location_t, tree, tree);
+ 
+-/* Read SOURCE_DATE_EPOCH from environment to have a deterministic
+-   timestamp to replace embedded current dates to get reproducible
+-   results.  Returns -1 if SOURCE_DATE_EPOCH is not defined.  */
+-extern time_t get_source_date_epoch (void);
+-
+ #endif /* ! GCC_C_COMMON_H */
+diff --git a/gcc/c-family/c-lex.c b/gcc/c-family/c-lex.c
+index bf1db6c0252..42a4135d339 100644
+--- a/gcc/c-family/c-lex.c
++++ b/gcc/c-family/c-lex.c
+@@ -80,6 +80,7 @@ init_c_lex (void)
+   cb->valid_pch = c_common_valid_pch;
+   cb->read_pch = c_common_read_pch;
+   cb->has_attribute = c_common_has_attribute;
++  cb->get_source_date_epoch = cb_get_source_date_epoch;
+ 
+   /* Set the debug callbacks if we can use them.  */
+   if ((debug_info_level == DINFO_LEVEL_VERBOSE
+@@ -385,9 +386,6 @@ c_lex_with_flags (tree *value, location_t *loc, unsigned char *cpp_flags,
+   enum cpp_ttype type;
+   unsigned char add_flags = 0;
+   enum overflow_type overflow = OT_NONE;
+-  time_t source_date_epoch = get_source_date_epoch ();
+-
+-  cpp_init_source_date_epoch (parse_in, source_date_epoch);
+ 
+   timevar_push (TV_CPP);
+  retry:
+diff --git a/gcc/doc/cppenv.texi b/gcc/doc/cppenv.texi
+index e958e93e97e..8cefd529aa3 100644
+--- a/gcc/doc/cppenv.texi
++++ b/gcc/doc/cppenv.texi
+@@ -81,7 +81,6 @@ main input file is omitted.
+ @end ifclear
+ 
+ @item SOURCE_DATE_EPOCH
+-
+ If this variable is set, its value specifies a UNIX timestamp to be
+ used in replacement of the current date and time in the @code{__DATE__}
+ and @code{__TIME__} macros, so that the embedded timestamps become
+@@ -89,8 +88,9 @@ reproducible.
+ 
+ The value of @env{SOURCE_DATE_EPOCH} must be a UNIX timestamp,
+ defined as the number of seconds (excluding leap seconds) since
+-01 Jan 1970 00:00:00 represented in ASCII, identical to the output of
+-@samp{@command{date +%s}}.
++01 Jan 1970 00:00:00 represented in ASCII; identical to the output of
++@samp{@command{date +%s}} on GNU/Linux and other systems that support the
++@code{%s} extension in the @code{date} command.
+ 
+ The value should be a known timestamp such as the last modification
+ time of the source or package and it should be set by the build
+diff --git a/gcc/gcc.c b/gcc/gcc.c
+index cfa074d4e43..f88596219bc 100644
+--- a/gcc/gcc.c
++++ b/gcc/gcc.c
+@@ -3541,6 +3541,29 @@ save_switch (const char *opt, size_t n_args, const char *const *args,
+   n_switches++;
+ }
+ 
++/* Set the SOURCE_DATE_EPOCH environment variable to the current time if it is
++   not set already.  */
++
++static void
++set_source_date_epoch_envvar ()
++{
++  /* Array size is 21 = ceil(log_10(2^64)) + 1 to hold string representations
++     of 64 bit integers.  */
++  char source_date_epoch[21];
++  time_t tt;
++
++  errno = 0;
++  tt = time (NULL);
++  if (tt < (time_t) 0 || errno != 0)
++    tt = (time_t) 0;
++
++  snprintf (source_date_epoch, 21, "%llu", (unsigned long long) tt);
++  /* Using setenv instead of xputenv because we want the variable to remain
++     after finalizing so that it's still set in the second run when using
++     -fcompare-debug.  */
++  setenv ("SOURCE_DATE_EPOCH", source_date_epoch, 0);
++}
++
+ /* Handle an option DECODED that is unknown to the option-processing
+    machinery.  */
+ 
+@@ -3840,6 +3863,7 @@ driver_handle_option (struct gcc_options *opts,
+       else
+ 	compare_debug_opt = arg;
+       save_switch (compare_debug_replacement_opt, 0, NULL, validated, true);
++      set_source_date_epoch_envvar ();
+       return true;
+ 
+     case OPT_fdiagnostics_color_:
+diff --git a/gcc/testsuite/gcc.dg/cpp/source_date_epoch-1.c b/gcc/testsuite/gcc.dg/cpp/source_date_epoch-1.c
+new file mode 100644
+index 00000000000..f6aa1a360ff
+--- /dev/null
++++ b/gcc/testsuite/gcc.dg/cpp/source_date_epoch-1.c
+@@ -0,0 +1,11 @@
++/* { dg-do run } */
++/* { dg-set-compiler-env-var SOURCE_DATE_EPOCH "630333296" } */
++
++int
++main(void)
++{
++  __builtin_printf ("%s %s\n", __DATE__, __TIME__);
++  return 0;
++}
++
++/* { dg-output "^Dec 22 1989 12:34:56\n$" } */
+diff --git a/gcc/testsuite/gcc.dg/cpp/source_date_epoch-2.c b/gcc/testsuite/gcc.dg/cpp/source_date_epoch-2.c
+new file mode 100644
+index 00000000000..ae18362ae87
+--- /dev/null
++++ b/gcc/testsuite/gcc.dg/cpp/source_date_epoch-2.c
+@@ -0,0 +1,12 @@
++/* { dg-do compile } */
++/* { dg-set-compiler-env-var SOURCE_DATE_EPOCH "AAA" } */
++
++/* Make sure that SOURCE_DATE_EPOCH is only parsed once */
++
++int
++main(void)
++{
++  __builtin_printf ("%s %s\n", __DATE__, __TIME__); /* { dg-error "SOURCE_DATE_EPOCH must expand" } */
++  __builtin_printf ("%s %s\n", __DATE__, __TIME__);
++  return 0;
++}
+diff --git a/gcc/testsuite/lib/gcc-dg.exp b/gcc/testsuite/lib/gcc-dg.exp
+index 9dd909b0985..822d2fbb3b9 100644
+--- a/gcc/testsuite/lib/gcc-dg.exp
++++ b/gcc/testsuite/lib/gcc-dg.exp
+@@ -450,6 +450,38 @@ proc restore-target-env-var { } {
+     }
+ }
+ 
++proc dg-set-compiler-env-var { args } {
++    global set_compiler_env_var
++    global saved_compiler_env_var
++    if { [llength $args] != 3 } {
++	error "dg-set-compiler-env-var: need two arguments"
++	return
++    }
++    set var [lindex $args 1]
++    set value [lindex $args 2]
++    if [info exists ::env($var)] {
++      lappend saved_compiler_env_var [list $var 1 $::env($var)]
++    } else {
++      lappend saved_compiler_env_var [list $var 0]
++    }
++    setenv $var $value
++    lappend set_compiler_env_var [list $var $value]
++}
++
++proc restore-compiler-env-var { } {
++    global saved_compiler_env_var
++    for { set env_vari [llength $saved_compiler_env_var] } {
++          [incr env_vari -1] >= 0 } {} {
++	set env_var [lindex $saved_compiler_env_var $env_vari]
++	set var [lindex $env_var 0]
++	if [lindex $env_var 1] {
++	    setenv $var [lindex $env_var 2]
++	} else {
++	    unsetenv $var
++	}
++    }
++}
++
+ # Utility routines.
+ 
+ #
+@@ -873,6 +905,11 @@ if { [info procs saved-dg-test] == [list] } {
+ 	if [info exists set_target_env_var] {
+ 	    unset set_target_env_var
+ 	}
++	if [info exists set_compiler_env_var] {
++	    restore-compiler-env-var
++	    unset set_compiler_env_var
++	    unset saved_compiler_env_var
++	}
+ 	if [info exists keep_saved_temps_suffixes] {
+ 	    unset keep_saved_temps_suffixes
+ 	}
+diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
+index 4998b3a8ab8..9d70cc856ef 100644
+--- a/libcpp/include/cpplib.h
++++ b/libcpp/include/cpplib.h
+@@ -594,6 +594,9 @@ struct cpp_callbacks
+ 
+   /* Callback that can change a user builtin into normal macro.  */
+   bool (*user_builtin_macro) (cpp_reader *, cpp_hashnode *);
++
++  /* Callback to parse SOURCE_DATE_EPOCH from environment.  */
++  time_t (*get_source_date_epoch) (cpp_reader *);
+ };
+ 
+ #ifdef VMS
+@@ -784,9 +787,6 @@ extern void cpp_init_special_builtins (cpp_reader *);
+ /* Set up built-ins like __FILE__.  */
+ extern void cpp_init_builtins (cpp_reader *, int);
+ 
+-/* Initialize the source_date_epoch value.  */
+-extern void cpp_init_source_date_epoch (cpp_reader *, time_t);
+-
+ /* This is called after options have been parsed, and partially
+    processed.  */
+ extern void cpp_post_options (cpp_reader *);
+diff --git a/libcpp/init.c b/libcpp/init.c
+index f5ff85b3bae..e78b3206def 100644
+--- a/libcpp/init.c
++++ b/libcpp/init.c
+@@ -257,6 +257,9 @@ cpp_create_reader (enum c_lang lang, cpp_hash_table *table,
+   /* Do not force token locations by default.  */
+   pfile->forced_token_location_p = NULL;
+ 
++  /* Initialize source_date_epoch to -2 (not yet set).  */
++  pfile->source_date_epoch = (time_t) -2;
++
+   /* The expression parser stack.  */
+   _cpp_expand_op_stack (pfile);
+ 
+@@ -533,13 +536,6 @@ cpp_init_builtins (cpp_reader *pfile, int hosted)
+     _cpp_define_builtin (pfile, "__OBJC__ 1");
+ }
+ 
+-/* Initialize the source_date_epoch value.  */
+-void
+-cpp_init_source_date_epoch (cpp_reader *pfile, time_t source_date_epoch)
+-{
+-  pfile->source_date_epoch = source_date_epoch; 
+-}
+-
+ /* Sanity-checks are dependent on command-line options, so it is
+    called as a subroutine of cpp_read_main_file.  */
+ #if CHECKING_P
+diff --git a/libcpp/internal.h b/libcpp/internal.h
+index e3eb26b1f27..cea32ec73c6 100644
+--- a/libcpp/internal.h
++++ b/libcpp/internal.h
+@@ -503,7 +503,8 @@ struct cpp_reader
+   const unsigned char *time;
+ 
+   /* Externally set timestamp to replace current date and time useful for
+-     reproducibility.  */
++     reproducibility.  It should be initialized to -2 (not yet set) and
++     set to -1 to disable it or to a non-negative value to enable it.  */
+   time_t source_date_epoch;
+ 
+   /* EOF token, and a token forcing paste avoidance.  */
+diff --git a/libcpp/macro.c b/libcpp/macro.c
+index c2a83764660..a3b8348a23f 100644
+--- a/libcpp/macro.c
++++ b/libcpp/macro.c
+@@ -358,9 +358,13 @@ _cpp_builtin_macro_text (cpp_reader *pfile, cpp_hashnode *node,
+ 	  struct tm *tb = NULL;
+ 
+ 	  /* Set a reproducible timestamp for __DATE__ and __TIME__ macro
+-	     usage if SOURCE_DATE_EPOCH is defined.  */
+-	  if (pfile->source_date_epoch != (time_t) -1)
+-	     tb = gmtime (&pfile->source_date_epoch);
++	     if SOURCE_DATE_EPOCH is defined.  */
++	  if (pfile->source_date_epoch == (time_t) -2
++	      && pfile->cb.get_source_date_epoch != NULL)
++	    pfile->source_date_epoch = pfile->cb.get_source_date_epoch (pfile);
++
++	  if (pfile->source_date_epoch >= (time_t) 0)
++	    tb = gmtime (&pfile->source_date_epoch);
+ 	  else
+ 	    {
+ 	      /* (time_t) -1 is a legitimate value for "number of seconds
+-- 
+2.14.1
+
-- 
2.14.1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: ‘core-updates’ is back!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 2023 bytes --]

Marius Bakke <mbakke@fastmail.com> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Marius Bakke <mbakke@fastmail.com> skribis:
>>
>>> Efraim Flashner <efraim@flashner.co.il> writes:
>>>
>>>> On Wed, Aug 30, 2017 at 09:10:42PM +0200, Marius Bakke wrote:
>>>>> Since we're on the topic, I would like to switch to GCC 6 or 7 soon...
>>>>> Are we agile enough to use the very latest GCC by default yet? :-)
>>>>
>>>> That would be nice to at least move to GCC 6. My aarch64 board is
>>>> currently idle, I can see how well it works on my machine.
>>>
>>> The main issue with GCC 6 is that we need to port the SOURCE_DATE_EPOCH
>>> patches again.  But we might want to do that even if switching to 7.
>>
>> I think we should upgrade.  My preference would be GCC 6, which I think
>> may trigger fewer build failures than GCC 7, but maybe GCC 7 would be
>> fine.
>>
>> Are you sure the SOURCE_DATE_EPOCH thing isn’t already in GCC 6?
>
> I just checked out the gcc-6_4_0-release tag and ran `git grep
> SOURCE_DATE_EPOCH`.  No results :/
>
> However I tried cherry-picking the two commits and there was only one
> trivial conflict in gcc/c-family/c-common.h (apart from ChangeLog
> updates, which were omitted).  Patch attached and building!

It works!

$ cat sde.c
#include <stdio.h>
int main()
{
    printf( "__DATE__ says: %s \n", __DATE__ );
    printf( "__TIME__ says: %s \n", __TIME__ );
    return 0;
}
$ ./pre-inst-env guix environment --pure --ad-hoc gcc@6 binutils glibc
bash: tty: command not found
[env]$ gcc sde.c && ./a.out
__DATE__ says: Aug 31 2017 
__TIME__ says: 17:09:07 
[env]$ SOURCE_DATE_EPOCH=1 gcc sde.c && ./a.out
__DATE__ says: Jan  1 1970 
__TIME__ says: 00:00:01 
[env]$ SOURCE_DATE_EPOCH=-1 gcc sde.c && ./a.out
sde.c: In function 'main':
sde.c:4:5: error: environment variable SOURCE_DATE_EPOCH must expand to a non-negative integer less than or equal to 253402300799
     printf( "__DATE__ says: %s \n", __DATE__ );
     ^~~~~~

\o/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: ‘core-updates’ is back!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1438 bytes --]

Marius Bakke <mbakke@fastmail.com> writes:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> Marius Bakke <mbakke@fastmail.com> skribis:
>>>
>>>> Efraim Flashner <efraim@flashner.co.il> writes:
>>>>
>>>>> On Wed, Aug 30, 2017 at 09:10:42PM +0200, Marius Bakke wrote:
>>>>>> Since we're on the topic, I would like to switch to GCC 6 or 7 soon...
>>>>>> Are we agile enough to use the very latest GCC by default yet? :-)
>>>>>
>>>>> That would be nice to at least move to GCC 6. My aarch64 board is
>>>>> currently idle, I can see how well it works on my machine.
>>>>
>>>> The main issue with GCC 6 is that we need to port the SOURCE_DATE_EPOCH
>>>> patches again.  But we might want to do that even if switching to 7.
>>>
>>> I think we should upgrade.  My preference would be GCC 6, which I think
>>> may trigger fewer build failures than GCC 7, but maybe GCC 7 would be
>>> fine.
>>>
>>> Are you sure the SOURCE_DATE_EPOCH thing isn’t already in GCC 6?
>>
>> I just checked out the gcc-6_4_0-release tag and ran `git grep
>> SOURCE_DATE_EPOCH`.  No results :/
>>
>> However I tried cherry-picking the two commits and there was only one
>> trivial conflict in gcc/c-family/c-common.h (apart from ChangeLog
>> updates, which were omitted).  Patch attached and building!
>
> It works!

Is it okay to push this GCC-6 SOURCE_DATE_EPOCH patch to 'master'?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: ‘core-updates’ is back!

[Ludovic Courtès]

Hey Marius,

Marius Bakke <mbakke@fastmail.com> skribis:

> However I tried cherry-picking the two commits and there was only one
> trivial conflict in gcc/c-family/c-common.h (apart from ChangeLog
> updates, which were omitted).  Patch attached and building!

Woohoo!

> I also prefer going through GCC 6 before 7.  Let's go for it, assuming
> this patch works.
>
> From 329e016e35c51426ccf30aa2335bfd3ef12678a6 Mon Sep 17 00:00:00 2001
> From: Marius Bakke <mbakke@fastmail.com>
> Date: Thu, 31 Aug 2017 18:13:08 +0200
> Subject: [PATCH] gnu: gcc@6: Respect SOURCE_DATE_EPOCH in __DATE__ and
>  __TIME__ macros.
>
> * gnu/packages/patches/gcc-6-source-date-epoch-1.patch,
> gnu/packages/patches/gcc-6-source-date-epoch-2.patch: New files.
> * gnu/local.mk (dist_patch_DATA): Register them.
> * gnu/packages/gcc.scm (gcc-6)[source]: Use them.

Looks OK to me for your favorite branch!  (Could be both ‘master’ and
‘core-updates’ after all.)

For the switch to GCC 6 in ‘core-updates’, you can get inspiration from
commit b810a85019ab3c4ee1f889d0751b8eb06157dadc, and then do something
like “guix build sed” for testing.

Ludo’.

Re: ‘core-updates’ is back!

[Ricardo Wurmus]

Marius Bakke <mbakke@fastmail.com> writes:

> Probably unrelated, but we also have to change the '--enable-kernel'
> configure flag to say '3.2.0' since that is the minimum supported by
> glibc 2.26.

Does this mean that binaries built with Guix will need at least Linux
3.2.0 to be executed?

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: ‘core-updates’ is back!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 640 bytes --]

Ricardo Wurmus <rekado@elephly.net> writes:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> Probably unrelated, but we also have to change the '--enable-kernel'
>> configure flag to say '3.2.0' since that is the minimum supported by
>> glibc 2.26.
>
> Does this mean that binaries built with Guix will need at least Linux
> 3.2.0 to be executed?

I believe so, according to the release notes[0]:

* Linux kernel 3.2 or later is required at runtime, on all architectures
  supported by that kernel.  (This is a change from version 2.25 only for
  x86-32 and x86-64.)

[0] https://lists.gnu.org/archive/html/info-gnu/2017-08/msg00000.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: ‘core-updates’ is back!

[Andy Wingo]

On Tue 29 Aug 2017 00:01, ludo@gnu.org (Ludovic Courtès) writes:

> Yup, I just created a new ‘core-updates’ branch by pushing
> <https://bugs.gnu.org/27849>.  Enjoy!
>
> Let’s freeze in one month, say Oct. 1st?

It would be pretty cool if we could fix our O(n^2) problems in search
paths in this core-updates -- basically whenever you go to create an
environment, instead of making e.g. VAR=A:B:C:..., for all VARs
(LIBRARY_PATH, PKG_CONFIG_PATH, etc), instead we make a union directory
Z containing the union of A, B, C, etc and set VAR=Z.  The goal would be
to fix quadratic run-time lookup costs by replacing it with a
compile-time computation.  This applies to many lookups: PATH, -rpath,
etc.

For specific variables we might also have to rewrite some internal
references, for example to replace the -L line in pkg-config files to
point to the union directory.  Dunno if there are more examples or if
PKG_CONFIG_PATH is the only one.

Andy

Re: ‘core-updates’ is back!

[Ricardo Wurmus]

Andy Wingo <wingo@igalia.com> writes:

> On Tue 29 Aug 2017 00:01, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Yup, I just created a new ‘core-updates’ branch by pushing
>> <https://bugs.gnu.org/27849>.  Enjoy!
>>
>> Let’s freeze in one month, say Oct. 1st?
>
> It would be pretty cool if we could fix our O(n^2) problems in search
> paths in this core-updates

I agree, this would be very nice!

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: ‘core-updates’ is back!

[宋文武]

Andy Wingo <wingo@igalia.com> writes:

> On Tue 29 Aug 2017 00:01, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Yup, I just created a new ‘core-updates’ branch by pushing
>> <https://bugs.gnu.org/27849>.  Enjoy!
>>
>> Let’s freeze in one month, say Oct. 1st?
>
> It would be pretty cool if we could fix our O(n^2) problems in search
> paths in this core-updates -- basically whenever you go to create an
> environment, instead of making e.g. VAR=A:B:C:..., for all VARs
> (LIBRARY_PATH, PKG_CONFIG_PATH, etc), instead we make a union directory
> Z containing the union of A, B, C, etc and set VAR=Z.  The goal would be
> to fix quadratic run-time lookup costs by replacing it with a
> compile-time computation.  This applies to many lookups: PATH, -rpath,
> etc.

The runtime references scanner of daemon will be inefficient if the
union gets recorded with all the inputs contained, and we likely won't
apply this to packages with multiple outputs (eg: fcitx) which will make
every output including the union into its runtime reference.

Re: ‘core-updates’ is back!

[Ludovic Courtès]

iyzsong@member.fsf.org (宋文武) skribis:

> Andy Wingo <wingo@igalia.com> writes:
>
>> On Tue 29 Aug 2017 00:01, ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> Yup, I just created a new ‘core-updates’ branch by pushing
>>> <https://bugs.gnu.org/27849>.  Enjoy!
>>>
>>> Let’s freeze in one month, say Oct. 1st?
>>
>> It would be pretty cool if we could fix our O(n^2) problems in search
>> paths in this core-updates -- basically whenever you go to create an
>> environment, instead of making e.g. VAR=A:B:C:..., for all VARs
>> (LIBRARY_PATH, PKG_CONFIG_PATH, etc), instead we make a union directory
>> Z containing the union of A, B, C, etc and set VAR=Z.  The goal would be
>> to fix quadratic run-time lookup costs by replacing it with a
>> compile-time computation.  This applies to many lookups: PATH, -rpath,
>> etc.
>
> The runtime references scanner of daemon will be inefficient if the
> union gets recorded with all the inputs contained, and we likely won't
> apply this to packages with multiple outputs (eg: fcitx) which will make
> every output including the union into its runtime reference.

Indeed, good point.

Ludo’.

Re: ‘core-updates’ is back!

[Ludovic Courtès]

Andy Wingo <wingo@igalia.com> skribis:

> On Tue 29 Aug 2017 00:01, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Yup, I just created a new ‘core-updates’ branch by pushing
>> <https://bugs.gnu.org/27849>.  Enjoy!
>>
>> Let’s freeze in one month, say Oct. 1st?
>
> It would be pretty cool if we could fix our O(n^2) problems in search
> paths in this core-updates -- basically whenever you go to create an
> environment, instead of making e.g. VAR=A:B:C:..., for all VARs
> (LIBRARY_PATH, PKG_CONFIG_PATH, etc), instead we make a union directory
> Z containing the union of A, B, C, etc and set VAR=Z.  The goal would be
> to fix quadratic run-time lookup costs by replacing it with a
> compile-time computation.  This applies to many lookups: PATH, -rpath,
> etc.

A possible alternative solution for ld.so is at
<https://bugs.gnu.org/26048>.

I’m not entirely sure about the idea of creating union directories
everywhere.  A problem is that we’d be retaining lots of union
directories, and even if they’re cheap, that could become non-negligible
(for every package we’d have to download/build an extra derivation).  It
would also add an extra level of symlinks to go through when one is
debugging things.  Also, some tools might not notice that
/abc…/lib/libfoo, /def…/lib/libfoo, and /123…/lib/libfoo actually are
the same thing.

Thoughts?

Ludo’.

Re: ‘core-updates’ is back!

[Ricardo Wurmus]

Andy wrote:

> It would be pretty cool if we could fix our O(n^2) problems in search
> paths in this core-updates -- basically whenever you go to create an
> environment, instead of making e.g. VAR=A:B:C:..., for all VARs
> (LIBRARY_PATH, PKG_CONFIG_PATH, etc), instead we make a union directory
> Z containing the union of A, B, C, etc and set VAR=Z.  The goal would be
> to fix quadratic run-time lookup costs by replacing it with a
> compile-time computation.  This applies to many lookups: PATH, -rpath,
> etc.

Ludo wrote:

> A possible alternative solution for ld.so is at
> <https://bugs.gnu.org/26048>.

Do you know of any negative side effects this would have?  For example,
would it not become impossible to override libraries at runtime with
LD_LIBRARY_PATH or LD_PRELOAD when “-lfoo” is replaced by the absolute
file name of the “foo” library?

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#26048: ‘core-updates’ is back!

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Andy wrote:
>
>> It would be pretty cool if we could fix our O(n^2) problems in search
>> paths in this core-updates -- basically whenever you go to create an
>> environment, instead of making e.g. VAR=A:B:C:..., for all VARs
>> (LIBRARY_PATH, PKG_CONFIG_PATH, etc), instead we make a union directory
>> Z containing the union of A, B, C, etc and set VAR=Z.  The goal would be
>> to fix quadratic run-time lookup costs by replacing it with a
>> compile-time computation.  This applies to many lookups: PATH, -rpath,
>> etc.
>
> Ludo wrote:
>
>> A possible alternative solution for ld.so is at
>> <https://bugs.gnu.org/26048>.
>
> Do you know of any negative side effects this would have?  For example,
> would it not become impossible to override libraries at runtime with
> LD_LIBRARY_PATH or LD_PRELOAD when “-lfoo” is replaced by the absolute
> file name of the “foo” library?

Good question.  Per the ELF v1.2 spec¹ (page 82), setting DT_NEEDED to
an absolute file name would prevent overriding via LD_LIBRARY_PATH,
which is not desirable.

It turns out we cannot even set DT_NEEDED to an absolute file name in
the first place, because ld records the DT_SONAME of the library, when
it’s available (which is the case most of the time), rather than its
file name.

IOW, the approach suggested in <https://bugs.gnu.org/26048> isn’t really
applicable.  :-/

Ludo’.

¹ http://refspecs.linuxbase.org/elf/elf.pdf