emacs-dev: 'unpack' phase does not seem to preserve timestamps

emacs-dev: 'unpack' phase does not seem to preserve timestamps

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 3372 bytes --]


I'm trying to build an Emacs package of my local git checkout:

--8<---------------cut here---------------start------------->8---
(define-public emacs-dev
  (package
    (inherit emacs)
    (name "emacs-dev")
    (version "27.0.0")
    (source (local-file "/home/ambrevar/projects/emacs" #:recursive? #t))
    (arguments
     `(#:tests? #f               ; TODO: Enable tests?  Need to fix tramp first.
                #:phases
                (modify-phases %standard-phases
                  (delete 'reset-gzip-timestamps) ; TODO: Why does this fail?
                  (delete 'build))))
    (synopsis "Emacs (development version)")
    (license license:lgpl3+)))
--8<---------------cut here---------------end--------------->8---

The above declaration works on a clean checkout (e.g. after a `make
clean`).

When developing and building locally multiple times, it is more
efficient to only run the install phase during Guix packaging.

Emacs is based on the GNU build system which on Guix uses a separate
"source" and "build" folder.  Unfortunately, some Emacs files are
generated during the build phase inside the "source" folder no matter
what.  (Am I wrong about that?)

Since all file timestamps are set to Epoch by 'unpack', the build phase
tries to rebuild everything.
And since the files unpacked to "source" are read-only, `make` fails to
regenerate the files that go in "source".

Excerpt from the build log:

--8<---------------cut here---------------start------------->8---
make -C ../admin/charsets all
make[2]: Entering directory '/tmp/guix-build-emacs-dev-27.0.0.drv-0/build/admin/charsets'
  GEN      ../../../source/etc/charsets/JISX2131.map
/gnu/store/icz3hd36aqpjz5slyp4hhr8wsfbgiml1-bash-minimal-4.4.12/bin/bash: line 2: ../../../source/etc/charsets/JISX2131.map: Permission denied
--8<---------------cut here---------------end--------------->8---

And the offending files details (notice the permissions and the mtime):

- Source /home/ambrevar/projects/emacs/etc/charsets/JISX2131.map:
-rw-r--r--   1 ambrevar       users    120k 2018-05-17 15:08 JISX2131.map

- Destination /tmp/guix-build-emacs-dev-27.0.0.drv-0/source/etc/charsets/JISX2131.map:
-r--r--r--   1 ambrevar       users    120k 1970-01-01  1970 JISX2131.map

The same error occurs for all the generated files (mostly .map files and
stamp files).

From the source, 'unpack' is supposed to preserve the timestamp:

--8<---------------cut here---------------start------------->8---
(define* (unpack #:key source #:allow-other-keys)
  "Unpack SOURCE in the working directory, and change directory within the
source.  When SOURCE is a directory, copy it in a sub-directory of the current
working directory."
  (if (file-is-directory? source)
      (begin
        (mkdir "source")
        (chdir "source")

        ;; Preserve timestamps (set to the Epoch) on the copied tree so that
        ;; things work deterministically.
        (copy-recursively source "."
                          #:keep-mtime? #t)
        #t)
      (and (if (string-suffix? ".zip" source)
               (zero? (system* "unzip" source))
               (zero? (system* "tar" "xvf" source)))
           (chdir (first-subdirectory ".")))))
--8<---------------cut here---------------end--------------->8---

Bug?

--
Pierre Neidhardt

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: emacs-dev: 'unpack' phase does not seem to preserve timestamps

[Ludovic Courtès]

Hello,

Pierre Neidhardt <ambrevar@gmail.com> skribis:

> Since all file timestamps are set to Epoch by 'unpack', the build phase
> tries to rebuild everything.
> And since the files unpacked to "source" are read-only, `make` fails to
> regenerate the files that go in "source".

Yes, that’s annoying, but that’s expected.  Perhaps the ‘unpack’
phase could automatically make the copy that it creates with
‘copy-recursively’ writable?

> From the source, 'unpack' is supposed to preserve the timestamp:
>
> (define* (unpack #:key source #:allow-other-keys)
>   "Unpack SOURCE in the working directory, and change directory within the
> source.  When SOURCE is a directory, copy it in a sub-directory of the current
> working directory."
>   (if (file-is-directory? source)
>       (begin
>         (mkdir "source")
>         (chdir "source")
>
>         ;; Preserve timestamps (set to the Epoch) on the copied tree so that
>         ;; things work deterministically.
>         (copy-recursively source "."
>                           #:keep-mtime? #t)

It does preserve timestamps, but since SOURCE is in the store, its
timestamps are all set to the Epoch.

But that’s OK: Git doesn’t specify timestamps anyway.

Thanks,
Ludo’.

Re: emacs-dev: 'unpack' phase does not seem to preserve timestamps

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 503 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Yes, that’s annoying, but that’s expected.  

Right, but then how would it be possible to call `make install` without
rebuilding everything all over again?

> Perhaps the ‘unpack’
> phase could automatically make the copy that it creates with
> ‘copy-recursively’ writable?

That wouldn't solve the issue: it would build but the point is to skip
the compilation phase.

Can you think of a way around this?

-- 
Pierre Neidhardt

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: emacs-dev: 'unpack' phase does not seem to preserve timestamps

[Ludovic Courtès]

Pierre Neidhardt <ambrevar@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Yes, that’s annoying, but that’s expected.  
>
> Right, but then how would it be possible to call `make install` without
> rebuilding everything all over again?
>
>> Perhaps the ‘unpack’
>> phase could automatically make the copy that it creates with
>> ‘copy-recursively’ writable?
>
> That wouldn't solve the issue: it would build but the point is to skip
> the compilation phase.
>
> Can you think of a way around this?

Nope.  Again, Git does not “preserve” timestamps (rightly so IMO), so
there’s nothing we can do.

Ludo’.

Re: emacs-dev: 'unpack' phase does not seem to preserve timestamps

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 401 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Nope.  Again, Git does not “preserve” timestamps (rightly so IMO), so
> there’s nothing we can do.

I think there is a misunderstanding: I'm working on a local folder, this
has nothing to do with git.

From my 'emacs-dev' package definition:

	(source (local-file "/home/ambrevar/projects/emacs" #:recursive? #t))

-- 
Pierre Neidhardt

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: emacs-dev: 'unpack' phase does not seem to preserve timestamps

[Ludovic Courtès]

Pierre Neidhardt <ambrevar@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Nope.  Again, Git does not “preserve” timestamps (rightly so IMO), so
>> there’s nothing we can do.
>
> I think there is a misunderstanding: I'm working on a local folder, this
> has nothing to do with git.
>
> From my 'emacs-dev' package definition:
>
> 	(source (local-file "/home/ambrevar/projects/emacs" #:recursive? #t))

Oh sorry.  It’s a similar kind of issue: when that directory is added to
the store, its timestamps are lost.  One workaround would be to create a
tarball out of it, and *then* add that tarball to the store.  Not
difficult to do, but a bit cumbersome.

Ludo’.

Re: emacs-dev: 'unpack' phase does not seem to preserve timestamps

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1640 bytes --]


Cumbersome indeed.  I also thought of installing to a local "output"
folder and using this as source.  The issue is that the prefix (as in
"./configure --prefix=") must be set to the store output of emacs-dev.
Otherwise Emacs won't find its Lisp folders.

I can't think of a clean way to do it.

Let's go back to my original intention: I want to do is a clean way to
install work-in-progress projects.  On traditional Unices, this can be
done by calling `make install` to /usr/local (possibly with the help of
`stow`).

On Guix, installing to /usr/local won't help: in all cases we've got to
set all the *PATH variables to folders of the locally compiled project.
I initially thought that the Guix package manager could help but it
seems that I was wrong: Guix is made to _build_ packages reproducibly,
not to install some random files.

Now I'm thinking that a better approach to this issue is to install,
say, in ~/.local and to set all PATH variables appropriately, e.g.

--8<---------------cut here---------------start------------->8---
export C_INCLUDE_PATH="$HOME/.local/include":$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH="$HOME/.local/include":$CPLUS_INCLUDE_PATH
export LIBRARY_PATH="$HOME/.local/lib":$LIBRARY_PATH
export PKG_CONFIG_PATH="$HOME/.local/lib/pkgconfig":$PKG_CONFIG_PATH
export INFOPATH="$HOME/.local/share/info":$INFOPATH
export MANPATH="$HOME/.local/share/man":$MANPATH
# ...
--8<---------------cut here---------------end--------------->8---

Lots of work ahead! :p

For emacs, maybe a better way is to simply add
~/projects/emacs/src/emacs to the PATH.

Does anyone know of a better approach?

--
Pierre Neidhardt

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]