mtime of fresh .go

mtime of fresh .go

[Ludovic Courtès]

Hello!

Can you re-explain the rationale for this:

  commit 535fb833b34dfc3cc11a679d39390b06fd7e9180
  Author: Andy Wingo <wingo@pobox.com>
  Date:   Fri Jun 5 10:51:21 2009 +0200

      stamp .go with timestamp of .scm; a fresh go has same mtime of .scm

      * libguile/load.c (compiled_is_fresh): Rename from compiled_is_newer.
        Check that the mtines of the .go and .scm match exactly, so we don't
        get fooled by rsync-like modifications of the filesystem.

When packaging things “normally”, the .go has an mtime strictly greater
than that of the source file, so checking for equality doesn’t work.

(‘make’ implementations check target >= source, too.)

Also, the code is similar to ‘fresh-compiled-file-name’ in boot-9.scm.
Do you think they could be factorized?

Thanks,
Ludo’.

Re: mtime of fresh .go

[Andy Wingo]

Heya,

On Fri 16 Jul 2010 10:07, ludo@gnu.org (Ludovic Courtès) writes:

>   commit 535fb833b34dfc3cc11a679d39390b06fd7e9180
>   Author: Andy Wingo <wingo@pobox.com>
>   Date:   Fri Jun 5 10:51:21 2009 +0200
>
>       stamp .go with timestamp of .scm; a fresh go has same mtime of .scm
>
>       * libguile/load.c (compiled_is_fresh): Rename from compiled_is_newer.
>         Check that the mtines of the .go and .scm match exactly, so we don't
>         get fooled by rsync-like modifications of the filesystem.
>
> When packaging things “normally”, the .go has an mtime strictly greater
> than that of the source file, so checking for equality doesn’t work.

But when reinstalling code from a binary packaging system, sometimes the
mtime can go backwards. In a perfect world target >= source should be
sufficient but this is how all caches that I know of (the gtk icon
cache, the gstreamer registry, etc) do things.

> Also, the code is similar to ‘fresh-compiled-file-name’ in boot-9.scm.
> Do you think they could be factorized?

Tough to tell without a patch ;-) IIRC it's not the same code due to
boot considerations.

Andy
-- 
http://wingolog.org/

Re: mtime of fresh .go

[Ludovic Courtès]

Hey!

Andy Wingo <wingo@pobox.com> writes:

> On Fri 16 Jul 2010 10:07, ludo@gnu.org (Ludovic Courtès) writes:
>
>>   commit 535fb833b34dfc3cc11a679d39390b06fd7e9180
>>   Author: Andy Wingo <wingo@pobox.com>
>>   Date:   Fri Jun 5 10:51:21 2009 +0200
>>
>>       stamp .go with timestamp of .scm; a fresh go has same mtime of .scm
>>
>>       * libguile/load.c (compiled_is_fresh): Rename from compiled_is_newer.
>>         Check that the mtines of the .go and .scm match exactly, so we don't
>>         get fooled by rsync-like modifications of the filesystem.
>>
>> When packaging things “normally”, the .go has an mtime strictly greater
>> than that of the source file, so checking for equality doesn’t work.
>
> But when reinstalling code from a binary packaging system, sometimes the
> mtime can go backwards.

Please forgive my ignorance, but can you give an example of how this can
happen?

Besides, what do you think packages that install scm and go files should
do?

A package of mine basically lists .go’s in ‘nodist_foobar_DATA’ and
.scm’s in ‘foobar_SOURCES’ (or similar).  Having to change the mtime of
.go’s in ‘install-hook’ seems inconvenient and fragile to me.

Thanks,
Ludo’.

Re: mtime of fresh .go

[Andy Wingo]

Greets,

On Mon 19 Jul 2010 00:31, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> writes:
>
>> On Fri 16 Jul 2010 10:07, ludo@gnu.org (Ludovic Courtès) writes:
>>
>>>   commit 535fb833b34dfc3cc11a679d39390b06fd7e9180
>>>   Author: Andy Wingo <wingo@pobox.com>
>>>   Date:   Fri Jun 5 10:51:21 2009 +0200
>>>
>>>       stamp .go with timestamp of .scm; a fresh go has same mtime of .scm
>>>
>>>       * libguile/load.c (compiled_is_fresh): Rename from compiled_is_newer.
>>>         Check that the mtines of the .go and .scm match exactly, so we don't
>>>         get fooled by rsync-like modifications of the filesystem.
>>>
>>> When packaging things “normally”, the .go has an mtime strictly greater
>>> than that of the source file, so checking for equality doesn’t work.
>>
>> But when reinstalling code from a binary packaging system, sometimes the
>> mtime can go backwards.
>
> Please forgive my ignorance, but can you give an example of how this can
> happen?

Well there are two cases really:

 * RPM or Deb systems typically timestamp their files based on the times
   they were built, not installed, and we don't have guarantees that
   upgrading a package won't actually move the mtime into the past.

 * When installing source code (via _SOURCES / _DATA), you don't have
   any guarantees about the relations between the times of the SOURCE
   and DATA files, *as installed*.

I think, anyway.

> Besides, what do you think packages that install scm and go files should
> do?

Preserve the timestamps as built, as Guile does.

> A package of mine basically lists .go’s in ‘nodist_foobar_DATA’ and
> .scm’s in ‘foobar_SOURCES’ (or similar).  Having to change the mtime of
> .go’s in ‘install-hook’ seems inconvenient and fragile to me.

I agree, FWIW. I don't know of a better option right now, though. We
should look to see what PLT/Racket does, as our current system has a
number of other disadvantages -- if a macro used by a module changes,
the module should be recompiled. Same goes for procedures called by a
macro used by a module, etc. They do it by linking to openssl (gross)
and doing SHA sums of various files, AFAIK, but I don't know all the
details.

Andy
-- 
http://wingolog.org/

Re: mtime of fresh .go

[Ludovic Courtès]

Hello!

Andy Wingo <wingo@pobox.com> writes:

> On Mon 19 Jul 2010 00:31, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Andy Wingo <wingo@pobox.com> writes:
>>
>>> On Fri 16 Jul 2010 10:07, ludo@gnu.org (Ludovic Courtès) writes:
>>>
>>>>   commit 535fb833b34dfc3cc11a679d39390b06fd7e9180
>>>>   Author: Andy Wingo <wingo@pobox.com>
>>>>   Date:   Fri Jun 5 10:51:21 2009 +0200
>>>>
>>>>       stamp .go with timestamp of .scm; a fresh go has same mtime of .scm
>>>>
>>>>       * libguile/load.c (compiled_is_fresh): Rename from compiled_is_newer.
>>>>         Check that the mtines of the .go and .scm match exactly, so we don't
>>>>         get fooled by rsync-like modifications of the filesystem.
>>>>
>>>> When packaging things “normally”, the .go has an mtime strictly greater
>>>> than that of the source file, so checking for equality doesn’t work.
>>>
>>> But when reinstalling code from a binary packaging system, sometimes the
>>> mtime can go backwards.
>>
>> Please forgive my ignorance, but can you give an example of how this can
>> happen?
>
> Well there are two cases really:
>
>  * RPM or Deb systems typically timestamp their files based on the times
>    they were built, not installed, and we don't have guarantees that
>    upgrading a package won't actually move the mtime into the past.

What matters is that the mtime(.go) >= mtime(.scm), and I’d expect it to
be preserved by packaging systems, even after an upgrade, no?

(My understanding is that RPMs/Debs are tar-like archives containing
meta-data such as the mtime, which is restored upon extraction.)

>  * When installing source code (via _SOURCES / _DATA), you don't have
>    any guarantees about the relations between the times of the SOURCE
>    and DATA files, *as installed*.

Apparently “make install” preserves mtimes on my system.  IOW, since
uninstalled .gos have mtime >= uninstalled .scms, the same holds for
installed .gos and .scms.

We’d need to ask the Automake people to check whether this is always the
case.

Thanks,
Ludo’.

Re: mtime of fresh .go

[Andy Wingo]

Hi,

On Tue 20 Jul 2010 00:17, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> writes:
>
>>  * RPM or Deb systems typically timestamp their files based on the times
>>    they were built, not installed, and we don't have guarantees that
>>    upgrading a package won't actually move the mtime into the past.
>
> What matters is that the mtime(.go) >= mtime(.scm), and I’d expect it to
> be preserved by packaging systems, even after an upgrade, no?

Not all packages with Scheme files will have .go files, so you get
compilation by the user in some cases. But even if the version
increases, you are not guaranteed that the mtime increases. This is
besides the issue of downgrades, of course.

>>  * When installing source code (via _SOURCES / _DATA), you don't have
>>    any guarantees about the relations between the times of the SOURCE
>>    and DATA files, *as installed*.
>
> Apparently “make install” preserves mtimes on my system.  IOW, since
> uninstalled .gos have mtime >= uninstalled .scms, the same holds for
> installed .gos and .scms.

AFAIR it does not on mine -- I wouldn't have added the install-hook in
Guile otherwise...

Andy
-- 
http://wingolog.org/

Re: mtime of fresh .go

[Andy Wingo]

Continuing this, it seems that you really want automake to handle Guile
better. Why not submit a patch to automake instead?

Andy
-- 
http://wingolog.org/

Re: mtime of fresh .go

[Ludovic Courtès]

Hello,

Andy Wingo <wingo@pobox.com> writes:

> On Tue 20 Jul 2010 00:17, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Andy Wingo <wingo@pobox.com> writes:
>>
>>>  * RPM or Deb systems typically timestamp their files based on the times
>>>    they were built, not installed, and we don't have guarantees that
>>>    upgrading a package won't actually move the mtime into the past.
>>
>> What matters is that the mtime(.go) >= mtime(.scm), and I’d expect it to
>> be preserved by packaging systems, even after an upgrade, no?
>
> Not all packages with Scheme files will have .go files, so you get
> compilation by the user in some cases. But even if the version
> increases, you are not guaranteed that the mtime increases.

Do you mean that installing a new version of a Guile-using package that
does not ship .go files may not trigger a recompilation?  (Whereas you
would want automatic recompilation.)

>>>  * When installing source code (via _SOURCES / _DATA), you don't have
>>>    any guarantees about the relations between the times of the SOURCE
>>>    and DATA files, *as installed*.
>>
>> Apparently “make install” preserves mtimes on my system.  IOW, since
>> uninstalled .gos have mtime >= uninstalled .scms, the same holds for
>> installed .gos and .scms.
>
> AFAIR it does not on mine -- I wouldn't have added the install-hook in
> Guile otherwise...

Oh, OK.  I’ll mail bug-automake then.

Thanks,
Ludo’.

Re: mtime of fresh .go

[Ludovic Courtès]

Andy Wingo <wingo@pobox.com> writes:

> Continuing this, it seems that you really want automake to handle Guile
> better. Why not submit a patch to automake instead?

Heh.  :-)  I’ve had this on my to-do list for a while, but with a low
priority.

Thanks,
Ludo’.

Re: mtime of fresh .go

[Andy Wingo]

On Tue 20 Jul 2010 18:13, ludo@gnu.org (Ludovic Courtès) writes:

> Hello,
>
> Andy Wingo <wingo@pobox.com> writes:
>
>> Not all packages with Scheme files will have .go files, so you get
>> compilation by the user in some cases. But even if the version
>> increases, you are not guaranteed that the mtime increases.
>
> Do you mean that installing a new version of a Guile-using package that
> does not ship .go files may not trigger a recompilation?  (Whereas you
> would want automatic recompilation.)

Yes, that is what I mean.

>> AFAIR [“make install” does not preserve mtimes] on my system -- I
>> wouldn't have added the install-hook in Guile otherwise...
>
> Oh, OK.  I’ll mail bug-automake then.

I saw your mail, and it was good; but I was actually thinking that we
should have native Guile support. Automake has native support for lots
of languages, but not Guile. Not yet :)

Andy
-- 
http://wingolog.org/

Re: mtime of fresh .go

[Ludovic Courtès]

Howdy!

Andy Wingo <wingo@pobox.com> writes:

> On Tue 20 Jul 2010 18:13, ludo@gnu.org (Ludovic Courtès) writes:

>> Andy Wingo <wingo@pobox.com> writes:

[...]

>>> AFAIR [“make install” does not preserve mtimes] on my system -- I
>>> wouldn't have added the install-hook in Guile otherwise...
>>
>> Oh, OK.  I’ll mail bug-automake then.
>
> I saw your mail, and it was good; but I was actually thinking that we
> should have native Guile support. Automake has native support for lots
> of languages, but not Guile. Not yet :)

Sure, but one thing at a time.  :-)

Ludo’.