feature/native-comp, comp-deferred-compilation: no recompilation when .elc newer than .eln?

feature/native-comp, comp-deferred-compilation: no recompilation when .elc newer than .eln?

[Gregor Zattler]

Dear Andrea, emacs developers,

I gave feature/native-comp in combination with
`comp-deferred-compilation` a try.  I use org-mode from git
master and it's .elc files eventually were compiled to .eln
files.  Works like a charm.

Then I updated org-mode (make update) since there was a (for
me) important bug fix.  Now org.elc is newer than org.el
which in turn is newer than org.eln.

But nonetheless the org.eln file does not get re-created
according from the newer org.elc file.

Is this on purpose?

Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

Re: feature/native-comp, comp-deferred-compilation: no recompilation when .elc newer than .eln?

[Andrea Corallo]

Gregor Zattler <telegraph@gmx.net> writes:

> Dear Andrea, emacs developers,
>
> I gave feature/native-comp in combination with
> `comp-deferred-compilation` a try.  I use org-mode from git
> master and it's .elc files eventually were compiled to .eln
> files.  Works like a charm.
>
> Then I updated org-mode (make update) since there was a (for
> me) important bug fix.  Now org.elc is newer than org.el
> which in turn is newer than org.eln.
>
> But nonetheless the org.eln file does not get re-created
> according from the newer org.elc file.
>
> Is this on purpose?
>
> Ciao; Gregor

Hi Gregor,

I'm not sure the sequence of events is clear to me, especially how the
the new org was loaded after it was compiled calling make.

Deferred compilation logic to date works as follow: if an elc is being
loaded, is lexical and the corresponding source is found, then an async
compilation is queued.  No file date is taken in account.

Is it possible that the old eln is still being loaded because
load-prefer-newer is nil?

Thanks for testing it, ciao!

Andrea

-- 
akrl@sdf.org

Re: feature/native-comp, comp-deferred-compilation: no recompilation when .elc newer than .eln?

[Gregor Zattler]

Hi Andrea,
* Andrea Corallo <akrl@sdf.org> [2020-05-15; 19:43]:
> Gregor Zattler <telegraph@gmx.net> writes:
>> I gave feature/native-comp in combination with
>> `comp-deferred-compilation` a try.  I use org-mode from git
>> master and it's .elc files eventually were compiled to .eln
>> files.  Works like a charm.
>>
>> Then I updated org-mode (make update) since there was a (for
>> me) important bug fix.  Now org.elc is newer than org.el
>> which in turn is newer than org.eln.
>>
>> But nonetheless the org.eln file does not get re-created
>> according from the newer org.elc file.
>>
>> Is this on purpose?


> I'm not sure the sequence of events is clear to me, especially how the
> the new org was loaded after it was compiled calling make.

I use org-mode from git.  Org-mode has a build system and
esypecially `make up1` does a git pull, compile , builds
documentation and runs checks.  I do this from time to time.

The relevant part of the git repo is in my emacs load-path:

(add-to-list 'load-path (expand-file-name "~/src/org-mode/lisp"))
(setq load-path (cons "~/src/org-mode/contrib/lisp/" load-path))

> Deferred compilation logic to date works as follow: if an elc is being
> loaded, is lexical and the corresponding source is found, then an async
> compilation is queued.  No file date is taken in account.
>
> Is it possible that the old eln is still being loaded because
> load-prefer-newer is nil?

load-prefer-newer is t in my case.  The stale org.eln file
was loaded instead of the newer org.elc and the newer
org.elc wasn't compiled to a newer org.eln file.  I realized
because I experienced a specific bug although there was a
patch with a fix in the repo.

Thanks, Gregor

Re: feature/native-comp, comp-deferred-compilation: no recompilation when .elc newer than .eln?

[Andrea Corallo]

Gregor Zattler <telegraph@gmx.net> writes:

>> I'm not sure the sequence of events is clear to me, especially how the
>> the new org was loaded after it was compiled calling make.
>
> I use org-mode from git.  Org-mode has a build system and
> esypecially `make up1` does a git pull, compile , builds
> documentation and runs checks.  I do this from time to time.
>
> The relevant part of the git repo is in my emacs load-path:
>
> (add-to-list 'load-path (expand-file-name "~/src/org-mode/lisp"))
> (setq load-path (cons "~/src/org-mode/contrib/lisp/" load-path))
>
>> Deferred compilation logic to date works as follow: if an elc is being
>> loaded, is lexical and the corresponding source is found, then an async
>> compilation is queued.  No file date is taken in account.
>>
>> Is it possible that the old eln is still being loaded because
>> load-prefer-newer is nil?
>
> load-prefer-newer is t in my case.  The stale org.eln file
> was loaded instead of the newer org.elc and the newer
> org.elc wasn't compiled to a newer org.eln file.  I realized
> because I experienced a specific bug although there was a
> patch with a fix in the repo.
>
> Thanks, Gregor

Hi Gregor,

the bit I'm missing is how the load was performed after org is updated.
Restarting Emacs or calling `load'?

Thanks

  Andrea

-- 
akrl@sdf.org

Re: feature/native-comp, comp-deferred-compilation: no recompilation when .elc newer than .eln?

[Gregor Zattler]

Hi Andrea,
* Andrea Corallo <akrl@sdf.org> [2020-05-16; 07:30]:
> Gregor Zattler <telegraph@gmx.net> writes:
>>> Is it possible that the old eln is still being loaded because
>>> load-prefer-newer is nil?
>>
>> load-prefer-newer is t in my case.  The stale org.eln file
>> was loaded instead of the newer org.elc and the newer
>> org.elc wasn't compiled to a newer org.eln file.  I realized
>> because I experienced a specific bug although there was a
>> patch with a fix in the repo.

> the bit I'm missing is how the load was performed after org is updated.
> Restarting Emacs or calling `load'?

Sorry for not being more precise: I restart
emacs after upgrading emacs or org-mode (or notmuch).


Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

Re: feature/native-comp, comp-deferred-compilation: no recompilation when .elc newer than .eln?

[Andrea Corallo]

Gregor Zattler <telegraph@gmx.net> writes:

> Hi Andrea,
> * Andrea Corallo <akrl@sdf.org> [2020-05-16; 07:30]:
>> Gregor Zattler <telegraph@gmx.net> writes:
>>>> Is it possible that the old eln is still being loaded because
>>>> load-prefer-newer is nil?
>>>
>>> load-prefer-newer is t in my case.  The stale org.eln file
>>> was loaded instead of the newer org.elc and the newer
>>> org.elc wasn't compiled to a newer org.eln file.  I realized
>>> because I experienced a specific bug although there was a
>>> patch with a fix in the repo.
>
>> the bit I'm missing is how the load was performed after org is updated.
>> Restarting Emacs or calling `load'?
>
> Sorry for not being more precise: I restart
> emacs after upgrading emacs or org-mode (or notmuch).

I reproduced here.  The problem is apparently that the .eln is always
preferred to the .elc for being loaded regardless `load-prefer-newer'
value, therefore the .eln is loaded directly and no deferred compilation
is triggerd.

I'll have look for a patch.

Thanks

  Andrea

-- 
akrl@sdf.org

Re: feature/native-comp, comp-deferred-compilation: no recompilation when .elc newer than .eln?

[Andrea Corallo]

Gregor Zattler <telegraph@gmx.net> writes:

> Hi Andrea,
> * Andrea Corallo <akrl@sdf.org> [2020-05-16; 07:30]:
>> Gregor Zattler <telegraph@gmx.net> writes:
>>>> Is it possible that the old eln is still being loaded because
>>>> load-prefer-newer is nil?
>>>
>>> load-prefer-newer is t in my case.  The stale org.eln file
>>> was loaded instead of the newer org.elc and the newer
>>> org.elc wasn't compiled to a newer org.eln file.  I realized
>>> because I experienced a specific bug although there was a
>>> patch with a fix in the repo.
>
>> the bit I'm missing is how the load was performed after org is updated.
>> Restarting Emacs or calling `load'?
>
> Sorry for not being more precise: I restart
> emacs after upgrading emacs or org-mode (or notmuch).
>
>
> Ciao; Gregor

Hi Gregor,

This issue should be fixed by Nico's patch e38678b268 "Reduce the number
of files probed when finding a lisp file."

Hope works for you, in case does not please complain :)

Ciao

  Andrea

-- 
akrl@sdf.org

Re: feature/native-comp, comp-deferred-compilation: no recompilation when .elc newer than .eln?

[Gregor Zattler]

Hi Andrea,
* Andrea Corallo <akrl@sdf.org> [2020-06-06; 21:51]:
> This issue should be fixed by Nico's patch e38678b268 "Reduce the number
> of files probed when finding a lisp file."
>
> Hope works for you, in case does not please complain :)

I will test in the next days.


Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

Re: feature/native-comp, comp-deferred-compilation: no recompilation when .elc newer than .eln?

[Gregor Zattler]

Hi Andrea,
* Gregor Zattler <telegraph@gmx.net> [2020-06-09; 08:17]:
> * Andrea Corallo <akrl@sdf.org> [2020-06-06; 21:51]:
>> This issue should be fixed by Nico's patch e38678b268 "Reduce the number
>> of files probed when finding a lisp file."
>>
>> Hope works for you, in case does not please complain :)
>
> I will test in the next days.

it worked, but only after the 3rd invokation of emacs:

1. I recompiled emacs, checked out an old version of org
   mode with a certain bug, compiled that, used it with the
   new emacs, this produced an org.eln file with the
   specific bug.  Which I experienced while using org-mode.

2. I killed emacs, checked out newest org-mode, compiled it,
   started emacs, used org-mode, org.el was not shown in the
   async compile buffer, and the bug was still present and
   the eln file was older than the el an elc files.

3. I killed emacs, started it again, used an org-file, now
   org.el showed up in the async compile buffer, but the eln
   file was as old as before and the bug was present.

4. I killed emacs.  Had a look again for the org.el[cn]?
   files: now the eln file was newest, I used org-mode and
   the bug was gone.

I understand, that a new elc file is used in emacs, the
compilation to eln happens async and when finished the eln
file is not loaded over the elc file.  Therefore I have to
restart emacs in order to actually use the eln file.

I do not understand why my org-usage in step 2 didn't
trigger the compilation of the eln file.

Reloading the eln file instead of the already loaded elc in
3 would be cool, but not that important, since the elc file
is feature-wise equivalent with the eln fiie.


Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

Re: feature/native-comp, comp-deferred-compilation: no recompilation when .elc newer than .eln?

[Andrea Corallo]

Gregor Zattler <telegraph@gmx.net> writes:

> Hi Andrea,
> * Gregor Zattler <telegraph@gmx.net> [2020-06-09; 08:17]:
>> * Andrea Corallo <akrl@sdf.org> [2020-06-06; 21:51]:
>>> This issue should be fixed by Nico's patch e38678b268 "Reduce the number
>>> of files probed when finding a lisp file."
>>>
>>> Hope works for you, in case does not please complain :)
>>
>> I will test in the next days.
>
> it worked, but only after the 3rd invokation of emacs:
>
> 1. I recompiled emacs, checked out an old version of org
>    mode with a certain bug, compiled that, used it with the
>    new emacs, this produced an org.eln file with the
>    specific bug.  Which I experienced while using org-mode.
>
> 2. I killed emacs, checked out newest org-mode, compiled it,
>    started emacs, used org-mode, org.el was not shown in the
>    async compile buffer, and the bug was still present and
>    the eln file was older than the el an elc files.
>
> 3. I killed emacs, started it again, used an org-file, now
>    org.el showed up in the async compile buffer, but the eln
>    file was as old as before and the bug was present.
>
> 4. I killed emacs.  Had a look again for the org.el[cn]?
>    files: now the eln file was newest, I used org-mode and
>    the bug was gone.
>
> I understand, that a new elc file is used in emacs, the
> compilation to eln happens async and when finished the eln
> file is not loaded over the elc file.  Therefore I have to
> restart emacs in order to actually use the eln file.

Well the .eln should be loaded over the .elc, but happen in a
transparent way.

> I do not understand why my org-usage in step 2 didn't
> trigger the compilation of the eln file.

Is a little hard do undestand what is going on from here.

Actually this mechanism (the one that triggers the compilation) will be
radically simplified once the dynamic scope support comes in.  I've a
branch for that already and I've 'just' to make it work :)

I suggest we rediscuss this with the new mechanism in place given it
should substitute the current one.

Thanks for trying the branch!

  Andrea

-- 
akrl@sdf.org