bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Sévère Durand]

[-- Attachment #1: Type: text/plain, Size: 4471 bytes --]

Hello:

I recently found that sometimes an argument of a function is
unexpectedly set to nil when the function is natively compiled.

A recipe to reproduce the behaviour is as follows.

```elisp
(defun test (&optional start)
  "Test the values of START."
  (message "start is %S" start)
  (cond
   ((not (integer-or-marker-p start))
    (message "entered first cond variant")
    (setq start nil))
   ((and
     (markerp start)
     (not (equal (marker-buffer start) (current-buffer))))
    (message "entered second cond variant")
    (setq start nil)))
  (message "start is %S" start))
(native-compile 'test)
(test 12)
```

When I evaluate the above forms in sequence, I got the following two
messages:

start is 12
start is nil

As can be seen from the above, no `cond` variants are triggered, so the
variable START should have the value 12, while it becomes nil at the
end.

Sorry if this example is not minimal: I found this behaviour in my
package, so copied the relevant parts here.  I am not sure what a
minimal example would be.

Hope this report is helpful.
---
In GNU Emacs 29.3.50 (build 1, x86_64-apple-darwin23.4.0, NS
 appkit-2487.50 Version 14.4.1 (Build 23E224)) of 2024-06-12 built on
 MacBook-Pro-de-Severe.local
Repository revision: 59261e6f4fef0ec03c8127b29fe107ed19280a3b
Repository branch: emacs-29
Windowing system distributor 'Apple', version 10.3.2487
System Description:  macOS 14.4.1

Configured using:
 'configure --with-ns --without-tree-sitter
 --with-native-compilation=yes'

Configured features:
ACL DBUS GIF GLIB GMP GNUTLS JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY KQUEUE NS PDUMPER PNG RSVG SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS WEBP XIM ZLIB

Important settings:
  value of $LC_ALL: fr_FR.UTF-8
  value of $LC_COLLATE: fr_FR.UTF-8
  value of $LC_CTYPE: fr_FR.UTF-8
  value of $LC_MESSAGES: fr_FR.UTF-8
  value of $LC_MONETARY: fr_FR.UTF-8
  value of $LC_NUMERIC: fr_FR.UTF-8
  value of $LC_TIME: fr_FR.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils time-date cl-print byte-opt debug
backtrace find-func misearch multi-isearch vc-git diff-mode easy-mmode
vc-dispatcher shortdoc text-property-search thingatpt help-fns
radix-tree format-spec comp comp-cstr warnings icons subr-x rx cl-seq
cl-macs gv cl-extra help-mode cl-loaddefs cl-lib bytecomp byte-compile
face-remap rmc iso-transl tooltip cconv eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind kqueue cocoa ns
lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 115764 6238)
 (symbols 48 8233 1)
 (strings 32 30416 2847)
 (string-bytes 1 896655)
 (vectors 16 21051)
 (vector-slots 8 395280 10306)
 (floats 8 95 77)
 (intervals 56 866 15)
 (buffers 976 16))
---
Durand

[-- Attachment #2: Type: text/html, Size: 5278 bytes --]

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Gerd Möllmann]

Sévère Durand <mmemmew@gmail.com> writes:

> Hello:
>
> I recently found that sometimes an argument of a function is
> unexpectedly set to nil when the function is natively compiled.
>
> A recipe to reproduce the behaviour is as follows.
>
> ```elisp
> (defun test (&optional start)
>   "Test the values of START."
>   (message "start is %S" start)
>   (cond
>    ((not (integer-or-marker-p start))
>     (message "entered first cond variant")
>     (setq start nil))
>    ((and
>      (markerp start)
>      (not (equal (marker-buffer start) (current-buffer))))
>     (message "entered second cond variant")
>     (setq start nil)))
>   (message "start is %S" start))
> (native-compile 'test)
> (test 12)
> ```
>
> When I evaluate the above forms in sequence, I got the following two
> messages:
>
> start is 12
> start is nil
>
> As can be seen from the above, no `cond` variants are triggered, so the
> variable START should have the value 12, while it becomes nil at the
> end.
>
> Sorry if this example is not minimal: I found this behaviour in my
> package, so copied the relevant parts here.  I am not sure what a 
> minimal example would be.
>
> Hope this report is helpful.
> ---
> In GNU Emacs 29.3.50 (build 1, x86_64-apple-darwin23.4.0, NS
>  appkit-2487.50 Version 14.4.1 (Build 23E224)) of 2024-06-12 built on
>  MacBook-Pro-de-Severe.local

I can reproduce this on macOS 14.5 x86-64 on the emacs-29 branch but not
on master. Maybe Andrea (CC'd) can help.

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Andrea Corallo]

Sévère Durand <mmemmew@gmail.com> writes:

> Hello:
>
> I recently found that sometimes an argument of a function is
> unexpectedly set to nil when the function is natively compiled.
>
> A recipe to reproduce the behaviour is as follows.
>
> ```elisp
> (defun test (&optional start)
>   "Test the values of START."
>   (message "start is %S" start)
>   (cond
>    ((not (integer-or-marker-p start))
>     (message "entered first cond variant")
>     (setq start nil))
>    ((and
>      (markerp start)
>      (not (equal (marker-buffer start) (current-buffer))))
>     (message "entered second cond variant")
>     (setq start nil)))
>   (message "start is %S" start))
> (native-compile 'test)
> (test 12)
> ```
>
> When I evaluate the above forms in sequence, I got the following two
> messages:
>
> start is 12
> start is nil
>
> As can be seen from the above, no `cond` variants are triggered, so the
> variable START should have the value 12, while it becomes nil at the
> end.
>
> Sorry if this example is not minimal: I found this behaviour in my
> package, so copied the relevant parts here.  I am not sure what a 
> minimal example would be.
>
> Hope this report is helpful.

Hi Sévère,

I can't reproduce on master so the bug there is fixed.

AFAIK we don't plan to release a new 29 so I'm not sure it's worth the
effort of investigating and producing a backport for this bug.

Eli WDYT?

Thanks

  Andrea

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Sévère Durand]

[-- Attachment #1: Type: text/plain, Size: 368 bytes --]

Hello Mr.Andrea:
Thanks for your reply and that of Mr.Gerd!
I did not know that this was fixed on the master branch.  In this case, I
think I can just add a notification to my package's readme, warning users
that if they want to natively compile the package, then they should use the
master branch (or the version 30) to avoid the bug.
Thanks for the help!
---
Durand

[-- Attachment #2: Type: text/html, Size: 466 bytes --]

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Andrea Corallo]

Sévère Durand <mmemmew@gmail.com> writes:

> Hello Mr.Andrea:
> Thanks for your reply and that of Mr.Gerd!
> I did not know that this was fixed on the master branch.  In this case, I think I can just add a notification to my
> package's readme, warning users that if they want to natively compile the package, then they should use the master branch
> (or the version 30) to avoid the bug.
> Thanks for the help!
> ---
> Durand

Hi Sévère,

another workaround that should work would be to add to your function:

(declare (speed 0))

This should work around the issue on all the already released releases.

Could you give it a try?

Thanks

  Andrea

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Sévère Durand]

[-- Attachment #1: Type: text/plain, Size: 544 bytes --]

Hello Mr.Andrea:
Thanks for the suggestion.
Unfortunately when I tried the afore-mentionned snippet with `(declare
(speed 0))` added after the documentation string of the `test` function,
the execution results still indicate that the START variable is
unexpectedly set to nil, for some reason.
I surely have no opinions against closing this bug, as it seems to be fixed
on the master branch, though I have not tried yet.
Thanks to all for all the great work, especially to Mr.Andrea for the
excellent native-compilation feature.  =)
---
Durand

[-- Attachment #2: Type: text/html, Size: 859 bytes --]

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Andrea Corallo]

Sévère Durand <mmemmew@gmail.com> writes:

> Hello Mr.Andrea:
> Thanks for the suggestion.
> Unfortunately when I tried the afore-mentionned snippet with `(declare (speed 0))` added after the documentation string
> of the `test` function, the execution results still indicate that the START variable is unexpectedly set to nil, for some
> reason.

I think it's because you are giving to the native compiler an already
defined function to compile.

If you use M-x emacs-lisp-native-compile-and-load or (load
(native-compile FILE)) it will work.  I tried the first on 29 and work
around the original issue successfully.

BTW I believe this different handling of the declare could be worth a
separate bug.

Thanks

  Andrea

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Andrea Corallo]

Andrea Corallo <acorallo@gnu.org> writes:

> Sévère Durand <mmemmew@gmail.com> writes:
>
>> Hello Mr.Andrea:
>> Thanks for the suggestion.
>> Unfortunately when I tried the afore-mentionned snippet with `(declare (speed 0))` added after the documentation string
>> of the `test` function, the execution results still indicate that the START variable is unexpectedly set to nil, for some
>> reason.
>
> I think it's because you are giving to the native compiler an already
> defined function to compile.
>
> If you use M-x emacs-lisp-native-compile-and-load or (load
> (native-compile FILE)) it will work.  I tried the first on 29 and work
> around the original issue successfully.
>
> BTW I believe this different handling of the declare could be worth a
							^^^
                                                       should
> separate bug.
>
> Thanks
>
>   Andrea

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Eli Zaretskii]

> From: Andrea Corallo <acorallo@gnu.org>
> Cc: 71555@debbugs.gnu.org, eliz@gnu.org
> Date: Mon, 17 Jun 2024 03:25:09 -0400
> 
> I can't reproduce on master so the bug there is fixed.
> 
> AFAIK we don't plan to release a new 29 so I'm not sure it's worth the
> effort of investigating and producing a backport for this bug.
> 
> Eli WDYT?

If the problem happens on emacs-29, but not on master, it is good
enough for me, and we can close this bug.

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <acorallo@gnu.org>
>> Cc: 71555@debbugs.gnu.org, eliz@gnu.org
>> Date: Mon, 17 Jun 2024 03:25:09 -0400
>> 
>> I can't reproduce on master so the bug there is fixed.
>> 
>> AFAIK we don't plan to release a new 29 so I'm not sure it's worth the
>> effort of investigating and producing a backport for this bug.
>> 
>> Eli WDYT?
>
> If the problem happens on emacs-29, but not on master, it is good
> enough for me, and we can close this bug.

Thanks closing then.  Happy to re-open if we feel is necessary.

  Andrea

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Mattias Engdegård]

> I can't reproduce on master so the bug there is fixed.

We don't actually know that, do we? The bug could just be hidden by other changes on master; the underlying error could still be present.

You know the native compiler code better than anyone else (and it's your time, of course), but it might be worth investigating. Do you have a way to dump intermediate representations at various stages?

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Andrea Corallo]

Mattias Engdegård <mattias.engdegard@gmail.com> writes:

>> I can't reproduce on master so the bug there is fixed.
>
> We don't actually know that, do we? The bug could just be hidden by other changes on master; the underlying error could still be present.

The type inference on master compared to 29 had many
changes/improvements so I'm not surprised at all of seeing this fixed.

> You know the native compiler code better than anyone else (and it's
> your time, of course), but it might be worth investigating. Do you
> have a way to dump intermediate representations at various stages?

Indeed see 'native-comp-verbose'.

  Andrea

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Mattias Engdegård]

17 juni 2024 kl. 12.15 skrev Andrea Corallo <acorallo@gnu.org>:

> The type inference on master compared to 29 had many
> changes/improvements so I'm not surprised at all of seeing this fixed.

Yes, maybe it's not a good use of your time at this point. As an engineer I'm still curious about the failure modes of your compiler, though (and the underlying causes).

> Indeed see 'native-comp-verbose'.

Thanks -- maybe something for Sévère to try.

bug#71555: 29.3.50; Native-compilation sets some variable to nil unexpectedly.

[Eli Zaretskii]

> From: Mattias Engdegård <mattias.engdegard@gmail.com>
> Date: Mon, 17 Jun 2024 12:08:51 +0200
> Cc: Sévère Durand <mmemmew@gmail.com>,
>  71555@debbugs.gnu.org,
>  Eli Zaretskii <eliz@gnu.org>
> 
> > I can't reproduce on master so the bug there is fixed.
> 
> We don't actually know that, do we? The bug could just be hidden by other changes on master; the underlying error could still be present.

In that case, it will be reported again soon enough, and we can reopen
the bug.