From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Jimmy Yuen Ho Wong Newsgroups: gmane.emacs.devel Subject: Re: How does nativecomp compile circular lists? Date: Wed, 28 Jul 2021 14:49:02 +0100 Message-ID: References: <6247f78b-282f-27e7-e0cf-6bc3b1cdee26@gmail.com> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000e87c4005c82f4182" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18574"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Emacs-Devel devel To: Andrea Corallo Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Jul 28 15:51:00 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m8jxb-0004bF-Fl for ged-emacs-devel@m.gmane-mx.org; Wed, 28 Jul 2021 15:50:59 +0200 Original-Received: from localhost ([::1]:54932 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m8jxa-0005ms-Cz for ged-emacs-devel@m.gmane-mx.org; Wed, 28 Jul 2021 09:50:58 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:52166) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m8jwN-0002nU-EE for emacs-devel@gnu.org; Wed, 28 Jul 2021 09:49:43 -0400 Original-Received: from mail-lj1-x232.google.com ([2a00:1450:4864:20::232]:44587) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m8jwK-0006vJ-Se for emacs-devel@gnu.org; Wed, 28 Jul 2021 09:49:43 -0400 Original-Received: by mail-lj1-x232.google.com with SMTP id a7so3129057ljq.11 for ; Wed, 28 Jul 2021 06:49:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=oxRwM5SoWTxpE6NbLed7EVrwDsEn7dwS74o12HKuZ8U=; b=U+Zg2eS0Up++7wKCAgu8dpf0+ZjSS4hLWArkFN15hsU1+xMA9gBCegzogS3j5PKnVo G1HXZbQEKHFCjprYFFeYST+xxKOPqo97yulMWuJSiUF24kLU87Nq31kFy+ZdwN2Qh8eJ 8pb/rm+e+rdKSEBmzEfmsuHLbobXwHQyoS4n8zkfCYNWe9/0mcVUBgeuiQSwhxoelYYs ad0kFPBSOaHj3YbnJLCorc4+fprN9K86I042zM6uvv1JLGxiDiokC+zJUN+sBFbNPFBx QQve8ddoup3TJtYTPdFlTlHSVGyiqWnqUxVuZrHEf3TbXfnL5ZLMkc2D2zKDhr4A+tBd AqjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oxRwM5SoWTxpE6NbLed7EVrwDsEn7dwS74o12HKuZ8U=; b=l/3HZLtxGS/YH5Ci7cv9cWBiHo00roYcvWaA0INwOJ67gzS1rEsfEd6jwvMgw3RGrv tA0ANSc+WggLgtLifqgRch53DaPdn4Ki0VjWmj3IYz7kuDTK+90u022VNBaBdmhJ5LOl OhHHyn5uY4aYXWu9RJ77VVcCkvpb2PuF0frBmEXw6id0/sA2cAYD7PhF2jCQo+WTbKhe Fe5u8a3mKDpVGGRBKTQMGiqF7tt7YpTeN+S76RqekNCs91pi2w+O7eamKg8ksG6pOEdV a3KaBONa60d0aAlp5rIAQ46NlflgoTDhhvN4yo+IKSpTC+/hQh7aicQiA1+dtm2vgaI5 uJTA== X-Gm-Message-State: AOAM530z1IcPliy5ho6WXZxSmP5+E0+aU+u9EZIE2vMd7BwCGZTXH5bq KrlLEi4M6/plZ3EYCL0UdP+0tsRyBSavdVP7ies= X-Google-Smtp-Source: ABdhPJy+d7Bvt0zXOvzBGGnqldin/kPHdwooH0JEmdTO3qqf3SrT9pXMZz3aiTPYXbpCSXCYVuvePFBF3xaogRrHIjQ= X-Received: by 2002:a2e:86c5:: with SMTP id n5mr18872662ljj.398.1627480178325; Wed, 28 Jul 2021 06:49:38 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=2a00:1450:4864:20::232; envelope-from=wyuenho@gmail.com; helo=mail-lj1-x232.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:271743 Archived-At: --000000000000e87c4005c82f4182 Content-Type: text/plain; charset="UTF-8" On Wed, Jul 28, 2021 at 8:20 AM Andrea Corallo wrote: > Jimmy Yuen Ho Wong writes: > > > On 28/07/2021 3:09 AM, Jimmy Yuen Ho Wong wrote: > > > > I think I've found out what that stacktrace means. The regular > expression in disassemble-internal hasn't taken into > > account LLVM's objdump output format for mach-o objects. The first > couple of lines of the *Disassemble* buffer looks > > like this on macOS: > > > > > /Users/wyuenho/.emacs.d/eln-cache/28_0_50-83d1a9b9/markdown-mode-4888e153-14d4f01e.eln: > file format mach-o 64-bit > > x86-64 > > > > Disassembly of section __TEXT,__text: > > > > 0000000000002fa0 <_maybe_gc_quit>: > > > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/objdump: > warning: > > > '/Users/wyuenho/.emacs.d/eln-cache/28_0_50-83d1a9b9/markdown-mode-4888e153-14d4f01e.eln': > failed to parse debug > > information for > /Users/wyuenho/.emacs.d/eln-cache/28_0_50-83d1a9b9/markdown-mode-4888e153-14d4f01e.eln > > 2fa0: 8b 05 86 9f 08 00 movl 565126(%rip), %eax # > 8cf2c <_quitcounter> > > 2fa6: 83 c0 01 addl $1, %eax > > > > Jimmy > > > > On Tue, Jul 27, 2021 at 10:28 AM Andrea Corallo wrote: > > > > Jimmy Yuen Ho Wong writes: > > > > > I'm not exactly sure how to even ask this question, as I've seen > various packages fail to natively compile > > circular lists > > > or optimize it wrongly. So I've come up with a simple test case for > me to be able to disassemble the bytecode > > and the > > > native code, and see if I could spot the problem, but disassembling > natively compiled code doesn't seem to > > work. > > > > > > Test case: > > > > > > (defun test-circular-list () > > > (let ((l (list 1 2 3))) > > > (setcdr (last l) l) > > > l)) > > > > > > (print (test-circular-list)) > > > > > > (provide 'test-circular-list) > > > > > > M-x emacs-lisp-byte-compile-and-load > > > M-x disassemble RET test-circular-list (works) > > > > > > M-x emacs-lisp-native-compile-and-load > > > M-x disassemble RET test-circular-list (doesn't work) > > > > > > Debugger entered--Lisp error: (search-failed > "^.* > > > re-search-forward("^.* > > disassemble-internal(test-circular-list 0 nil) > > > disassemble(test-circular-list nil 0 t) > > > funcall-interactively(disassemble test-circular-list nil 0 t) > > > command-execute(disassemble record) > > > execute-extended-command(nil "disassemble" nil) > > > funcall-interactively(execute-extended-command nil "disassemble" > nil) > > > command-execute(execute-extended-command) > > > > > > Furthermore, this test case, though superficially similar, doesn't > seem to be able to reproduce the same > > failure to > > > natively compile issue as seen in powerline. > > > > > > There seems to be more than one thing wrong with this journey into my > deep dive into how native compiling > > circular lists > > > work. > > > > > > Jimmy Yuen Ho Wong > > > > > > > Hi Jimmy, > > > > I'm failing to reproduce this issue. > > > > Disassembling following your instructions works here. Also the compiled > > `test-circular-list' function as expected here. > > > > Am I missing something? > > > > Best Regards > > > > Andrea > > > > Sorry wrong patch from the last email. This is the correct one. > > [re-adding the list] > > Hi Jimmy, > > thanks for the patch. > > So I guess is clear this is unrelated to circular lists and probably > disassemble is just broken for every native compiled function in this > configuration. > > We might have other architecture+OS where function labels start with '_' > but so far the patch LGTM, I just suggest a small nit. > > > diff --git a/lisp/emacs-lisp/disass.el b/lisp/emacs-lisp/disass.el > > index 6ac76f1c19..d0e3bf7bf4 100644 > > --- a/lisp/emacs-lisp/disass.el > > +++ b/lisp/emacs-lisp/disass.el > > @@ -95,6 +95,7 @@ disassemble-internal > > (re-search-forward (concat "^.*" > > (regexp-quote > > (concat "<" > > + (if (eq system-type > 'darwin) "_") > ^^^ > I'd use `when' here. > > (comp-c-func-name > > (subr-name obj) "F" > t) > > ">:")))) > > Do you aready have copyright assignment? > > Best Regards > > Andrea > I do have an existing copyright assignment. No objection to using a when there, would you like me to submit a new patch or you are fine with editing it yourself? As to whether this is related to circular lists, I'm not so sure yet. There's definitely something in native compilation that compiles code that behaves differently when circular lists are involved, I just haven't found out what interaction with circular lists will result in compilation failure or optimizing the result away. Any clues for what I should try is much appreciated. --000000000000e87c4005c82f4182 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

On Wed, Jul 28, 2021 at 8:20 AM Andrea Co= rallo <akrl@sdf.org> wrote:
Jimmy Yuen Ho Wong &l= t;wyuenho@gmail.com<= /a>> writes:

> On 28/07/2021 3:09 AM, Jimmy Yuen Ho Wong wrote:
>
>=C2=A0 I think I've found out what that stacktrace means. The regul= ar expression in disassemble-internal hasn't taken into
>=C2=A0 account LLVM's objdump output format for mach-o objects. The= first couple of lines of the *Disassemble* buffer looks
>=C2=A0 like this on macOS:
>
>=C2=A0 /Users/wyuenho/.emacs.d/eln-cache/28_0_50-83d1a9b9/markdown-mode= -4888e153-14d4f01e.eln: file format mach-o 64-bit
>=C2=A0 x86-64
>
>=C2=A0 Disassembly of section __TEXT,__text:
>
>=C2=A0 0000000000002fa0 <_maybe_gc_quit>:
>=C2=A0 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefau= lt.xctoolchain/usr/bin/objdump: warning:
>=C2=A0 '/Users/wyuenho/.emacs.d/eln-cache/28_0_50-83d1a9b9/markdown= -mode-4888e153-14d4f01e.eln': failed to parse debug
>=C2=A0 information for /Users/wyuenho/.emacs.d/eln-cache/28_0_50-83d1a9= b9/markdown-mode-4888e153-14d4f01e.eln
>=C2=A0 =C2=A0 =C2=A0 2fa0: 8b 05 86 9f 08 00=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0movl 565126(%rip), %eax=C2=A0 # 8cf2c <_quitcounter= >
>=C2=A0 =C2=A0 =C2=A0 2fa6: 83 c0 01=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0addl $1, %eax
>
>=C2=A0 Jimmy
>
>=C2=A0 On Tue, Jul 27, 2021 at 10:28 AM Andrea Corallo <akrl@sdf.org> wrote:
>
>=C2=A0 Jimmy Yuen Ho Wong <wyuenho@gmail.com> writes:
>
>=C2=A0 > I'm not exactly sure how to even ask this question, as = I've seen various packages fail to natively compile
>=C2=A0 circular lists
>=C2=A0 > or optimize it wrongly. So I've come up with a simple t= est case for me to be able to disassemble the bytecode
>=C2=A0 and the
>=C2=A0 > native code, and see if I could spot the problem, but disas= sembling natively compiled code doesn't seem to
>=C2=A0 work.
>=C2=A0 >
>=C2=A0 > Test case:
>=C2=A0 >
>=C2=A0 > (defun test-circular-list ()
>=C2=A0 >=C2=A0 =C2=A0(let ((l (list 1 2 3)))
>=C2=A0 >=C2=A0 =C2=A0 =C2=A0(setcdr (last l) l)
>=C2=A0 >=C2=A0 =C2=A0 =C2=A0l))
>=C2=A0 >
>=C2=A0 > (print (test-circular-list))
>=C2=A0 >
>=C2=A0 > (provide 'test-circular-list)
>=C2=A0 >
>=C2=A0 > M-x emacs-lisp-byte-compile-and-load
>=C2=A0 > M-x disassemble RET test-circular-list (works)
>=C2=A0 >
>=C2=A0 > M-x emacs-lisp-native-compile-and-load
>=C2=A0 > M-x disassemble RET test-circular-list (doesn't work) >=C2=A0 >
>=C2=A0 > Debugger entered--Lisp error: (search-failed "^.*<F= 746573742d63697263756c61722d6c697374_test_cir...")
>=C2=A0 >=C2=A0 =C2=A0re-search-forward("^.*<F746573742d63697= 263756c61722d6c697374_test_cir...")
>=C2=A0 >=C2=A0 =C2=A0disassemble-internal(test-circular-list 0 nil)<= br> >=C2=A0 >=C2=A0 =C2=A0disassemble(test-circular-list nil 0 t)
>=C2=A0 >=C2=A0 =C2=A0funcall-interactively(disassemble test-circular= -list nil 0 t)
>=C2=A0 >=C2=A0 =C2=A0command-execute(disassemble record)
>=C2=A0 >=C2=A0 =C2=A0execute-extended-command(nil "disassemble&= quot; nil)
>=C2=A0 >=C2=A0 =C2=A0funcall-interactively(execute-extended-command = nil "disassemble" nil)
>=C2=A0 >=C2=A0 =C2=A0command-execute(execute-extended-command)
>=C2=A0 >
>=C2=A0 > Furthermore, this test case, though superficially similar, = doesn't seem to be able to reproduce the same
>=C2=A0 failure to
>=C2=A0 > natively compile issue as seen in powerline.
>=C2=A0 >
>=C2=A0 > There seems to be more than one thing wrong with this journ= ey into my deep dive into how native compiling
>=C2=A0 circular lists
>=C2=A0 > work.
>=C2=A0 >
>=C2=A0 > Jimmy Yuen Ho Wong
>=C2=A0 >
>
>=C2=A0 Hi Jimmy,
>
>=C2=A0 I'm failing to reproduce this issue.
>
>=C2=A0 Disassembling following your instructions works here. Also the c= ompiled
>=C2=A0 `test-circular-list' function as expected here.
>
>=C2=A0 Am I missing something?
>
>=C2=A0 Best Regards
>
>=C2=A0 =C2=A0 Andrea
>
> Sorry wrong patch from the last email. This is the correct one.

[re-adding the list]

Hi Jimmy,

thanks for the patch.

So I guess is clear this is unrelated to circular lists and probably
disassemble is just broken for every native compiled function in this
configuration.

We might have other architecture+OS where function labels start with '_= '
but so far the patch LGTM, I just suggest a small nit.

> diff --git a/lisp/emacs-lisp/disass.el b/lisp/emacs-lisp/disass.el
> index 6ac76f1c19..d0e3bf7bf4 100644
> --- a/lisp/emacs-lisp/disass.el
> +++ b/lisp/emacs-lisp/disass.el
> @@ -95,6 +95,7 @@ disassemble-internal
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (re-search-forw= ard (concat "^.*"
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0(regexp-quote
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 (concat "<"
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (if (eq system-type 'darwin) "_&qu= ot;)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0^^^
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0I'd use `when' here.
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (comp-c-func-name
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(subr-name obj) "F" = t)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ">:"))))

Do you aready have copyright assignment?

Best Regards

=C2=A0 Andrea

I do have an existing cop= yright assignment. No objection to using a when there, would you like me to= submit a new patch or you are fine with editing it yourself?

As to = whether this is related to circular lists, I'm not so sure yet. There&#= 39;s definitely something in native compilation that compiles code that beh= aves differently when circular lists are involved, I just haven't found= out what interaction with circular lists will result in compilation failur= e or optimizing the result away. Any clues for what I should try is much ap= preciated.
--000000000000e87c4005c82f4182--