From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= Newsgroups: gmane.emacs.bugs Subject: bug#51982: Erroneous handling of local variables in byte-compiled nested lambdas Date: Sat, 20 Nov 2021 19:34:03 +0100 Message-ID: References: <87y25jo2q1.fsf@web.de> <29C3A3F8-CD9F-4AF2-A731-3304FC30E380@acm.org> Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.21\)) Content-Type: multipart/mixed; boundary="Apple-Mail=_D7447F0A-66F7-4083-8BF3-8B7D56A60FC3" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="21518"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Michael Heerdegen , Stefan Monnier , 51982@debbugs.gnu.org To: Paul Pogonyshev Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Nov 20 19:35:24 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1moVCu-0005Jb-D3 for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 20 Nov 2021 19:35:24 +0100 Original-Received: from localhost ([::1]:51772 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1moVCs-00072Z-F2 for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 20 Nov 2021 13:35:22 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:51346) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1moVCY-00072Q-VR for bug-gnu-emacs@gnu.org; Sat, 20 Nov 2021 13:35:03 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:60347) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1moVCY-0001oz-Ig for bug-gnu-emacs@gnu.org; Sat, 20 Nov 2021 13:35:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1moVCX-0002WR-OU for bug-gnu-emacs@gnu.org; Sat, 20 Nov 2021 13:35:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 20 Nov 2021 18:35:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51982 X-GNU-PR-Package: emacs Original-Received: via spool by 51982-submit@debbugs.gnu.org id=B51982.16374332569627 (code B ref 51982); Sat, 20 Nov 2021 18:35:01 +0000 Original-Received: (at 51982) by debbugs.gnu.org; 20 Nov 2021 18:34:16 +0000 Original-Received: from localhost ([127.0.0.1]:43660 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1moVBn-0002VD-QE for submit@debbugs.gnu.org; Sat, 20 Nov 2021 13:34:16 -0500 Original-Received: from mail71c50.megamailservers.eu ([91.136.10.81]:53646 helo=mail92c50.megamailservers.eu) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1moVBi-0002Uu-73 for 51982@debbugs.gnu.org; Sat, 20 Nov 2021 13:34:14 -0500 X-Authenticated-User: mattiase@bredband.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=megamailservers.eu; s=maildub; t=1637433246; bh=3ZOdRkoVHZelMCoDxIpcjEAtXrn2GU1yyXUZWLS1Dzs=; h=From:Subject:Date:In-Reply-To:Cc:To:References:From; b=XikoX4eQBxs1u/I3CVzRYLpnzDDY7ejVXzPiQeFaOHhJVQewvkgooj2Rts7K9UBFR 1b31dXy0rUGmcX5DfM/wUtK0sBk1PlwBAfOB7TA2MbcBMHbVqlwn2t6uZvjxJ7XXam Qg5QMXz0QDFQWdpjIAF5f3EekM+RBufFYjrKllrQ= Feedback-ID: mattiase@acm.or Original-Received: from [192.168.0.4] (c188-150-171-71.bredband.tele2.se [188.150.171.71]) (authenticated bits=0) by mail92c50.megamailservers.eu (8.14.9/8.13.1) with ESMTP id 1AKIY3XD031696; Sat, 20 Nov 2021 18:34:05 +0000 In-Reply-To: X-Mailer: Apple Mail (2.3445.104.21) X-CTCH-RefID: str=0001.0A742F1C.61993F9E.004E, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0 X-CTCH-VOD: Unknown X-CTCH-Spam: Unknown X-CTCH-Score: 0.000 X-CTCH-Flags: 0 X-CTCH-ScoreCust: 0.000 X-CSC: 0 X-CHA: v=2.4 cv=Y72bDjSN c=1 sm=1 tr=0 ts=61993f9e a=SF+I6pRkHZhrawxbOkkvaA==:117 a=SF+I6pRkHZhrawxbOkkvaA==:17 a=M51BFTxLslgA:10 a=pGLkceISAAAA:8 a=qeIFS-pKyf6qnnE2i3MA:9 a=QEXdDO2ut3YA:10 a=yO2wDlikK09HlSVJ6-MA:9 a=De_Ol2h6w80A:10 X-Origin-Country: SE X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:220515 Archived-At: --Apple-Mail=_D7447F0A-66F7-4083-8BF3-8B7D56A60FC3 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 20 nov. 2021 kl. 18.22 skrev Paul Pogonyshev : > > you cannot count on an eventual fix to be back-ported to Emacs 28 >=20 > Uh, it's not even released yet. Are the stabilization rules so strict = that even a *fix* cannot go in? Unless it is going to be a rewrite of = half the file... Sorry, I'd love to have it in 28 but I don't make the rules. Meanwhile, here's a lightly tested attempt at a fix. I'm not very = satisfied by it because it forces an up-front access of a captured = variable; I'd rather sink it to each application of the =CE=BB-lifted = function. --Apple-Mail=_D7447F0A-66F7-4083-8BF3-8B7D56A60FC3 Content-Disposition: attachment; filename=cconv-bug51982.diff Content-Type: application/octet-stream; x-unix-mode=0644; name="cconv-bug51982.diff" Content-Transfer-Encoding: 7bit diff --git a/lisp/emacs-lisp/cconv.el b/lisp/emacs-lisp/cconv.el index 03e109f250..17deb8b90a 100644 --- a/lisp/emacs-lisp/cconv.el +++ b/lisp/emacs-lisp/cconv.el @@ -431,7 +431,8 @@ cconv-convert (let ((closedsym (make-symbol (format "closed-%s" var)))) (setq new-env (cconv--remap-llv new-env var closedsym)) (setq new-extend (cons closedsym (remq var new-extend))) - (push `(,closedsym ,var) binders-new))) + (let ((var-def (or (cdr (assq var env)) var))) + (push `(,closedsym ,var-def) binders-new)))) ;; We push the element after redefined free variables are ;; processed. This is important to avoid the bug when free @@ -456,7 +457,8 @@ cconv-convert (closedsym (make-symbol (format "closed-%s" var)))) (setq new-env (cconv--remap-llv new-env var closedsym)) (setq new-extend (cons closedsym (remq var new-extend))) - (push `(,closedsym ,var) binders-new))))) + (let ((var-def (or (cdr (assq var env)) var))) + (push `(,closedsym ,var-def) binders-new)))))) `(,letsym ,(nreverse binders-new) . ,(mapcar (lambda (form) diff --git a/test/lisp/emacs-lisp/bytecomp-tests.el b/test/lisp/emacs-lisp/bytecomp-tests.el index dbc0aa3db4..ed596ef3b4 100644 --- a/test/lisp/emacs-lisp/bytecomp-tests.el +++ b/test/lisp/emacs-lisp/bytecomp-tests.el @@ -643,6 +643,15 @@ bytecomp-tests--test-cases (cond) (mapcar (lambda (x) (cond ((= x 0)))) '(0 1)) + + ;; This gives different results in lexbind and dynbind modes, + ;; but in each the compiler and interpreter should agree! + (let ((f (lambda (x) + (lambda () + (let ((g (lambda () x))) + (let ((x 'a)) + (list x (funcall g)))))))) + (funcall (funcall f 'b))) ) "List of expressions for cross-testing interpreted and compiled code.") diff --git a/test/lisp/emacs-lisp/cconv-tests.el b/test/lisp/emacs-lisp/cconv-tests.el index 4290571735..b6a789fcfe 100644 --- a/test/lisp/emacs-lisp/cconv-tests.el +++ b/test/lisp/emacs-lisp/cconv-tests.el @@ -205,5 +205,88 @@ cconv-convert-lambda-lifted nil 99) 42))) +(defun cconv-tests--intern-all (x) + "Intern all symbols in X." + (cond ((symbolp x) (intern (symbol-name x))) + ((consp x) (cons (cconv-tests--intern-all (car x)) + (cconv-tests--intern-all (cdr x)))) + ;; Assume we don't need to deal with vectors etc. + (t x))) + +(ert-deftest cconv-closure-convert-remap-var () + ;; Verify that we correctly remap shadowed lambda-lifted variables. + + ;; We intern all symbols for ease of comparison; this works because + ;; the `cconv-closure-convert' result should contain no pair of + ;; distinct symbols having the same name. + + ;; Sanity check: captured variable, no lambda-lifting or shadowing: + (should (equal (cconv-tests--intern-all + (cconv-closure-convert + '#'(lambda (x) + #'(lambda () x)))) + '#'(lambda (x) + (internal-make-closure + nil (x) nil + (internal-get-closed-var 0))))) + + ;; Basic case: + ;; - with `let': + (should (equal (cconv-tests--intern-all + (cconv-closure-convert + '#'(lambda (x) + (let ((f #'(lambda () x))) + (let ((x 'b)) + (list x (funcall f))))))) + '#'(lambda (x) + (let ((f #'(lambda (x) x))) + (let ((x 'b) + (closed-x x)) + (list x (funcall f closed-x))))))) + ;; - with `let*': + (should (equal (cconv-tests--intern-all + (cconv-closure-convert + '#'(lambda (x) + (let ((f #'(lambda () x))) + (let* ((x 'b)) + (list x (funcall f))))))) + '#'(lambda (x) + (let ((f #'(lambda (x) x))) + (let* ((closed-x x) + (x 'b)) + (list x (funcall f closed-x))))))) + + ;; With the lambda-lifted shadowed variable also being captured: + ;; - with `let': + (should (equal (cconv-tests--intern-all + (cconv-closure-convert + '#'(lambda (x) + #'(lambda () + (let ((f #'(lambda () x))) + (let ((x 'a)) + (list x (funcall f)))))))) + '#'(lambda (x) + (internal-make-closure + nil (x) nil + (let ((f #'(lambda (x) x))) + (let ((x 'a) + (closed-x (internal-get-closed-var 0))) + (list x (funcall f closed-x)))))))) + ;; - with `let*': + (should (equal (cconv-tests--intern-all + (cconv-closure-convert + '#'(lambda (x) + #'(lambda () + (let ((f #'(lambda () x))) + (let* ((x 'a)) + (list x (funcall f)))))))) + '#'(lambda (x) + (internal-make-closure + nil (x) nil + (let ((f #'(lambda (x) x))) + (let* ((closed-x (internal-get-closed-var 0)) + (x 'a)) + (list x (funcall f closed-x))))))))) + (provide 'cconv-tests) ;;; cconv-tests.el ends here --Apple-Mail=_D7447F0A-66F7-4083-8BF3-8B7D56A60FC3--