From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#51982: Erroneous handling of local variables in byte-compiled nested lambdas Date: Tue, 30 Nov 2021 09:12:20 -0500 Message-ID: References: <87y25jo2q1.fsf@web.de> <29C3A3F8-CD9F-4AF2-A731-3304FC30E380@acm.org> <87wnl23pnd.fsf@web.de> <59A729EF-C4D4-47EB-9ADC-19FE8EBE7F10@acm.org> <877dd0bi17.fsf@web.de> Reply-To: Stefan Monnier Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="17671"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Cc: Michael Heerdegen , Paul Pogonyshev , 51982@debbugs.gnu.org To: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Nov 30 15:13:57 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ms3tM-0004KL-9J for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 30 Nov 2021 15:13:56 +0100 Original-Received: from localhost ([::1]:37854 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ms3tL-0003X8-AE for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 30 Nov 2021 09:13:55 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:35622) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ms3sU-0002Ck-57 for bug-gnu-emacs@gnu.org; Tue, 30 Nov 2021 09:13:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:57152) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ms3sT-0007Jo-Sr for bug-gnu-emacs@gnu.org; Tue, 30 Nov 2021 09:13:01 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ms3sT-0004tu-MH for bug-gnu-emacs@gnu.org; Tue, 30 Nov 2021 09:13:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 30 Nov 2021 14:13:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51982 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 51982-submit@debbugs.gnu.org id=B51982.163828155318798 (code B ref 51982); Tue, 30 Nov 2021 14:13:01 +0000 Original-Received: (at 51982) by debbugs.gnu.org; 30 Nov 2021 14:12:33 +0000 Original-Received: from localhost ([127.0.0.1]:40465 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ms3s1-0004t7-2v for submit@debbugs.gnu.org; Tue, 30 Nov 2021 09:12:33 -0500 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:38285) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ms3ry-0004sr-AU for 51982@debbugs.gnu.org; Tue, 30 Nov 2021 09:12:31 -0500 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 5D5B24413A5; Tue, 30 Nov 2021 09:12:24 -0500 (EST) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id ECCF8441392; Tue, 30 Nov 2021 09:12:21 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1638281541; bh=xDNKp6+GvEScxtLHSp3kGfkMcDJ43Eb/8NP2sgODffQ=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=mQxmbu6yGwiiFr0qVtsgKROD6KaJsLX4NiDPeeDIFs4H50bR7YLdxc/b10xcQLuQC ZinjfUSX7pUB6WtcmnNCwTdxx+V4Wo4wNJxyh38a7kFTHuaHIXaHmnQbz4uAcCRpJ7 LG3NAVFPY/9k/XkRt4W+cmxWXJczY2Qgm1DPU32xTZ1XH4eQvhotpg/dJWdJm43eOT 1i084c6+Dh7F+mJBtxRp1cu5VgKi6vAdEla60Rinolp2uXtZ40Mf55KIBxkifno5VX iT3UGbvE5FCxT9vzNXeyty3MpTbu+7Td6R1xmb6IhKzAPyCxHkES9hyeQxEv4Myusn SrUX3+5V6XBaQ== Original-Received: from pastel (unknown [216.154.30.173]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id BB087120124; Tue, 30 Nov 2021 09:12:21 -0500 (EST) In-Reply-To: ("Mattias =?UTF-8?Q?Engdeg=C3=A5rd?="'s message of "Mon, 22 Nov 2021 18:35:18 +0100") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:221121 Archived-At: Sorry 'bout the delay, and thanks Mattias for finding the bug and the fix. > @@ -428,10 +428,26 @@ cconv-convert > ;; One of the lambda-lifted vars is shadowed, so add > ;; a reference to the outside binding and arrange to use > ;; that reference. > - (let ((closedsym (make-symbol (format "closed-%s" var))= )) > - (setq new-env (cconv--remap-llv new-env var closedsym= )) > - (setq new-extend (cons closedsym (remq var new-extend= ))) > - (push `(,closedsym ,var) binders-new))) > + (let* ((mapping (cdr (assq var env))) > + (var-def > + (pcase-exhaustive mapping > + (`(internal-get-closed-var . ,_) > + ;; The variable is captured. > + mapping) > + (`(car-safe (internal-get-closed-var . ,_)) > + ;; The variable is mutably captured; skip > + ;; the indirection step because the variable= is > + ;; passed "by rerefence" to the =CE=BB-lifte= d function. > + (cadr mapping)) > + ((or '() `(car-safe ,(pred symbolp))) > + ;; The variable is not captured. Add a > + ;; reference to the outside binding and arra= nge > + ;; to use that reference. > + var)))) > + (let ((closedsym (make-symbol (format "closed-%s" var= )))) > + (setq new-env (cconv--remap-llv new-env var closeds= ym)) > + (setq new-extend (cons closedsym (remq var new-exte= nd))) > + (push `(,closedsym ,var-def) binders-new)))) >=20=20 > ;; We push the element after redefined free variables are > ;; processed. This is important to avoid the bug when fr= ee > @@ -449,14 +465,28 @@ cconv-convert > ;; before we know that the var will be in `new-extend' (bug#241= 71). > (dolist (binder binders-new) > (when (memq (car-safe binder) new-extend) > - ;; One of the lambda-lifted vars is shadowed, so add > - ;; a reference to the outside binding and arrange to use > - ;; that reference. > + ;; One of the lambda-lifted vars is shadowed. > (let* ((var (car-safe binder)) > - (closedsym (make-symbol (format "closed-%s" var)))) > - (setq new-env (cconv--remap-llv new-env var closedsym)) > - (setq new-extend (cons closedsym (remq var new-extend))) > - (push `(,closedsym ,var) binders-new))))) > + (mapping (cdr (assq var env))) > + (var-def > + (pcase-exhaustive mapping > + (`(internal-get-closed-var . ,_) > + ;; The variable is captured. > + mapping) > + (`(car-safe (internal-get-closed-var . ,_)) > + ;; The variable is mutably captured; skip > + ;; the indirection step because the variable is > + ;; passed "by rerefence" to the =CE=BB-lifted fu= nction. > + (cadr mapping)) > + ((or '() `(car-safe ,(pred symbolp))) > + ;; The variable is not captured. Add a > + ;; reference to the outside binding and > + ;; arrange to use that reference. > + var)))) > + (let ((closedsym (make-symbol (format "closed-%s" var)))) > + (setq new-env (cconv--remap-llv new-env var closedsym)) > + (setq new-extend (cons closedsym (remq var new-extend))) > + (push `(,closedsym ,var-def) binders-new)))))) Can we avoid this duplication by moving that code to a separate function? > + (let ((f (lambda (x) > + (let ((g (lambda () x)) > + (h (lambda () (setq x x)))) > + (let ((x 'b)) > + (list x (funcall g) (funcall h))))))) > + (funcall (funcall f 'b))) > + (let ((f (lambda (x) > + (let ((g (lambda () x)) > + (h (lambda () (setq x x)))) > + (let* ((x 'b)) > + (list x (funcall g) (funcall h))))))) > + (funcall (funcall f 'b))) These two tests are identical aren't they? Also, can we change the (setq x x) into something like (setq x (list x x)) and avoid using the same `b` value for both `x` vars, so as to catch more potential errors? > @@ -428,10 +428,27 @@ cconv-convert > ;; One of the lambda-lifted vars is shadowed, so add > ;; a reference to the outside binding and arrange to use > ;; that reference. > - (let ((closedsym (make-symbol (format "closed-%s" var))= )) > - (setq new-env (cconv--remap-llv new-env var closedsym= )) > - (setq new-extend (cons closedsym (remq var new-extend= ))) > - (push `(,closedsym ,var) binders-new))) > + (let* ((mapping (cdr (assq var env))) > + (remap-to > + (pcase-exhaustive mapping > + (`(internal-get-closed-var . ,_) > + ;; The variable is captured; remap. > + mapping) > + (`(car-safe (internal-get-closed-var . ,_)) > + ;; The variable is mutably captured; remap, = but skip > + ;; the indirection step because the variable= is > + ;; passed "by rerefence" to the =CE=BB-lifte= d function. > + (cadr mapping)) > + ((or '() `(car-safe ,(pred symbolp))) > + ;; The variable is not captured. Add a > + ;; reference to the outside binding and arra= nge > + ;; to use that reference. > + (let ((closedsym > + (make-symbol (format "closed-%s" var)= ))) > + (push `(,closedsym ,var) binders-new) > + closedsym))))) > + (setq new-env (cconv--remap-llv new-env var remap-to)) > + (setq new-extend (cons remap-to (remq var new-extend)= )))) >=20=20 > ;; We push the element after redefined free variables are > ;; processed. This is important to avoid the bug when fr= ee Looks good (better than patch A). You say "On the other hand, patch B does abuse the cconv data structures a little (but it works!)" so the code should say something about this abuse. A least I failed to see where the abuse lies. > @@ -449,14 +466,29 @@ cconv-convert > ;; before we know that the var will be in `new-extend' (bug#241= 71). > (dolist (binder binders-new) > (when (memq (car-safe binder) new-extend) > - ;; One of the lambda-lifted vars is shadowed, so add > - ;; a reference to the outside binding and arrange to use > - ;; that reference. > + ;; One of the lambda-lifted vars is shadowed. > (let* ((var (car-safe binder)) > - (closedsym (make-symbol (format "closed-%s" var)))) > - (setq new-env (cconv--remap-llv new-env var closedsym)) > - (setq new-extend (cons closedsym (remq var new-extend))) > - (push `(,closedsym ,var) binders-new))))) > + (mapping (cdr (assq var env))) > + (remap-to > + (pcase-exhaustive mapping > + (`(internal-get-closed-var . ,_) > + ;; The variable is captured; remap. > + mapping) > + (`(car-safe (internal-get-closed-var . ,_)) > + ;; The variable is mutably captured; remap, but = skip > + ;; the indirection step because the variable is > + ;; passed "by rerefence" to the =CE=BB-lifted fu= nction. > + (cadr mapping)) > + ((or '() `(car-safe ,(pred symbolp))) > + ;; The variable is not captured. Add a > + ;; reference to the outside binding and arrange > + ;; to use that reference. > + (let ((closedsym > + (make-symbol (format "closed-%s" var)))) > + (push `(,closedsym ,var) binders-new) > + closedsym))))) > + (setq new-env (cconv--remap-llv new-env var remap-to)) > + (setq new-extend (cons remap-to (remq var new-extend)))))= )) Same comment as before about the code duplication. Stefan