From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#54079: 29.0.50; Method dispatching eratically fails Date: Wed, 09 Mar 2022 17:04:00 -0500 Message-ID: References: <87o82ypfxz.fsf@web.de> <875youbhbu.fsf@web.de> <87k0d7257t.fsf@web.de> Reply-To: Stefan Monnier Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="15757"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Cc: Michael Heerdegen , Lars Ingebrigtsen , 54079@debbugs.gnu.org To: Alan Mackenzie Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Mar 09 23:05:17 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nS4Qm-0003uZ-CN for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 09 Mar 2022 23:05:16 +0100 Original-Received: from localhost ([::1]:38494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nS4Qk-0001ox-QD for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 09 Mar 2022 17:05:14 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:53174) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nS4QY-0001oo-8e for bug-gnu-emacs@gnu.org; Wed, 09 Mar 2022 17:05:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:38854) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nS4QX-0006CN-Rw for bug-gnu-emacs@gnu.org; Wed, 09 Mar 2022 17:05:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nS4QX-0004eF-Km for bug-gnu-emacs@gnu.org; Wed, 09 Mar 2022 17:05:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 09 Mar 2022 22:05:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 54079 X-GNU-PR-Package: emacs Original-Received: via spool by 54079-submit@debbugs.gnu.org id=B54079.164686345917801 (code B ref 54079); Wed, 09 Mar 2022 22:05:01 +0000 Original-Received: (at 54079) by debbugs.gnu.org; 9 Mar 2022 22:04:19 +0000 Original-Received: from localhost ([127.0.0.1]:60984 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nS4Pn-0004cz-9W for submit@debbugs.gnu.org; Wed, 09 Mar 2022 17:04:19 -0500 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:55115) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nS4Ph-0004cf-8l for 54079@debbugs.gnu.org; Wed, 09 Mar 2022 17:04:14 -0500 Original-Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 472EC8012F; Wed, 9 Mar 2022 17:04:03 -0500 (EST) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 77277802FF; Wed, 9 Mar 2022 17:04:01 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1646863441; bh=COpiJ7DR8BwqOfBUPX+p9vcnIjYiW7BmmTAR1Gw2aF4=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=W41vY1GRBMdfxaCQCQqusmhN04Hm9IVRFzsSTLjMBZYsPNbjpedimM3EG1BWXquir /tdCayEClZI6SGYILV8JP2obeIf09M7yycvM7pR1Gxaf/6JOEs5tFg4u3yWBZjuxoh IPbZGHmkLYOs2+pOBa5zehoIxNJ+C9GcTkCaWfAoRmqUsomBmfdkCnHHCaM4sYKFMU c76duFV1f1gDjUuOXNHE+TtCafm0RnJm5F7Npl/n4TewBsd0d3EUpCniFa64JXFjFj tZ6OdR+Wt6BrjY5JpeN6KZdd8BdOL8iMtpRxEn8wlDl3aGHaPk9zGVQoZPthktYby1 ls1hQ0bVILh7g== Original-Received: from alfajor (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 5D5D81202B9; Wed, 9 Mar 2022 17:04:01 -0500 (EST) In-Reply-To: (Alan Mackenzie's message of "Wed, 9 Mar 2022 20:32:59 +0000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:228176 Archived-At: Alan Mackenzie [2022-03-09 20:32:59] wrote: > On Wed, Mar 09, 2022 at 13:06:11 -0500, Stefan Monnier wrote: >> >> I don't understand the scenario you're thinking of. >> >> Are you thinking of something like `(eval-when-compile (byte-compile ...))? >> > Yes. >> >> Does that ever happen in real life? >> > Probably exceedingly seldomly. >> > What's to be gained by not catering to this unusual case? What do we >> > lose? > >> We lose making it work right for the 99% other cases that *do* occur? > How would it not work right for such a case? Can you give an example? I'm not sure what "such a case" you're thinking of. But in general, evaluation of code doesn't expect symbols to have positions: it may test `eq` between symbols and it may be run without having the magical `symbols-with-pos-enabled`. So as a general rule we should strip symbols before we pass it to `eval`. >> >> >> And why bother stripping the result of `byte-compile-eval`? >> >> > Because it might be the result of evaluating a defun (or defvar or >> >> > defconst). > >> >> AFAIK sympos should only appear within the compiler pipeline between the >> >> "read" and the "emit resulting bytecode". They may be passed to various >> >> functions and macros along the way, but I can't think of any scenario >> >> where they'd end up returned by `(byte-compile-)eval`. > >> >> > This was the situation which gave rise to the bug. > >> >> Could you give some details about how it played out? >> >> [ Either here or as a comment in the code. ] > >> > Michael byte compiled cl-generic.el. This created cl-generic.elc >> > correctly, but also left uncompiled forms in the function cells of the >> > symbols defun'd inside an eval-{when,and}-compile. These forms >> > contained symbols with positions. > >> Hmm... we're talking about stripping the result of `byte-compile-eval`. >> This function is only used for `eval-when-compile`, not `eval-and-compile`. >> And nothing in your above description indicates that the sympos appeared >> in the resulting value of `eval-when-compile` (as opposed to appearing >> in the slot of functions and variables that were set during the course >> of the evaluation). > > OK, sorry, I was mistaken. These forms with SWPs arose from > evan-AND-compile, which doesn't use byte-compile-eval. OK, now can we get back to the question: And why bother stripping the result of `byte-compile-eval`? >> >> >> Fundamentally, `eval` should always strip before doing its job. >> >> > Except when what it's evaluating is a defun, defmacrro, defsubst, etc. >> >> Why? >> > Because that evaluated form might later be byte compiled, and the SWPs >> > will be needed for that. > >> I don't understand the scenario you're thinking of. >> Are thinking of a case like: > >> - something causes the execution of (eval '(defun foo ...)) >> - the user types `M-x byte-compile RET foo RET` > > Sorry again, I've lost the thread here. Weren't we talking about > eval-{when,and}-compile, not eval? No, the text you cited even included my original statement: Fundamentally, `eval` should always strip before doing its job. > Inside these two special forms, we should preserve the SWPs as long as > possible (i.e. as long as they won't cause any errors). We should preserve them while macroexpanding and compiling their contents, yes, but then we should strip the symbols before we pass the result to `eval`. >> If so, then: >> - I don't think we should care about this case because it's extremely >> rare and fundamentally broken (the symbol's function cell contains >> a function *value* (i.e. a closure) and not a function's source code, >> so in general we need `byte-compile--reify-function` which implements >> a heuristic to go back to something like a source form, which can >> break in various ways in corner cases). > > Really? After evaluating a defun, etc., we have a lisp form in the > function cell, which may be a closure. A closure is not "a Lisp form". In general passing a closure to `eval` may signal an error because it may very well be an invalid form. The body of a closure is a Lips form, yes. But that's not what's inside a symbol's function cell. > The function byte-compile compiles an arbitrary form, doesn't it? Try the following: ;; -*- lexical-binding: t -*- (let ((x 0)) (defun my-inc1 () (interactive) (message "x=%S" (setq x (1+ x)))) (defun my-inc2 () (interactive) (message "x=%S" (setq x (1+ x))))) load that file. Then try `M-x my-inc1` and `M-x my-inc2` and you'll see that they correctly increment the same counter. Now do `M-: (byte-compile 'my-inc1)`. And try `M-x my-inc1` and `M-x my-inc2` and you'll see that suddenly they each have their own counter. That's because it's one of the corner cases that `byte-compile--reify-function` can't handle correctly. >> - If we don't strip before calling the `M-x byte-compile` then the code >> may/will bisbehave because of the presence of the sympos. > How? byte-compile is designed to use SWPs. The misbehavior I'm referring to is what happens when you call the function before you byte-compile it (or, more likely, when you never end up byte-compiling it), because the presence of sympos in the function will mess up its semantics (because `symbols-with-pos-enabled` won't be set any more when it's called). Stefan