From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.bugs Subject: bug#54079: 29.0.50; Method dispatching eratically fails Date: Fri, 11 Mar 2022 22:01:41 +0000 Message-ID: References: <87k0d7257t.fsf@web.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="7712"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Michael Heerdegen , acm@muc.de, Lars Ingebrigtsen , 54079@debbugs.gnu.org To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Mar 11 23:02:39 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nSnLK-0001hQ-W5 for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 11 Mar 2022 23:02:39 +0100 Original-Received: from localhost ([::1]:38928 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nSnLJ-0005k3-Hf for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 11 Mar 2022 17:02:37 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:53460) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nSnKk-0005jm-AK for bug-gnu-emacs@gnu.org; Fri, 11 Mar 2022 17:02:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:44658) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nSnKk-0001Eg-1W for bug-gnu-emacs@gnu.org; Fri, 11 Mar 2022 17:02:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nSnKj-0004lC-Us for bug-gnu-emacs@gnu.org; Fri, 11 Mar 2022 17:02:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 11 Mar 2022 22:02:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 54079 X-GNU-PR-Package: emacs Original-Received: via spool by 54079-submit@debbugs.gnu.org id=B54079.164703611218281 (code B ref 54079); Fri, 11 Mar 2022 22:02:01 +0000 Original-Received: (at 54079) by debbugs.gnu.org; 11 Mar 2022 22:01:52 +0000 Original-Received: from localhost ([127.0.0.1]:38555 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nSnKa-0004km-Ap for submit@debbugs.gnu.org; Fri, 11 Mar 2022 17:01:52 -0500 Original-Received: from colin.muc.de ([193.149.48.1]:13547 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1nSnKY-0004kX-1K for 54079@debbugs.gnu.org; Fri, 11 Mar 2022 17:01:51 -0500 Original-Received: (qmail 20729 invoked by uid 3782); 11 Mar 2022 22:01:42 -0000 Original-Received: from acm.muc.de (p4fe157a5.dip0.t-ipconnect.de [79.225.87.165]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Fri, 11 Mar 2022 23:01:42 +0100 Original-Received: (qmail 9193 invoked by uid 1000); 11 Mar 2022 22:01:41 -0000 Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) X-Primary-Address: acm@muc.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:228236 Archived-At: Hello, Stefan. On Wed, Mar 09, 2022 at 17:04:00 -0500, Stefan Monnier wrote: > Alan Mackenzie [2022-03-09 20:32:59] wrote: > > On Wed, Mar 09, 2022 at 13:06:11 -0500, Stefan Monnier wrote: > >> >> I don't understand the scenario you're thinking of. > >> >> Are you thinking of something like `(eval-when-compile (byte-compile ...))? > >> > Yes. > >> >> Does that ever happen in real life? > >> > Probably exceedingly seldomly. > >> > What's to be gained by not catering to this unusual case? What do we > >> > lose? > >> We lose making it work right for the 99% other cases that *do* occur? > > How would it not work right for such a case? Can you give an example? > I'm not sure what "such a case" you're thinking of. One of the "99% other cases that *do* occur" that you referred to in your previous paragraph. You say that these wouldn't "work right". I'm asking you for an example of such a "not working right". > But in general, evaluation of code doesn't expect symbols to have > positions: The evaluation is completely indifferent to the SWPs, when symbols-with-pos-enabled is t. This is the case whilst eval-{when,and}-compile is being evaluated during a compilation. > ... it may test `eq` between symbols and it may be run without having > the magical `symbols-with-pos-enabled`. I've lost the thread here. What scenario are you considering? I thought we were talking only about the `eval' within eval-{when,and}-compile. > So as a general rule we should strip symbols before we pass it to `eval`. I don't see this, due to all the confusion we're experiencing. As a general rule, within byte compilation, symbol stripping should be postponed as long as possible whilst a compilation of the form is still possible. But, looking at the code, I don't think byte-compile binds symbols-with-pos-enabled to t. This could be a bug. > >> >> >> And why bother stripping the result of `byte-compile-eval`? > >> >> > Because it might be the result of evaluating a defun (or defvar or > >> >> > defconst). > >> >> AFAIK sympos should only appear within the compiler pipeline between the > >> >> "read" and the "emit resulting bytecode". They may be passed to various > >> >> functions and macros along the way, but I can't think of any scenario > >> >> where they'd end up returned by `(byte-compile-)eval`. > >> >> > This was the situation which gave rise to the bug. > >> >> Could you give some details about how it played out? > >> >> [ Either here or as a comment in the code. ] > >> > Michael byte compiled cl-generic.el. This created cl-generic.elc > >> > correctly, but also left uncompiled forms in the function cells of the > >> > symbols defun'd inside an eval-{when,and}-compile. These forms > >> > contained symbols with positions. > >> Hmm... we're talking about stripping the result of `byte-compile-eval`. > >> This function is only used for `eval-when-compile`, not `eval-and-compile`. > >> And nothing in your above description indicates that the sympos appeared > >> in the resulting value of `eval-when-compile` (as opposed to appearing > >> in the slot of functions and variables that were set during the course > >> of the evaluation). > > OK, sorry, I was mistaken. These forms with SWPs arose from > > evan-AND-compile, which doesn't use byte-compile-eval. > OK, now can we get back to the question: > And why bother stripping the result of `byte-compile-eval`? (This is in eval-when-compile only.) That result may contain SWPs. For example: (eval-when-compile (cons 'foo 'bar)) .. I don't think the stripping happens anywhere else, though I might be mistaken, here. > >> >> >> Fundamentally, `eval` should always strip before doing its job. > >> >> > Except when what it's evaluating is a defun, defmacrro, defsubst, etc. > >> >> Why? > >> > Because that evaluated form might later be byte compiled, and the SWPs > >> > will be needed for that. > >> I don't understand the scenario you're thinking of. > >> Are thinking of a case like: > >> - something causes the execution of (eval '(defun foo ...)) > >> - the user types `M-x byte-compile RET foo RET` > > Sorry again, I've lost the thread here. Weren't we talking about > > eval-{when,and}-compile, not eval? > No, the text you cited even included my original statement: > Fundamentally, `eval` should always strip before doing its job. You mean, by "always", you meant ALWAYS??? I understood you to mean "always, when in the context of eval-{when,and}-compile". If we're not inside a compilation, and thus there're no SWPs hanging around, stripping symbols from an expression will just mean a loss of time for a null operation. > > Inside these two special forms, we should preserve the SWPs as long as > > possible (i.e. as long as they won't cause any errors). > We should preserve them while macroexpanding and compiling their > contents, yes, but then we should strip the symbols before we pass the > result to `eval`. At the moment, I disagree with you. I don't think you have given an example of a form FOO which will only work if stripping is done before evaluation in (eval-when-compile FOO) or (eval-and-compile FOO) .. At the moment, I still think it is better to strip the positions after the evaluation. > >> If so, then: > >> - I don't think we should care about this case because it's extremely > >> rare and fundamentally broken (the symbol's function cell contains > >> a function *value* (i.e. a closure) and not a function's source code, > >> so in general we need `byte-compile--reify-function` which implements > >> a heuristic to go back to something like a source form, which can > >> break in various ways in corner cases). > > Really? After evaluating a defun, etc., we have a lisp form in the > > function cell, which may be a closure. > A closure is not "a Lisp form". In general passing a closure to `eval` > may signal an error because it may very well be an invalid form. > The body of a closure is a Lisp form, yes. But that's not what's inside > a symbol's function cell. > > The function byte-compile compiles an arbitrary form, doesn't it? > Try the following: > ;; -*- lexical-binding: t -*- > (let ((x 0)) > (defun my-inc1 () > (interactive) > (message "x=%S" (setq x (1+ x)))) > (defun my-inc2 () > (interactive) > (message "x=%S" (setq x (1+ x))))) > load that file. Then try `M-x my-inc1` and `M-x my-inc2` and you'll see > that they correctly increment the same counter. > Now do `M-: (byte-compile 'my-inc1)`. > And try `M-x my-inc1` and `M-x my-inc2` and you'll see that suddenly > they each have their own counter. > That's because it's one of the corner cases that > `byte-compile--reify-function` can't handle correctly. > >> - If we don't strip before calling the `M-x byte-compile` then the code > >> may/will bisbehave because of the presence of the sympos. > > How? byte-compile is designed to use SWPs. > The misbehavior I'm referring to is what happens when you call the > function before you byte-compile it (or, more likely, when you never end > up byte-compiling it), because the presence of sympos in the function > will mess up its semantics (because `symbols-with-pos-enabled` won't be > set any more when it's called). I'm puzzled. Are we still talking about eval-{when,and}-compile, here? If so, how can a form with SWPs get into a symbol's function cell? The positions are stripped inside the e-w/a-compile. If not, the only function which produces SWPs is read-positioning-symbols (in lread.c). (There are other functions which _can_ produce SWPs, but they're used only by the compiler, I think.) read-positioning-symbols is called only by the byte compiler, so how can an uncompiled form with SWPs get into a symbol's function cell? I think the only answer to either of the above two paragraphs is a bug. Ah, maybe not. I think you're thinking of something like (eval-when-compile (fset 'foo '(defun bar ....))) , where SWPs will escape into foo. Again, this is presumably a vanishingly unlikely thing to write, except for the writing of test suites. ;-) Maybe we will need to make strip-symbol-positions available to (advanced) users, after all (having renamed it from byte-run-strip-symbol-positions). > Stefan -- Alan Mackenzie (Nuremberg, Germany).