From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.bugs Subject: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Date: Wed, 27 Mar 2024 21:43:29 +0000 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="542"; mail-complaints-to="usenet@ciao.gmane.io" Cc: acm@muc.de, Eli Zaretskii , 67455@debbugs.gnu.org To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Mar 27 22:44:41 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rpb4a-000ASn-Dc for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 27 Mar 2024 22:44:40 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rpb42-0003Vi-DX; Wed, 27 Mar 2024 17:44:06 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rpb3z-0003VM-Er for bug-gnu-emacs@gnu.org; Wed, 27 Mar 2024 17:44:03 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rpb3y-0008W3-Q7 for bug-gnu-emacs@gnu.org; Wed, 27 Mar 2024 17:44:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rpb3y-0007Pc-NT for bug-gnu-emacs@gnu.org; Wed, 27 Mar 2024 17:44:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 27 Mar 2024 21:44:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67455 X-GNU-PR-Package: emacs Original-Received: via spool by 67455-submit@debbugs.gnu.org id=B67455.171157582428345 (code B ref 67455); Wed, 27 Mar 2024 21:44:02 +0000 Original-Received: (at 67455) by debbugs.gnu.org; 27 Mar 2024 21:43:44 +0000 Original-Received: from localhost ([127.0.0.1]:38461 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rpb3f-0007N4-G0 for submit@debbugs.gnu.org; Wed, 27 Mar 2024 17:43:44 -0400 Original-Received: from mail.muc.de ([193.149.48.3]:52188) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rpb3Z-0007MG-IK for 67455@debbugs.gnu.org; Wed, 27 Mar 2024 17:43:38 -0400 Original-Received: (qmail 43918 invoked by uid 3782); 27 Mar 2024 22:43:30 +0100 Original-Received: from acm.muc.de (pd953a0c3.dip0.t-ipconnect.de [217.83.160.195]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Wed, 27 Mar 2024 22:43:30 +0100 Original-Received: (qmail 9387 invoked by uid 1000); 27 Mar 2024 21:43:29 -0000 Content-Disposition: inline In-Reply-To: X-Submission-Agent: TMDA/1.3.x (Ph3nix) X-Primary-Address: acm@muc.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:282158 Archived-At: Hello, Stefan. On Wed, Mar 27, 2024 at 08:22:27 -0400, Stefan Monnier wrote: > > More precisely, to @dfn{posify} their containing forms by writing the > > position information into their doc strings. We do this in the byte > > compilation case, too. The difference is that in the "load from source" > > case we want to strip the SWP, in byte compilation, we don't. > [ Nitpick: in byte-compilation, we also do. We want to keep the SWPs > longer, but in the end we also want to strip them away necause we don't > want them in the resulting compiled code. ] In byte compilation we want to keeo the SWPs for AS LONG AS POSSIBLE. In other cases, we want to keep them for AS SHORT A TIME AS POSSIBLE. > > I don't think that is a good name. The byte compiler has no business > > setting "internal" variables for the posification processing. Instead it > > should announce it's running and expect the posification to respect that. > > I think byte-compile-in-progress is a good name for this. > AFAIK we want those SWPs stripped if and only if we're in a "load from > source" case. The compilation case is one of those where we don't want > to strip them, but it's not the only one, so the compiler should not do > anything w.r.t to that. Instead it's the code that does the "load from > source" which should set some indicator that stripping is requested. I think it is better to regard the byte compilation as the special case. Only in byte compilation do we want to preserve the SWPs on forms getting posified. > Also, I think as a general rule it's better for the caller to set > a callee variable that controls how the callee behaves, rather than for the > callee to check a caller variable to decide how to behave, because > it's normal for the caller to "know about" its callee (after all, it's > the caller which decides to call the callee), whereas it's not normal > for the callee to know about specific callers (it creates undesirable > dependencies). Byte compilation is NOT calling loading from source. We don't have a caller/callee relationship here. We are doing posification either from byte compilation or from somewhere else. It is analogous to testing lexical binding. Here, the variable is called lexical-binding; it is not named after a particular activity to be carried out differently for l-b and not l-b. > >> Better yet: to avoid the problem of dynamic scope extending "too far" > >> (i.e. accidentally applying to nested loads/evals/compile/...), you > >> could put the relevant info into `macroexpand-all-environment`. > >> [ That var is also dynamically bound, but we're already careful to > >> rebind it when needed so it doesn't apply to nested uses > >> of macroexpansion. ] > > That variable is only loaded in the 17th loaded Lisp file. The new > > facility should be working at the earliest stages of loading Lisp, as it > > does at the moment. > The earlier the better, in theory, but not at any cost. No, the earlier the better, full stop. The earlier an error happens in bootstrapping, the more important it is to provide debugging support. The new facilities were exceptionally helpful whilst debugging the partially finished code. > Having to write all that code within the very restrictive sublanguage > available before subr.el and backquote.el is a cost I don't think > justifies it. The cost has already been paid, by me. The cost of maintaining that code will be small by comparison; byte-run--posify-defining-form is _not_ all that difficult to understand and amend. > If we *really* want that, then we should explore other avenues, such as > keeping pre-macroexpanded versions of the files (for bootstrapping > purposes) but generating those files from files where a more normal > coding style can be used. > [ Something similar to the ldefs-boot.el. ] Possibly - but that will also introduce complications. > > Besides, macroexpand-all-environment is not > > documented anywhere, what it is, what it's for, etc. > Feel free to disregard my advice if you don't like it. > I'm just pointing out that it's probably the tool which gives you the > semantics you want. OK. > >> My crystal ball suggests that "currently" may be the wrong way to think > >> about it: maybe instead of thinking of "when" (as in "during the > >> definition of function FOO") what you're looking for might be "where" > >> (as in "within the body of FOO"). > >> [ That's the same difference as the difference between dynamic and > >> static scoping. ] > > I'm having trouble understanding what you're saying, here. > Is it because you don't understand the difference between dynamic > scoping and static scoping, or because you don't see the relationship > with that and your notion of "currently being defined"? The latter, I think. defining-symbol is entirely dynamically scoped. > The above citation is in the context of my question about what you mean > by "currently" in: > doc: /* The symbol currently being defined by a defining form. > I personally don't really understand it, and AFAICT, you don't really > understand it either because you haven't been able to describe it. I understand it fully. I'm puzzled by your failure to understand what I've written. The flow goes as follows: (i) defining-symbol gets bound to nil in readevalloop_eager_eval_loop. (ii) d-s gets setq'd to NAME in defun, defvar, cl-defgeneric, ..... Usually, the setq has been generated by byte-run--posify-defining-form. (iii) d-s gets used in byte-run-posify-doc-string, which writes its details to a new or existing doc string. By "currently", I mean that d-s holds NAME between (ii) and (iii) and until the process of defining the new form is complete. > >> > Ideally, I would like to have bound defining-symbol inside defun. > >> (defmacro my-defun (name args &rest body) > >> `(cl-macrolet ((defining-symbol () '',name)) > >> (defun ,name ,args ,@body))) > >> (my-defun my-foo (x) (list x (defining-symbol))) > >> (symbol-function 'my-foo) > >> ==> #f(lambda (x) [t] (list x 'my-foo)) > >> `cl-macrolet` uses `macroexpand-all-environment` for that. > > cl-macs gets loaded far too late for such an approach to be useful. > That's not really relevant since we're just trying to understand what > you mean by "currently". What is relevant is whether it gives > the intended semantics. I'm convinced it does. Can you suggest a scenario where the defining-symbol mechanism (outlined above) might fail? > But if you insist, here's the equivalent version without `cl-macs` > (using the same underlying technique as used by `cl-macrolet`): > (defmacro my-defun (name args &rest body) > (macroexpand-all > `(defun ,name ,args ,@body) > (cons > (cons 'defining-symbol (lambda () `',name)) > macroexpand-all-environment))) Thanks. > >> Do you have rough numbers comparing the cost of `read`, > >> `read-positioning-symbols`, and `read-positioning-DEFINED-symbols`? > > No, but they will be very close to eachother (and very cheap) > Then I think we should use `read-positioning-symbols`, which > requires fewer code changes. It won't. Required would be Lisp code to determine whether a particular SWP needs to be stripped or not. This is not going to be simple. It is likely to be about as complicated as the existing enhancements to read0. > >> Also, IIUC you don't have a separate phase to strip the SWPs when > >> loading from source, but instead you strip them as you "consume" their > >> info during macroexpansion. If so, how/when/where do you strip the > >> false positives that may occur inside quoted data or in code like: > >> (defmacro foo (lambda bar) ...) > >> (defmacro foo (defun bar) ...) > >> (let* ((lambda foo) > >> (defun bar)) > >> ...) > > There's a pcase arm right at the end of macroexp--expand-all which strips > > SWPs of their positions. Recursing through macroexp--all-forms will > > eventually hit this pcase arm for these lambdas. > Ah, so it's like a "strip phase" but "fused" into the macroexpansion phase. I suppose so. > >> Not at all. Those will remain without position, but only in > >> `src/bootstrap-emacs`. > > This would be a Bad Thing. > But your current code in byte-run.el is a Bad Thing as well. What, precisely, do you find bad about it? It may be possible to improve it without wholesale redesign. > It's all a question of trade-offs :-( > >> In the real `src/emacs` they will get the position because they'll come > >> from the `.el[cn]` file and by the time we get compile those files > >> `macro-declarations-alist` will be fully populated. > > The understanding we reached in November was that loading from source > > files would be handled, too. > I'm not suggesting to drop support for lambdas loaded from source. > I'm saying we don't need to support it for the first N files loaded into > `src/emacs-bootstrap`. You're suggesting dropping support for many source files, where that support is most needed. You're suggesting introducing awkward special cases where the code won't work. As currently implemented, the code DOES work. > Stefan -- Alan Mackenzie (Nuremberg, Germany).