From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.) Date: Mon, 25 Mar 2024 18:10:11 -0400 Message-ID: References: Reply-To: Stefan Monnier Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="28967"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Eli Zaretskii , 67455@debbugs.gnu.org To: Alan Mackenzie Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Mar 25 23:13:28 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rosZK-0007Kq-WE for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 25 Mar 2024 23:13:27 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rosZ0-0003cc-B9; Mon, 25 Mar 2024 18:13:06 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rosYy-0003cU-MT for bug-gnu-emacs@gnu.org; Mon, 25 Mar 2024 18:13:04 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rosYy-0003jg-Ef for bug-gnu-emacs@gnu.org; Mon, 25 Mar 2024 18:13:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rosYw-00037x-8t for bug-gnu-emacs@gnu.org; Mon, 25 Mar 2024 18:13:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 25 Mar 2024 22:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67455 X-GNU-PR-Package: emacs Original-Received: via spool by 67455-submit@debbugs.gnu.org id=B67455.171140476511995 (code B ref 67455); Mon, 25 Mar 2024 22:13:02 +0000 Original-Received: (at 67455) by debbugs.gnu.org; 25 Mar 2024 22:12:45 +0000 Original-Received: from localhost ([127.0.0.1]:36459 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rosYd-00037N-Jy for submit@debbugs.gnu.org; Mon, 25 Mar 2024 18:12:45 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:29651) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rosYY-000377-Bb for 67455@debbugs.gnu.org; Mon, 25 Mar 2024 18:12:42 -0400 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 24195442957; Mon, 25 Mar 2024 18:12:33 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1711404751; bh=XGWAsB2eDcmCpt2UnSunPdg2kBpey/KRKEblCztcei8=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=ei1voFaqlvQQ7o4PcqzHr+XC1UzLE4WkgsAqgkqUYHCA6rVPyVwBFUt/QCe/8Tklm AR8OIJD9QT7PeusLNKJzAFX46LY3TakUu7CkRXyX410WBpMRdZ/nwBRIJYQymwUYii 1g/NOriCYCN+VRW5WRXtknvJZhrnkccvvVAbc8hCNTrW4ukBKFHh4//DvHtKgiQZfW KS5R5+cN7zuIqdtci9gVXoTMqKHJWClOzxkLCkgRvAQrPC4MLhlHfSDebrrjbuOXT0 DRiS9/Fqy0afoVC0uDgR9q9T7Uq6sykqPbot6SILYhBo3OEm4RXHYkfavTpRV10bmr AluRysse6RGrw== Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 0F4DA442953; Mon, 25 Mar 2024 18:12:31 -0400 (EDT) Original-Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id F15B1120280; Mon, 25 Mar 2024 18:12:30 -0400 (EDT) In-Reply-To: (Alan Mackenzie's message of "Mon, 25 Mar 2024 21:03:22 +0000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:282079 Archived-At: >> That sounds rather ugly. I'd rather first try and define precisely >> what is we mean by "compilation in progress". > That the byte compiler is active, and all symbols (bar nil) get > positioned. That is much too vague to be usable. Which symbols are we talking about? What do we mean by "active"? I think we need to describe it in terms that link the symbol with code in bytecomp.el (not based on time, but based on function bytecomp-FOO calling BAR calling BAZ ... touching that symbol). Note that I also have no idea why we need to care about whether we're in the compiled case or the non-compiled case, so maybe my questions are "XY" style problems. > An alternative might be to pass an &optional boolean argument meaning > "preserve positions on symbols". >From where to where should we pass that argument? [ And yes, explicit arguments might be preferable unless the call-trace is too long for it to be practical or unless some of the functions along that call trace can't be changed easily to take such an extra argument. ] BTW, do you really mean "preserve" (meaning that the symbols were sympos to start with)? >> I see the same problem with: > >> DEFVAR_LISP ("defining-symbol", Vdefining_symbol, >> doc: /* The symbol currently being defined by a defining form. >> This variable is bound in the read-eval-print loop and certain >> high-level functions in the byte compiler. It is set to a value by >> functions and macros such as `defun', `defmacro', and `defvar'. */); > >> Lots and lots of things can happen "during the definition" of a form, >> including definition of lots of other forms. So I think we'd need to >> define much more precisely what you meant by "currently". >> In addition, a definition is "intemporal" (it's declarative), so >> "currently being defined" is almost like an oxymoron. > > By "currently", I mean that a defining form such as defun or defvar has > commenced, but not yet terminated; its functions currently occupy stack > frames. So you mean we're inside `Fdefalias` or `Fdefvar_1`? >> I'm trying to understand your code, but I clearly lack a high-level >> overview of the approach you decided to takes, so I don't understand >> what's going on there. > > Sorry about that. A quick summary: defined symbols (and lambda) get > positioned by the new reader function read-positioning-defined symbols. > The new declare clause defining-symbol marks a macro such as defun or > cl-defgeneric as a macro which defines such symbols. > > The conversion of these SWPs into position structures in doc strings > happens at macro expansion time, through byte-run-posify-lambda-form. So, IIUC (defmacro defun (name args docstring &rest body) (declare (defining-symbol 1)) ...) is akin to: (defmacro defun (name args docstring &rest body) (setq docstring (add-pos-to-docstring (symbol-with-pos-pos name) docstring)) ...) ? >> If both, could you split it into two, then? > I'm not sure that would be possible or sensible - both use a common > approach. So, IIUC a first part of the change was to make `load` use `read-positioning-symbols` just like the compiler? >> AFAICT doing it only for compiled functions should be significantly >> simpler than for interpreted functions, so it would be a good >> stepping stone. > The work has already been done, and there is working code. Just as a > matter of interest, the branch runs the test suite without errors (not > counting "expensive" tests ). I'm not talking about a separate branch. I'm talking about splitting your changes into understandable commits. >> On the cosmetic side, you have way too much code in `byte-run.el`. >> I think most of this code can be moved elsewhere, e.g. somewhere where >> backquote can be used > > Yes, I noticed this, too. A lot of the bulk is for diagnostic functions > for SWPs, and these can eventually be deleted. Or possibly moved into a > new file with-pos.el to be loaded before byte-run.el. I don't like the idea of adding another file before byte-run.el. > byte-run--posify-defining-symbol, the function with the extreme hand > expansion of backquotes is used as a declare clause handler, and is > needed by defun. Hence it couldn't really be moved to after the loading > of backquote.el. I think you can simply wait to add the entry to `macro-declarations-alist` until a later time, so the `defining-symbol` thingies will be ignored during the early bootstrap and once we have more infrastructure in place we can then register the handler on `macro-declarations-alist`. > There are some additional functions which batch-posify functions and > variables defined before the posification mechanism is in place. This > must be done ASAP, for the benefit of backtraces in early bootstrap. That complexifies the early bootstrap code. I'd rather keep that code simpler. During early bootstrap, all those functions are interpreted and I can't remember ever having difficulty tracking the origin of those interpreted lambdas during early bootstrap, so I'm rather opposed to such complexity just for the sake of maybe occasionally hypothetically giving a bit of help to the 3 of us who have to deal with those problems. I see functions-with-pos as something mostly targeted at "end users" who aren't expert enough to figure out the origin of anonymous functions by looking at the disassembly of the bytecode and taking an educated guess. Stefan