From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Update 1 on Bytecode Offset tracking Date: Fri, 17 Jul 2020 18:08:34 -0400 Message-ID: References: <87a700fk3j.fsf@gmail.com> <87blkfoz9v.fsf@gmail.com> <87wo31sxmu.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="12717"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: emacs-devel@gnu.org To: Zach Shaftel Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Jul 18 00:44:46 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jwZ5x-0003C0-MU for ged-emacs-devel@m.gmane-mx.org; Sat, 18 Jul 2020 00:44:45 +0200 Original-Received: from localhost ([::1]:60852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jwYXe-0005vD-Qq for ged-emacs-devel@m.gmane-mx.org; Fri, 17 Jul 2020 18:09:18 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:34174) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jwYX9-0005W1-1G for emacs-devel@gnu.org; Fri, 17 Jul 2020 18:08:47 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:55032) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jwYX6-0000kE-1E for emacs-devel@gnu.org; Fri, 17 Jul 2020 18:08:46 -0400 Original-Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 9E6F380B63; Fri, 17 Jul 2020 18:08:42 -0400 (EDT) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 05AEA80D35; Fri, 17 Jul 2020 18:08:36 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1595023716; bh=r9YIi+6yt6IEdQyKHpfyVZe7j5tBvq45ETGau34r5no=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=bsL8uXBUP8vF5sRU3WzRQ6Pf4YDDu93co6s37qg7OB+i+sOH/2cO6GYbls/6A/Uns /gIYXVNN1gdAmT4tKI8M/Uqw0yJNn0IfnxpIE6tKkWKwgugMi8iVT+pkuxSIb1KE9A SkKO063HnmjfFw5Lani3q0DWg3q/cyaNJeuDIUjrJGdr6SPsAwNunLWcwHmj1FGJ7Q a0H1hX57nMAuwIhl56KOECL0QESMArGwxSNeW+I0UlJCSoaUqh38OScm4AvOKnzTfJ aISWDDvWczNfAyRQH5VZO8Eam3iaR2Zk5gi5LA8plM0khM8X1coff5JKhz6ZPq8ZvO MLzekpbX7GzoA== Original-Received: from milanesa (76-10-180-175.dsl.teksavvy.com [76.10.180.175]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 616E3120439; Fri, 17 Jul 2020 18:08:35 -0400 (EDT) In-Reply-To: <87wo31sxmu.fsf@gmail.com> (Zach Shaftel's message of "Fri, 17 Jul 2020 16:19:05 -0400") Received-SPF: pass client-ip=132.204.25.50; envelope-from=monnier@iro.umontreal.ca; helo=mailscanner.iro.umontreal.ca X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/17 18:08:42 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:253053 Archived-At: >> While waiting for the paperwork to go through, you can prepare the patch >> and we can start discussing it. > Sure, does that just mean the 'git format-patch -1' emailed to > bug-gnu-emacs@gnu.org, as mentioned in CONTRIBUTE? If that's the gist of > it then I can do that shortly. Pretty much, yes. You can add some text to give extra background on the design, the motivation for some of the choices, or ask questions about particular details, but that's not indispensable. You can also send an email that just refers to a branch in emacs.git. But for the discussion to work well, it's usually better to make sure this branch is "small" so people aren't discouraged to read the large diff ;-) > I was able to speed that function up to the point that it's about the > same as one using `read`. Those functions are doing a whole lot of IO > (reading and writing hundreds of files) so it's not really a fair > comparison. I've done more tests with functions that just read a whole > buffer, collecting what they read into a list. In a 9600 line file with > just over 500 sexps, the `read` version took about ~.02-.04 seconds > (according to `benchmark-run-compiled`), and the `source-map-read` > version took ~.08 seconds when it didn't GC, but unlike with `read` it > did cause a GC 10-20% of the time. IME when the time is in the sub-second range the measurements are very imprecise, so better measure the time to repeat the same `read` N times so the total time is a few seconds (and since it's the same `read`, it won't suffer from extra IO overhead). >> For macros, OTOH, it's really fundamentally hard (or impossible, in >> general). > Helmut Eller mentioned before that most macros do use at least some of > the original code in their expansion. We can definitely hope to use some heuristics that will preserve "most" source info for "most" existing macros, yes. But it's still a fundamentally impossible problem in general ;-) >> We could/should introduce some new way to define macros which >> knows about "source code annotated with locations". > I've wondered about this too but don't know what the right approach > would be. The first step is to define a `defmacro2` which works like `defmacro` but is defined to take as arguments (and to return) annotated-sexps instead of "bare sexps". It'll be less convenient to use, but In Scheme "annotated sexps" are called "syntax objects". > I doubt anyone would want to use something like macro-cons/list/append > etc. functions, Scheme avoids the problem by defining additional higher-level layers, where macros are defined in a more restrictive way using templates, so for most macros the programmer doesn't need to use care very much about the difference between bare sexps and syntax objects. The main motivation for it was hygiene (the framework takes care of adding the needed `gensym`s where applicable) rather than tracking source-location, but fundamentally the issue is the same: an AST node is not just some random sexp. IOW "code and data aren't quite the same, after all" ;-) See for example `syntax-case` https://www.gnu.org/software/guile/manual/html_node/Syntax-Case.html Note that Scheme uses the #' notation for syntax objects. Adapting the example for `when` to an Elisp syntax could look like: (defmacro2 when (form) (elisp-case form ((_ test e e* ...) (elisp (if test (progn e e* ...)))))) [ Where I used `elisp` instead of Scheme's `syntax` since we already use the prefix "syntax-" for things related to syntax-tables. ] Notice how it's `elisp-case` which extracts `test`, `e`, and `e*` and then it's `syntax` which builds the new chunk of code, so all the replacement of `car` with `elisp-car` can be hidden within the definition of `elisp-case` and `elisp`. >> There's a lot of work on Scheme macros we could leverage for that. > Interesting, so far I've had some difficulty finding documentation about > how other Lisps track source locations. It's not really discussed, but the distinction between "sexp" and "syntax object" is the key. It's largely not discussed because Scheme macros have never officially included the equivalent of `defmacro` operating on raw sexps, so they've never really had to deal with the issue (tho Gambit does provide a `define-macro` which operates like our `defmacro` but it's rarely used so Gambit just punts on the source-location issue in that case). Stefan