From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Zach Shaftel Newsgroups: gmane.emacs.devel Subject: Re: Update 1 on Bytecode Offset tracking Date: Mon, 20 Jul 2020 20:28:40 -0400 Message-ID: <878sfdhft3.fsf@gmail.com> References: <87a700fk3j.fsf@gmail.com> <87blkfoz9v.fsf@gmail.com> <87wo31sxmu.fsf@gmail.com> <87zh7wjybj.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="10638"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.4.10; emacs 28.0.50 Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Jul 21 02:29:57 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jxgAP-0002gZ-0r for ged-emacs-devel@m.gmane-mx.org; Tue, 21 Jul 2020 02:29:57 +0200 Original-Received: from localhost ([::1]:52248 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jxgAO-000554-3r for ged-emacs-devel@m.gmane-mx.org; Mon, 20 Jul 2020 20:29:56 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:36752) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jxg9H-0004GQ-GI for emacs-devel@gnu.org; Mon, 20 Jul 2020 20:28:47 -0400 Original-Received: from mail-qk1-x744.google.com ([2607:f8b0:4864:20::744]:36898) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jxg9F-0003bX-MC for emacs-devel@gnu.org; Mon, 20 Jul 2020 20:28:47 -0400 Original-Received: by mail-qk1-x744.google.com with SMTP id k18so17707637qke.4 for ; Mon, 20 Jul 2020 17:28:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=references:user-agent:from:to:cc:subject:in-reply-to:date :message-id:mime-version; bh=yfhYMsIO8FIfMqK0gGeAV+hKL0U+g7nvXzreaGK5RO4=; b=lquc/syPnnVRqTWs6nTFGMBPL/YD5NVNEhUPFqDCsVaZHjjPg3ZoonUUkwXZ+seu71 Bovc3LDkEhiy8wY4ekiFNSqUxXp6RBBauKKwCI5uzISgZfShlFMAqh3lnOxIjrIDL8qm IX8fIiwP7U8fS9oVkqiiOq2pEmEe7DQwGFU9CqBJWxMnRKHyNUVksD8iEKRvp8nuGHjj eHWEqhF/vezuXQIN0eJLlOyp1jZgP4jxQwXm+EM6kJz8Xp9MYdGxB0vdWaj8uxFMSN3M R3nUdMyPJnjyJ0/l0jy/at2DRpTz/4CKcEXCR15+iUpSzyimeC1pymvP8xTeC7Uw2f/H yMcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:cc:subject :in-reply-to:date:message-id:mime-version; bh=yfhYMsIO8FIfMqK0gGeAV+hKL0U+g7nvXzreaGK5RO4=; b=m3BJg/ATpM03h619lKCoygVb8NG8zFIohOX/Niw5x3a7tsyoTm0be5VxKEp/YfgTL0 3C6LFBeqgY2EFvBWI8BCVLlwNUGVq1QrEZfP2bHeBYnMhCutZXpPWeEFgupH+ZZzUiXv fzswo4DLC70D5/sEiWZO06B+11+sqwC1kAB9iSa0Qf9OVTCpbNHxlHtm4mqILQBoUkfK M81YN/1tOSyKIsR+eQt6gyZs9tiT1r+3yQvDscCrqzXY51B6sJSDrJmSDsjaqIm4r40m wCMjAM1cRaAjaoX9lIxXGYtz9TgJv+K17z3Mmg7WMeX7GI7WahMQG65YTp57CdyaVqDz T98g== X-Gm-Message-State: AOAM533/zhpfgxP43NHt7FvV2iSBsRIIGE4eZgCnwK1wHGgKXEwRHHgx xttupOyjcLvn57ZsdHzXAHOiWTGXOEU= X-Google-Smtp-Source: ABdhPJxK2MFnShoOfIO6yulRq4/Xx04UFpKnsITDdgmayDH6n+FqBOnw9hYpx6P2nBI+Oa47Hr1D5A== X-Received: by 2002:a37:6285:: with SMTP id w127mr8599974qkb.204.1595291323904; Mon, 20 Jul 2020 17:28:43 -0700 (PDT) Original-Received: from arch-thinkpad ([217.138.200.230]) by smtp.gmail.com with ESMTPSA id h3sm1073129qkd.53.2020.07.20.17.28.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2020 17:28:43 -0700 (PDT) In-reply-to: Received-SPF: pass client-ip=2607:f8b0:4864:20::744; envelope-from=zshaftel@gmail.com; helo=mail-qk1-x744.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:253127 Archived-At: Stefan Monnier writes: >> Sure, I'll do some more exhaustive testing. So far though, the results >> aren't great, the biggest issue being memory usage. The >> `source-map-read` can GC over 5 times more often than `read`. > > Sounds fine for a prototype. > >> I think it's more sensible to accept that a real implementation will >> have to be in C and this reader will just remain a prototype. > > Indeed. > >> Aha, I had never even considered hygienic macros in Elisp (nor had I >> recognized how trivial it is to track their source-code). That would be >> an amazing development for Emacs Lisp, but is certainly a huge >> undertaking, not something I could fit into the GSoC timeline. > > No, I'm just discussing what the longer-run might look like. > >> I know that it has been done in Common Lisp (by Pascal Costanza), but >> I believe that implementation serves the sole purpose of capture >> avoidance and doesn't abstract syntax. For Emacs I assume this would >> have to be done in C, but I do wonder if an Elisp implementation would >> be possible. > > I haven't thought very much about it, but I can't see any reason why it > would need to be done in C, no (tho I wouldn't be surprised if it could > benefit from a bit of help from the C side, of course). This is my gut feeling as well, but in the few discussions I've seen about implementing hygiene in an unhygienic macro system people suggest that it's nigh impossible without rewriting core parts of the language. I haven't seen a convincing defense of that argument in any of those discussions though. >> Doing the similar thing in Elisp -- relegating source location tracking >> to code using only a specialized kind of macro, hygienic or otherwise -- >> would of course be a major loss, since it would take years for that new >> paradigm to become commonplace. > > Indeed, we'll need some fallback heuristic for all the existing > `defmacro`s. > > Part of the issue is "tracking source location" but another important > part is to take the annotated source code and "de-annotate" it > (recursively) to pass it to the macro, since the macro expects > a raw sexp. > > That's why we've been thinking about annotated representations > which are "transparent" (i.e. can be used as if they weren't annotated). > Either using "fat cons-cells" or using "fat symbols" or storing the > annotations in an eq-hash-table. A hash-table seems like the most straightforward approach, which I'm working on now. Doing something like in the scratch/accurate-warning-pos branch adds a whole lot of complexity to both C and Lisp, and toggling byte-compilation versions of subrs feels clunky to me (though that's obviously just a prototype). A hash-table of conses will hopefully be enough, with or without the `source-map` stuff. > Another way to attack the problem is to rely on the Edebug spec: you can > refrain from de-annotating all the parts marked as `form` or `body` (as > long as the annotations themselves look sufficiently like normal code, > at least). Interesting, that's not something I had thought about. I suspect flawed edebug specs are common enough that this can't be relied upon. -Zach