From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Sam Halliday Newsgroups: gmane.emacs.bugs Subject: bug#36432: 26.2; SMIE does not request forward tokens when point is at point-max Date: Sat, 29 Jun 2019 13:51:31 +0100 Message-ID: References: <83y31kfumo.fsf@gnu.org> <83woh4ftqk.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="195155"; mail-complaints-to="usenet@blaine.gmane.org" Cc: 36432@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Jun 29 14:52:23 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hhCq7-000oex-5n for geb-bug-gnu-emacs@m.gmane.org; Sat, 29 Jun 2019 14:52:23 +0200 Original-Received: from localhost ([::1]:39456 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hhCq5-00069t-DD for geb-bug-gnu-emacs@m.gmane.org; Sat, 29 Jun 2019 08:52:21 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:40049) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hhCpq-00068f-6Q for bug-gnu-emacs@gnu.org; Sat, 29 Jun 2019 08:52:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hhCpo-0005OO-Gw for bug-gnu-emacs@gnu.org; Sat, 29 Jun 2019 08:52:05 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:58272) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hhCpm-0005Mt-IW for bug-gnu-emacs@gnu.org; Sat, 29 Jun 2019 08:52:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hhCpm-0006m9-FN for bug-gnu-emacs@gnu.org; Sat, 29 Jun 2019 08:52:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Sam Halliday Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 29 Jun 2019 12:52:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36432 X-GNU-PR-Package: emacs Original-Received: via spool by 36432-submit@debbugs.gnu.org id=B36432.156181269926010 (code B ref 36432); Sat, 29 Jun 2019 12:52:02 +0000 Original-Received: (at 36432) by debbugs.gnu.org; 29 Jun 2019 12:51:39 +0000 Original-Received: from localhost ([127.0.0.1]:43582 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhCpP-0006lR-8w for submit@debbugs.gnu.org; Sat, 29 Jun 2019 08:51:39 -0400 Original-Received: from mail-vs1-f54.google.com ([209.85.217.54]:41608) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hhCpN-0006lF-8m for 36432@debbugs.gnu.org; Sat, 29 Jun 2019 08:51:37 -0400 Original-Received: by mail-vs1-f54.google.com with SMTP id 2so5899176vso.8 for <36432@debbugs.gnu.org>; Sat, 29 Jun 2019 05:51:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=JEp6MgH5fxyQHQD/lxx5Fl2UxG7wBl/mo1lTwDnIkYE=; b=WXvnRpOo6sQl652ydavog+wFUqj2apmzRhqeD0b0im7oOsqZV8iupMikch1KCCUFY5 CAJ4ATDcNkcqx52RG0bzIVZJXYu+ZPvdswq3uh3tKN3fyifhxLDflHBeXUHgr2x6QGTR dFOo1cQO8E1aEa3bYj5jYMdo808CB3wCRVK66r7CPTtkdem8oSyMbGQxk6lgF8jy42NK YXarNcz6qU2N6dNARhiiU5GB1Ua9bj1Mgxj0BXQqo/dIJgFVqKyAv8haXD34po0HwJqC Ka4EBnTwXsJJCiQWxN9iTXGllWaQc9VdqqfDRP9RyPzZ36wcMT1cd+0X6GRhbCWpPIpn EbSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=JEp6MgH5fxyQHQD/lxx5Fl2UxG7wBl/mo1lTwDnIkYE=; b=sAVxZ6At0hyhkvXqn5AAXm6JJu+fKvZIxIe7ZWZQ5CH7Ly/+PyK39MtYINsnxPp5Nn e3qPBmftJEM7Y7sgTEll0Ofco/y8TOkZnJPx+hB5IxXhJEMqM28ZO+E3kNv+F9F3NwMe grTG2j9Ci90qiN8+cGEQgvZyaKDEP734yuz4rjw5H5lwyCAyYeyXEUuTXN4hXRWG+rlc 7OInZuLosEYlPxQ2MCOntNyzD+wigecLSkSTMIUuYDicYCU2B1rXeGOu3f7P/r0+kSBH /1u0mzS9CYrEu47JO1xLfbc/re8Bl5fEhx7n3Ys8dREOUlK/fdi+DpcPUfsAl2z8CEpT 3rfg== X-Gm-Message-State: APjAAAUT+7Kwao7T80uxkb+Bi5flBLSrCXpT0xAYH4/tWLVgNUXbaTmO fx25b3CI7CDxkuVTawj8EFDhnrPh+h7OTwxojGGbdg== X-Google-Smtp-Source: APXvYqyBbzyxy/i56h7QO6Ilnq7faQ0lHAemD3PQ63dZHpLWGFTS9DclBoEz9YcBmYryS7tGn4QHQS4RpVpe5ZBq/kA= X-Received: by 2002:a05:6102:3c5:: with SMTP id n5mr9099505vsq.56.1561812691677; Sat, 29 Jun 2019 05:51:31 -0700 (PDT) Original-Received: by 2002:a9f:2427:0:0:0:0:0 with HTTP; Sat, 29 Jun 2019 05:51:31 -0700 (PDT) In-Reply-To: <83woh4ftqk.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:161822 Archived-At: On 29/06/2019, Eli Zaretskii wrote: >> From: Sam Halliday >> Date: Sat, 29 Jun 2019 13:34:06 +0100 >> Cc: 36432@debbugs.gnu.org >> >> > ??? There can be noting at point-max, as that position is beyond the >> > last buffer position. Did you mean the position just before that? Or >> > am I missing something here? >> > >> >> I mean at point-max. >> >> Consider this layout algorithm >> >> https://gitlab.com/tseenshe/haskell-tng.el/blob/tng/haskell-tng-layout.el >> >> and this lexer >> >> https://gitlab.com/tseenshe/haskell-tng.el/blob/tng/haskell-tng-lexer.el >> >> that can continue to produce tokens even when the point is at the very >> end of the buffer. > > So you create an illusion of characters beyond the EOB? > > How would Emacs know this is the case? When testing it is possible to keep polling the lexer until it returns nil when at point-max, rather than looking at `point-max` and giving up. I think that could work in general inside SMIE. https://gitlab.com/tseenshe/haskell-tng.el/blob/tng/test/haskell-tng-lexer-test.el I suspect the example forward lexer, from the documentation https://www.gnu.org/software/emacs/manual/html_mono/elisp.html#SMIE-Lexer would be ok in this situation. I'd be concerned that existing lexers would throw an error if they were polled when at the beginning/end of the buffer unexpectedly. BTW, this also happens at the start of the buffer. SMIE doesn't ask for backwards tokens when at the beginning. > Why don't you also override > point-max to make it consistent with those illusory characters? Hmm, that is a workaround worth exploring. I'm not sure what the consequences would be of changing something so fundamental. I think changing SMIE would probably be easier, even with a monkey patch of the relevant function or advice. I can have a go at trying to do that. I just need to figure out exactly which function is doing the check.