From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: =?UTF-8?Q?Bj=C3=B6rn_Lindqvist?= Newsgroups: gmane.emacs.help Subject: Re: SMIE examples or guides Date: Fri, 1 Jul 2016 21:28:01 +0200 Message-ID: References: NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1467401325 29001 80.91.229.3 (1 Jul 2016 19:28:45 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 1 Jul 2016 19:28:45 +0000 (UTC) Cc: help-gnu-emacs@gnu.org To: Stefan Monnier Original-X-From: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Fri Jul 01 21:28:39 2016 Return-path: Envelope-to: geh-help-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bJ47C-0002G3-1d for geh-help-gnu-emacs@m.gmane.org; Fri, 01 Jul 2016 21:28:38 +0200 Original-Received: from localhost ([::1]:35324 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bJ47B-0001me-8F for geh-help-gnu-emacs@m.gmane.org; Fri, 01 Jul 2016 15:28:37 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:59965) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bJ46e-0001mT-M0 for help-gnu-emacs@gnu.org; Fri, 01 Jul 2016 15:28:05 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bJ46d-0006x1-GU for help-gnu-emacs@gnu.org; Fri, 01 Jul 2016 15:28:04 -0400 Original-Received: from mail-wm0-x22f.google.com ([2a00:1450:400c:c09::22f]:36589) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bJ46d-0006wu-95 for help-gnu-emacs@gnu.org; Fri, 01 Jul 2016 15:28:03 -0400 Original-Received: by mail-wm0-x22f.google.com with SMTP id f126so37739403wma.1 for ; Fri, 01 Jul 2016 12:28:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=OVSBPS1uWDtk72Uor6Gud9i67Y3RsvpEciIRyao7wiA=; b=FUcolyf/+JrW7vHHvADsnch6gCDS4E1FlAB8j+1mIq4Awt1HmMAMRJfJWE1p2KGikK 0mESvzmBiU26H4VI5/1BjRw1/mIYk9HDD/YRJREzJovICvgnbGwvEMGqBP0yQKjvP6WE SnKo8pbS/UoUmyl9g7JjbKdHQ0HCjexhVVP37fXHO4osVmnJJbTwKOgy3sLdkFKAPLzP jFBKp7A3MQg8m8LWEtLibKO5Fr65oitEGMFlAcngl9MpyuGU24Q/ayJgt/uF6H3c+7/c 4ILFC/zfsok5lor0S+cegzA99sganZXBFYuZfdPamNcbIYqLtIIFgcVKHx9B/yBac+Gl EJ4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=OVSBPS1uWDtk72Uor6Gud9i67Y3RsvpEciIRyao7wiA=; b=Qt+3exB0Zd35jGLrY1nqj6UQVTP6388T+sXhbVLCTkvlTmFWvw8+UfJ5VsT3rC/nXO NIMpj+B5UhHsL6QLGDGm202ee8P+F670dJLt+f8UqljPASCicfunLQqCf2/tJXB/PMvy d3enapEdsxZQpgKQZ55rhM7LSybeQsmlSiB2cTEiFRJOxRp74EHJHDyT62/+AlUySsAk ay5a0gha4rqNHD4g1+sRmD3zuk4I8yrvxjHZxXnvBQJpTNxE99uGO+Rn2AnBOCb61xmH yui1lRXWqapKeHoy63piX53S06UscPP18sBKuQgnJvdxTdeXaowInXxrxZYUaCATqFOM zelQ== X-Gm-Message-State: ALyK8tIkV83TVp23O+uGD5hYGmN+fj3SPSpbnGPQ4TB/8FRmiEaJibn9olO36I2Mtle2NpiHPryyjtIkYo+Vkg== X-Received: by 10.28.212.211 with SMTP id l202mr1068591wmg.6.1467401282335; Fri, 01 Jul 2016 12:28:02 -0700 (PDT) Original-Received: by 10.194.45.202 with HTTP; Fri, 1 Jul 2016 12:28:01 -0700 (PDT) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::22f X-BeenThere: help-gnu-emacs@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Users list for the GNU Emacs text editor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "help-gnu-emacs" Xref: news.gmane.org gmane.emacs.help:110685 Archived-At: I managed to write the forward and backward tokenization functions: (defun factor-smie-token (dir) (pcase dir ('forward (forward-comment (point-max))) ('backward (forward-comment (- (point))))) (let ((tok (buffer-substring-no-properties (point) (let ((syntax "w_\\\"")) (pcase dir ('forward (skip-syntax-forward syntax)) ('backward (skip-syntax-backward syntax))) (point))))) ;; Normalizes different indent starters. (cond ((string-match factor-smie-indents-regex tok) ":") (t tok)))) (defun factor-smie-forward-token () (factor-smie-token 'forward)) (defun factor-smie-backward-token () (factor-smie-token 'backward)) It works in 99% of the cases. But since almost any character can be part of a token, it doesn't work perfectly. E.g te]st(ab3 would be a perfectly valid variable name. I tried changing the syntax to "w_()\\\"" and that fixes the tokenization but then I lose the useful automatic indentation smie adds to opening- and closing bracket characters. E.g I'm happy that smie indents: [ neat { nice } ( good ) ] It's very nice. But the following two lines are not right: hi[there two They should be: hi[there two 2016-07-01 9:13 GMT+02:00 Stefan Monnier : >> Many thanks! That appears to work. But there is one problem. In a >> buffer with just this content: > >> [ >> \ >> ] > >> I get the error: (error "Bumped into unknown token") > > You should either give a different syntax to the \ char in the > syntax-table, or change the :forward-token and :backward-token functions > so they do something with \ > >> (smie-bnf->prec2 >> '((exp (("HELLO" "HALLO" "CIAO" "SALUT") exp "BYE")))))) > > One way to do that is: > > (smie-bnf->prec2 > '((exp ("HELLO" exp "BYE") > ("HALLO" exp "BYE") > ("CIAO" exp "BYE") > ("SALUT" exp "BYE")))))) > > Another is to change the :forward-token and :backward-token functions so > they return the same string (e.g. "HELLO") when tokenizing any one of tho= se. > > > Stefan --=20 mvh/best regards Bj=C3=B6rn Lindqvist