From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Matt DeBoard Newsgroups: gmane.emacs.devel Subject: Re: SMIE Date: Wed, 9 Jul 2014 23:53:04 -0400 Message-ID: References: NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1404976880 889 80.91.229.3 (10 Jul 2014 07:21:20 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 10 Jul 2014 07:21:20 +0000 (UTC) Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Jul 10 09:21:15 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1X58fK-00087O-Ul for ged-emacs-devel@m.gmane.org; Thu, 10 Jul 2014 09:21:15 +0200 Original-Received: from localhost ([::1]:36188 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X581y-0005r7-4V for ged-emacs-devel@m.gmane.org; Thu, 10 Jul 2014 02:40:34 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:38702) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X55QZ-0003eh-Gw for emacs-devel@gnu.org; Wed, 09 Jul 2014 23:53:48 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1X55QX-00041g-W4 for emacs-devel@gnu.org; Wed, 09 Jul 2014 23:53:47 -0400 Original-Received: from mail-ig0-x235.google.com ([2607:f8b0:4001:c05::235]:34141) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X55QX-00041F-OG for emacs-devel@gnu.org; Wed, 09 Jul 2014 23:53:45 -0400 Original-Received: by mail-ig0-f181.google.com with SMTP id h15so2642524igd.8 for ; Wed, 09 Jul 2014 20:53:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=bSPG0UxbxjfihNcUaKrOZRbhvqjdcR+2/q0uWWozDFM=; b=0F+C4iG/GLCOLmfLWkqyeTIfJj8Kdm4XdL8xquN1m8G23l0OwGktXkyNfjPh0Ihj0N /NEM0U3TD0ZFqLM0VHnAHmmyevezNZZWDUXCTgZDs8xlqpAC+wiNrU3Q1LnVibcMH8Nr b3Hi9H75/8RT6sbABVF7Pkcs7KQzIVmtfyM5OtzPLGK9M/ELMeakpQBjl9OCAmNKJi7S gSVObNga+AFz2+0S1aUQXSPKGGMYP+9uw6H+hRzCmJp3mDS38zVqBVwu5Q2D2J5vjEjx oMIqOMTBSThNkwgVS5Bukgpq8E9PV3eGFRQQCs9++McGuvDM5ppspKgiOcd2d/uqTxyN eJ0g== X-Received: by 10.50.66.179 with SMTP id g19mr18260312igt.29.1404964424669; Wed, 09 Jul 2014 20:53:44 -0700 (PDT) Original-Received: by 10.50.96.163 with HTTP; Wed, 9 Jul 2014 20:53:04 -0700 (PDT) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2607:f8b0:4001:c05::235 X-Mailman-Approved-At: Thu, 10 Jul 2014 02:40:26 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:172926 Archived-At: In general I=E2=80=99m having a hard time connecting the dots between the B= NF grammar table creation, the smie-rules (i.e. :before, :after, etc.), tokenization, indentation, and so forth, and how it all comes together to make this indentation machine work. It=E2=80=99s not that the manual is poorly written. In fact between the man= ual and the code comments it=E2=80=99s pretty comprehensive. It=E2=80=99s just = a high volume of new information. I think it=E2=80=99ll take me awhile to ingest everything, but when I do I=E2=80=99d be glad to contribute back. Regarding the specific issue I mentioned, I closed that out tonight. There was some overly =E2=80=9Cgreedy=E2=80=9D definitions in the syntax ta= ble, so there=E2=80=99s that. I=E2=80=99m slowly starting to pare things away. The = bit you wrote about :list-intro is interesting. When you say that it sees two or more concatenated expressions, how does that tie in to the BNF grammar definitions? On Wed, Jul 9, 2014 at 11:02 PM, Stefan Monnier wrote: >> Hi there. As the subject line says I=E2=80=99m writing for help with SMI= E. > > Cool! > >> I am currently working on elixir-mode >> , having (apparently) takin= g >> over the mode as the latest in a line of contributors. > > I'd love to include this in GNU ELPA. Interested? > >> Specifically I=E2=80=99m having trouble understanding the mental model f= or how >> tokenisation & indentation works. For example, in this >> issue, indentati= on >> errors seem to crop up only after separating lines of code with blank >> lines. >> I have spent, seriously, hundreds of hours trying to sort out what=E2=80= =99s >> happening here and I am at my wits=E2=80=99 end. > > IIUC, Elixir syntax does not treat all whitespace as "irrelevant", > contrary to the default tokenizer of SMIE. > >> Does this issue ring any bells with issues you=E2=80=99ve dealt with in >> the past? > > Yes, indeed. Octave and sh are two other languages that use SMIE and > where some whitespace is syntactically significant. > > What you need to do is to change the tokenizer so that instead of > skipping all whitespace, it turns the syntactically-significant > whitespace into a token (you can name it any way you like; in the above > languages, it turns out to be syntactically equivalent to a semi-colon, > so we call it ";"). > > I know absolutely nothing about Elixir or its syntax, so I can't give > you specific details, but you can look at octave.el and sh-script.el > for examples. Feel free to email me back with more details if you need > further help. > >> Final question, how is it determined if a token is a :list-intro token? > > Not sure I understand the question. The issue is for the indentation > rules, when it sees two (or more) concatenated expressions (e.g. "exp1 > exp2"), should it assume that exp2 is something like an argument to the > exp1 function (and hence exp2 (and exp3, ...) should be indented like > a function argument) or are all those "expressions" just a list, where > the first is not more special than the second? > This usually depends on the context. E.g. in a situation like > > fun x1 x2 x3 =3D> > > x2 is not an argument passed to the function x1; Instead x1, x2, and x3 > are "siblings" and should be indented to the same level. So to decide > how to indent x2 and x3 w.r.t x1, SMIE calls the smie-rule-function with > (:list-intro . "fun") so smie-rule-function can tell it that "fun" > introduces a *list* of "things" rather than being followed by a "normal > expression". > > Does that make more sense? > >> I have read the SMIE manual ten times, at least, but I=E2=80=99m really >> struggling. I would truly appreciate your help. > > I'm not very good at writing manuals, sorry. But I promise to do my > best to help you get SMIE working well. In return, I would appreciate > if you could help me improve the doc by giving, if not actual patches, > at least suggestions of how to rewrite the doc, or what to add to it > (usually, you can only make such suggestions after you finally > understand what's going on, and at the same time it's > important/necessary/useful to try and remember what it was that you > didn't understand). > > > Stefan