From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Tim Cross Newsgroups: gmane.emacs.devel Subject: Re: Christmas wish: Literate Elisp Date: Sat, 21 Dec 2019 12:16:07 +1100 Message-ID: References: <87h81vhrek.fsf@russet.org.uk> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000009fb90b059a2c8aaa" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="11473"; mail-complaints-to="usenet@blaine.gmane.org" Cc: arthur miller , emacs-devel To: Phillip Lord Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Dec 21 02:17:09 2019 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1iiTOG-0002pu-4C for ged-emacs-devel@m.gmane.org; Sat, 21 Dec 2019 02:17:08 +0100 Original-Received: from localhost ([::1]:35824 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iiTOE-0000uQ-Bc for ged-emacs-devel@m.gmane.org; Fri, 20 Dec 2019 20:17:06 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:53977) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iiTNX-0000Se-Ej for emacs-devel@gnu.org; Fri, 20 Dec 2019 20:16:26 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iiTNU-0002vb-So for emacs-devel@gnu.org; Fri, 20 Dec 2019 20:16:23 -0500 Original-Received: from mail-ot1-x332.google.com ([2607:f8b0:4864:20::332]:40047) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iiTNU-0002qT-J1 for emacs-devel@gnu.org; Fri, 20 Dec 2019 20:16:20 -0500 Original-Received: by mail-ot1-x332.google.com with SMTP id w21so6353696otj.7 for ; Fri, 20 Dec 2019 17:16:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=k6zd7hkVimeMjYS4ppM+69L+8dO3OuYZUOqE1Lx9CM8=; b=JJyoaYSBK5CLAy14hpRgulrp6a+CwXmBjMO78r+rUD7TobJcBXhAJRxbhPmUryLOXE Y12dq9HgDTHWfz1elVIowxsQnmAA4Qf31vwv8TwceFaWpZpWyTzn6qIhj2OjokCIgC8K yDSd4ljF19yLOqG8Q122pKSkqfXQ8tDsPdy7pmfLpNt7ezOpkcrnVc8GF+9PgVJKkad5 Yv9Qf3x0HdCITUZOMU93S72ZcK0l/xozDgirFPAuuINQo/H7HsAqci32nCx146y07TjJ l4T49BOTjCI7J3DD6Fg94zpaENO82p+Qn0d6CYnGaC04o3pRqPvAZWwuRCJbtlTNL/On Lc3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=k6zd7hkVimeMjYS4ppM+69L+8dO3OuYZUOqE1Lx9CM8=; b=ja/OCIROgsrF1ydzQShFSOxh9shxNbssnwK+KbQL9ipxs/irQms3VPydCTlcKucHYd kAaFd8AvyheHkiREg3HWSk1Ia/Ua4qxORU7j45mKyc+MTV/kwNHuivBEbDs5eoP4VEWn GJc/HzVybRwnJyMRkisFpoRLUYxvHOlvVgL8pWgjwUMXWuBvp09ADgdv9zzeDvYZtfMf V3V2BXf8AtPcqaLjCtPIL0jUWR8l52TlP4M+nPfrVV4HssLVhcXlPhOHHCsBxnQdao+0 PV5zyi7n8DNET5qE59a0bHTe8gohnQdlcIinXIiXIxGvl285/+2exFT8MdrrmzlacTGf 2Gwg== X-Gm-Message-State: APjAAAVp1UlBreupIthNARxJbcr5ECZHAtcKkWPoibvSen3GHZgyFmc/ g5F8z50HTLDG+0ky6XU3HbvznoH070dWYAtCnahy5g== X-Google-Smtp-Source: APXvYqxFk3P/68BihJVUSCiiMzWshsaPTayYxYheUDtHE4JTBy5jfepf49zJdFRGg3Or+mVfgLPUqpUOGzA18CubmG4= X-Received: by 2002:a05:6830:1bda:: with SMTP id v26mr17712243ota.314.1576890978513; Fri, 20 Dec 2019 17:16:18 -0800 (PST) In-Reply-To: <87h81vhrek.fsf@russet.org.uk> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::332 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:243528 Archived-At: --0000000000009fb90b059a2c8aaa Content-Type: text/plain; charset="UTF-8" I've been watching this thread and resisting adding to it, but feel now might be when I can contribute something useful. This whole discussion seems to centre around some views that literate programming for lisp is a positive way forward and therefore, making changes to both C and Elisp parsing is a good idea if it means we can move to a 'modern' literate programming model with greater ease. As someone who has done literate programming for non-trivial projects (i.e. something other than my .emacs file), I think this assumption needs to be challenged. Literate programming has been around for a long time, but has failed to gain traction. Why? Is it due to the existing tools or is it something more fundamental? I have been a fan of Donald Knuth's work for a long time. When I first came across literate programming in the mid-90s, I thought it sounded really interesting and a potentially great approach to writing code. A few of us adopted this approach in our project and embarked on this new path of discovery. Unfortunately, the reality just did not hold up to the promise. Some of this was due to limitations in tooling (relating to the 'weave' and 'tangle' tools of the time), but a big part really just failed because people don't move between prose and code that easily and we don't maintain the prose as well as the code. In the end, you just have lots of out-of-date prose mixed in with your code, which is often more misleading than no prose at all. The tools at the time were particularly poor when it came to working within a REPL type environment. This was partly because they were a two step process (tangling) and you could only go in one direction (getting chagnes made in the REPL back into your source was distracting and often forgotten, so work was frequently lost). Dealing with scope, namespaces etc was a pain in a system which relied on 'tangling' of source files in order to get the final output and nearly all supporting utilities (code formatters, highlighting, linters etc) were broken. Like others who have posted in this thread, we thought it could all be improved if you could eliminate the weaving/tangling and be free to just write prose within your code, so we tried to do this with the lisp we were using (sorry, can't remember what lisp flavor it was). At first glance, it looked very easy to do - a few tweaks to the parser, some new reader macros and we should be fine. On one level, it did fix some of the issues, but we did find lots of unforeseen edge cases and it added sufficient additional complexity to the whole system that in the end, we decided it just didn't deliver the benefits we had hoped for. At the end of the day, it wasn't the tools or the implementation which killed it - it was simply that literate programming simply didn't fit with how people like to code. It was one of those things which sounds nice in theory, but for most, fails in practice. We did find the approach works better for some languages than others. For example, languages like C, where you have a code, compile, run, test cycle, it wasn't oo bad. However, for languages where you often included a more REPL driven style of devleopment, where lots of devleopment is done as experimentation/refinement at the repl, the paradigm added very little and more often than not, just got in the way. We did find it was useful for smaller tasks and configuration management, but for larger projects, especially with teams of developers, it didn't stand up. So, my advice, for what it is worth is Implement a solution which does NOT require any modifications to either the elisp parser or C code and use it to develop some non-trivial elisp packages. You could probably base it on org-mode. My suspicion is that you will find that after some time, it really doesn't deliver what you hoped. However, if I'm wrong, it will provide the necessary experience and familiarity to help guide a more refined and targeted model for working with literate programming and elisp. Don't under estimate the effort or impact making changes to either the elisp parser or C sources will have. There has been a number of comments in this thread which makes it sound like you would just need to 'tweak' the parser and everything will work. That 'tweaking' is far more complicated than it might seem on the surface. One of the main benefits of lisp dialects is there simple syntax and the idea of code as data. Any changes made to either the syntax or parsing of elisp is going to impact on lots of things, many of which none of us have even thought about yet. My feeling is that if literate programming is a great way forward, this will be evident in the number of people who start using an elisp literate programming mode. If this does turn out to be the case, then we can use the experiences from many people who are using that mode to see what can be done to develop the paradigm further. This may involve low level C or elsip parser or syntax changes, but they will be based on experience and demand. On Sat, 21 Dec 2019 at 03:56, Phillip Lord wrote: > > > Stefan Monnier writes: > > >> My proposal is to slightly change Elisp parser to treat lines that start > >> with any other printable character but '(' as a start of comment and to > >> simply ignore the line, just as it treats ';' as a comment. > > > > The `sexpresso` Haskell package follows the same idea ;-) > > > > As for using it in Elisp: I don't think there's anything stopping anyone > > from making such a `literate-elisp-mode` and even arrange for `load` to > > handle such a file (just like there is already a package that lets > > `load` work directly on .org files). > > > > I'd welcome such a package in GNU ELPA. > > > I'm very late to this thread, for which apologies. > > My own package, lentic, could achieve this. It's gives you two > visualisations of the same file; so one with comment characters, and one > without. You can then do the literate part in one mode and the coding > part in another. > > It already supports org-mode delination markers; it could be made to > support brackets as a delineator too. > > Phil > > -- regards, Tim -- Tim Cross --0000000000009fb90b059a2c8aaa Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I've been watching this thread and resisting addi= ng to it, but feel now might be when I can contribute something useful.

This whole discussion seems to centre around som= e views that literate programming for lisp is a positive way forward and th= erefore, making changes to both C and Elisp parsing is a good idea if it me= ans we can move to a 'modern' literate programming model with great= er ease.

As someone who has done literate pro= gramming for non-trivial projects (i.e. something other than my .emacs file= ), I think this assumption needs to be challenged. Literate programming has= been around for a long time, but has failed to gain traction. Why? Is it d= ue to the existing tools or is it something more fundamental?

I have been a fan of Donald Knuth's work for a long ti= me. When I first came across literate programming in the mid-90s, I thought= it sounded really interesting and a potentially great approach to writing = code. A few of us adopted this approach in our project and embarked on this= new path of discovery. Unfortunately, the reality just did not hold up to = the promise. Some of this was due to limitations in tooling (relating to th= e 'weave' and 'tangle' tools of the time), but a big part r= eally just failed because people don't move between prose and code that= easily and we don't maintain the prose as well as the code. In the end= , you just have lots of out-of-date prose mixed in with your code, which is= often more misleading than no prose at all.

= The tools at the time were particularly poor when it came to working within= a REPL type environment. This was partly because they were a two step proc= ess (tangling) and you could only go in one direction (getting chagnes made= in the REPL back into your source was distracting and often forgotten, so = work was frequently lost). Dealing with scope, namespaces etc was a pain in= a system which relied on 'tangling' of source files in order to ge= t the final output and nearly all supporting utilities (code formatters, hi= ghlighting, linters etc) were broken.=C2=A0 Like others who have posted in = this thread, we thought it could all be improved if you could eliminate the= weaving/tangling and be free to just write prose within your code, so we t= ried to do this with the lisp we were using (sorry, can't remember what= lisp flavor it was). At first glance, it looked very easy to do - a few tw= eaks to the parser, some new reader macros and we should be fine. On one le= vel, it did fix some of the issues, but we did find lots of unforeseen edge= cases and it added sufficient additional complexity to the whole system th= at in the end, we decided it just didn't deliver the benefits we had ho= ped for.=C2=A0 At the end of the day, it wasn't the tools or the implem= entation which killed it - it was simply that literate programming simply d= idn't fit with how people like to code. It was one of those things whic= h sounds nice in theory, but for most, fails in practice.
We did find the approach works better for some languages than = others. For example, languages like C, where you have a code, compile, run,= test cycle, it wasn't oo bad. However, for languages where you often i= ncluded a more REPL driven style of devleopment, where lots of devleopment = is done as experimentation/refinement at the repl, the paradigm added very = little and more often than not, just got in the way.=C2=A0 We did find it w= as useful for smaller tasks and configuration management, but for larger pr= ojects, especially with teams of developers, it didn't stand up.

So, my advice, for what it is worth is
Implement a solution which does NOT require any modifications = to either the elisp parser or C code and use it to develop some non-trivial= elisp packages. You could probably base it on org-mode. My suspicion is th= at you will find that after some time, it really doesn't deliver what y= ou hoped. However, if I'm wrong, it will provide the necessary experien= ce and familiarity to help guide a more refined and targeted model for work= ing with literate programming and elisp.

Don&= #39;t under estimate the effort or impact making changes to either the elis= p parser or C sources will have. There has been a number of comments in thi= s thread which makes it sound like you would just need to 'tweak' t= he parser and everything will work. That 'tweaking' is far more com= plicated than it might seem on the surface. One of the main benefits of lis= p dialects is there simple syntax and the idea of code as data. Any changes= made to either the syntax or parsing of elisp is going to impact on lots o= f things, many of which none of us have even thought about yet.
<= div>
My feeling is that if literate programming is a great wa= y forward, this will be evident in the number of people who start using an = elisp literate programming mode. If this does turn out to be the case, then= we can use the experiences from many people who are using that mode to see= what can be done to develop the paradigm further. This may involve low lev= el C or elsip parser or syntax changes, but they will be based on experienc= e and demand.

On Sat, 21 Dec 2019 at 03:56, Phillip Lord <phillip.lord@russet.org.uk>= ; wrote:


Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> My proposal is to slightly change Elisp parser to treat lines that= start
>> with any other printable character but '(' as a start of c= omment and to
>> simply ignore the line, just as it treats ';' as a comment= .
>
> The `sexpresso` Haskell package follows the same idea ;-)
>
> As for using it in Elisp: I don't think there's anything stopp= ing anyone
> from making such a `literate-elisp-mode` and even arrange for `load` t= o
> handle such a file (just like there is already a package that lets
> `load` work directly on .org files).
>
> I'd welcome such a package in GNU ELPA.


I'm very late to this thread, for which apologies.

My own package, lentic, could achieve this. It's gives you two
visualisations of the same file; so one with comment characters, and one without. You can then do the literate part in one mode and the coding
part in another.

It already supports org-mode delination markers; it could be made to
support brackets as a delineator too.

Phil



--
regards,

Tim=

--
Tim Cross

--0000000000009fb90b059a2c8aaa--