Re: Christmas wish: Literate Elisp

[Tim Cross <theophilusx@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 12160 bytes --]

I certainly don't want to discourage you from experimenting and trying out
ideas - this is what makes Emacs great. What I would encourage is, in this
case, using org-mode to develop something non-trivial (possibly your
literate elsip idea) as it will give you the elisp experience you need and
more importantly, a taste of a form of literate programming, which will
both highlight the limitations of the current implementation (ie.e.
org-mode) and possibly, the literate programming approach itself. The key
point is that it needs to be non-trivial. Most of the limitations I found
with literate programming really only become evident in larger projects.
For smaller things, it works great and it is how I maintain my own init.el
file. It is only when you do something more complex that weaknesses become
evident.

I would also highly recommend re-visiting Stepan's posts after you have
gained some more elisp experience. Stephan is one of the most experienced
and knowledgeable contributors on this list. His advice and guidance are
valuable and should be considered carefully. While it may not be obvious
why he is making a recommendation, you can be fairly confident it is based
on experience and deep knowledge of Emacs. Spending the time to analyze his
suggestions and looking into them in detail is likely to save you a lot of
time going down dead ends or having to re-design/re-factor things as you
discover them the hard way.

I hope you don't become too discouraged and do work at implementing your
idea. At the very least, you may be able to find ways to improve or enhance
org-mode, which is always a valuable contribution.

On Sat, 21 Dec 2019 at 15:24, arthur miller <arthur.miller@live.com> wrote:

> Very interesting and nice read, thank you for taking your time Tim.
>
> I have no idea if literate programmign is very good or not.I certainly
> don't do it much. I never intended to present it as some kind of modern,
> silver bullet or something. I hope I didn't sound like that. I am aware
> that it is not the new idea either.
>
> Personally I really like ORG-mode and ipossibility to mix languages
> in one and same file. I was just looking at something I was working
> with and thought it had so much noise around stuff that matters. I
> wondered if it could be simplified. Then I got the idea that elisp could
> actually be written without special markers, since in elisp we put
> all code between () (with some exceptions). As I stated in my very first
> mail where I asked if we could have this in Emacs, I am not sure myself
> if this is somethign very usefull or just a cool gimmick. I personally can
> see some nice use cases, but it might not be of value to somebody else.
> I can maybe make a video where I play with it a bit after the hollidays it
> it is interesting.
>
> Yes, you are right about tools, checkers and so on. If we call it "literal
> elisp"
> then one thing should be clear: literal elisp and elisp are not same. All
> elisp is literal elisp, but all literal elsip would not be elisp.
> Relevance of
> this should be obvious to anyone, all current tools work with elisp, not
> literal elisp. So if you open a buffer and type comments without ';' in
> front, of course, identation, font locking etc will be confused. This
> should
> be self evident fact. Literal elisp would be a superset of elisp (with only
> one extra feature - different handling of comments). This is regardless
> of implementation language, just as elisp would be same language
> regardless if interpreter is written in C or Rust or Elisp for that
> matter.
>
> While yes, I have hacked C code, it does not mean I have altered how
> Elisp works. Elisp is still the very same, so even with the patch all Elisp
> should still work and parse the very same in Emacs and all external
> tools as it does since the beginning of the time. Look at the patch, apply
> it, run it. There is even boolean flag that eliminates that code at run
> time
> if desired. I don't know how your lisp and your patches worked, not all
> Lisps are created equal. It is interesting experience you share, and I do
> understand what you mean and I agree with you. I am very conservative
> myself when it comes to making changes, what is not broken should not
> be repaired. I am also very pragmatic. I don't do changes just for
> changes
> sake, I do it only if it does something useful.
>
> Also, don't misstake me, I don't insist on change be done in C. I have done
> those changes the way I did because I am new to Emacs internals hacking.
> I am not new to programming, but I have no idea how they do in Emacs code,
> what API calls are there and so on. I also have limited with time. So I
> have
> hacked where Emacs self help functionality and debugger has thrown me in.
>
> It turned to be some elisp and some C. I am sure it is possible to do this
> in
> different ways, and if somebody is willing to implement some other way,
> please go ahead. Of those two ideas that Stefan outlined, I am not willing
> to implement them. First one is super brutish and inneficient, the second
> one is sane but involves too much work. I don't have knowledge and time
> to put into it. But I am sure it can be done.
>
> ------------------------------
> *Från:* Tim Cross <theophilusx@gmail.com>
> *Skickat:* den 21 december 2019 02:16
> *Till:* Phillip Lord <phillip.lord@russet.org.uk>
> *Kopia:* arthur miller <arthur.miller@live.com>; emacs-devel <
> emacs-devel@gnu.org>
> *Ämne:* Re: Christmas wish: Literate Elisp
>
> I've been watching this thread and resisting adding to it, but feel now
> might be when I can contribute something useful.
>
> This whole discussion seems to centre around some views that literate
> programming for lisp is a positive way forward and therefore, making
> changes to both C and Elisp parsing is a good idea if it means we can move
> to a 'modern' literate programming model with greater ease.
>
> As someone who has done literate programming for non-trivial projects
> (i.e. something other than my .emacs file), I think this assumption needs
> to be challenged. Literate programming has been around for a long time, but
> has failed to gain traction. Why? Is it due to the existing tools or is it
> something more fundamental?
>
> I have been a fan of Donald Knuth's work for a long time. When I first
> came across literate programming in the mid-90s, I thought it sounded
> really interesting and a potentially great approach to writing code. A few
> of us adopted this approach in our project and embarked on this new path of
> discovery. Unfortunately, the reality just did not hold up to the promise.
> Some of this was due to limitations in tooling (relating to the 'weave' and
> 'tangle' tools of the time), but a big part really just failed because
> people don't move between prose and code that easily and we don't maintain
> the prose as well as the code. In the end, you just have lots of
> out-of-date prose mixed in with your code, which is often more misleading
> than no prose at all.
>
> The tools at the time were particularly poor when it came to working
> within a REPL type environment. This was partly because they were a two
> step process (tangling) and you could only go in one direction (getting
> chagnes made in the REPL back into your source was distracting and often
> forgotten, so work was frequently lost). Dealing with scope, namespaces etc
> was a pain in a system which relied on 'tangling' of source files in order
> to get the final output and nearly all supporting utilities (code
> formatters, highlighting, linters etc) were broken.  Like others who have
> posted in this thread, we thought it could all be improved if you could
> eliminate the weaving/tangling and be free to just write prose within your
> code, so we tried to do this with the lisp we were using (sorry, can't
> remember what lisp flavor it was). At first glance, it looked very easy to
> do - a few tweaks to the parser, some new reader macros and we should be
> fine. On one level, it did fix some of the issues, but we did find lots of
> unforeseen edge cases and it added sufficient additional complexity to the
> whole system that in the end, we decided it just didn't deliver the
> benefits we had hoped for.  At the end of the day, it wasn't the tools or
> the implementation which killed it - it was simply that literate
> programming simply didn't fit with how people like to code. It was one of
> those things which sounds nice in theory, but for most, fails in practice.
>
> We did find the approach works better for some languages than others. For
> example, languages like C, where you have a code, compile, run, test cycle,
> it wasn't oo bad. However, for languages where you often included a more
> REPL driven style of devleopment, where lots of devleopment is done as
> experimentation/refinement at the repl, the paradigm added very little and
> more often than not, just got in the way.  We did find it was useful for
> smaller tasks and configuration management, but for larger projects,
> especially with teams of developers, it didn't stand up.
>
> So, my advice, for what it is worth is
>
> Implement a solution which does NOT require any modifications to either
> the elisp parser or C code and use it to develop some non-trivial elisp
> packages. You could probably base it on org-mode. My suspicion is that you
> will find that after some time, it really doesn't deliver what you hoped.
> However, if I'm wrong, it will provide the necessary experience and
> familiarity to help guide a more refined and targeted model for working
> with literate programming and elisp.
>
> Don't under estimate the effort or impact making changes to either the
> elisp parser or C sources will have. There has been a number of comments in
> this thread which makes it sound like you would just need to 'tweak' the
> parser and everything will work. That 'tweaking' is far more complicated
> than it might seem on the surface. One of the main benefits of lisp
> dialects is there simple syntax and the idea of code as data. Any changes
> made to either the syntax or parsing of elisp is going to impact on lots of
> things, many of which none of us have even thought about yet.
>
> My feeling is that if literate programming is a great way forward, this
> will be evident in the number of people who start using an elisp literate
> programming mode. If this does turn out to be the case, then we can use the
> experiences from many people who are using that mode to see what can be
> done to develop the paradigm further. This may involve low level C or elsip
> parser or syntax changes, but they will be based on experience and demand.
>
> On Sat, 21 Dec 2019 at 03:56, Phillip Lord <phillip.lord@russet.org.uk>
> wrote:
>
>
>
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> >> My proposal is to slightly change Elisp parser to treat lines that start
> >> with any other printable character but '(' as a start of comment and to
> >> simply ignore the line, just as it treats ';' as a comment.
> >
> > The `sexpresso` Haskell package follows the same idea ;-)
> >
> > As for using it in Elisp: I don't think there's anything stopping anyone
> > from making such a `literate-elisp-mode` and even arrange for `load` to
> > handle such a file (just like there is already a package that lets
> > `load` work directly on .org files).
> >
> > I'd welcome such a package in GNU ELPA.
>
>
> I'm very late to this thread, for which apologies.
>
> My own package, lentic, could achieve this. It's gives you two
> visualisations of the same file; so one with comment characters, and one
> without. You can then do the literate part in one mode and the coding
> part in another.
>
> It already supports org-mode delination markers; it could be made to
> support brackets as a delineator too.
>
> Phil
>
>
>
> --
> regards,
>
> Tim
>
> --
> Tim Cross
>
>

-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 21330 bytes --]