Regarding outline headings in emacs-lisp libraries

Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

[-- Attachment #1: Type: text/plain, Size: 4747 bytes --]

Hello everybody!

I find it very useful when source files are divided into sections and
occasionally subsections as well.  It helps me understand the structure
of not just the file itself but ideally also of the functionality that
it implements.

This is also useful when reading code that one is not familiar with, but
unfortunately not all files are split into sections.  When I come across
a library that lacks explicit headings and I then often add the headings
myself and open a pull-request.

Yesterday I proposed such changes to Emacs itself and there is a detail
that Eli wanted to discuss here first, thus this thread.

But first some words about `outline-minor-mode', which is the section
facility that Emacs provides for use in arbitrary major-modes, for the
people who are not familiar with it yet or haven't actively used it in
a while:

  `outline-minor-mode' lets users navigate sections and lets them change
  their visibility, in pretty much the same way that `org-mode' does;
  the crucial difference being that it works in any major mode including
  programming modes.

  If you are not familiar with that minor-mode yet, then I would
  encourage you to give it try.  You might want to look at some of
  my libraries, which are very likely to actually being divided into
  sub*sections.

  From `org-mode' you might be familiar with local as well as global
  visibility cycling.  `outline-minor-mode' does not provide that out
  of the box, but (interesting historic fact:) the author of `org-mode'
  actually implemented that in the `outline-magic' package before moving
  on to writing `org-mode'.

  Nowadays `outline-magic' is moribund, but I have implemented
  `bicycle', which does the same thing but with a twist: it can also
  changes the visibility of top-level sexps, by wrapping around not only
  `outline' but also `hideshow' -- thus the "bi" prefix.

Eli pointed out that I split emacs-lisp libraries into (sub-)sections in
a way that violates one of the relevant conventions and that we should
discuss that here first.

To begin with, here are the relevant conventions:

,---- From (info "(elisp) Comment Tips")
| ‘;;;’
|      Comments that start with three semicolons, ‘;;;’, should start at
|      the left margin.  We use them for comments which should be
|      considered a heading by Outline minor mode.  By default, comments
|      starting with at least three semicolons (followed by a single space
|      and a non-whitespace character) are considered headings, comments
|      starting with two or fewer are not.  Historically, triple-semicolon
|      comments have also been used for commenting out lines within a
|      function, but this use is discouraged.
| 
|      When commenting out entire functions, use two semicolons.
| 
| ‘;;;;’
|      Comments that start with four (or more) semicolons, ‘;;;;’, should
|      be aligned to the left margin and are used for headings of major
|      sections of a program.  For example:
| 
|           ;;;; The kill ring
| 
|      If you wish to have sub-headings under these heading, use more
|      semicolons to nest these sub-headings.
`----

----------

TL;DR I want to use just *three* semicolons for "major sections of a
program", i.e. I want to consider "Code:" to be one of them instead of
their parent.

Okay then, here's the long version:

----------

When showing nothing but the top-level headings, then a library, which
is split up the way that I do it, looks like this:

,----
| ;;; foo.el --- Fooing support
| ;;; Commentary:...
| ;;; Code:...
| ;;; Options...
| ;;; Mode...
| ;;; Commands...
| ;;; Integrations...
| ;;; foo.el ends here
`----

But according to the second part of the conventions quoted above it
should look like this:

,----
| ;;; foo.el --- Fooing support
| ;;; Commentary:...
| ;;; Code:...
| ;;; foo.el ends here
`----

And the user would have to expand "Code:" explicitly to see all the
"major sections of the program", i.e. to see this:

,----
| ;;; foo.el --- Fooing support
| ;;; Commentary:...
| ;;; Code:...
| ;;;; Options...
| ;;;; Mode...
| ;;;; Commands...
| ;;;; Integrations...
| ;;; foo.el ends here
`----

Now, I agree that it makes sense to do it that way but the problem is
that it only does so in theory, in practice this approach comes with
several annoyances.

Before we discuss those, it is worth knowing that global visibility
cycling knows four states.  I am showing them here, using "my style":

(I am also attaching screenshots of these states.  Note that the look
of the headings is due to the use of my `outline-minor-faces' and
`backline' packages.)


[-- Attachment #2: 1-overview --]
[-- Type: image/png, Size: 24212 bytes --]

[-- Attachment #3: 2-toc --]
[-- Type: image/png, Size: 37569 bytes --]

[-- Attachment #4: 3-trees --]
[-- Type: image/png, Size: 75036 bytes --]

[-- Attachment #5: 4-all --]
[-- Type: image/png, Size: 109623 bytes --]

[-- Attachment #6: Type: text/plain, Size: 4471 bytes --]


,----
| 1. OVERVIEW: Show only top-level heading.
+----
| ;;; foo.el --- Fooing support
| ;;; Commentary:...
| ;;; Code:...
| ;;; Options...
| ;;; Mode...
| ;;; Commands...
| ;;; Integrations...
| ;;; foo.el ends here
`----

,----
| 2. TOC:      Show all headings, without treating top-level
|              code blocks as sections.
+----
| ;;; foo.el --- Fooing support...
| ;;; Commentary:...
| ;;; Code:...
| ;;; Options
| ;;; Mode
| ;;; Commands
| ;;;; List Commands
| ;;;; Misc Commands
| ;;; Integrations
| ;;; foo.el ends here
`----

,----
| 3. TREES:    Show all headings, treaing top-level code blocks
|              as sections (i.e. their first line is treated as
|              a heading).
+----
| ;;; foo.el --- Fooing support...
| ;;; Commentary:...
| ;;; Code:...
| (require 'bar)
| (declare-function 'baz-ing "baz" (arg))
| ;;; Options
| (defcustom foo-option nil...
| ;;; Mode
| (define-derived-mode foo-mode special-mode...
| ;;; Commands
| ;;;; List Commands
| (defun foo-list (arg)...
| (defun foo-list-all ()...
| ;;;; Misc Commands
| (defun foo-do-stuff (arg)...
| ;;; Integrations
| (defun foo-bookmark ()...
| (defun foo-helm ()...
| (provide 'foo)...
| ;;; foo.el ends here
`----

,----
| 4. ALL:      Show everything, except code blocks that have been
|              collapsed individually (using a `hideshow' command
|              or function).
+----
| ;;; foo.el --- Fooing support
|
| ;; Copyright (C) 2020 Me
|
| ;; Author: Me
|
| ;;; Commentary:
| ;;
| ;; This package implements support for fooing.
|
| ;;; Code:
|
| (require 'bar)
|
| (declare-function helm "helm")
|
| ;;; Options
|
| (defcustom foo-option nil
|   "bla")
|
| ;;; Mode
|
| (define-derived-mode foo-mode special-mode
|   "blabla")
|
| ;;; Commands
| ;;;; List Commands
|
| (defun foo-list (arg)
|   (interactive (read))
|   (things happen here))
|
| (defun foo-list-all ()
|   (interactive)
|   (things happen here too))
|
| ;;;; Misc Commands
|
| (defun foo-do-stuff (arg)
|   (interactive (read))
|   (some code)))
|
| ;;; Integrations
|
| (defun foo-bookmark ()
|   (more code))
|
| (defun foo-helm ()
|   (require 'helm)
|   (helm stuff))
|
| (provide 'foo)
|
| ;;; foo.el ends here
`----

And here are my reasons why "Options" et al. should not be subsections
of "Code:":

A) The OVERVIEW state would become almost useless.  Personally I would
   always skip it and go straight for TOC.  When following the
   convention, then the OVERVIEW state looks like this for *every*
   library.

   ,----
   | ;;; NAME.el --- DESCRIPTION
   | ;;; Commentary:...
   | ;;; Code:...
   | ;;; NAME.el ends here
   `----

   That just isn't useful.  If one wants to see the DESCRIPTION, then
   one does not have to hide any sections; one just has to make sure
   that the first line is visible by moving to the beginning of the
   buffer.

B) The OVERVIEW shown above isn't just not useful, for more complex
   libraries (which are split into more (sub*)sections) having a useful
   OVERVIEW is quite important.

   For such libraries the TOC just isn't a suitable substitute for
   OVERVIEW.  It could be deeply nested and if one only wants a list
   of the "major sections of a program", then a deeply nested tree of
   sub*sections just isn't the same.

C) If "Options" et al. are subsections of "Code:", then there likely is
   code between the section heading "Code:" and the heading of the first
   subsection "Options".  This usually includes calls to `require' and
   `declare-function' and such.

   One could deal with that by adding an additional subsection called--
   I don't know -- "Frontmatter", but I feel that just adds unnecessary
   noise.

   It is worth noting that the TOC state does not show these pre-first-
   subsection sexps.  IMO that is a problem, as it makes is very easy
   for a user to overlook these forms.

If we put "Options" et al. at the same level as "Code:", then the latter
itself becomes the "Frontmatter" section mentioned in (C).

Sorry this has gotten so long.

     Cheers,
     Jonas

Ps: In my own libraries I put the `provide' form and the ";; Local
    Variables:\n...\n;; End:" block, if any, in their own section,
    which I name just "_" (i.e. ";;; _\n").  But I don't dare suggest
    we do the same in Emacs.

Pps: When one often collapses sections, then the ";;; NAME.el ends here"
     becomes very annoying.  I am glad to have learned recently that as
     early as Emacs 31.1 that line won't be mandatory anymore. ;P

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

Hi Jonas,

> Now, I agree that it makes sense to do it that way but the problem is
> that it only does so in theory, in practice this approach comes with
> several annoyances.

[ Note: I use `outline-minor-mode` systematically in all my Elisp files.  ]

I don't have a strong opinion on this, but FWIW, I have a slight
preference for the ";;;;" version over the ";;;" version (but it's very
slight; I'm really OK either way.  In most cases I don't even notice
the difference).

The annoyance you mention doesn't affect me, because I use

    (hide-sublevels (if (eq outline-level 'lisp-outline-level) 1000 1))

to fold most of the actual code when opening the file but none of the
headings.  More specifically, I get something that might look like:

    ;;; foo.el --- Blabla
    ;;; Commentary:
    ;;; Code:
    (require 'cl-lib)
    ;;;; Custom vars:
    (defcustom foo-var nil...
    ;;;; Aux functions
    (defun foo-fun1 (arg)...
    (defun foo-fun2 (arg)...
    ;;; foo.el ends here

[ And I (almost) never toggle the folding or fiddle with it by hand,
  I only use `reveal-mode` to auto-unfold/refold as I move about the buffer.  ]


        Stefan

Re: Regarding outline headings in emacs-lisp libraries

[T.V Raman]

I'd definitely find this useful.

I started using a version of folding.el that was posted to Usenet in the
early 90's and still use it -- updated to a version from melpa a couple
of years ago. The emacspeak codebase uses this in all files.

Other elisp libraries that bring similar functionality:
1. allout  -- has been part of emacs for a long time, but has its own
   drawbacks

   2. More recent: outshine   which gives you similar functionality to
      what you outline.

      3. Lacking anything else, I use emacs' selective-display to
         collapse code below a specific indentation level (C-x $).

         All that said, having something common across Emacs libraries
         would be a big win 
-- 

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

T.V Raman [2020-07-17 18:41:16] wrote:
> I'd definitely find this useful.

I don't know what "this" refers to.


        Stefan

Re: Regarding outline headings in emacs-lisp libraries

[T.V Raman]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

this == "being able to collapse expand and navigate code by sectioning and subsectioning"> T.V Raman [2020-07-17 18:41:16] wrote:
>> I'd definitely find this useful.
>
> I don't know what "this" refers to.
>
>
>         Stefan
>

-- 

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

> this == "being able to collapse expand and navigate code by sectioning and
> subsectioning"

I still don't understand: we've had that "for ever", AFAICT.


        Stefan


> T.V Raman [2020-07-17 18:41:16] wrote:
>>> I'd definitely find this useful.
>>
>> I don't know what "this" refers to.
>>
>>
>>         Stefan
>>

Re: Regarding outline headings in emacs-lisp libraries

[T.V Raman]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

We've had that for ever -- yes but via disparate means that  power users
 have discovered. I read the original proposal on this thread as getting
 one step closer to exposing that power to more users via a set of
 consistent mechanisms. >> this == "being able to collapse expand and navigate code by sectioning and
>> subsectioning"
>
> I still don't understand: we've had that "for ever", AFAICT.
>
>
>         Stefan
>
>
>> T.V Raman [2020-07-17 18:41:16] wrote:
>>>> I'd definitely find this useful.
>>>
>>> I don't know what "this" refers to.
>>>
>>>
>>>         Stefan
>>>
>

-- 

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

"T.V Raman" <raman@google.com> writes:

> We've had that for ever -- yes but via disparate means that power
> users have discovered. I read the original proposal on this thread as
> getting one step closer to exposing that power to more users via a set
> of consistent mechanisms. >> this == "being able to collapse expand
> and navigate code by sectioning and subsectioning"

Those facilities already exist, whether you limit yourself to just
`outline-minor-mode' and its hands-on commands or additionally use
something like `bicycle' (there have been similar approaches).

If as a side-effect of this conversation you have learned about some
facilities, then that is a side-effect, but that's not what wasn't
really my goal.  (I begin with an introduction to how all this works,
but that really was to make sure that all who voice an opinion on my
proposal have a common understanding of what we are talking about.

What I would like to talk about is my proposal, which is: lets make the
"major code sections" siblings, not children, of the "Code:" section.

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

[ sorry, I somehow missed the end of your message on first reading.  ]

> B) The OVERVIEW shown above isn't just not useful, for more complex
>    libraries (which are split into more (sub*)sections) having a useful
>    OVERVIEW is quite important.
>
>    For such libraries the TOC just isn't a suitable substitute for
>    OVERVIEW.  It could be deeply nested and if one only wants a list
>    of the "major sections of a program", then a deeply nested tree of
>    sub*sections just isn't the same.

I guess what I was saying in my previous message is that I haven't found
a case of a file where the TOC is so large that it warrants the OVERVIEW.
And I'd even claim that such a file would be just too large and should
likely benefit from splitting it into a few files.

> Pps: When one often collapses sections, then the ";;; NAME.el ends here"
>      becomes very annoying.  I am glad to have learned recently that as
>      early as Emacs 31.1 that line won't be mandatory anymore. ;P

I think you missed the episode where they explain that it was
re-introduced in Emacs-31.2 because of all the complaints from people
who couldn't find the end of their code any more.


        Stefan

RE: Regarding outline headings in emacs-lisp libraries

[Drew Adams]

> > B) The OVERVIEW shown above isn't just not useful, for more complex
> >    libraries (which are split into more (sub*)sections) having a
> >    useful OVERVIEW is quite important.
> >
> >    For such libraries the TOC just isn't a suitable substitute for
> >    OVERVIEW.  It could be deeply nested and if one only wants a list
> >    of the "major sections of a program", then a deeply nested tree of
> >    sub*sections just isn't the same.
> 
> ... I haven't found a case of a file where the TOC is so
> large that it warrants the OVERVIEW.  And I'd even claim
> that such a file would be just too large and should
> likely benefit from splitting it into a few files.

FWIW -

In my Elisp libraries I often put an Index in the
Commentary, with links to the major sections.  For
this I use the simple library `linkd.el', which is
here:

https://www.emacswiki.org/emacs/download/linkd.el

When in minor-mode `linkd-mode', the links and
their targets are highlighted.

The major sections I use differ, depending on the
library.  Here's an example:

;;  (@> "Things Defined Here")
;;  (@> "Documentation")
;;  (@> "Macros")
;;  (@> "Faces (Customizable)")
;;  (@> "User Options (Customizable)")
;;  (@> "Internal Variables")
;;  (@> "New Commands")
;;  (@> "Replacements for Existing Functions")
;;  (@> "Non-Interactive Functions")

Each section begins with corresponding link-target
(anchor) text.  E.g., the "Macros" section has this:

;;(@* "Macros")

(I typically also put a formfeed (^L) char before
that line.  That lets me use `C-x ]' and `C-x [',
and with library `pp-c-l.el' I get clear visual
indication of ^L chars, i.e., of the sections.)

Org Mode offers similar linking, of course, but
Linkd is simple.

RE: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Drew Adams <drew.adams@oracle.com> writes:

> FWIW -
>
> In my Elisp libraries I often put an Index in the
> Commentary, with links to the major sections.  For
> this I use the simple library `linkd.el'

I used that library too for a while.  It's neat but ultimately I decided
it wasn't *necessary*.

1. Emacs already provided a convention and implementation for outlines
   that I liked.
2. I didn't want to manually maintain an index in the commentary like
   you do.  (And I realize that I don't have to do that.)
3. Links are orthogonal to sections and don't have to be implemented
   in the same package.

But most importantly we would never be able to convince "everybody"
to use this library.  I expect `outline-minor-mode' to be less
controversial and thus the easier sell.

> Org Mode offers similar linking, of course, but
> Linkd is simple.

I ended up implementing `orglink.el' which allows you to "use Org Mode
links in other modes". ;P

RE: Regarding outline headings in emacs-lisp libraries

[Drew Adams]

> > FWIW -
> > In my Elisp libraries I often put an Index in the
> > Commentary, with links to the major sections.  For
> > this I use the simple library `linkd.el'
> 
> I used that library too for a while.  It's neat but ultimately I
> decided it wasn't *necessary*.
> 
> 1. Emacs already provided a convention and implementation for outlines
>    that I liked.
> 2. I didn't want to manually maintain an index in the commentary like
>    you do.  (And I realize that I don't have to do that.)
> 3. Links are orthogonal to sections and don't have to be implemented
>    in the same package.
> 
> But most importantly we would never be able to convince "everybody"
> to use this library.  I expect `outline-minor-mode' to be less
> controversial and thus the easier sell.

I just mentioned it as a FWIW.  Nothing is
"*necessary*".  I wasn't proposing to convince
everybody, or anybody for that matter, to use
`linkd.el'.  There's no contest with Org or
`outline-minor-mode' or anything else.

Using text anchors, or form-feed chars, or both,
for links are alternative ways to define sections
to the way you propose.  They're not orthogonal
to it. 

Any way of defining sections is orthogonal to
folding or otherwise focusing on a section or
hiding sections.

And any way of defining sections is orthogonal
to having a separate index/TOC that links to
sections.  And to having commands that go to
the next/previous section.

I personally don't need/use folding for Elisp
files, and I personally do want easy ways to
move among sections.

You'll say that folding provides, in a way, an
index/TOC, which is true.  But an explicit one
doesn't require anyone to know about or use
folding (or Linkd, for that matter).

> > Org Mode offers similar linking, of course,
> > but Linkd is simple.
> 
> I ended up implementing `orglink.el' which allows you to "use Org Mode
> links in other modes". ;P

Good.  So it sounds like it has some of the Linkd
features (the only ones I use, perhaps), but it
uses Org mode-style links.  Either style is OK by
me.  Not much difference between the two, anyway:

Unrendered:

 (* "Linkd-style anchor")
 (@ "Linkd-style link")

 <<Org-style anchor>>
 [[Org-style link]]

Rendered (and highlighted):

 * Linkd anchor    *, text in diff faces, + mouse-face
 > Linkd link      >, text in diff faces, + mouse-face

 <<Org anchor>>    in a face
 Org link          in a face, + mouse-face

Now that `orglink.el' exists, maybe I'll switch to
using that, who knows?  But it does require loading
Org - and dash.el, which is pretty heavyweight for
just what it offers, no?

BTW, I just downloaded it, loaded a version of dash.el
(required, unfortunately), loaded orglink.el, and got
this (Emacs 26.3) when I tried `M-x orglink-mode' in
buffer orglink.el:

Debugger entered--Lisp error: (void-variable org-link-descriptive)
  (set (make-local-variable (quote org-link-descriptive)) org-link-descriptive)
  (cond (orglink-mode (org-load-modules-maybe) (add-hook ...
  (let ((last-message (current-message))) (setq orglink-mode ...
  orglink-mode(toggle)
  funcall-interactively(orglink-mode toggle)
  call-interactively(orglink-mode record nil)
  command-execute(orglink-mode record)
  execute-extended-command(nil "orglink-mode" nil)
  funcall-interactively(execute-extended-command nil "orglink-mode" nil)
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

HTH.

RE: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Drew Adams <drew.adams@oracle.com> writes:

>> > FWIW -
> I just mentioned it as a FWIW.  Nothing is
> "*necessary*".

Yes, that's whey I didn't remove the FWIW from the quoted text. ;)

> Now that `orglink.el' exists, maybe I'll switch to
> using that, who knows?  But it does require loading
> Org - and dash.el, which is pretty heavyweight for
> just what it offers, no?

Org is indeed a heavy dependency.

Unless in cases where it has some big benefit I am slowly moving away
from dash in my packages.  This seems to be one of the cases where I
should do that.  Dash is only used once!

> BTW, I just downloaded it, loaded a version of dash.el
> (required, unfortunately), loaded orglink.el, and got
> this (Emacs 26.3) when I tried `M-x orglink-mode' in
> buffer orglink.el:

You need Org >= 9.3 as specified in the Package-Requires.

RE: Regarding outline headings in emacs-lisp libraries

[Drew Adams]

> > Now that `orglink.el' exists, maybe I'll switch to
> > using that, who knows?  But it does require loading
> > Org - and dash.el, which is pretty heavyweight for
> > just what it offers, no?
> 
> Org is indeed a heavy dependency.
> 
> Unless in cases where it has some big benefit I am slowly moving away
> from dash in my packages.  This seems to be one of the cases where I
> should do that.  Dash is only used once!

Commendable!

> You need Org >= 9.3 as specified in the Package-Requires.

Got it.

Is the dependency on Org really necessary?
Seems pretty heavyweight for just the features
that orglink.el provides (which is similar to
what linkd.el provides).

I can understand, for example, having orglink.el
do more, i.e., optionally, if Org is loaded, but
is a hard-require really needed for the basic
behavior?  Maybe consider conditionalizing some
behavior on (featurep 'org)?

(Just a naive suggestion, without looking at the code.)

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I don't have a strong opinion on this, but FWIW, I have a slight
> preference for the ";;;;" version over the ";;;" version (but it's very
> slight; I'm really OK either way.  In most cases I don't even notice
> the difference).

I definitely prefer the Code:-sibling approach, but if Emacs continues
to use and advertise the Code:-children approach, then I can live with
that.

What I care about more is that as many of Emacs' libraries as possible
use are split into sections.  I am happy to help with that task and if
I have to compromise on the nesting, so be it.

(For my own libraries will continue to use the sibling approach and if
Emacs sticks with the children approach, then I will probably have to
add a buffer-local variable to teach `bicycle-cycle-global' to skip the
overview state.)

> The annoyance you mention doesn't affect me, because I use

I think we will have to wait for more feedback then.  We would probably
want to hear from more than one person (me) that uses this in a way that
is even affected by whether we nest more or less.

> [ sorry, I somehow missed the end of your message on first reading.  ]

Ah... spaced out somewhere along the way. ;)

>> B) The OVERVIEW shown above isn't just not useful, for more complex
>>    libraries (which are split into more (sub*)sections) having a useful
>>    OVERVIEW is quite important.
>>
>>    For such libraries the TOC just isn't a suitable substitute for
>>    OVERVIEW.  It could be deeply nested and if one only wants a list
>>    of the "major sections of a program", then a deeply nested tree of
>>    sub*sections just isn't the same.
>
> I guess what I was saying in my previous message is that I haven't found
> a case of a file where the TOC is so large that it warrants the OVERVIEW.
> And I'd even claim that such a file would be just too large and should
> likely benefit from splitting it into a few files.

Okay, let's assume the benefit is tiny in almost all non-pathological
cases.  It's still a tiny benefit at least to have both the overview
and toc views.

Let's look at it from another perspective: what are the *benefits* of
making the "major code parts" children of "Code:"?  The only one that
I can think of is that the name of that section "Code" implies that it
contains all the code.  I would argue that is a just a tiny benefit as
well.  Also it might be that when the name of that section was chosen
little thought was given to the issue we are discussing now, so maybe
that name should not hold too much weight when deciding on this issue.

I think it comes down to: when it doubt, then do *not* add additional
hierarchies.  Only do that when there is some clear benefit.

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

> What I care about more is that as many of Emacs' libraries as possible
> use are split into sections.  I am happy to help with that task and if
> I have to compromise on the nesting, so be it.

Right.  Like you I stated my preferred color, but really I'm much more
interested in the shed than its color.  I'm pretty bad at sectioning the
files on which I work, but I do appreciate when files are
properly sectioned.  If using ';;;' can increase your motivation, then
I'll gladly support that.


        Stefan

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Date: Sun, 19 Jul 2020 01:26:01 +0200
> Cc: emacs-devel@gnu.org
> 
> Let's look at it from another perspective: what are the *benefits* of
> making the "major code parts" children of "Code:"?  The only one that
> I can think of is that the name of that section "Code" implies that it
> contains all the code.  I would argue that is a just a tiny benefit as
> well.

But if we agree to your proposal, what would be the purpose of the
"Code:" section?  It no longer contains all of the code, but what does
it contain?

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Jonas Bernoulli <jonas@bernoul.li> writes:

> I think we will have to wait for more feedback then.  We would probably
> want to hear from more than one person (me) that uses this in a way that
> is even affected by whether we nest more or less.

No protests so far so here is what I would like to use to replace the
last two items in (info "(elisp)Comment Tips"):

‘;;;’ and more
     Comments that start with three (or more) semicolons, ‘;;;’, should
     start at the left margin.  We use them for comments that should be
     considered a heading by Outline minor mode.  By default, comments
     starting with at least three semicolons (followed by a single space
     and a non-whitespace character) are considered section headings,
     comments starting with two or fewer are not.
     
     (Historically, triple-semicolon comments have also been used for
     commenting out lines within a function, but this use is discouraged
     in favor of using just two semicolons.  This also applies when
     commenting out entire functions; when doing that use two semicolons
     as well.)

     Three semicolons are used for top-level sections, four for
     sub-sections, five for sub-sub-sections and so on.

     Typically libraries have at least four top-level sections.  For
     example when the bodies of all of these sections are hidden:

          ;;; backquote.el --- implement the ` Lisp construct...
	  ;;; Commentary:...
          ;;; Code:...
          ;;; backquote.el ends here

     (In a sense the last line is not a section heading as it must
     never be followed by any text; after all it marks the end of the
     file.)

     Historically, we recommended that all sections that contain code
     should be subsections of the ‘;;; Code:’ section, but because
     that needlessly introduces an additional nesting level we now
     recommend that use top-level sections for all the major code
     sections of the library.  The ‘;;; Code:’ section itself should
     be used for the ‘require’ forms and for other such "front
     matter".

     Finally, we recommend that you don't end headings with a colon or
     any other punctuation for that matter.  For historic reasons the
     ‘;;; Code:’ and ‘;;; Commentary:’ headings end with a colon, but
     we recommend that you don't do the same for other headings anyway.

Personally I would also recommend that people use a "back matter"
section similar to the "front matter" section that is ";;; Code:", but
I fear my preferred heading for that, ";;; _", might not find too many
fans.  But then again, I might be wrong, what do you think about ending
with this instead:

          ;;; backquote.el ends here

     Historically, we recommended that all sections that contain code
     should be subsections of the ‘;;; Code:’ section, but because
     that needlessly introduces an additional nesting level we now
     recommend that use top-level sections for all the major code
     sections of the library.

     The ‘;;; Code:’ section itself should be used for the ‘require’
     forms and for other such "front matter".

     If you would like to analogously place the ‘provide’ form and other
     "back matter" in their own section, then do not abuse the ";;; FILE
     ends here" line for that purpose.  In a sense that is not a heading
     because it must never be followed by any text; after all its
     purpose is to mark the very end of the file.  We recommend that you
     use just ‘;;; _’ as the heading of the "back matter" section.

     Finally, we recommend that you don't end headings with a colon or
     any other punctuation for that matter.  For historic reasons the
     ‘;;; Code:’ and ‘;;; Commentary:’ headings end with a colon, but
     we recommend that you don't do the same for other headings anyway.

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Date: Sun, 26 Jul 2020 15:49:26 +0200
> Cc: emacs-devel@gnu.org
> 
> Jonas Bernoulli <jonas@bernoul.li> writes:
> 
> > I think we will have to wait for more feedback then.  We would probably
> > want to hear from more than one person (me) that uses this in a way that
> > is even affected by whether we nest more or less.
> 
> No protests so far so here is what I would like to use to replace the
> last two items in (info "(elisp)Comment Tips"):

I think I asked a question about the proposed use of the "Code"
heading under your proposal, and I'd still like to discuss that issue,
because it sounds like that heading will no longer have any role.

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Eli Zaretskii <eliz@gnu.org> writes:

> I think I asked a question about the proposed use of the "Code"
> heading under your proposal, and I'd still like to discuss that issue,
> because it sounds like that heading will no longer have any role.

Yes you did but I though that this part of what you are replying to
addressed that:

> The ‘;;; Code:’ section itself should
> be used for the ‘require’ forms and for other such "front
> matter".

I am also listing a few more things explicitly in my response to Stefan,
i.e.:

> This section may also contain other
> things that "have to happen early on" and/or "don't fit anywhere
> else".  For example the definition of `PACKAGE-version' or internal
> variables and macros that are used early on.

We all have seen plenty of libraries and know the messy bits that tend
to accumulate at the beginning of their code portion.  That's the stuff
that I would leave inside the "Code:" section.  The bits that come
before the first `defgroup' (if any).

     Cheers,
     Jonas

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Date: Sun, 26 Jul 2020 17:43:13 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I think I asked a question about the proposed use of the "Code"
> > heading under your proposal, and I'd still like to discuss that issue,
> > because it sounds like that heading will no longer have any role.
> 
> Yes you did but I though that this part of what you are replying to
> addressed that:
> 
> > The ‘;;; Code:’ section itself should
> > be used for the ‘require’ forms and for other such "front
> > matter".

That's okay, but we will have to rename it to something else, then.

> I am also listing a few more things explicitly in my response to Stefan,
> i.e.:
> 
> > This section may also contain other
> > things that "have to happen early on" and/or "don't fit anywhere
> > else".  For example the definition of `PACKAGE-version' or internal
> > variables and macros that are used early on.
> 
> We all have seen plenty of libraries and know the messy bits that tend
> to accumulate at the beginning of their code portion.  That's the stuff
> that I would leave inside the "Code:" section.  The bits that come
> before the first `defgroup' (if any).

"Preliminaries"?

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jonas Bernoulli <jonas@bernoul.li>
>> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
>> Date: Sun, 26 Jul 2020 17:43:13 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > I think I asked a question about the proposed use of the "Code"
>> > heading under your proposal, and I'd still like to discuss that issue,
>> > because it sounds like that heading will no longer have any role.
>> 
>> Yes you did but I though that this part of what you are replying to
>> addressed that:
>> 
>> > The ‘;;; Code:’ section itself should
>> > be used for the ‘require’ forms and for other such "front
>> > matter".
>
> That's okay, but we will have to rename it to something else, then.
>
>> I am also listing a few more things explicitly in my response to Stefan,
>> i.e.:
>> 
>> > This section may also contain other
>> > things that "have to happen early on" and/or "don't fit anywhere
>> > else".  For example the definition of `PACKAGE-version' or internal
>> > variables and macros that are used early on.
>> 
>> We all have seen plenty of libraries and know the messy bits that tend
>> to accumulate at the beginning of their code portion.  That's the stuff
>> that I would leave inside the "Code:" section.  The bits that come
>> before the first `defgroup' (if any).
>
> "Preliminaries"?

Sounds like an interesting option, but lets discuss in the sibling
thread where the names of both sections are being discussed.

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

> Personally I would also recommend that people use a "back matter"
> section similar to the "front matter" section that is ";;; Code:", but
> I fear my preferred heading for that, ";;; _", might not find too many
> fans.  But then again, I might be wrong, what do you think about ending
> with this instead:

Indeed, I don't much like `;;; _` simply because it is non-descriptive.

Could we try and find a good name for those sections?
What do you put in those things (other than file-local vars)?


        Stefan

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Personally I would also recommend that people use a "back matter"
>> section similar to the "front matter" section that is ";;; Code:", but
>> I fear my preferred heading for that, ";;; _", might not find too many
>> fans.  But then again, I might be wrong, what do you think about ending
>> with this instead:
>
> Indeed, I don't much like `;;; _` simply because it is non-descriptive.
>
> Could we try and find a good name for those sections?
> What do you put in those things (other than file-local vars)?

There's actually two sections that we might want to find better names
for.  Let's use "front matter" and "back matter" as working titles at
least.

- Currently we use "Code:" as the name of that "front matter" section.

  It usually contains a bunch of `require' forms and maybe some function
  and variable declaration when we want to avoid loading the respective
  libraries up front.

  We could therefore name it "Dependencies".  But that does not always
  match the content perfectly.  This section may also contain other
  things that "have to happen early on" and/or "don't fit anywhere
  else".  For example the definition of `PACKAGE-version' or internal
  variables and macros that are used early on.

  Sticking to "Code:" has the advantage that we can justify that by
  "that's what it used to be called and since we cannot think of
  anything better we might as well stick to it.  Of course if we
  actually could think of something better...

- We currently don't put the `provide' form, the local variable
  definitions and other "stuff that goes at the end" into a dedicated
  "back matter" section.  Instead we put these things at the end
  whatever other completely unrelated section happens to come last in
  the file.

  In my own libraries I name that section "_" and (beside the things
  just mentioned that have to go there because they have to be placed
  near the end of the file) I also use it for "administrative things",
  like adding functions to third-party hooks, test definitions,
  font-lock keyword definitions for forms defined across the file.

  Some of these things may define separate sections or in same cases
  they should be placed elsewhere further up in the file along-side
  related stuff.  But sometimes/often that doesn't work / is overkill
  and it is nice to have a "All the crap that remains" section, that
  has a non-descriptive name such as "_".

  I tend to think of the non-descriptiveness as a feature.

The best I can come up right now is "Front matter" and "Back matter".
Or else "Code:" and "_" as suggested before.  But hopefully someone else
can think of something better.

     Jonas

Re: Regarding outline headings in emacs-lisp libraries

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 954 bytes --]



> On Jul 26, 2020, at 11:32 AM, Jonas Bernoulli <jonas@bernoul.li> wrote:
> 
> 
> - We currently don't put the `provide' form, the local variable
>  definitions and other "stuff that goes at the end" into a dedicated
>  "back matter" section.  Instead we put these things at the end
>  whatever other completely unrelated section happens to come last in
>  the file.
> 
>  In my own libraries I name that section "_" and (beside the things
>  just mentioned that have to go there because they have to be placed
>  near the end of the file) I also use it for "administrative things",
>  like adding functions to third-party hooks, test definitions,
>  font-lock keyword definitions for forms defined across the file.

Since we normally put stuff that has side-effects (other than definitions) in the end of a package (provide, hooks, etc) maybe we can call it ;;; Side effect or simply ;;; Effect? Or Something in the same line.

Yuan

[-- Attachment #2: Type: text/html, Size: 9600 bytes --]

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

> The best I can come up right now is "Front matter" and "Back matter".
> Or else "Code:" and "_" as suggested before.  But hopefully someone else
> can think of something better.

I definitely don't want to rename "Code:" to something else.
It would be a rather gratuitous change, since "Code:" is basically the
only section that appears in virtually all Elisp files and renaming it
would provide very little concrete benefits.

As for the end: I'd rather simply not define a convention for it.
I'm not even sure what would be the benefit of such a convention
(unless there's really something substantial that will often end up
there, compared to the current situation where for most files it would
only contain the `provide` statement).


        Stefan

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Sun, 26 Jul 2020 14:23:40 -0400
> Cc: emacs-devel@gnu.org
> 
> > The best I can come up right now is "Front matter" and "Back matter".
> > Or else "Code:" and "_" as suggested before.  But hopefully someone else
> > can think of something better.
> 
> I definitely don't want to rename "Code:" to something else.
> It would be a rather gratuitous change, since "Code:" is basically the
> only section that appears in virtually all Elisp files and renaming it
> would provide very little concrete benefits.

But after the proposed changes, almost none of the code will be under
"Code:", so the name will be a misnomer, no?

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

> But after the proposed changes, almost none of the code will be under
> "Code:", so the name will be a misnomer, no?

Could be.  But that's a very minor cosmetic problem, AFAIC, and none of
the suggested replacements for it sound particularly useful/enlightening
either.  Which is why I wrote that "renaming it would provide very
little concrete benefits".

The costs/downsides of a rename are much more immediately apparent OTOH.


        Stefan

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> But after the proposed changes, almost none of the code will be under
>> "Code:", so the name will be a misnomer, no?
>
> Could be.  But that's a very minor cosmetic problem, AFAIC, and none of
> the suggested replacements for it sound particularly useful/enlightening
> either.  Which is why I wrote that "renaming it would provide very
> little concrete benefits".
>
> The costs/downsides of a rename are much more immediately apparent OTOH.

So does that mean that you approve of my first
variant of the updated description?

And Eli, are you okay with sticking to "Code:"?

I have two minor copy edits so far:
- s/that use/that you use/
- s/anyway././
Did you spot anything else?

     Jonas

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Cc: emacs-devel@gnu.org
> Date: Tue, 28 Jul 2020 15:41:27 +0200
> 
> And Eli, are you okay with sticking to "Code:"?

I very much dislike that possibility, sorry.

I cannot say I understand why Stefan objects so much: your proposal
breaks long-standing traditions anyway.

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jonas Bernoulli <jonas@bernoul.li>
>> Cc: emacs-devel@gnu.org
>> Date: Tue, 28 Jul 2020 15:41:27 +0200
>> 
>> And Eli, are you okay with sticking to "Code:"?
>
> I very much dislike that possibility, sorry.
>
> I cannot say I understand why Stefan objects so much: your proposal
> breaks long-standing traditions anyway.

Likewise Stefan and I don't understand why you dislike sticking to
"Code:" so much.  ;)

My proposal just changes which style we *recommend*.  Both styles
are being used in the wild.

     Jonas

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Date: Tue, 28 Jul 2020 16:42:32 +0200
> 
> > I cannot say I understand why Stefan objects so much: your proposal
> > breaks long-standing traditions anyway.
> 
> Likewise Stefan and I don't understand why you dislike sticking to
> "Code:" so much.  ;)

Because it would be a lie.

> My proposal just changes which style we *recommend*.  Both styles
> are being used in the wild.

IMO, our proposed style must make sense.

Anyway, that's just my opinion, feel free to ignore it.

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

> So does that mean that you approve of my first
> variant of the updated description?

Yes.


        Stefan

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

> I definitely don't want to rename "Code:" to something else.
> It would be a rather gratuitous change, since "Code:" is basically the
> only section that appears in virtually all Elisp files and renaming it
> would provide very little concrete benefits.
>
> As for the end: I'd rather simply not define a convention for it.
> I'm not even sure what would be the benefit of such a convention
> (unless there's really something substantial that will often end up
> there, compared to the current situation where for most files it would
> only contain the `provide` statement).
>
>         Stefan

I agree with all of that.

     Jonas

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Jonas Bernoulli <jonas@bernoul.li> writes:
> The best I can come up right now is "Front matter" and "Back matter".

Eli suggested "Preliminaries" for the former.

We should probably think about two names that go well together.

What about "prolog" and "epilog"?
Or similar, "Preamble" and "postamble"?

Re: [SPAM UNSURE] Re: Regarding outline headings in emacs-lisp libraries

[Stephen Leake]

Jonas Bernoulli <jonas@bernoul.li> writes:

> No protests so far so here is what I would like to use to replace the
> last two items in (info "(elisp)Comment Tips"):
>
> ‘;;;’ and more
>      Comments that start with three (or more) semicolons, ‘;;;’, should
>      start at the left margin.  We use them for comments that should be
>      considered a heading by Outline minor mode.  

It might be more helpful to spell this "outline-minor-mode", so users
know what command to invoke.

-- 
-- Stephe

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Date: Fri, 17 Jul 2020 23:48:04 +0200

Thanks for posting this extensive and detailed description.

> Eli pointed out that I split emacs-lisp libraries into (sub-)sections in
> a way that violates one of the relevant conventions and that we should
> discuss that here first.

To be clear, I wanted this discussed because if people prefer the
method proposed by Jonas, and we decide to adopt it, we should modify
the ELisp manual to match it, and use it as our recommendation
henceforth.

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Let me summarize the conversation so far:

1. I make some arguments as to why it is better for the sections that
   contain code to be top-level sections instead of sub-sections of a
   single top-level section.

   One can of course disagree with those arguments but that has not
   really happened.  Stefan stated that he ever so slightly prefers
   the nested approach but also that he is fine with either style.

2. Eli approves the change as long as we adjust the documentation and
   rename the "Code:" section to something else because if it does not
   contain all the code anymore, then keeping the old name would be a
   misnomer.

   > But after the proposed changes, almost none of the code will be
   > under "Code:", so the name will be a misnomer, no?

3. Stefan does not want to rename "Code:" to something else because it
   is the one bit that in nearly all elisp files.

   > I definitely don't want to rename "Code:" to something else.  It
   > would be a rather gratuitous change, since "Code:" is basically the
   > only section that appears in virtually all Elisp files and renaming
   > it would provide very little concrete benefits.

   Stefan also thinks that what Eli calls a "lie" is just a "very minor
   cosmetic problem".

Is that what they call a Mexican Standoff?

I think the conversation should be about whether my arguments as to
*why* we should change the recommended style are sound, but we discuss
whether "Code:" should be renamed or not.  (I tend to agree with Stefan
that it {should not / does not have to be} renamed.)  Nothing wrong with
discussing that detail, but I fear that disagreement about it is what
will ultimately derail my proposal; not disagreement about the merit of
my arguments as to why it is preferable to skip one nesting level.

     Jonas

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Date: Tue, 28 Jul 2020 21:18:03 +0200
> 
> Is that what they call a Mexican Standoff?

According to https://en.wikipedia.org/wiki/Mexican_standoff,

  A "Mexican standoff" is a confrontation in which no strategy exists
  that allows any party to achieve victory.

I don't think we are in that situation here, because several exit
strategies exist:

 . you can disregard my opinion and go with Stefan's
 . you can disregard Stefan's opinion and go with mine
 . we can still try reaching some compromise (from my POV I'd like to
   understand better why renaming "Code" would be a problem; the fact
   that it's present in every ELisp file doesn't seem to explain that)

> I think the conversation should be about whether my arguments as to
> *why* we should change the recommended style are sound, but we discuss
> whether "Code:" should be renamed or not.

I don't think there's disagreement about the essence of your proposal,
that's why we are not talking about it.

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jonas Bernoulli <jonas@bernoul.li>
>> Date: Tue, 28 Jul 2020 21:18:03 +0200
>> 
>> Is that what they call a Mexican Standoff?
>
> According to https://en.wikipedia.org/wiki/Mexican_standoff,
>
>   A "Mexican standoff" is a confrontation in which no strategy exists
>   that allows any party to achieve victory.
>
> I don't think we are in that situation here, because several exit
> strategies exist:
>
>  . you can disregard my opinion and go with Stefan's
>  . you can disregard Stefan's opinion and go with mine
>  . we can still try reaching some compromise (from my POV I'd like to
>    understand better why renaming "Code" would be a problem; the fact
>    that it's present in every ELisp file doesn't seem to explain that)

I mostly agree, but disregarding either your or Stefan's opinion is not
something I would be comfortable with, so the only option that I saw was
to somehow get one of you to change their mind.

>> I think the conversation should be about whether my arguments as to
>> *why* we should change the recommended style are sound, but we discuss
>> whether "Code:" should be renamed or not.
>
> I don't think there's disagreement about the essence of your proposal,
> that's why we are not talking about it.

Ah good, thanks for clarifying.

The common ground that we are starting from is that we all agree that
the _section title_ "Code:" is a misnomer if it stops containing all
the code.  (Though it still contains _some of_ the code. ;)

Writing the following lines helped me understand my own perspective
better and I am changing my stated position from something like "'Code:'
is good enough to stick to, if only for historic reasons" to "this is a
good name and actually the best name we could possibly come up with".

I primarily think of "Code:" as a section heading and was going to say
that it could also (as in "other people do that") be though of as a
separator, similar to \f, only to realize that I myself actually think
of it in both ways simultaneously.

The ";;; Code:" line makes a distinction between the code on one side
(and the non-code on the other).  A line "End of X" in the same spot
would serve a similar purpose, but naming X would be much harder.  All
that follows this line is code and all code goes after this line, but
what comes before this line is not the same in all files.  Usually there
is a copyright notice and a permission statement, some header keywords,
and some documentation, but any of these parts could be missing, so X
would not always be the same.

But the other side of the distinction/line is always the same, its the
code.  If we _had_ to pick a name for the non-code side that always
works, then it would be ... "metadata" ... shudder.

Sticking to "Code:" also has practical advantages along the same lines.

Often this section is preceded by "Commentary:".  If that section gets
long, then it may be split into sub-sections.  In other cases it may
be followed by other top-level non-code sections such as "History",
"Install" or "TODO".

There are many reasons why one would prefer to use sibling- instead of,
sub-sections, all that matters here is that there are _legit_ reasons
to have additional top-level non-code sections between "Commentary" and
The Section (Formerly Known as Code).

Whatever alternative name we choose for "Code:", nothing else can so
clearly say "this is the first section that contains code".  Given any
other name it would not be 100% clear whether it contains prose or code,
i.e. whether this line marks the boarder between the prose and the code.

It would be even worse if we did not insist that every library's first
code section must have a certain common name.  In such a scenario, and
when sections are collapsed imagine that "Commentary" is followed by
"Dependencies" -- there is no way of knowing whether "Dependencies" is
a prose section that talks about external required executables and/or
C libraries or if it basically contains a bunch of `require' forms.

     Jonas

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Cc: emacs-devel@gnu.org
> Date: Wed, 29 Jul 2020 20:34:01 +0200
> 
> Whatever alternative name we choose for "Code:", nothing else can so
> clearly say "this is the first section that contains code".

"Code starts here" is better (assuming that we want to have a marker,
not a heading).

Anyway, my main problem with the proposal to leave "Code:" is that it
works against the very heart of outlining: it breaks the structure of
the document.  And it gives me the creeps when I see documents whose
structure is broken.  (I actually don't understand how you reconcile
that with the rest of your proposal.)

Re: Regarding outline headings in emacs-lisp libraries

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

The definition of "lie" is
a false statement, made knowing it is false,
and with intent to deceive.

I don't think anyone here is trying to deceive anyone.
So let's not use the harshly critical word, "lie".
The question here is whether "Code:" is an accurate description
in certain circumstances.

I suggest we allow two ways of structuring the code into sections.

1. A single top-level section called "Code", which can have subsections.

2. Multiple top-level sections, none called just "Code", but their
names should contain the word "Code".

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Fri, 31 Jul 2020 22:53:46 -0400
> Cc: emacs-devel@gnu.org
> 
> I suggest we allow two ways of structuring the code into sections.
> 
> 1. A single top-level section called "Code", which can have subsections.
> 
> 2. Multiple top-level sections, none called just "Code", but their
> names should contain the word "Code".

Both possibilities are fine with me.

However, AFAIU Jonas didn't like 1, and both Stefan and Jonas didn't
like 2 (although the way 2 was discussed, we didn't consider having
"Code" in each of the top-level sections, so maybe this will make the
difference).

Thanks.

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Richard Stallman <rms@gnu.org>
>> Date: Fri, 31 Jul 2020 22:53:46 -0400
>> Cc: emacs-devel@gnu.org
>> 
>> I suggest we allow two ways of structuring the code into sections.
>> 
>> 1. A single top-level section called "Code", which can have subsections.
>> 
>> 2. Multiple top-level sections, none called just "Code", but their
>> names should contain the word "Code".
>
> Both possibilities are fine with me.
>
> However, AFAIU Jonas didn't like 1, and both Stefan and Jonas didn't
> like 2 (although the way 2 was discussed, we didn't consider having
> "Code" in each of the top-level sections, so maybe this will make the
> difference).

I find that even less appealing.

While I would prefer if we could just stick to "Code:", I am not
fundamentally opposed to using some other name.

I think we should not enforce some new standard name for the first code
section.  "Dependencies" would often make sense, and we could suggest
that name but when it doesn't make sense, then people should be free to
use something else.

I have mentioned that if we don't enforce some common name for the first
code section, then one could no longer determine with 100% accuracy
which section is the first code section from the section headings alone.
That is a bit unfortunate, but not really a big deal IMO.

Stefan seems to dislike the idea of doing away with the "Code:" name
more strongly than I do.  Maybe we could grandfather that name as a
compromise?  I.e. existing libraries could keep using "Code:" but new
libraries and when splitting up an existing library into sections for
the first time, then some other name should be picked.

     Peace,
     Jonas

Re: Regarding outline headings in emacs-lisp libraries

[T.V Raman]

Also, I believe the current code that extracts the Commentary section
from a file relies on ;;; Code: as the end-marker for the Commentary.

-- 

RE: Regarding outline headings in emacs-lisp libraries

[Drew Adams]

> Also, I believe the current code that extracts the Commentary section
> from a file relies on ;;; Code: as the end-marker for the Commentary.

Not exactly.

It looks for a match for "^;;;;* \\(.*\\):[ \011]*$".

That could be ";;; Code:" or it could be (and often
has been, in the past) ";;; Change Log".  It could
be any match for that regexp.


(Don't take this as support for changing something.
I'm just clarifying what's done currently by functions
such as `finder-commentary', which uses `lm-commentary',
which uses `lm-commentary-end'.)

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Cc: emacs-devel@gnu.org
> Date: Sat, 01 Aug 2020 23:13:31 +0200
> 
> Stefan seems to dislike the idea of doing away with the "Code:" name
> more strongly than I do.

And I still don't understand why.

> Maybe we could grandfather that name as a compromise?  I.e. existing
> libraries could keep using "Code:" but new libraries and when
> splitting up an existing library into sections for the first time,
> then some other name should be picked.

Fine with me, assuming that old libraries will not use the other
sections you proposed, but instead will be left in their current form.

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

>> Stefan seems to dislike the idea of doing away with the "Code:" name
>> more strongly than I do.
> And I still don't understand why.

I have the strong suspicion that renaming all those `;;; Code:` in all
those files *will* lead to regressions, since that convention's been
with us for so many years and followed by such a wide percentage of all
distributed Elisp files, that I expect it's used by several scripts we
don't know about.

And of course, we won't be able to rename them all, since too many of
them aren't under our control anyway.  So ";;; Core:" will be with us
for the next 10 years whether we like it or not.

Now, if the benefit were significant, I wouldn't mind, but AFAICT, the
benefit of renaming is very small (at least, given the alternative names
I've seen proposed so far which all seemed rather bland).  


        Stefan

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Jonas Bernoulli <jonas@bernoul.li>,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Sun, 02 Aug 2020 23:22:01 -0400
> 
> >> Stefan seems to dislike the idea of doing away with the "Code:" name
> >> more strongly than I do.
> > And I still don't understand why.
> 
> I have the strong suspicion that renaming all those `;;; Code:` in all
> those files *will* lead to regressions, since that convention's been
> with us for so many years and followed by such a wide percentage of all
> distributed Elisp files, that I expect it's used by several scripts we
> don't know about.

If we don't know about such scripts, do we really need to assume they
exist?

And what kind of scripts did you have in mind -- scripts which do what
jobs?

> And of course, we won't be able to rename them all, since too many of
> them aren't under our control anyway.  So ";;; Core:" will be with us
> for the next 10 years whether we like it or not.

How about if we leave "Code:", but just as a heading with absolutely
no body, i.e. if we add right after it some more appropriate heading
("Prerequisites:" or somesuch)?  Would that be a good compromise?

> Now, if the benefit were significant, I wouldn't mind, but AFAICT, the
> benefit of renaming is very small (at least, given the alternative names
> I've seen proposed so far which all seemed rather bland).  

It's IMO as significant when compared to the rest of Jonas's proposal.

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

> If we don't know about such scripts, do we really need to assume they
> exist?
>
> And what kind of scripts did you have in mind -- scripts which do what
> jobs?

The same kinds of scripts as those running on elpa.gnu.org.
I've had scripts that looked for `;;; Code:`.  I've replaced them with
code that relies on other features of the code since then, but that
makes me expect that there are such scripts out there.

>> Now, if the benefit were significant, I wouldn't mind, but AFAICT, the
>> benefit of renaming is very small (at least, given the alternative names
>> I've seen proposed so far which all seemed rather bland).  
> It's IMO as significant when compared to the rest of Jonas's proposal.

Fair point,


        Stefan

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Eli Zaretskii <eliz@gnu.org> writes:

> How about if we leave "Code:", but just as a heading with absolutely
> no body, i.e. if we add right after it some more appropriate heading
> ("Prerequisites:" or somesuch)?  Would that be a good compromise?

IMO that's an acceptable (but somewhat silly) compromise but what
matters more is whether Stefan can agree to this and even though he
replied to the rest of your message he did not reply to this question.

Re: Regarding outline headings in emacs-lisp libraries

[Stefan Monnier]

>> How about if we leave "Code:", but just as a heading with absolutely
>> no body, i.e. if we add right after it some more appropriate heading
>> ("Prerequisites:" or somesuch)?  Would that be a good compromise?
>
> IMO that's an acceptable (but somewhat silly) compromise but what
> matters more is whether Stefan can agree to this and even though he
> replied to the rest of your message he did not reply to this question.

[ Sorry, I just missed it.  ]
Yes, that's OK with me.


        Stefan

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> How about if we leave "Code:", but just as a heading with absolutely
>>> no body, i.e. if we add right after it some more appropriate heading
>>> ("Prerequisites:" or somesuch)?  Would that be a good compromise?
>>
>> IMO that's an acceptable (but somewhat silly) compromise but what
>> matters more is whether Stefan can agree to this and even though he
>> replied to the rest of your message he did not reply to this question.
>
> [ Sorry, I just missed it.  ]
> Yes, that's OK with me.

Bingo!

I'll rephrase the conventions and adjust my commits accordingly later
today.

     Jonas

Re: Regarding outline headings in emacs-lisp libraries

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   I.e. existing libraries could keep using "Code:" but new
  > libraries and when splitting up an existing library into sections for
  > the first time, then some other name should be picked.

What's wrong with having just one Code: section in a new library?
Multiple sections can be useful, but they are not always needed.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Regarding outline headings in emacs-lisp libraries

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > 1. A single top-level section called "Code", which can have subsections.
  > > 
  > > 2. Multiple top-level sections, none called just "Code", but their
  > > names should contain the word "Code".

  > Both possibilities are fine with me.

  > However, AFAIU Jonas didn't like 1,

If there ia a fairly simple file, and its code is in one section named 
"Code" (which is the convention now), what can be wrong with that?

That's how our files are set up now.  Is someone saying that that
is unacceptable and they _must_ all be changed?  That seems like an
unnecessary nuisance.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Richard Stallman <rms@gnu.org> writes:

>   > > 1. A single top-level section called "Code", which can have subsections.
...
>   > However, AFAIU Jonas didn't like 1,
>
> If there ia a fairly simple file, and its code is in one section named 
> "Code" (which is the convention now), what can be wrong with that?

A fairly simple (i.e. short) file does not have to split up the "Code:"
section at all.  If the file gets longer, then the convention calls for
the "Code:" section to be split into sub-section.

I would rather splitting the "Code:" section into multiple sibling
sections, because of the reasons given in the topmost post.

> That's how our files are set up now.  Is someone saying that that
> is unacceptable and they _must_ all be changed?  That seems like an
> unnecessary nuisance.

I am only suggesting that using sibling sections should also allowed and
that it maybe could even become the recommended approach going forward.

Using child sections would continue to be allowed and am not suggesting
that we adjust all existing files to use the sibling style instead.  I
don't think anyone else is suggesting that either.

     Jonas

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Jonas Bernoulli <jonas@bernoul.li> writes:

> Let me summarize the conversation so far:
>
> 1. I make some arguments as to why it is better for the sections that
>    contain code to be top-level sections instead of sub-sections of a
>    single top-level section.
>
>    One can of course disagree with those arguments but that has not
>    really happened.  Stefan stated that he ever so slightly prefers
>    the nested approach but also that he is fine with either style.
>
> 2. Eli approves the change as long as we adjust the documentation and
>    rename the "Code:" section to something else because if it does not
>    contain all the code anymore, then keeping the old name would be a
>    misnomer.
>
>    > But after the proposed changes, almost none of the code will be
>    > under "Code:", so the name will be a misnomer, no?
>
> 3. Stefan does not want to rename "Code:" to something else because it
>    is the one bit that in nearly all elisp files.
>
>    > I definitely don't want to rename "Code:" to something else.  It
>    > would be a rather gratuitous change, since "Code:" is basically the
>    > only section that appears in virtually all Elisp files and renaming
>    > it would provide very little concrete benefits.
>
>    Stefan also thinks that what Eli calls a "lie" is just a "very minor
>    cosmetic problem".
>
> Is that what they call a Mexican Standoff?
>
> I think the conversation should be about whether my arguments as to
> *why* we should change the recommended style are sound, but we discuss
> whether "Code:" should be renamed or not.  (I tend to agree with Stefan
> that it {should not / does not have to be} renamed.)  Nothing wrong with
> discussing that detail, but I fear that disagreement about it is what
> will ultimately derail my proposal; not disagreement about the merit of
> my arguments as to why it is preferable to skip one nesting level.
>
>      Jonas

One round of arguments later, I think the above sadly still summarizes
the situation pretty well.

  I have one last argument:

What I have proposed is not some crazy new idea of mine, which may or
may not proof to be useful.  It is already common practice.  I wasn't
even aware that this is controversial.  I have seen so many libraries
that split the code across multiple top-level sections that I though
this was accepted practice (even though I was aware that the tips say
that everything should go within "Code:").

These are the statistics for Emacs itself:

  658 libraries use child sections exclusively.
  730 libraries use sibling sections exclusively.
   36 libraries use both child and sibling sections.

It's about fifty/fifty!

(There are also 22 libraries that lack a "Code:" section and 14 sections
 that lack a "... ends here" footer; and I am ignoring those here.)

(Some sections may only appear to use multiple top-level code sections
 because the use the obsolete method of commenting out parts of functions
 with three semicolons instead of just two.  But even if we generously
 account for that and guess that that affects about a hundred libraries,
 then it still is about fifty/fifty.)

Eli, this is common practice and it has been for years -- decades
probably.  Please consider just accepting the fact that many people like
to use multiple top-level code sections while not minding that a section
named "Code:" exists that does not contain *all* the code.

I think this common practice should be officially allowed by updating
the conventions to say that both forms are acceptable.

     Legalize it!
     Jonas

Re: Regarding outline headings in emacs-lisp libraries

[Eli Zaretskii]

> From: Jonas Bernoulli <jonas@bernoul.li>
> Date: Sat, 08 Aug 2020 20:37:56 +0200
> 
> Eli, this is common practice and it has been for years -- decades
> probably.  Please consider just accepting the fact that many people like
> to use multiple top-level code sections while not minding that a section
> named "Code:" exists that does not contain *all* the code.

I already said, long ago, that you can disregard my opinion.  Why do
you insist that I agree?

Re: Regarding outline headings in emacs-lisp libraries

[Jonas Bernoulli]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jonas Bernoulli <jonas@bernoul.li>
>> Date: Sat, 08 Aug 2020 20:37:56 +0200
>> 
>> Eli, this is common practice and it has been for years -- decades
>> probably.  Please consider just accepting the fact that many people like
>> to use multiple top-level code sections while not minding that a section
>> named "Code:" exists that does not contain *all* the code.
>
> I already said, long ago, that you can disregard my opinion.  Why do
> you insist that I agree?

(As mentioned before) I am just not quite comfortable simply ignoring
you, the maintainer.  I probably just have to get over that.  But it is
all good, Stefan just agreed to your last suggested compromise of using
a "Code:" section with no content.

Re: Regarding outline headings in emacs-lisp libraries

[Thorsten Jolitz]

Jonas Bernoulli <jonas@bernoul.li> writes:

Hallo,

> Let me summarize the conversation so far:
>
> 1. I make some arguments as to why it is better for the sections that
>    contain code to be top-level sections instead of sub-sections of a
>    single top-level section.
>
>    One can of course disagree with those arguments but that has not
>    really happened.  Stefan stated that he ever so slightly prefers
>    the nested approach but also that he is fine with either style.
>
> 2. Eli approves the change as long as we adjust the documentation and
>    rename the "Code:" section to something else because if it does not
>    contain all the code anymore, then keeping the old name would be a
>    misnomer.
>
>    > But after the proposed changes, almost none of the code will be
>    > under "Code:", so the name will be a misnomer, no?
>
> 3. Stefan does not want to rename "Code:" to something else because it
>    is the one bit that in nearly all elisp files.
>
>    > I definitely don't want to rename "Code:" to something else.  It
>    > would be a rather gratuitous change, since "Code:" is basically the
>    > only section that appears in virtually all Elisp files and renaming
>    > it would provide very little concrete benefits.
>
>    Stefan also thinks that what Eli calls a "lie" is just a "very minor
>    cosmetic problem".
>
> Is that what they call a Mexican Standoff?
>
> I think the conversation should be about whether my arguments as to
> *why* we should change the recommended style are sound, but we discuss
> whether "Code:" should be renamed or not.  (I tend to agree with Stefan
> that it {should not / does not have to be} renamed.)  Nothing wrong with
> discussing that detail, but I fear that disagreement about it is what
> will ultimately derail my proposal; not disagreement about the merit of
> my arguments as to why it is preferable to skip one nesting level.

Quite some time ago when writing outshine.el (with outorg.el and
navi.el) I saw a lot of strange behaviour wrt comments and outlining in
Emacs core elisp files, and wrote quite an extensive analysis about it:

https://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00112.html

Maybe that is still of some interest today. 
I critizised that ;;;Code: convention too in that mail, since it leads
to extremely unbalanced outlining. But during that discussion in 2013, it was
concluded too that a change is not worth the pain. 

I just wanted to mention this, since this old thread from 2013 might
already be forgetten. I apologize, if it has been mentioned already and
I did not read the current thread from the start. 

-- 
cheers,
Thorsten