Modern Conventions for Emacs Lisp files?

Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

Hi List, 

there are new ways (libraries) to make Emacs Lisp buffers similar to
Org-mode buffers in looks and (outline) functionality. 

I made a screencast showing these new libraries for 'Org-mode outside
Org-mode' (outshine, outorg, poporg, navi-mode) in action:

,----------------------------
| http://youtu.be/nqE6YxlY0rw
`----------------------------

(Watch in HD)

Would it make sense to define 'modern conventions for Emacs Lisp files'
based on all this new functionality and the popularity of Org-mode?
Something like this:

1. File structuring with outcommented Org-mode headlines (;; * Headline)
2. Meta information not as free text anymore but as Org-mode properties,
maybe like this:

,---------------------------------------------------------
| * navi-mode.el --- major-mode for easy buffer-navigation
|   :PROPERTIES:
|   :copyright: Thorsten Jolitz
|   :copyright-years: 2013
|   :version:  0.9
|   :licence:  GPL 2 or later (free software)
|   :licence-url: http://www.gnu.org/licenses/
|   :part-of-emacs: no
|   :author: Thorsten Jolitz
|   :author_email: tjolitz AT gmail DOT com
|   :inspiration:  occur-mode org-mode
|   :keywords: emacs outline lisp
|   :END:
`---------------------------------------------------------

3. A file template somehow similar to this:

,---------------------------------------------------------
| * navi-mode.el --- major-mode for easy buffer-navigation
| ** Commentary
| *** About navi-mode
| *** Usage
| *** Installation
| *** Emacs Version
| ** ChangeLog
| * Requires
| * Mode Definitions
| * Variables
| ** Consts
| ** Vars
| ** Hooks
| ** Fonts
| ** Customs
| *** Custom Groups 
| *** Custom Vars
| * Defuns
| ** Functions
| ** Commands
| * Menus and Keys
| ** Menus
| ** Keys
| * Run Hooks and Provide
`---------------------------------------------------------

-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Alan Mackenzie]

Morning, Thorsten!

On Mon, Apr 08, 2013 at 02:23:54AM +0200, Thorsten Jolitz wrote:

> Hi List, 

> there are new ways (libraries) to make Emacs Lisp buffers similar to
> Org-mode buffers in looks and (outline) functionality. 

> I made a screencast showing these new libraries for 'Org-mode outside
> Org-mode' (outshine, outorg, poporg, navi-mode) in action:

> ,----------------------------
> | http://youtu.be/nqE6YxlY0rw
> `----------------------------

I must admit, I spent more time listening to the music than paying
attention to the screen.  ;-)

> (Watch in HD)

> Would it make sense to define 'modern conventions for Emacs Lisp files'
> based on all this new functionality and the popularity of Org-mode?

Maybe for new elisp files, and those old ones which somebody is prepared
to convert.  But not as a standard applicable to all elisp files.

> Something like this:

> 1. File structuring with outcommented Org-mode headlines (;; * Headline)
> 2. Meta information not as free text anymore but as Org-mode properties,
> maybe like this:

> ,---------------------------------------------------------
> | * navi-mode.el --- major-mode for easy buffer-navigation
> |   :PROPERTIES:
> |   :copyright: Thorsten Jolitz
> |   :copyright-years: 2013
> |   :version:  0.9
> |   :licence:  GPL 2 or later (free software)
> |   :licence-url: http://www.gnu.org/licenses/
> |   :part-of-emacs: no
> |   :author: Thorsten Jolitz
> |   :author_email: tjolitz AT gmail DOT com
> |   :inspiration:  occur-mode org-mode
> |   :keywords: emacs outline lisp
> |   :END:
> `---------------------------------------------------------

> 3. A file template somehow similar to this:

> ,---------------------------------------------------------
> | * navi-mode.el --- major-mode for easy buffer-navigation
> | ** Commentary
> | *** About navi-mode
> | *** Usage
> | *** Installation
> | *** Emacs Version
> | ** ChangeLog
> | * Requires
> | * Mode Definitions
> | * Variables
> | ** Consts
> | ** Vars
> | ** Hooks
> | ** Fonts
> | ** Customs
> | *** Custom Groups 
> | *** Custom Vars
> | * Defuns
> | ** Functions
> | ** Commands
> | * Menus and Keys
> | ** Menus
> | ** Keys
> | * Run Hooks and Provide
> `---------------------------------------------------------

I would be against this bit.  I think that semantically close defined
entities should be close together.  For example, cc-engine.el is divided
up into pages by ^Ls, and all the def{const,var,macro,subst,fun}s to do
with "A system for finding noteworthy parens before the point" are
together in that page.  It is convenient to `narrow-to-page' to enclose
all these definitions together.  This facility would be lost if all
definitions in the file (over 200 of them) were ordered as suggested
above.

> -- 
> cheers,
> Thorsten

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Modern Conventions for Emacs Lisp files?

[Stefan Monnier]

> 1. File structuring with outcommented Org-mode headlines (;; * Headline)

There is already a standard for that:

   ;;; Headline
   ;;;; SubHeadline
   ;;;;; SubSubHeadline

outline-minor-mode understands it.
   
> 2. Meta information not as free text anymore but as Org-mode properties,
> maybe like this:
> ,---------------------------------------------------------
> | * navi-mode.el --- major-mode for easy buffer-navigation
> |   :PROPERTIES:
> |   :copyright: Thorsten Jolitz
> |   :copyright-years: 2013
> |   :version:  0.9
> |   :licence:  GPL 2 or later (free software)
> |   :licence-url: http://www.gnu.org/licenses/
> |   :part-of-emacs: no
> |   :author: Thorsten Jolitz
> |   :author_email: tjolitz AT gmail DOT com
> |   :inspiration:  occur-mode org-mode
> |   :keywords: emacs outline lisp
> |   :END:
> `---------------------------------------------------------

There is already a non-free-text section with such meta-info,
manipulated with lisp-mnt.el functions.  It looks like

   ;; Author: Chinua Achebe <achebe@chinua.ng>
   ;; Maintainer: FSF
   ;; Package: emacs

It's not the same format, but that shouldn't be a major issue.


        Stefan

Re: Modern Conventions for Emacs Lisp files?

[Andreas Röhler]

Am 08.04.2013 15:22, schrieb Stefan Monnier:
>> 1. File structuring with outcommented Org-mode headlines (;; * Headline)
>
> There is already a standard for that:
>
>     ;;; Headline
>     ;;;; SubHeadline
>     ;;;;; SubSubHeadline
>
> outline-minor-mode understands it.
>
>> 2. Meta information not as free text anymore but as Org-mode properties,
>> maybe like this:
>> ,---------------------------------------------------------
>> | * navi-mode.el --- major-mode for easy buffer-navigation
>> |   :PROPERTIES:
>> |   :copyright: Thorsten Jolitz
>> |   :copyright-years: 2013
>> |   :version:  0.9
>> |   :licence:  GPL 2 or later (free software)
>> |   :licence-url: http://www.gnu.org/licenses/
>> |   :part-of-emacs: no
>> |   :author: Thorsten Jolitz
>> |   :author_email: tjolitz AT gmail DOT com
>> |   :inspiration:  occur-mode org-mode
>> |   :keywords: emacs outline lisp
>> |   :END:
>> `---------------------------------------------------------
>
> There is already a non-free-text section with such meta-info,
> manipulated with lisp-mnt.el functions.  It looks like
>
>     ;; Author: Chinua Achebe <achebe@chinua.ng>
>     ;; Maintainer: FSF
>     ;; Package: emacs
>
> It's not the same format, but that shouldn't be a major issue.
>
>
>          Stefan
>
>

Reads like another argument not to hard-code "*" as headers

Andreas

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> 1. File structuring with outcommented Org-mode headlines (;; * Headline)
>
> There is already a standard for that:
>
>    ;;; Headline
>    ;;;; SubHeadline
>    ;;;;; SubSubHeadline
>
> outline-minor-mode understands it.

ok, lets put it like this: I wanted to let fellow Emacs/Org-mode users
know that they can have the same thing in Org-mode style now.

And since 'outshine.el' and 'outorg.el' simply comment and uncomment
Org-mode headlines, it works in other modes too:

,--------------
| LaTeX
| % * Headline
| 
| PicoLisp
| ## * Headline
| 
| etc.
`--------------

The headline is calculated from the comment-syntax of the major-mode and
the Org-mode headline syntax.

>> 2. Meta information not as free text anymore but as Org-mode properties,
>> maybe like this:
>> ,---------------------------------------------------------
>> | * navi-mode.el --- major-mode for easy buffer-navigation
>> |   :PROPERTIES:
>> |   :copyright: Thorsten Jolitz
>> |   :copyright-years: 2013
>> |   :version:  0.9
>> |   :licence:  GPL 2 or later (free software)
>> |   :licence-url: http://www.gnu.org/licenses/
>> |   :part-of-emacs: no
>> |   :author: Thorsten Jolitz
>> |   :author_email: tjolitz AT gmail DOT com
>> |   :inspiration:  occur-mode org-mode
>> |   :keywords: emacs outline lisp
>> |   :END:
>> `---------------------------------------------------------
>
> There is already a non-free-text section with such meta-info,
> manipulated with lisp-mnt.el functions.  It looks like
>
>    ;; Author: Chinua Achebe <achebe@chinua.ng>
>    ;; Maintainer: FSF
>    ;; Package: emacs
>
> It's not the same format, but that shouldn't be a major issue.

Ok, again I probably wanted to let fellow Emacs/Org-mode users know that
they can now quickly open their elisp comment-sections in temporary
Org-mode edit buffers (with outorg.el) and thus effortlessly use the
special Org-mode stuff like table-editing and property-drawers in their
Emacs Lisp comment headers.

-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

Andreas Röhler <andreas.roehler@online.de> writes:

> Am 08.04.2013 15:22, schrieb Stefan Monnier:
>>> 1. File structuring with outcommented Org-mode headlines (;; * Headline)

> Reads like another argument not to hard-code "*" as headers

Actually, in outshine.el, in contrast to Org-mode, "*" is not hard-coded
but a customizable variable.

,-------------------------------------------------
| (defcustom outshine-outline-regexp-base " [*]+ "
|   "Base for calculating the outline-regexp"
|   :group 'outshine
|   :type 'regexp)
`-------------------------------------------------

-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

Alan Mackenzie <acm@muc.de> writes:

> I must admit, I spent more time listening to the music than paying
> attention to the screen.  ;-)

I was surprised too that Youtube offers me such wonderful completely
unknown (Japanese!) jazz as a playback track I can add after uploading
the screencast.

>> Would it make sense to define 'modern conventions for Emacs Lisp files'
>> based on all this new functionality and the popularity of Org-mode?
>
> Maybe for new elisp files, and those old ones which somebody is prepared
> to convert.  But not as a standard applicable to all elisp files.

I guess that people who spend much time in Org-mode buffers anyway day
by day (like me) might appreciate this 'Org-mode look & feel' in their
Emacs lisp buffers too.

>> 3. A file template somehow similar to this:
>
>> ,---------------------------------------------------------
>> | * navi-mode.el --- major-mode for easy buffer-navigation
>> | ** Commentary
>> | *** About navi-mode
>> | *** Usage
>> | *** Installation
>> | *** Emacs Version
>> | ** ChangeLog
>> | * Requires
>> | * Mode Definitions
>> | * Variables
>> | ** Consts
>> | ** Vars
>> | ** Hooks
>> | ** Fonts
>> | ** Customs
>> | *** Custom Groups 
>> | *** Custom Vars
>> | * Defuns
>> | ** Functions
>> | ** Commands
>> | * Menus and Keys
>> | ** Menus
>> | ** Keys
>> | * Run Hooks and Provide
>> `---------------------------------------------------------
>
> I would be against this bit.  I think that semantically close defined
> entities should be close together.  For example, cc-engine.el is divided
> up into pages by ^Ls, and all the def{const,var,macro,subst,fun}s to do
> with "A system for finding noteworthy parens before the point" are
> together in that page.  It is convenient to `narrow-to-page' to enclose
> all these definitions together.  This facility would be lost if all
> definitions in the file (over 200 of them) were ordered as suggested
> above.

Thats a valid argument, so consider the above template just as something
that turned out to be useful for me. It avoids ever increasing chaos in
a developing library, but sometimes I thought too that it would actually
be nice to have all definitions that belong together in one place.

-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Stefan Monnier]

>>> 1. File structuring with outcommented Org-mode headlines (;; * Headline)
>> There is already a standard for that:
>> ;;; Headline
>> ;;;; SubHeadline
>> ;;;;; SubSubHeadline
>> outline-minor-mode understands it.
> ok, lets put it like this: I wanted to let fellow Emacs/Org-mode users
> know that they can have the same thing in Org-mode style now.

I was responding to:

> Would it make sense to define 'modern conventions for Emacs Lisp files'
> based on all this new functionality and the popularity of Org-mode?

Your message wasn't quite clear on what those "modern" conventions would
look like.  I'd be happy to refine/extend/tweak the existing
conventions, but switching to different ones would be a lot
more problematic.


        Stefan

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>>> 1. File structuring with outcommented Org-mode headlines (;; * Headline)
>>> There is already a standard for that:
>>> ;;; Headline
>>> ;;;; SubHeadline
>>> ;;;;; SubSubHeadline
>>> outline-minor-mode understands it.
>> ok, lets put it like this: I wanted to let fellow Emacs/Org-mode users
>> know that they can have the same thing in Org-mode style now.
>
> I was responding to:
>
>> Would it make sense to define 'modern conventions for Emacs Lisp files'
>> based on all this new functionality and the popularity of Org-mode?
>
> Your message wasn't quite clear on what those "modern" conventions would
> look like.  I'd be happy to refine/extend/tweak the existing
> conventions, but switching to different ones would be a lot
> more problematic.

The modern conventions would just be that headlines are outcommented
Org-mode headlines with whatever comment syntax the major-mode at hand
uses. 

Thus the info about 'this is a headline' and 'this headline is level X'
is entirely in the Org-mode headers, the comment-characters are only
for that - outcommenting these headers. 

Thats the way outshine.el and outorg.el work, so they should in theory
work with all kinds of major-modes where 'comment-region' and
'uncomment-region' are defined, even those not yet written. 

That are of course different conventions, and I did not want to push any
big problematic moves or so. But with all the momentum and popularity
Org-mode enjoys, maybe such an 'Org-mode' style structuring of source
code files could become an accepted second choice (and if only in
libraries that are not part of Emacs). 

-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Stefan Monnier]

> The modern conventions would just be that headlines are outcommented
> Org-mode headlines with whatever comment syntax the major-mode at hand
> uses.

But that is incompatible with the current convention to use ";;;+" where
the number of semi-colons indicates depth.  So it would require massive
rewrites of pretty much all files.
A non-starter.

> That are of course different conventions, and I did not want to push any
> big problematic moves or so. But with all the momentum and popularity
> Org-mode enjoys, maybe such an 'Org-mode' style structuring of source
> code files could become an accepted second choice (and if only in
> libraries that are not part of Emacs).

Couldn't "Org-mode" be made to work with the current convention?
That would be a lot more useful since it would "just work" on all the
existing files.


        Stefan

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

Stefan Monnier <monnier@iro.umontreal.ca> writes:


> Couldn't "Org-mode" be made to work with the current convention?
> That would be a lot more useful since it would "just work" on all the
> existing files.

I will give it a try (not Org-mode, but the trio 'outshine', 'outorg'
and 'navi-mode'. I could make them work with Org-mode, maybe they work
with 'semicolons only' too. 

-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Pascal J. Bourguignon]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> 1. File structuring with outcommented Org-mode headlines (;; * Headline)
>
> There is already a standard for that:
>
>    ;;; Headline
>    ;;;; SubHeadline
>    ;;;;; SubSubHeadline

But the lisp convention is the opposite:

------------------------------------------------------------------------
;;;; -*- mode:lisp -*-


;;;; 
;;;; File comment
;;;; 

;;;
;;; Section comment
;;;

(defun code ()
  ;; indented comment
  (form)                 ; inline comment
  (form2)                ; long inline comment
  ;; indented comment
  (form3))

------------------------------------------------------------------------


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

"Pascal J. Bourguignon" <pjb@informatimago.com> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> 1. File structuring with outcommented Org-mode headlines (;; * Headline)
>>
>> There is already a standard for that:
>>
>>    ;;; Headline
>>    ;;;; SubHeadline
>>    ;;;;; SubSubHeadline
>
> But the lisp convention is the opposite:
>
> ------------------------------------------------------------------------
> ;;;; -*- mode:lisp -*-
>
>
> ;;;;
> ;;;; File comment
> ;;;;
>
> ;;;
> ;;; Section comment
> ;;;
>
> (defun code ()
>   ;; indented comment
>   (form)                 ; inline comment
>   (form2)                ; long inline comment
>   ;; indented comment
>   (form3))
>
> ------------------------------------------------------------------------

If thats true (and I think it actually might be) it would be a very
inconvenient convention, because one would need to know the number of
heading levels in advance, and would have to change all other headers if
a new level is introduced at the bottom:

,-------------------------------------------
| > ;;;; -*- mode:lisp -*-     ; * level 1
| >
| > ;;;;
| > ;;;; File comment          ; * level 1
| > ;;;;
| >
| > ;;;
| > ;;; Section comment        ; ** level 3
| > ;;;
`-------------------------------------------


,----------------------------------------------
| > ;;;;; -*- mode:lisp -*-       ; * level 1
| >
| > ;;;;;
| > ;;;;; File comment            ; * level 1
| > ;;;;;
| >
| > ;;;;
| > ;;;; Section comment          ; ** level 2
| > ;;;;
| >
| > ;;;
| > ;;; Subsection comment        ; *** level 3
| > ;;;
`----------------------------------------------

--
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Stefan Monnier]

> But the lisp convention is the opposite:

Right, that's because Lisp didn't think it through.
Luckily, we're only dealing with Elisp,


        Stefan

Re: Modern Conventions for Emacs Lisp files?

[Andreas Röhler]

Am 08.04.2013 21:51, schrieb Thorsten Jolitz:
> Andreas Röhler <andreas.roehler@online.de> writes:
>
>> Am 08.04.2013 15:22, schrieb Stefan Monnier:
>>>> 1. File structuring with outcommented Org-mode headlines (;; * Headline)
>
>> Reads like another argument not to hard-code "*" as headers
>
> Actually, in outshine.el, in contrast to Org-mode, "*" is not hard-coded
> but a customizable variable.
>
> ,-------------------------------------------------
> | (defcustom outshine-outline-regexp-base " [*]+ "
> |   "Base for calculating the outline-regexp"
> |   :group 'outshine
> |   :type 'regexp)
> `-------------------------------------------------
>

Ah, thanks. Thus it will keep legibility of native code.

Andreas

Re: Modern Conventions for Emacs Lisp files?

[Bastien]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Couldn't "Org-mode" be made to work with the current convention?

Not really.  But you can make org-mode work on top of current
conventions, provided users set `orgstruct-heading-prefix-regexp'
to "^;;;+ " as a local file variable in their Elisp files, and
use `orgstruct-mode' for them.

In that case, they can fold/unfold lines starting with "^;;;+ \*+".

(This is in the next release.)

> That would be a lot more useful since it would "just work" on all
> the existing files.

Right now we can use `outline-minor-mode' for existing files that do
not "*" characters in the comment sections, and `orgstruct-mode' for
those who do.

Personally, I'm not in favor of enforcing a new convention for using
"*" in comment sections: this is aesthetically dubious, it relies on a
nice little hack that provides Org-like folding.

I'd be more in favor of making it easier to use `outline-minor-mode'
with current conventions (i.e. more accessible keybindings, etc.)

-- 
 Bastien

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> But the lisp convention is the opposite:
>
> Right, that's because Lisp didn't think it through.
> Luckily, we're only dealing with Elisp,

So what is the official Elisp convention then:

,------------------------
| ;;;       * level 1
| ;;;;      ** level 2
| ;;;;;     *** level 3
`------------------------

or

,-------------------------
| ;;;;;      * level 1
| ;;;;       ** level 2
| ;;;        *** level 3
`-------------------------

?

-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Leo Liu]

On 2013-04-08 08:23 +0800, Thorsten Jolitz wrote:
> there are new ways (libraries) to make Emacs Lisp buffers similar to
> Org-mode buffers in looks and (outline) functionality.

Generally speaking this is a bad idea.

Leo

Re: Modern Conventions for Emacs Lisp files?

[Stefan Monnier]

> So what is the official Elisp convention then:

As mentioned earlier:

>    ;;; Headline
>    ;;;; SubHeadline
>    ;;;;; SubSubHeadline


        Stefan

Re: Modern Conventions for Emacs Lisp files?

[Stefan Monnier]

>> Couldn't "Org-mode" be made to work with the current convention?
> Not really.  But you can make org-mode work on top of current
> conventions, provided users set `orgstruct-heading-prefix-regexp'
> to "^;;;+ " as a local file variable in their Elisp files, and
> use `orgstruct-mode' for them.
> In that case, they can fold/unfold lines starting with "^;;;+ \*+".
> (This is in the next release.)

I'm obviously not familiar enough with Org, because I don't understand
what you're saying.  You seem to start by saying no, and then to go on and
say yes.


        Stefan

Re: Modern Conventions for Emacs Lisp files?

[Bastien]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I'm obviously not familiar enough with Org, because I don't understand
> what you're saying.  You seem to start by saying no, and then to go on and
> say yes.

Let me try again: 

No, we cannot make Org add folding to Elisp files with the current
conventions.

Yes, you can get Org folding in Elisp files if (1) you use stars
after ;;;+ and (2) turn `orgstruct-mode' on in those files and
(3) set `orgstruct-heading-prefix-regexp' to "^;;;+ ".

PS: The star(s) for starting a headline are hardcoded in Org.

-- 
 Bastien

Re: Modern Conventions for Emacs Lisp files?

[Andreas Röhler]

Am 09.04.2013 15:11, schrieb Bastien:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
[ ... ]
> PS: The star(s) for starting a headline are hardcoded in Org.
>

Hi Bastien,

as this was discussed some weeks ago: That restriction is easy to remove.
If kept, it should not show up last time.

Best,

Andreas

Re: Modern Conventions for Emacs Lisp files?

[Bastien]

Hi Andreas,

Andreas Röhler <andreas.roehler@online.de> writes:

> as this was discussed some weeks ago: That restriction is easy to remove.
> If kept, it should not show up last time.

You sent the start of a patch for this (untested and uncomplete), it
did not really help to convince me that removing this restriction is
easy.  I know why you can believe this is easy, but I'm not convinced
it is -- don't want to discourage you, though.

-- 
 Bastien

Re: Modern Conventions for Emacs Lisp files?

[Barry OReilly]

[-- Attachment #1: Type: text/plain, Size: 695 bytes --]

>>> 1. File structuring with outcommented Org-mode headlines (;; * Headline)
>>
>> There is already a standard for that:
>>
>>    ;;; Headline
>>    ;;;; SubHeadline
>>    ;;;;; SubSubHeadline
>
> But the lisp convention is the opposite:

Perhaps the Elisp manual needs an update, because it doesn't at all
describe what Stefan does.

http://www.gnu.org/software/emacs/manual/html_mono/elisp.html#Comment-Tips

"Comments that start with four semicolons, ‘;;;;’ ... are used for headings
of major sections".

"Comments that start with three semicolons, ‘;;;’ ... are used,
occasionally, for comments within functions [and] sometimes for comments
that are between functions."

[-- Attachment #2: Type: text/html, Size: 905 bytes --]

Re: Modern Conventions for Emacs Lisp files?

[Nicolas Richard]

Barry OReilly <gundaetiapo@gmail.com> writes:
>>> There is already a standard for that:
>>>
>>> ;;; Headline
>>> ;;;; SubHeadline
>>> ;;;;; SubSubHeadline
>
> Perhaps the Elisp manual needs an update, because it doesn't at all
> describe what Stefan does.
>
> http://www.gnu.org/software/emacs/manual/html_mono/elisp.html#Comment-Tips

The number of spaces after the semicolons is important too.

,----
| By default, Outline minor mode does not consider a comment to be a
| heading (even if it starts with at least three semicolons) if the
| semicolons are followed by at least two spaces.
`----

-- 
N.

Re: Modern Conventions for Emacs Lisp files?

[Pascal J. Bourguignon]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> But the lisp convention is the opposite:
>
> Right, that's because Lisp didn't think it through.
> Luckily, we're only dealing with Elisp,

I think it was thought through: don't use more than four levels!

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

"Pascal J. Bourguignon" <pjb@informatimago.com> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> But the lisp convention is the opposite:
>>
>> Right, that's because Lisp didn't think it through.
>> Luckily, we're only dealing with Elisp,
>
> I think it was thought through: don't use more than four levels!

But, citing your earlier post, the following example gives you only 2
levels for headlines plus two types of comments, and taking into account
that the file comment only appears once at the beginning of the file,
there is only one level left (;;;) for headlines. Thus in the actual
main (code) section of the library no hierarchical tree structure can be
build. I would not want to structure a 4000+ lines .emacs file with only
one level of headlines, in such big files I rather end up using 4 or 5
level or so.

------------------------------------------------------------------------
;;;; -*- mode:lisp -*-


;;;; 
;;;; File comment
;;;; 

;;;
;;; Section comment
;;;

(defun code ()
  ;; indented comment
  (form)                 ; inline comment
  (form2)                ; long inline comment
  ;; indented comment
  (form3))

------------------------------------------------------------------------


-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> The modern conventions would just be that headlines are outcommented
>> Org-mode headlines with whatever comment syntax the major-mode at hand
>> uses.
>
> But that is incompatible with the current convention to use ";;;+" where
> the number of semi-colons indicates depth.  So it would require massive
> rewrites of pretty much all files.
> A non-starter.
>
>> That are of course different conventions, and I did not want to push any
>> big problematic moves or so. But with all the momentum and popularity
>> Org-mode enjoys, maybe such an 'Org-mode' style structuring of source
>> code files could become an accepted second choice (and if only in
>> libraries that are not part of Emacs).
>
> Couldn't "Org-mode" be made to work with the current convention?
> That would be a lot more useful since it would "just work" on all the
> existing files.

I modified 'outshine.el', 'outorg.el' and 'navi-mode.el' so that they
now deal with the special case of "oldschool" elisp headers (";;;+").

Using this library-trio the way described in their comment-sections or
in this article

,-----------------------------------------------------------
| http://orgmode.org/worg/org-tutorials/org-outside-org.html
`-----------------------------------------------------------

gives any Emacs Lisp buffer with (modern or oldschool) headlines the
looks of an Org-mode buffer and easy keybindings for outline
functionality. Each subtree or the whole buffer can be edited in
temporary Org-mode buffers, and a permanent 'remote-control' *Navi*
buffer can be opened.

I uploaded a short (4 min) demo-video on youtube using 'org-elements.el'
as example file:

,-------------------------------------------
| http://www.youtube.com/watch?v=I0hxTAAlGZQ
`-------------------------------------------

I tried this on several arbitrary .el files from Emacs itself and from
Org-mode, and it worked. Testing several files revealed that many Elisp
libraries did not make much use of outline structuring, or used it in an
inconsistent way. Well structured files I found were e.g. 'ox.el' and
'org-elements.el' from Org-mode, but it seems as if there is room for
improvement in this area.

And while its useless to talk about the esthetics of 'org-style' headers
vs. 'oldschool' headers, my impression is that there is a clear
usability problem with the old conventions. 

This is still easy to spot, while a bit strange (3 times ; signal the
1st level):

,--------------
| ;;; 1st level
`--------------

but this one? Would you recognize the level at first sight?

,-----------------
| ;;;;;; header
`-----------------

(its 4th level)

Are these two at the same or at different levels?

,------------------
| ;;;;;;; header1
| text text text
| text text text
| text text text
| ;;;;;;;; header2
`------------------

Compare the conventional headers to Org-mode style headers:

,--------------
| ;; * 1st level
`--------------

,-----------------
| ;; **** header
`-----------------

,---------------
| ;; ***** header1
| text text text
| text text text
| text text text
| ;; ****** header2
`---------------

But anyway, if this cannot be changed with reasonable effort, its
fruitless to discuss it, and now the new libraries discussed in this
thread can be used for both kinds of header styles.

PS
There is one strange problem I have:

I can not open org.el any more, now that the oldschool elisp headers are
recognized by 'outshine' too. Its a big file, but size can't be the
problem since other quite big files open within 1 or 2 seconds or so.
When I try to open org.el, Emacs apparently goes into an infinite loop
and uses 100pc CPU - I have to kill the Emacs process and restart Emacs
to recover from that.

What might be different in org.el than in all the other elisp files I
tested so far to cause this problem?

-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Stefan Monnier]

> I modified 'outshine.el', 'outorg.el' and 'navi-mode.el' so that they
> now deal with the special case of "oldschool" elisp headers (";;;+").

Great, thank you!


        Stefan

Re: Modern Conventions for Emacs Lisp files?

[Stefan Monnier]

> I modified 'outshine.el', 'outorg.el' and 'navi-mode.el' so that they
> now deal with the special case of "oldschool" elisp headers (";;;+").

I see that the lines that match "^(" aren't treated as "deepest
headers", as they are in outline-minor-mode.

Since the format of those headers is very different, it might not be
easy to support them, of course.

> I tried this on several arbitrary .el files from Emacs itself and from
> Org-mode, and it worked. Testing several files revealed that many Elisp
> libraries did not make much use of outline structuring,

Indeed, which is why it's not worth the trouble introducing a new convention.


        Stefan

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I modified 'outshine.el', 'outorg.el' and 'navi-mode.el' so that they
>> now deal with the special case of "oldschool" elisp headers (";;;+").
>
> I see that the lines that match "^(" aren't treated as "deepest
> headers", as they are in outline-minor-mode.
>
> Since the format of those headers is very different, it might not be
> easy to support them, of course.

Well, the project 'Org-mode outside Org-mode' started with the very
simple yet generic idea to give source-code files the look&feel of
Org-mode files, and (in theory) it should work with all kinds of
major-modes - headlines are just outcommented Org-mode headlines with
whatever comment-syntax the major-mode at hand uses. 

Now I added two hacks - one for Org-mode itself (to enable the use of
'navi-mode' with Org-mode), the other for the oldschool lisp headers.
But its still oriented at Org-mode - only the headlines are matched, not
"^(" or so. 

But with 'navi-mode' its easy to get an overview over the all
definitions (or maybe just the function or just the variable
definitions) in a buffer. These are the keyword searches I defined for
Emacs Lisp in 'navi-mode':

,-------------------------------------------
| [KEY] : [SEARCH]
| ================
|                         a : ALL
|                         f : FUN
|                         v : VAR
|                         x : OBJ
|                         b : DB
|                         F : defun
|                         V : defvar
|                         C : defconst
|                         G : defgroup
|                         U : defcustom
|                         A : defadvice
|                         M : defmarcro
|                         D : defface
|                         S : defstruct
|                         L : defclass
|                         K : global-set-key
|                         T : add-to-list
|                         Q : setq
|                         H : add-hook
|                         O : hook
|                         X : lambda
|                         R : require
`-------------------------------------------

but this is customizable, try

,---------------------------------------------
| M-x customize-variable RET navi-key-mappings
| M-x customize-variable RET navi-keywords
`---------------------------------------------

for examples and explanations

One special thing about 'navi-mode' is that all 'free' ASCII printing
characters are by default bound to 'navi-generic-command', thus after
substracting the reserved one-key bindings from navi-mode itself, the
user still has lots of keys left to define his/her own keyword-searches,
and these key definitions are done by language. Here is e.g. what I
defined for picolisp-mode:

,------------------------------------
| [KEY] : [SEARCH]
| ================
|                         a : ALL
|                         f : FUN
|                         v : VAR
|                         x : OBJ
|                         b : DB
|                         D : de
|                         F : def
|                         C : class
|                         M : dm
|                         R : rel
|                         V : var
|                         X : extend
|                         O : obj
|                         J : object
|                         N : new
|                         S : symbols
|                         L : pool
|                         T : tree
|                         U : clause
|                         G : goal
|                         B : be
|                         P : prove
`------------------------------------
 
>> I tried this on several arbitrary .el files from Emacs itself and from
>> Org-mode, and it worked. Testing several files revealed that many Elisp
>> libraries did not make much use of outline structuring,
>
> Indeed, which is why it's not worth the trouble introducing a new convention.

I'm of course a bit biased towards the Ord-mode style headers, but with
the direct comparison to the 'oldschool headers' possible now, I still
find them much better when it comes to signaling 'this is a headline'
and 'this headline has level X'. But if one of those days one of my
libraries would make it into Emacs, it wouldn't be a problem to convert
them back to the oldschool syntax.

-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> PS
> There is one strange problem I have:
>
> I can not open org.el any more, now that the oldschool elisp headers are
> recognized by 'outshine' too. Its a big file, but size can't be the
> problem since other quite big files open within 1 or 2 seconds or so.
> When I try to open org.el, Emacs apparently goes into an infinite loop
> and uses 100pc CPU - I have to kill the Emacs process and restart Emacs
> to recover from that.
>
> What might be different in org.el than in all the other elisp files I
> tested so far to cause this problem?

FYI:

I figured out whats different in org.el and causes the problem described
above: its the one long line consisting only of ';' in the preamble,
just above the 'Commentary:' section.

,------------------------------------------------------------------------------
| ;;; org.el --- Outline-based notes management and organizer
| 
| ;; Carstens outline-mode for keeping track of everything.
| ;; Copyright (C) 2004-2013 Free Software Foundation, Inc.
| ;;
| ;; Author: Carsten Dominik <carsten at orgmode dot org>
| ;; Maintainer: Bastien Guerry <bzg at gnu dot org>
| ;; Keywords: outlines, hypermedia, calendar, wp
| ;; Homepage: http://orgmode.org
| ;;
| ;; This file is part of GNU Emacs.
| 
| [...]
| 
| ;; You should have received a copy of the GNU General Public License
| ;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
| ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
| ;;
| ;;; Commentary:
| ;;
| 
| [...]
`------------------------------------------------------------------------------

I would prefer in this case not to investigate why this line causes such
problems but rather to consider writing such a line a bug. 

So maybe Bastien could consider this a bug-report and remove this line?
Because otherwise org.el with its more than 23000 lines is just a
perfect example were the use of outline/outshine and navi-mode for file
structuring, high-level (over)views on the file and easy navigation
makes sense.

-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Eli Zaretskii]

> From: Thorsten Jolitz <tjolitz@gmail.com>
> Date: Mon, 15 Apr 2013 10:37:21 +0200
> Cc: Bastien <bzg@gnu.org>
> 
> | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> | ;;
> | ;;; Commentary:
> | ;;
> | 
> | [...]
> `------------------------------------------------------------------------------
> 
> I would prefer in this case not to investigate why this line causes such
> problems but rather to consider writing such a line a bug. 
> 
> So maybe Bastien could consider this a bug-report and remove this line?
> Because otherwise org.el with its more than 23000 lines is just a
> perfect example were the use of outline/outshine and navi-mode for file
> structuring, high-level (over)views on the file and easy navigation
> makes sense.

There are about 60 non-Org *.el files in Emacs which have similar
lines, and about 40 in lisp/org/.  So IMO asking people not to use
that runs a risk of being a quixotic battle.

Re: Modern Conventions for Emacs Lisp files?

[Bastien]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> So maybe Bastien could consider this a bug-report and remove this
> line?

Done, thanks.

-- 
 Bastien

Re: Modern Conventions for Emacs Lisp files?

[Thorsten Jolitz]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Thorsten Jolitz <tjolitz@gmail.com>
>> Date: Mon, 15 Apr 2013 10:37:21 +0200
>> Cc: Bastien <bzg@gnu.org>
>> 
>> | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>> | ;;
>> | ;;; Commentary:
>> | ;;
>> | 
>> | [...]
>> `------------------------------------------------------------------------------
>> 
>> I would prefer in this case not to investigate why this line causes such
>> problems but rather to consider writing such a line a bug. 
>> 
>> So maybe Bastien could consider this a bug-report and remove this line?
>> Because otherwise org.el with its more than 23000 lines is just a
>> perfect example were the use of outline/outshine and navi-mode for file
>> structuring, high-level (over)views on the file and easy navigation
>> makes sense.
>
> There are about 60 non-Org *.el files in Emacs which have similar
> lines, and about 40 in lisp/org/.  So IMO asking people not to use
> that runs a risk of being a quixotic battle.

This line actually shows another disadvantage of using only
comment-chars for signalling headlines (^;;;+ ): people feel free to do
whatever they want with comment-chars, even drawing ascii art or invent
their own section separators or so, which then interferes with the
matching of headlines.

I don't want to take the risk of fighting a quixotic battle. The problem
lies in the following function from outshine.el which is actually copied
from Fabrice Niessen who probably copied/adapted it from Org-mode:

,---------------------------------------------------------------------------------
| (defun outshine-fontify-headlines (outline-regexp)
|   ;; highlight the headings
|   ;; see http://www.gnu.org/software/emacs/manual/html_node/emacs/Font-Lock.html
|   ;; use `M-x customize-apropos-faces' to customize faces
|   ;; to find the corresponding face for each outline level, see
|   ;; `org-faces.el'
| 
|   ;; Added `\n?', after having read the following chunk of code (from org.el):
|   ;; `(,(if org-fontify-whole-heading-line
|   ;;        "^\\(\\**\\)\\(\\* \\)\\(.*\n?\\)"
|   ;;      "^\\(\\**\\)\\(\\* \\)\\(.*\\)")
| 
|   (let ((outshine-fontify-whole-heading-line "") ; "\n?")
|         (heading-1-regexp
|          (concat (substring outline-regexp 0 -1) 
|                  "\\{1\\} \\(.*" outshine-fontify-whole-heading-line "\\)"))
|          [... snip ...]
|         (heading-8-regexp
|          (concat (substring outline-regexp 0 -1)
|                  "\\{8,\\} \\(.*" outshine-fontify-whole-heading-line "\\)")))
|     (font-lock-add-keywords
|      nil
|      `((,heading-1-regexp 1 'outshine-level-1 t)
|         [... snip ...]
|        (,heading-8-regexp 1 'outshine-level-8 t)))))
`---------------------------------------------------------------------------------

Its the '-1' that causes the problem:

,----------------------------------------
| (concat (substring outline-regexp 0 -1)
`----------------------------------------

=> Result: "^;;[;]+\\{1\\} \\(.*\\)"

When I delete it

,----------------------------------------
| (concat (substring outline-regexp 0)
`----------------------------------------

=> Result: "^;;[;]+ \\{1\\} \\(.*\\)"

the problem is gone - but fontification doesn't work anymore. 

This is probably more suited for emacs-help, because I would need some
help on this, but since this thread is already on emacs-devel I ask it
here:

How can I get the best of both worlds in this case (no infinite loops
but working fontification)?

-- 
cheers,
Thorsten

Re: Modern Conventions for Emacs Lisp files?

[Stefan Monnier]

> => Result: "^;;[;]+\\{1\\} \\(.*\\)"

This is OK (tho a bit inefficient) but things like

    "^;;[;]+\\{3\\} \\(.*\\)"
or
    "^;;[;]+\\{8,\\} \\(.*\\)"

are problematic because we use a backtracking regexp-matcher, so when
matching ";;;;;" against "[;]+\\{4\\}", it may consider 4 different ways
to match it.  For ";;;;;;"  that are 4+6=10 ways to match it.
For ";;;;;;;" I already prefer not to count them and for 40 semi-colons
the number of different ways to match it is staggering.

> How can I get the best of both worlds in this case (no infinite loops
> but working fontification)?

Just match outline-regexp and then call outline-level to find the
level number.


        Stefan