[PATCH] Add new option 'org-imenu-flatten'

[PATCH] Add new option 'org-imenu-flatten'

[Morgan Smith]

* lisp/org/org-compat.el: Add definition of 'org-imenu-flatten'.
(org-imenu-get-tree): Use 'org-imenu-flatten'.
---

Hello!

I've been using this patch for a bit and I quite like it.  One would think
there would be an option in imenu itself to flatten trees but that does not
seem to be the case.  I copied the defcustom from 'doc-view-imenu-flatten'.

Thanks,

Morgan

 lisp/org-compat.el | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index 2697342d6..f2112e4e9 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -1333,6 +1333,10 @@ Pass COLUMN and FORCE to `move-to-column'."
 This also applied for speedbar access."
   :type 'integer)
 
+(defcustom org-imenu-flatten nil
+  "Whether to flatten the list of sections in an imenu or show it nested."
+  :type 'boolean)
+
 ;;;; Imenu
 
 (defvar-local org-imenu-markers nil
@@ -1353,7 +1357,8 @@ This also applied for speedbar access."
 			(org-link-display-format (org-get-heading t t t t)))))
 	 (when (and (<= level org-imenu-depth) (org-string-nw-p headline))
 	   (let* ((m (point-marker))
-		  (item (propertize headline 'org-imenu-marker m 'org-imenu t)))
+		  (item (propertize headline 'org-imenu-marker m 'org-imenu t))
+                  (level (if org-imenu-flatten 1 level)))
 	     (push m org-imenu-markers)
 	     (if (>= level last-level)
 		 (push (cons item m) (aref subs level))
-- 
2.41.0

Re: [PATCH] Add new option 'org-imenu-flatten'

[Ihor Radchenko]

Morgan Smith <Morgan.J.Smith@outlook.com> writes:

> * lisp/org/org-compat.el: Add definition of 'org-imenu-flatten'.
> (org-imenu-get-tree): Use 'org-imenu-flatten'.
> ---
>
> Hello!
>
> I've been using this patch for a bit and I quite like it.  One would think
> there would be an option in imenu itself to flatten trees but that does not
> seem to be the case.  I copied the defcustom from 'doc-view-imenu-flatten'.

Have you considered adding a "flatten" option to imenu itself?
That way, you could automatically get the functionality for free
everywhere, not just in Org mode.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add new option 'org-imenu-flatten'

[Morgan Smith]

Ihor Radchenko <yantar92@posteo.net> writes:

>
> Have you considered adding a "flatten" option to imenu itself?
> That way, you could automatically get the functionality for free
> everywhere, not just in Org mode.

I have considered that but gave up with minimal investigation because it
seemed harder then this solution.  It's possible imenu did actually have
this functionality sometime before 1998 (see commit
fe2908be7b09f4c765ebdaf16fe07b0a77f78ba8).

The doc-view imenu-flatten stuff was added 2022-09-28 (see commit
fe002cc8ce38efb256a2a60660ee626c2b2cdf81).  This makes me feel like
maybe that person thought adding it to imenu directly would be hard.

I might at some point investigate doing that but likely not soon.  Also
if that feature was ever added, it would still be compatible with the
patch I sent.  For those reasons, I advocate my patch should still be
applied even though it is clear that it is a sub-optimal solution.

Re: [PATCH] Add new option 'org-imenu-flatten'

[William Denton]

[-- Attachment #1: Type: text/plain, Size: 806 bytes --]

On 8 December 2023, Morgan Smith wrote:

>> Have you considered adding a "flatten" option to imenu itself?
>> That way, you could automatically get the functionality for free
>> everywhere, not just in Org mode.
>
> I have considered that but gave up with minimal investigation because it
> seemed harder then this solution.  It's possible imenu did actually have
> this functionality sometime before 1998 (see commit
> fe2908be7b09f4c765ebdaf16fe07b0a77f78ba8).

One of the things I love about Emacs is the longevity, which leads to someone 
musing about a feature that might have existed 25 (!) years ago.


Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada
CO₂: 420.74 ppm (Mauna Loa Observatory, 2023-12-07)

[FR] Allow flattened imenu index (was: [PATCH] Add new option 'org-imenu-flatten')

[Ihor Radchenko]

Morgan Smith <Morgan.J.Smith@outlook.com> writes:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> Have you considered adding a "flatten" option to imenu itself?
>> That way, you could automatically get the functionality for free
>> everywhere, not just in Org mode.
>
> I have considered that but gave up with minimal investigation because it
> seemed harder then this solution.  It's possible imenu did actually have
> this functionality sometime before 1998 (see commit
> fe2908be7b09f4c765ebdaf16fe07b0a77f78ba8).
>
> The doc-view imenu-flatten stuff was added 2022-09-28 (see commit
> fe002cc8ce38efb256a2a60660ee626c2b2cdf81).  This makes me feel like
> maybe that person thought adding it to imenu directly would be hard.
>
> I might at some point investigate doing that but likely not soon.  Also
> if that feature was ever added, it would still be compatible with the
> patch I sent.  For those reasons, I advocate my patch should still be
> applied even though it is clear that it is a sub-optimal solution.

I'd prefer to ask Emacs upstream first.

We are discussing adding a new feature to Org imenu - an option to
flatten the menu, so that all the nested index entries are displayed at
top level.

This feature is also present in doc-view via `doc-view-imenu-flatten',
and in python.el via `python-imenu-create-flat-index'

I am wondering if it makes more sense to add this "flatten" option
globally into imenu instead.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [FR] Allow flattened imenu index (was: [PATCH] Add new option 'org-imenu-flatten')

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: emacs-orgmode@gnu.org, 58131@debbugs.gnu.org, emacs-devel@gnu.org 
> Date: Sat, 09 Dec 2023 10:57:15 +0000
> 
> Morgan Smith <Morgan.J.Smith@outlook.com> writes:
> 
> > Ihor Radchenko <yantar92@posteo.net> writes:
> >
> >> Have you considered adding a "flatten" option to imenu itself?
> >> That way, you could automatically get the functionality for free
> >> everywhere, not just in Org mode.
> >
> > I have considered that but gave up with minimal investigation because it
> > seemed harder then this solution.  It's possible imenu did actually have
> > this functionality sometime before 1998 (see commit
> > fe2908be7b09f4c765ebdaf16fe07b0a77f78ba8).
> >
> > The doc-view imenu-flatten stuff was added 2022-09-28 (see commit
> > fe002cc8ce38efb256a2a60660ee626c2b2cdf81).  This makes me feel like
> > maybe that person thought adding it to imenu directly would be hard.
> >
> > I might at some point investigate doing that but likely not soon.  Also
> > if that feature was ever added, it would still be compatible with the
> > patch I sent.  For those reasons, I advocate my patch should still be
> > applied even though it is clear that it is a sub-optimal solution.
> 
> I'd prefer to ask Emacs upstream first.
> 
> We are discussing adding a new feature to Org imenu - an option to
> flatten the menu, so that all the nested index entries are displayed at
> top level.
> 
> This feature is also present in doc-view via `doc-view-imenu-flatten',
> and in python.el via `python-imenu-create-flat-index'
> 
> I am wondering if it makes more sense to add this "flatten" option
> globally into imenu instead.

I'm not sure I understand what you are asking.  As we don't seem to
have an active maintainer of imenu on board, more details are needed
to understand the request.  Of course, patches are even more welcome.

Also, should this be a new bug report? the one mentioned in the CC is
already closed and archived.

Thanks.

Re: [FR] Allow flattened imenu index (was: [PATCH] Add new option 'org-imenu-flatten')

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> I am wondering if it makes more sense to add this "flatten" option
>> globally into imenu instead.
>
> I'm not sure I understand what you are asking.  As we don't seem to
> have an active maintainer of imenu on board, more details are needed
> to understand the request.  Of course, patches are even more welcome.

Normally, `imenu' supports nested menus, when users select the target
location in steps, like item3 <RET> -> item3.1 <RET> -> ...

What is proposed is to list all the sub-menus together, as an option, so
that the users can choose, for example, item.3.1 directly, without going
through parent item3.

> Also, should this be a new bug report? the one mentioned in the CC is
> already closed and archived.

Maybe. I was just not sure how I can create a new bug report, while
still replying to Org mode feature request.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [FR] Allow flattened imenu index

[Juri Linkov]

> Normally, `imenu' supports nested menus, when users select the target
> location in steps, like item3 <RET> -> item3.1 <RET> -> ...
>
> What is proposed is to list all the sub-menus together, as an option, so
> that the users can choose, for example, item.3.1 directly, without going
> through parent item3.

Shouldn't each sub-item keep the parent menu item as a prefix?
So a nested menu like

  menu1
    sub-item1
    sub-item2

could be flattened to

  menu1 -> sub-item1
  menu1 -> sub-item2

Or this should be optional?

Re: [FR] Allow flattened imenu index

[João Távora]

On Sat, Dec 9, 2023 at 5:39 PM Juri Linkov <juri@linkov.net> wrote:

>   menu1
>     sub-item1
>     sub-item2
>
> could be flattened to
>
>   menu1 -> sub-item1
>   menu1 -> sub-item2


By the way, this seems to be exactly what the breadcrumb-jump command
in my breadcrumb.el package does.  Goes reasonably well with a flex/fuzzy
completion style.

You can look at it for an implementation idea.  Just be sure to do this
flattening at the presentation level (i.e. M-x imenu), not at the
internal representation level.

It shouldn't be needed here at all, but in case anyone's thinking
about it, please avoid messing with imenu's internal representation of
hierarchies as that structure is relied upon by many extensions (not just
mine, but several others).  Even certain things supported by certain
imenu-presenting frontends (like "special elements") are not supported by
other frontends.  It's a bit of a mess.  The symbols holding/describing
this representation (imenu--index-alist, maybe others) are incorrectly
named '--' but they are most definitely externally visible and used
customization points.

João

Re: [FR] Allow flattened imenu index

[Juri Linkov]

>>   menu1
>>     sub-item1
>>     sub-item2
>>
>> could be flattened to
>>
>>   menu1 -> sub-item1
>>   menu1 -> sub-item2
>
> By the way, this seems to be exactly what the breadcrumb-jump command
> in my breadcrumb.el package does.  Goes reasonably well with a flex/fuzzy
> completion style.

It would be great to have some form of breadcrumb-jump in imenu.el
since it's useful on its own even for someone who doesn't use breadcrumbs.

> You can look at it for an implementation idea.  Just be sure to do this
> flattening at the presentation level (i.e. M-x imenu), not at the
> internal representation level.

Here are some observations while testing on
emacs/test/manual/etags/ruby-src/test.rb.

Both ruby-mode and ruby-ts-mode provide a list that is already flat:

ruby-mode:
  ModuleExample
  ModuleExample#ModuleExample.module_class_method
  ModuleExample#module_instance_method
  ModuleExample::ClassExample
  ModuleExample::ClassExample#+
  ModuleExample::ClassExample#ClassExample.class_method
  ModuleExample::ClassExample#instance_method

ruby-ts-mode:
  ModuleExample
  ModuleExample#module_instance_method
  ModuleExample.module_class_method
  ModuleExample::ClassExample
  ModuleExample::ClassExample#+
  ModuleExample::ClassExample#instance_method
  ModuleExample::ClassExample.class_method

When eglot is enabled then imenu-create-index-function returns a tree.

What is interesting is that breadcrumb-jump already correctly handles both
a flat list and a tree:

a flat list in breadcrumb-jump completions is exactly the same as in 'imenu':
  ModuleExample
  ModuleExample#module_instance_method
  ModuleExample.module_class_method
  ModuleExample::ClassExample
  ModuleExample::ClassExample#+
  ModuleExample::ClassExample#instance_method
  ModuleExample::ClassExample.class_method

a tree from eglot in breadcrumb-jump completions:
  Class > ModuleExample > ClassExample
  Method > ClassExample > class_method
  Method > ModuleExample > module_class_method
  Method > ModuleExample > module_instance_method
  Method > ModuleExample::ClassExample > +
  Method > ModuleExample::ClassExample > instance_method
  Module >  > ModuleExample

is still usable even without special characters like "#".

> It shouldn't be needed here at all, but in case anyone's thinking
> about it, please avoid messing with imenu's internal representation of
> hierarchies as that structure is relied upon by many extensions (not just
> mine, but several others).  Even certain things supported by certain
> imenu-presenting frontends (like "special elements") are not supported by
> other frontends.  It's a bit of a mess.  The symbols holding/describing
> this representation (imenu--index-alist, maybe others) are incorrectly
> named '--' but they are most definitely externally visible and used
> customization points.

Indeed, it's unfortunate that imenu--make-index-alist and imenu--index-alist
are named as internal.

Re: [FR] Allow flattened imenu index

[João Távora]

On Mon, Dec 11, 2023 at 5:21 PM Juri Linkov <juri@linkov.net> wrote:

> > By the way, this seems to be exactly what the breadcrumb-jump command
> > in my breadcrumb.el package does.  Goes reasonably well with a flex/fuzzy
> > completion style.
>
> It would be great to have some form of breadcrumb-jump in imenu.el
> since it's useful on its own even for someone who doesn't use breadcrumbs.

You can copy it over, it  doesn't really need breadcrumbs, though,
it just needs imenu.

The only things I'm seeing in its implementation in a penchant
for a particular text property `breadcrumb-region`, which
has information that imenu's structure didn't have a good
place to place.  But it's not required, as you noticed, I  suppose
it just gives it increased accuracy.

> When eglot is enabled then imenu-create-index-function returns a tree.
>
> What is interesting is that breadcrumb-jump already correctly handles both
> a flat list and a tree:

Yes, that's the point.  Though when using Eglot, the exact tree
structure -- of any -- depends on the language server.

João

Re: [FR] Allow flattened imenu index

[João Távora]

On Mon, Dec 11, 2023 at 5:43 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> > is still usable even without special characters like "#".
>
> Until a class contains both an instance and a class method with the same
> name, I suppose.

If that happens, the LS will have to provide to distinct paths
to these two distinct items and breadcrumb will have two distinct
strings, so no need to invent #-things.

Re: [FR] Allow flattened imenu index

[Ihor Radchenko]

Juri Linkov <juri@linkov.net> writes:

>> It shouldn't be needed here at all, but in case anyone's thinking
>> about it, please avoid messing with imenu's internal representation of
>> hierarchies as that structure is relied upon by many extensions (not just
>> mine, but several others).  Even certain things supported by certain
>> imenu-presenting frontends (like "special elements") are not supported by
>> other frontends.  It's a bit of a mess.  The symbols holding/describing
>> this representation (imenu--index-alist, maybe others) are incorrectly
>> named '--' but they are most definitely externally visible and used
>> customization points.
>
> Indeed, it's unfortunate that imenu--make-index-alist and imenu--index-alist
> are named as internal.

Should they be renamed to `imenu-make-index-alist' and `imenu-index-alist' then?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [FR] Allow flattened imenu index

[João Távora]

On Mon, Dec 11, 2023 at 7:27 PM Ihor Radchenko <yantar92@posteo.net> wrote:

> > Indeed, it's unfortunate that imenu--make-index-alist and imenu--index-alist
> > are named as internal.
>
> Should they be renamed to `imenu-make-index-alist' and `imenu-index-alist' then?

Maybe, but it's much more urgent IMO to review the representation
they describe, since it's all over the place.  Ideally do this
before any rename.

Incidentally, I just remembered why breadcrumb-jump uses the
'breadcrumb-region' text property cookie: it's because without it
the current imenu tree structure has no way to for backends to
store locations to non-leaf nodes, which are often among the places
you want to jump to.

text-properties + standard cons trees + "special elements" is a
bit of a ugly mess, but OTOH it works and is backwards compatible
to existing frontends and backends.

So one of the possibilities that whoever conducts this review
may consider is an 'imenu-region' text property similar to
'breadcrumb-region', describe it imenu--index-alist, and
make M-x imenu take advantage of it.

João

Re: [FR] Allow flattened imenu index

[João Távora]

On Mon, Dec 11, 2023 at 7:24 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> It's no invention, it's docstring syntax for reference to a method.

In Ruby right?

> But if the LS will produce distinct strings, good.

All imenu backends, at least all the ones I've seen, produce
trees, not strings.  If you collect all the paths from the root to
all the nodes into lists and make strings thereof, the resulting
set will always be a distinct strings.  So I don't understand the
problem you were surfacing: any particular imenu backend in mind?

João

Re: [FR] Allow flattened imenu index

[João Távora]

On Mon, Dec 11, 2023 at 11:23 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> >> But if the LS will produce distinct strings, good.
> > All imenu backends, at least all the ones I've seen, produce
> > trees, not strings.
>
> There are exceptions, like the previously mentioned one.

What is the imenu backend that produces strings?  A list of
strings is still a tree, albeit very flat.  Some LS's produce
such things.  Are there imenu backends that produce lists of
strings with duplicates?   Where, when, how is that not a
problem with the existing imenu UI already, and have we heard
of it all these years?

> > If you collect all the paths from the root to
> > all the nodes into lists and make strings thereof, the resulting
> > set will always be a distinct strings.  So I don't understand the
> > problem you were surfacing: any particular imenu backend in mind?
>
> Do you have an example of such tree produced by Eglot when a class
> contains an instance and a singleton method with the same name?

No.  I thought you did, else why would you bring it up?  Grab a
Ruby LS and try it.  Or maybe say what the ruby-mode imenu backend
does? No rush, but as usual I think there's no point discussing
odd conjectures without something palpable in front.

Thanks,
João

Re: [FR] Allow flattened imenu index

[João Távora]

On Mon, Dec 11, 2023 at 11:41 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 12/12/2023 01:35, João Távora wrote:
> > Or maybe say what the ruby-mode imenu backend
> > does?
>
> Juri just posted a long example of what ruby-mode does (and ruby-ts-mode
> too) in his last message in this thread.

OK, and doesn't it answer _your_ question?  With Eglot

Method > ModuleExample::ClassExample > instance_method
Method > ClassExample > class_method

If both methods were named "foo" no problem, right?
When run without Eglot, also no problem, no duplicate strings,
right?

??

Re: [FR] Allow flattened imenu index

[João Távora]

On Mon, Dec 11, 2023 at 11:54 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 12/12/2023 01:48, João Távora wrote:
> > On Mon, Dec 11, 2023 at 11:41 PM Dmitry Gutov<dmitry@gutov.dev>  wrote:
> >> On 12/12/2023 01:35, João Távora wrote:
> >>> Or maybe say what the ruby-mode imenu backend
> >>> does?
> >> Juri just posted a long example of what ruby-mode does (and ruby-ts-mode
> >> too) in his last message in this thread.
> > OK, and doesn't it answer_your_  question?  With Eglot
> >
> > Method > ModuleExample::ClassExample > instance_method
> > Method > ClassExample > class_method
> >
> > If both methods were named "foo" no problem, right?
>
> But this doesn't look right: if 'class_method' is in a class called
> ModuleExample::ClassExample, shouldn't its entry look like
>
>    Method > ModuleExample::ClassExample > class_method
>
> as well?

Take it up with the LS.  I didn't write it.

> And then, if both 'instance_method' and 'class_method' actually are
> called 'foo', then we'd have a problem.

Bizarrely, you wouldn't have a tree to start with, which probably
explains why the devs of the LS Juri is using opted for that
layout.

> This entry from his message also looks odd:
>
>    Module >  > ModuleExample

If you can reproduce it, show the repro.

João

Re: [FR] Allow flattened imenu index

[Spencer Baugh]

Ihor Radchenko <yantar92@posteo.net> writes:
> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> I am wondering if it makes more sense to add this "flatten" option
>>> globally into imenu instead.
>>
>> I'm not sure I understand what you are asking.  As we don't seem to
>> have an active maintainer of imenu on board, more details are needed
>> to understand the request.  Of course, patches are even more welcome.
>
> Normally, `imenu' supports nested menus, when users select the target
> location in steps, like item3 <RET> -> item3.1 <RET> -> ...
>
> What is proposed is to list all the sub-menus together, as an option, so
> that the users can choose, for example, item.3.1 directly, without going
> through parent item3.

I would love to have this feature.  I have wanted this for OCaml, where
the default imenu index produced by the "merlin" tool is annoyingly
deeply nested, which makes it awkward to use.

I have an alternative proposal though, which might be better: We could
enhance the imenu completion table to understand completion boundaries.
Then the imenu hierarchy could be completed over in a single
completing-read instead of a sequence of multiple completing-reads.  It
would work the same way as read-file-name.

So, for example, if the completion boundary was /, and we had a hierarchy
containing these elements:
aaa/bbb/ddd
aaa/bbb/eee
aaa/ccc/eee
aaa/ccc/fff

You could choose aaa/bbb/ddd with this sequence of minibuffer contents:
(input in [brackets], point at |)

[a]
a|
[<TAB>]
aaa/| ; completion now shows bbb and ccc as options
[b]
aaa/b/d|
[TAB]
aaa/bbb/ddd|
[RET]

Alternatively, you could choose aaa/bbb/eee with this sequence:

[*/*/e]
*/*/e|
[TAB]
aaa/|/eee ; completion now shows bbb and ccc as options, point is at |
[b]
aaa/b|/eee
[TAB]
aaa/bbb/eee|
[RET]

To be very clear, these completion features already exist (and are
enabled by default!), all we would need to do is enhance the imenu
completion table to understand completion boundaries, and then imenu
would have access to these features too.

This new completion table could be turned on by default, since it would
be strictly more powerful than the current imenu UI.

Plus, if some alternative completion UI wanted to flatten the hierarchy
anyway, this more advanced imenu completion table would allow that.