New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library)

New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library)

[Ihor Radchenko]

Emanuel Berg <incal@dataswamp.org> writes:

> Ihor Radchenko wrote:
>
>> Yes, of course. I was hoping that Emanual (who is apparently
>> interested in this topic, or at least in improving Elisp)
>> can be the one.
> ...
> You need to reach out to the young guys studying math at
> universities today, one thinks all they do all day is math but
> almost _all_ of them are into computers as well, how many will
> use Emacs I don't know but this is the person you want for
> this job :)

Hmm. This reminds me of Google Summer of Code.
Eli, have you considered using GSC to get volunteers for implementing
some moderately difficult Emacs features? Since GSC provides financial
incentive, it may be possible to get student volunteers.

Of course, someone still needs to mentor the students, but that's less
effort compared to actually writing all the code.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library)

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: emacs-devel@gnu.org
> Date: Mon, 12 Aug 2024 06:28:21 +0000
> 
> Hmm. This reminds me of Google Summer of Code.
> Eli, have you considered using GSC to get volunteers for implementing
> some moderately difficult Emacs features? Since GSC provides financial
> incentive, it may be possible to get student volunteers.
> 
> Of course, someone still needs to mentor the students, but that's less
> effort compared to actually writing all the code.

People, including the other co-maintainers, are welcome to pursue
this, but I personally have no free time or resources to do that.

I can only say that it would be harder for a casual contributor to do
a GSoC-style job for Emacs than for other projects, because Emacs has
a lot of specialized needs and requirements that take time to learn.
But maybe this is not an obstacle.

Re: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library)

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Eli, have you considered using GSC to get volunteers for implementing
>> some moderately difficult Emacs features? Since GSC provides financial
>> incentive, it may be possible to get student volunteers.
>> 
>> Of course, someone still needs to mentor the students, but that's less
>> effort compared to actually writing all the code.

> I can only say that it would be harder for a casual contributor to do
> a GSoC-style job for Emacs than for other projects, because Emacs has
> a lot of specialized needs and requirements that take time to learn.
> But maybe this is not an obstacle.

May you elaborate?
The interested students will likely also be at least casual Emacs users.
So, some degree of familiarity is expected.
Other than that, how is Emacs dramatically different from working with
any other large codebase?

> People, including the other co-maintainers, are welcome to pursue
> this, but I personally have no free time or resources to do that.

Adding Stefan and Andrea to CC.

To add more details, FSF/GNU qualifies as a valid mentor organization
for GSoC. Org mode project even participated in it in the past:
https://orgmode.org/worg/archive/gsoc2012/orgmode-gsoc2012-ideas.html
AFAIR, there was a request from GNU this spring to submit proposals.

So, what I am talking about is not a theoretical idea. It can be done.

What is needed is a formulation of projects/features that are desired.
Mentors do not have to be maintainers. Experienced Emacs contributors
can be the mentors (also, mentoring a student can be a good addition to
some types of CVs).

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library)

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: emacs-devel@gnu.org, Stefan Kangas <stefankangas@gmail.com>, Andrea
>  Corallo <acorallo@gnu.org>
> Date: Mon, 12 Aug 2024 19:17:05 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Eli, have you considered using GSC to get volunteers for implementing
> >> some moderately difficult Emacs features? Since GSC provides financial
> >> incentive, it may be possible to get student volunteers.
> >> 
> >> Of course, someone still needs to mentor the students, but that's less
> >> effort compared to actually writing all the code.
> 
> > I can only say that it would be harder for a casual contributor to do
> > a GSoC-style job for Emacs than for other projects, because Emacs has
> > a lot of specialized needs and requirements that take time to learn.
> > But maybe this is not an obstacle.
> 
> May you elaborate?

Sorry, I don't know what else to say.  I think the above is clear
enough.  Granted, it's my opinion, and others could disagree, but I
didn't write that by accident, I wrote that with full intent.

> The interested students will likely also be at least casual Emacs users.
> So, some degree of familiarity is expected.

User-level familiarity doesn't help in these matters, IME.

> Other than that, how is Emacs dramatically different from working with
> any other large codebase?

In a nutshell, Emacs is much larger than most other projects, and its
features are much less localized than those of other large projects.

> So, what I am talking about is not a theoretical idea. It can be done.

I didn't say it couldn't be done.

> What is needed is a formulation of projects/features that are desired.
> Mentors do not have to be maintainers. Experienced Emacs contributors
> can be the mentors (also, mentoring a student can be a good addition to
> some types of CVs).

We have etc/TODO which could be used as a source of ideas.

Re: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library)

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> The interested students will likely also be at least casual Emacs users.
>> So, some degree of familiarity is expected.
>
> User-level familiarity doesn't help in these matters, IME.

>> Other than that, how is Emacs dramatically different from working with
>> any other large codebase?
>
> In a nutshell, Emacs is much larger than most other projects, and its
> features are much less localized than those of other large projects.

But not every feature, right? Some parts are easier to hack than others.
I imagine that many Elisp parts are somewhat easier compared to C code.

>> What is needed is a formulation of projects/features that are desired.
>> Mentors do not have to be maintainers. Experienced Emacs contributors
>> can be the mentors (also, mentoring a student can be a good addition to
>> some types of CVs).
>
> We have etc/TODO which could be used as a source of ideas.

Are there any specific todo items there that you view as more suitable
for people with limited experience in Emacs codebase?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library)

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: emacs-devel@gnu.org, stefankangas@gmail.com, acorallo@gnu.org
> Date: Sun, 18 Aug 2024 11:09:54 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> The interested students will likely also be at least casual Emacs users.
> >> So, some degree of familiarity is expected.
> >
> > User-level familiarity doesn't help in these matters, IME.
> 
> >> Other than that, how is Emacs dramatically different from working with
> >> any other large codebase?
> >
> > In a nutshell, Emacs is much larger than most other projects, and its
> > features are much less localized than those of other large projects.
> 
> But not every feature, right? Some parts are easier to hack than others.
> I imagine that many Elisp parts are somewhat easier compared to C code.

Only if you are lucky, and only if you are familiar with the code in
question.

> >> What is needed is a formulation of projects/features that are desired.
> >> Mentors do not have to be maintainers. Experienced Emacs contributors
> >> can be the mentors (also, mentoring a student can be a good addition to
> >> some types of CVs).
> >
> > We have etc/TODO which could be used as a source of ideas.
> 
> Are there any specific todo items there that you view as more suitable
> for people with limited experience in Emacs codebase?

Possibly, but someone needs to look at them with this in mind.

Re: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library)

[Stefan Kangas]

Ihor Radchenko <yantar92@posteo.net> writes:

>> We have etc/TODO which could be used as a source of ideas.
>
> Are there any specific todo items there that you view as more suitable
> for people with limited experience in Emacs codebase?

I agree with many of the reservations Eli has, but on the other hand
this depends highly on what the projects will be.  Perhaps this could be
a good way to attract new contributors, some of which will be found in
the already existing Emacs community (parts of which do not interact
much with emacs-devel).

So I'm basically all in favor of GSoC, but someone needs to do the work
to get the ball rolling, for example with regards to suitable proposals.
If someone were to propose a list of projects, then it would be easier
for others to comment on it.  This could potentially also be used to
expand the text that is currently in etc/TODO, to improve that file.
If that's possible, the effort will never be wasted.

Another thing that would be useful in this regard is if people could
indicate their interest in mentoring students, even tentatively.
As for me, I _might_ have the time to act as mentor, but that is highly
tentative as I didn't consider or think about that very closely.  It
would also depend on the project idea, of course.

Re: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library)

[Ihor Radchenko]

Stefan Kangas <stefankangas@gmail.com> writes:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>>> We have etc/TODO which could be used as a source of ideas.
>>
>> Are there any specific todo items there that you view as more suitable
>> for people with limited experience in Emacs codebase?
>
> I agree with many of the reservations Eli has, but on the other hand
> this depends highly on what the projects will be.  Perhaps this could be
> a good way to attract new contributors, some of which will be found in
> the already existing Emacs community (parts of which do not interact
> much with emacs-devel).

Yup. I want to have some kind of shortlist of tasks that the
maintainers familiar with the code can suggest as student projects.

Then, we can eventually go out to reddit/mastodon/reach Sacha
Chua/Emacsconf to ask the community.

> So I'm basically all in favor of GSoC, but someone needs to do the work
> to get the ball rolling, for example with regards to suitable proposals.
> If someone were to propose a list of projects, then it would be easier
> for others to comment on it.  This could potentially also be used to
> expand the text that is currently in etc/TODO, to improve that file.
> If that's possible, the effort will never be wasted.

+1

> Another thing that would be useful in this regard is if people could
> indicate their interest in mentoring students, even tentatively.
> As for me, I _might_ have the time to act as mentor, but that is highly
> tentative as I didn't consider or think about that very closely.  It
> would also depend on the project idea, of course.

I am also considering mentoring, but that will be for Org mode.

I think that it is a good idea to shortlist things from TODO file before
asking potential mentors. Simply to prioritize projects that are deemed
important by the maintainers.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library)

[arthur miller]

[-- Attachment #1: Type: text/plain, Size: 1835 bytes --]

>Eli Zaretskii <eliz@gnu.org> writes:
>
>>> The interested students will likely also be at least casual Emacs users.
>>> So, some degree of familiarity is expected.
>>
>> User-level familiarity doesn't help in these matters, IME.
>
>>> Other than that, how is Emacs dramatically different from working with
>>> any other large codebase?
>>
>> In a nutshell, Emacs is much larger than most other projects, and its
>> features are much less localized than those of other large projects.
>
>But not every feature, right? Some parts are easier to hack than others.
>I imagine that many Elisp parts are somewhat easier compared to C code.
>
>>> What is needed is a formulation of projects/features that are desired.
>>> Mentors do not have to be maintainers. Experienced Emacs contributors
>>> can be the mentors (also, mentoring a student can be a good addition to
>>> some types of CVs).
>>
>> We have etc/TODO which could be used as a source of ideas.
>
>Are there any specific todo items there that you view as more suitable
>for people with limited experience in Emacs codebase?

I am not sure if this is something, I hope you don't mind me asking, but could a
work to modularize Org, be an appropriate subject?

For example turn displaying pretty text (bold, italics etc), pretty links,
tables, dates, and perhaps some other stuff into, from Org-mode, independent,
minor modes that could be used in other parts of Emacs and more independently of
Org mode. I think both Org-mode and Hyperbole, and perhaps some other libraries
(button.el, help-mode, info), could use some minor mode that works with
links. For us users, it would mean less cruft loaded into Emacs, if those big
players could share some code.

I haven't done much research on this, just something I had in my head for a long
time.

[-- Attachment #2: Type: text/html, Size: 6330 bytes --]

Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[Ihor Radchenko]

arthur miller <arthur.miller@live.com> writes:

>> ...
>>Are there any specific todo items there that you view as more suitable
>>for people with limited experience in Emacs codebase?
>
> I am not sure if this is something, I hope you don't mind me asking, but could a
> work to modularize Org, be an appropriate subject?

Yes, it can be.

RMS previously asked us to do exactly this:
https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org

Also, see previous discussion on re-integrating org-capture into remember.el:
https://yhetil.org/emacs-devel/87bka7c2e0.fsf@localhost
Similar for org-protocol.el.

> For example turn displaying pretty text (bold, italics etc), pretty links,
> tables, dates, and perhaps some other stuff into, from Org-mode, independent,
> minor modes that could be used in other parts of Emacs and more independently of
> Org mode. I think both Org-mode and Hyperbole, and perhaps some other libraries
> (button.el, help-mode, info), could use some minor mode that works with
> links. For us users, it would mean less cruft loaded into Emacs, if those big
> players could share some code.
>
> I haven't done much research on this, just something I had in my head for a long
> time.

I would like to hear in more details what you have in mind. As is, your
proposal appears to be for multiple projects. Or am I mistaken?

For links in particular, Hyperbole already implements the described
minor mode, or what am I missing?

For tables, dates, and other markup, it is not clear what you want to
achieve. Do you want Org markup to be displayed in non-Org buffers?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[arthur miller]

[-- Attachment #1: Type: text/plain, Size: 6662 bytes --]

>>> ...
>>>Are there any specific todo items there that you view as more suitable
>>>for people with limited experience in Emacs codebase?
>>
>> I am not sure if this is something, I hope you don't mind me asking, but
>> could a
>> work to modularize Org, be an appropriate subject?
>
>Yes, it can be.
>
>RMS previously asked us to do exactly this:
>https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org
>
>Also, see previous discussion on re-integrating org-capture into remember.el:
>https://yhetil.org/emacs-devel/87bka7c2e0.fsf@localhost
>Similar for org-protocol.el.
>
>> For example turn displaying pretty text (bold, italics etc), pretty links,
>> tables, dates, and perhaps some other stuff into, from Org-mode, independent,
>> minor modes that could be used in other parts of Emacs and more independently
>> of
>> Org mode. I think both Org-mode and Hyperbole, and perhaps some other
>> libraries
>> (button.el, help-mode, info), could use some minor mode that works with
>> links. For us users, it would mean less cruft loaded into Emacs, if those big
>> players could share some code.
>>
>> I haven't done much research on this, just something I had in my head for a
>> long
>> time.
>
>I would like to hear in more details what you have in mind. As is, your
>proposal appears to be for multiple projects. Or am I mistaken?
>
>For links in particular, Hyperbole already implements the described
>minor mode, or what am I missing?
>
>For tables, dates, and other markup, it is not clear what you want to
>achieve.

To be honest I am not sure myself :)

>         Do you want Org markup to be displayed in non-Org buffers?

Well, part of it. Or more to make some parts of org-markup configurable, and
usable as minor modes so they can be easier used outside of org-mode.

For example about links, there could be a mode "text-link-mode" or
"pretty-links-mode" or something, that understands what a link description is,
and what a link itself is. The minor mode would have some mean to parse
description and link parts, and when on it would do what org-toggle-link-display
does. For example org uses angle brackets for link desc and url, whereas
markdown uses angle brackets and parenthesis. Thus link-mode could/should be
enough customizable so that modes could be clients of this minor mode, as well
as for user to be able to setup a regex or set a function that recognizes some
custom syntax for descriptions and links. Also a minor mode can come with a key
map and some actions, for example to follow link, to insert a link etc. I think
of org-links, but a bit more generalized and usable without org-mode
itself. Org-mode could use those under the hood.

Similarly for italics, bolds, underlines, etc. Those could be slightly
generalized and taken out into minor mode. Clients could setup their own
start/end markers and use them to enrich the text on the display instead of
perhaps defining own faces and regexes. I don't know how useful and desirable it
might be in other modes, perhaps in comments in programming languages, or
similar. Just a thought.

org-timestamp interactiveity could be usable elsewhere than just in org
mode. For example I can insert  org-timestamp in any mode, but it does work to
use C-left/right to change the date. It could be refactored into small tiny
minor mode timestamp-mode or something, that comes with tis mode map and enable
this interactive stuff.

ascii tables or org tables started as its own mode but got consumed by org. They
are still usable outside of org mode. I can create table with org-table-create
and I can align table with org-table-align, but by default I don't have this
functionality bound in some keymap if I am not in org-mode. Perhaps I am just
not familiar with it. But this could be a minor mode also.

I guess org-capture is a candidate, and probably some others.

By refactoring out those things, org-mode loose nothing, since everything is
still available in Emacs, and everything could be still used in org-mode under
the hoos, but it would be easier to use those things in other major modes. It
might also be easier to work with org-mode if it is less monolithic.

I am sorry if it is too scatchy, and uninformed, it is just some thoughts I had,
but I was never really motivated to do those things myself.
________________________________
Från: Ihor Radchenko <yantar92@posteo.net>
Skickat: den 20 augusti 2024 19:40
Till: arthur miller <arthur.miller@live.com>
Kopia: emacs-devel@gnu.org <emacs-devel@gnu.org>; emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Ämne: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

arthur miller <arthur.miller@live.com> writes:

>> ...
>>Are there any specific todo items there that you view as more suitable
>>for people with limited experience in Emacs codebase?
>
> I am not sure if this is something, I hope you don't mind me asking, but could a
> work to modularize Org, be an appropriate subject?

Yes, it can be.

RMS previously asked us to do exactly this:
https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org

Also, see previous discussion on re-integrating org-capture into remember.el:
https://yhetil.org/emacs-devel/87bka7c2e0.fsf@localhost
Similar for org-protocol.el.

> For example turn displaying pretty text (bold, italics etc), pretty links,
> tables, dates, and perhaps some other stuff into, from Org-mode, independent,
> minor modes that could be used in other parts of Emacs and more independently of
> Org mode. I think both Org-mode and Hyperbole, and perhaps some other libraries
> (button.el, help-mode, info), could use some minor mode that works with
> links. For us users, it would mean less cruft loaded into Emacs, if those big
> players could share some code.
>
> I haven't done much research on this, just something I had in my head for a long
> time.

I would like to hear in more details what you have in mind. As is, your
proposal appears to be for multiple projects. Or am I mistaken?

For links in particular, Hyperbole already implements the described
minor mode, or what am I missing?

For tables, dates, and other markup, it is not clear what you want to
achieve. Do you want Org markup to be displayed in non-Org buffers?

--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[-- Attachment #2: Type: text/html, Size: 17302 bytes --]

Re: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[Ihor Radchenko]

arthur miller <arthur.miller@live.com> writes:

>>         Do you want Org markup to be displayed in non-Org buffers?
>
> Well, part of it. Or more to make some parts of org-markup configurable, and
> usable as minor modes so they can be easier used outside of org-mode.

This is not the direction Org mode project is going. We do the reverse -
Org markup is getting more stable, with the aim to simplify developing
third-party parsers and eventually registering Org syntax in IANA MIME db.

However, stable Org markup does not mean that we cannot implement Org
mode features outside Org mode. Another general direction we are going to
is using parser APIs across Org mode. The idea is to encapsulate the
fine details of Org markup into the parser, only leaving some very basic
structures like recursiveness of markup objects exposed. In theory, one
may eventually just replace the parser with, say, Markdown parser (with
appropriate API), and get Org mode features in Markdown (well... except
that Markdown has fewer kinds of syntax and certain things do not exist
there)

> For example about links, there could be a mode "text-link-mode" or
> "pretty-links-mode" or something, that understands what a link description is,
> and what a link itself is. The minor mode would have some mean to parse
> description and link parts, and when on it would do what org-toggle-link-display
> does. For example org uses angle brackets for link desc and url, whereas
> markdown uses angle brackets and parenthesis. Thus link-mode could/should be
> enough customizable so that modes could be clients of this minor mode, as well
> as for user to be able to setup a regex or set a function that recognizes some
> custom syntax for descriptions and links. Also a minor mode can come with a key
> map and some actions, for example to follow link, to insert a link etc. I think
> of org-links, but a bit more generalized and usable without org-mode
> itself. Org-mode could use those under the hood.

I am not sure if many places (except Markdown) have a notion of link +
description. And the rest is supported by, say, Hyperbole.

What might be more interesting is generalizing Org previews: Org can
preview LaTeX and images, but we can think of it more generally - any
kind of text object might have alternative (possibly image)
representation. See https://list.orgmode.org/orgmode/87edbhljr7.fsf@hyperspace/

> Similarly for italics, bolds, underlines, etc. Those could be slightly
> generalized and taken out into minor mode. Clients could setup their own
> start/end markers and use them to enrich the text on the display instead of
> perhaps defining own faces and regexes. I don't know how useful and desirable it
> might be in other modes, perhaps in comments in programming languages, or
> similar. Just a thought.

You are describing font-lock here. Users can already add custom
fontification in buffers by supplying regexps + face to be used for
specific text fragments. What is the novelty?

> org-timestamp interactiveity could be usable elsewhere than just in org
> mode. For example I can insert  org-timestamp in any mode, but it does work to
> use C-left/right to change the date. It could be refactored into small tiny
> minor mode timestamp-mode or something, that comes with tis mode map and enable
> this interactive stuff.

This idea does sound useful.

> ascii tables or org tables started as its own mode but got consumed by org. They
> are still usable outside of org mode. I can create table with org-table-create
> and I can align table with org-table-align, but by default I don't have this
> functionality bound in some keymap if I am not in org-mode. Perhaps I am just
> not familiar with it. But this could be a minor mode also.

See orgtbl-mode.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Sv: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[arthur miller]

[-- Attachment #1: Type: text/plain, Size: 10007 bytes --]

>>>         Do you want Org markup to be displayed in non-Org buffers?
>>
>> Well, part of it. Or more to make some parts of org-markup configurable, and
>> usable as minor modes so they can be easier used outside of org-mode.
>
>This is not the direction Org mode project is going. We do the reverse -
>Org markup is getting more stable, with the aim to simplify developing
>third-party parsers and eventually registering Org syntax in IANA MIME db.

I am not subscribed to mailing lists due to lots of trafic going on there, so I
am not overly introduced in directions in which org goes, so forgive me if I am
not so familiar with the latest features.

>However, stable Org markup does not mean that we cannot implement Org
>mode features outside Org mode. Another general direction we are going to
>is using parser APIs across Org mode. The idea is to encapsulate the
>fine details of Org markup into the parser, only leaving some very basic
>structures like recursiveness of markup objects exposed. In theory, one
>may eventually just replace the parser with, say, Markdown parser (with
>appropriate API), and get Org mode features in Markdown (well... except
>that Markdown has fewer kinds of syntax and certain things do not exist
>there)

Well, yes something like that; make features available outside of org-mode, and
make org-mode use those features instead of having everything in a monolithical
library.

>> For example about links, there could be a mode "text-link-mode" or
>> "pretty-links-mode" or something, that understands what a link description is,
>> and what a link itself is. The minor mode would have some mean to parse
>> description and link parts, and when on it would do what
>> org-toggle-link-display
>> does. For example org uses angle brackets for link desc and url, whereas
>> markdown uses angle brackets and parenthesis. Thus link-mode could/should be
>> enough customizable so that modes could be clients of this minor mode, as well
>> as for user to be able to setup a regex or set a function that recognizes some
>> custom syntax for descriptions and links. Also a minor mode can come with a
>> key
>> map and some actions, for example to follow link, to insert a link etc. I
>> think
>> of org-links, but a bit more generalized and usable without org-mode
>> itself. Org-mode could use those under the hood.
>
>I am not sure if many places (except Markdown) have a notion of link +
>description. And the rest is supported by, say, Hyperbole.

The idea was that if such a minor mode has a list of regexes it recognizes,
neither org-, md- or hyperbole would need to keep their code for this, but could
relly on users to just (desc-links-mode +1). If I had this as global minor mode,
I could also enable it while writing mail and use say md syntax and have pretty
links, which admittedly would come out ugly on the other end, but that is an
agreement between the reader and the writer. In other words org links or md
links would be available in any format and up to users where they want to use them.

>What might be more interesting is generalizing Org previews: Org can
>preview LaTeX and images, but we can think of it more generally - any
>kind of text object might have alternative (possibly image)
>representation. See https://list.orgmode.org/orgmode/87edbhljr7.fsf@hyperspace/
>
>> Similarly for italics, bolds, underlines, etc. Those could be slightly
>> generalized and taken out into minor mode. Clients could setup their own
>> start/end markers and use them to enrich the text on the display instead of
>> perhaps defining own faces and regexes. I don't know how useful and desirable
>> it
>> might be in other modes, perhaps in comments in programming languages, or
>> similar. Just a thought.
>
>You are describing font-lock here. Users can already add custom
>fontification in buffers by supplying regexps + face to be used for
>specific text fragments. What is the novelty?

Not a novelty at all :). The same as for links. A user could (rich-text-mode
+1), and have org-markup (or their own) available in any mode of their choice. I
had font-lock and thingatpt in my mind, but I didn't want to talk about the
implementation. After all, org uses font-lock under the hood for this anyway (or
am I too unfamiliar now? :)). It is just a convenience. In all of this cases, I
am thinking of those small minor modes be a tiny small "frameworks", where a
user can setup like very few regexes, or whatever is needed for a parser, and
have a "customized" syntax instead of going on through writing their own minor
mode.

>> org-timestamp interactiveity could be usable elsewhere than just in org
>> mode. For example I can insert  org-timestamp in any mode, but it does work to
>> use C-left/right to change the date. It could be refactored into small tiny
>> minor mode timestamp-mode or something, that comes with tis mode map and
>> enable
>> this interactive stuff.
>
>This idea does sound useful.
>
>> ascii tables or org tables started as its own mode but got consumed by org.
>> They
>> are still usable outside of org mode. I can create table with org-table-create
>> and I can align table with org-table-align, but by default I don't have this
>> functionality bound in some keymap if I am not in org-mode. Perhaps I am just
>> not familiar with it. But this could be a minor mode also.
>
>See orgtbl-mode.

I see. Thanks. Something like that for other features.
________________________________
Från: Ihor Radchenko <yantar92@posteo.net>
Skickat: den 22 augusti 2024 14:23
Till: arthur miller <arthur.miller@live.com>
Kopia: emacs-devel@gnu.org <emacs-devel@gnu.org>; emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Ämne: Re: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

arthur miller <arthur.miller@live.com> writes:

>>         Do you want Org markup to be displayed in non-Org buffers?
>
> Well, part of it. Or more to make some parts of org-markup configurable, and
> usable as minor modes so they can be easier used outside of org-mode.

This is not the direction Org mode project is going. We do the reverse -
Org markup is getting more stable, with the aim to simplify developing
third-party parsers and eventually registering Org syntax in IANA MIME db.

However, stable Org markup does not mean that we cannot implement Org
mode features outside Org mode. Another general direction we are going to
is using parser APIs across Org mode. The idea is to encapsulate the
fine details of Org markup into the parser, only leaving some very basic
structures like recursiveness of markup objects exposed. In theory, one
may eventually just replace the parser with, say, Markdown parser (with
appropriate API), and get Org mode features in Markdown (well... except
that Markdown has fewer kinds of syntax and certain things do not exist
there)

> For example about links, there could be a mode "text-link-mode" or
> "pretty-links-mode" or something, that understands what a link description is,
> and what a link itself is. The minor mode would have some mean to parse
> description and link parts, and when on it would do what org-toggle-link-display
> does. For example org uses angle brackets for link desc and url, whereas
> markdown uses angle brackets and parenthesis. Thus link-mode could/should be
> enough customizable so that modes could be clients of this minor mode, as well
> as for user to be able to setup a regex or set a function that recognizes some
> custom syntax for descriptions and links. Also a minor mode can come with a key
> map and some actions, for example to follow link, to insert a link etc. I think
> of org-links, but a bit more generalized and usable without org-mode
> itself. Org-mode could use those under the hood.

I am not sure if many places (except Markdown) have a notion of link +
description. And the rest is supported by, say, Hyperbole.

What might be more interesting is generalizing Org previews: Org can
preview LaTeX and images, but we can think of it more generally - any
kind of text object might have alternative (possibly image)
representation. See https://list.orgmode.org/orgmode/87edbhljr7.fsf@hyperspace/

> Similarly for italics, bolds, underlines, etc. Those could be slightly
> generalized and taken out into minor mode. Clients could setup their own
> start/end markers and use them to enrich the text on the display instead of
> perhaps defining own faces and regexes. I don't know how useful and desirable it
> might be in other modes, perhaps in comments in programming languages, or
> similar. Just a thought.

You are describing font-lock here. Users can already add custom
fontification in buffers by supplying regexps + face to be used for
specific text fragments. What is the novelty?

> org-timestamp interactiveity could be usable elsewhere than just in org
> mode. For example I can insert  org-timestamp in any mode, but it does work to
> use C-left/right to change the date. It could be refactored into small tiny
> minor mode timestamp-mode or something, that comes with tis mode map and enable
> this interactive stuff.

This idea does sound useful.

> ascii tables or org tables started as its own mode but got consumed by org. They
> are still usable outside of org mode. I can create table with org-table-create
> and I can align table with org-table-align, but by default I don't have this
> functionality bound in some keymap if I am not in org-mode. Perhaps I am just
> not familiar with it. But this could be a minor mode also.

See orgtbl-mode.

--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[-- Attachment #2: Type: text/html, Size: 22272 bytes --]

Re: Sv: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[Ihor Radchenko]

arthur miller <arthur.miller@live.com> writes:

>>> Well, part of it. Or more to make some parts of org-markup configurable, and
>>> usable as minor modes so they can be easier used outside of org-mode.
>>
>>This is not the direction Org mode project is going. We do the reverse -
>>Org markup is getting more stable, with the aim to simplify developing
>>third-party parsers and eventually registering Org syntax in IANA MIME db.
>
> I am not subscribed to mailing lists due to lots of trafic going on there, so I
> am not overly introduced in directions in which org goes, so forgive me if I am
> not so familiar with the latest features.

Sorry, I did not intend my comment to sound personal. I just explained
why configurable markup is not something I see as a good idea.

> Well, yes something like that; make features available outside of org-mode, and
> make org-mode use those features instead of having everything in a monolithical
> library.
> ...

To conclude, we may use these ideas for GSoC projects, but maybe
separately. I doubt that "modularizing" in general would be a reasonable
task to finish within summer.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Sv: Sv: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[arthur miller]

[-- Attachment #1: Type: text/plain, Size: 2859 bytes --]

>arthur miller <arthur.miller@live.com> writes:
>
>>>> Well, part of it. Or more to make some parts of org-markup configurable, and
>>>> usable as minor modes so they can be easier used outside of org-mode.
>>>
>>>This is not the direction Org mode project is going. We do the reverse -
>>>Org markup is getting more stable, with the aim to simplify developing
>>>third-party parsers and eventually registering Org syntax in IANA MIME db.
>>
>> I am not subscribed to mailing lists due to lots of trafic going on there, so
>> I
>> am not overly introduced in directions in which org goes, so forgive me if I
>> am
>> not so familiar with the latest features.
>
>Sorry, I did not intend my comment to sound personal. I just explained
>why configurable markup is not something I see as a good idea.

No worries; I didn't took it personally at all; I was just letting you
know I am not very up-to date with latest development, and where Org is
going, if I suggest something that is diagonally opposite to Org development.
________________________________
Från: Ihor Radchenko <yantar92@posteo.net>
Skickat: den 31 augusti 2024 14:20
Till: arthur miller <arthur.miller@live.com>
Kopia: emacs-devel@gnu.org <emacs-devel@gnu.org>; emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Ämne: Re: Sv: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

arthur miller <arthur.miller@live.com> writes:

>>> Well, part of it. Or more to make some parts of org-markup configurable, and
>>> usable as minor modes so they can be easier used outside of org-mode.
>>
>>This is not the direction Org mode project is going. We do the reverse -
>>Org markup is getting more stable, with the aim to simplify developing
>>third-party parsers and eventually registering Org syntax in IANA MIME db.
>
> I am not subscribed to mailing lists due to lots of trafic going on there, so I
> am not overly introduced in directions in which org goes, so forgive me if I am
> not so familiar with the latest features.

Sorry, I did not intend my comment to sound personal. I just explained
why configurable markup is not something I see as a good idea.

> Well, yes something like that; make features available outside of org-mode, and
> make org-mode use those features instead of having everything in a monolithical
> library.
> ...

To conclude, we may use these ideas for GSoC projects, but maybe
separately. I doubt that "modularizing" in general would be a reasonable
task to finish within summer.

--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[-- Attachment #2: Type: text/html, Size: 6182 bytes --]

Re: Sv: Modularizing Org as GSoC project

[Björn Bidar]

arthur miller <arthur.miller@live.com> writes:

>>> For example turn displaying pretty text (bold, italics etc), pretty links,
>>> tables, dates, and perhaps some other stuff into, from Org-mode, independent,
>>> minor modes that could be used in other parts of Emacs and more independently
>>> of
>>> Org mode. I think both Org-mode and Hyperbole, and perhaps some other
>>> libraries
>>> (button.el, help-mode, info), could use some minor mode that works with
>>> links. For us users, it would mean less cruft loaded into Emacs, if those big
>>> players could share some code.
>>>
>>> I haven't done much research on this, just something I had in my head for a
>>> long
>>> time.
>>
>>I would like to hear in more details what you have in mind. As is, your
>>proposal appears to be for multiple projects. Or am I mistaken?
>>
>>For links in particular, Hyperbole already implements the described
>>minor mode, or what am I missing?
>>
>>For tables, dates, and other markup, it is not clear what you want to
>>achieve.
>
> To be honest I am not sure myself :)
>
>>         Do you want Org markup to be displayed in non-Org buffers?
>
> Well, part of it. Or more to make some parts of org-markup configurable, and
> usable as minor modes so they can be easier used outside of org-mode.
>
> For example about links, there could be a mode "text-link-mode" or
> "pretty-links-mode" or something, that understands what a link description is,
> and what a link itself is. The minor mode would have some mean to parse
> description and link parts, and when on it would do what org-toggle-link-display
> does. For example org uses angle brackets for link desc and url, whereas
> markdown uses angle brackets and parenthesis. Thus link-mode could/should be
> enough customizable so that modes could be clients of this minor mode, as well
> as for user to be able to setup a regex or set a function that recognizes some
> custom syntax for descriptions and links. Also a minor mode can come with a key
> map and some actions, for example to follow link, to insert a link etc. I think
> of org-links, but a bit more generalized and usable without org-mode
> itself. Org-mode could use those under the hood.

I think this is very much related to a similar problem that
bug-reference-mode tries to solve.
Certain patterns can mean certain things, does something match the
pattern -> buttonize.

Most modes roll their own thing of such a system e.g. Circe, Gnus,
Bug-reference-mode.
If there was a generalized system it could make it easier to configure.

The caveat is that some modes could have trouble using a generalized
solution for this either because of their own historic structure
(e.g. Gnus) or because it does not work for a specific modes structure
e.g. when  the link contains mode specific syntax.

E.g. Org-Links with ID properties would not work unless properties are
understood by other modes, much is woven into the org structure.

I think the best solution would be to add generalized helper functions
that modes can use but not to try to intercept the modes inner workings
by adding a custom mode on top.
However in instances where there is no special markup such a minor-mode
could work and would be very helpful to support instances where there's
a syntax in a syntax such as in programming comment lines.

Sooner or later this will get us to having another Outline or Outshine
mode.

Re: Sv: Modularizing Org as GSoC project

[Björn Bidar]

arthur miller <arthur.miller@live.com> writes:

>>> For example turn displaying pretty text (bold, italics etc), pretty links,
>>> tables, dates, and perhaps some other stuff into, from Org-mode, independent,
>>> minor modes that could be used in other parts of Emacs and more independently
>>> of
>>> Org mode. I think both Org-mode and Hyperbole, and perhaps some other
>>> libraries
>>> (button.el, help-mode, info), could use some minor mode that works with
>>> links. For us users, it would mean less cruft loaded into Emacs, if those big
>>> players could share some code.
>>>
>>> I haven't done much research on this, just something I had in my head for a
>>> long
>>> time.
>>
>>I would like to hear in more details what you have in mind. As is, your
>>proposal appears to be for multiple projects. Or am I mistaken?
>>
>>For links in particular, Hyperbole already implements the described
>>minor mode, or what am I missing?
>>
>>For tables, dates, and other markup, it is not clear what you want to
>>achieve.
>
> To be honest I am not sure myself :)
>
>>         Do you want Org markup to be displayed in non-Org buffers?
>
> Well, part of it. Or more to make some parts of org-markup configurable, and
> usable as minor modes so they can be easier used outside of org-mode.
>
> For example about links, there could be a mode "text-link-mode" or
> "pretty-links-mode" or something, that understands what a link description is,
> and what a link itself is. The minor mode would have some mean to parse
> description and link parts, and when on it would do what org-toggle-link-display
> does. For example org uses angle brackets for link desc and url, whereas
> markdown uses angle brackets and parenthesis. Thus link-mode could/should be
> enough customizable so that modes could be clients of this minor mode, as well
> as for user to be able to setup a regex or set a function that recognizes some
> custom syntax for descriptions and links. Also a minor mode can come with a key
> map and some actions, for example to follow link, to insert a link etc. I think
> of org-links, but a bit more generalized and usable without org-mode
> itself. Org-mode could use those under the hood.

I think this is very much related to a similar problem that
bug-reference-mode tries to solve.
Certain patterns can mean certain things, does something match the
pattern -> buttonize.

Most modes roll their own thing of such a system e.g. Circe, Gnus,
Bug-reference-mode.
If there was a generalized system it could make it easier to configure.

The caveat is that some modes could have trouble using a generalized
solution for this either because of their own historic structure
(e.g. Gnus) or because it does not work for a specific modes structure
e.g. when  the link contains mode specific syntax.

E.g. Org-Links with ID properties would not work unless properties are
understood by other modes, much is woven into the org structure.

I think the best solution would be to add generalized helper functions
that modes can use but not to try to intercept the modes inner workings
by adding a custom mode on top.
However in instances where there is no special markup such a minor-mode
could work and would be very helpful to support instances where there's
a syntax in a syntax such as in programming comment lines.

Sooner or later this will get us to having another Outline or Outshine
mode.