Improving documentation of Org Mode integration into Emacs.

Improving documentation of Org Mode integration into Emacs.

[Karl Fogel]

I'd like to improve our in-tree documentation of how Org Mode 
integration into GNU Emacs works.

Many of us know from experience that Org Mode is maintained in a 
separate repository at git://git.sv.gnu.org/emacs/org-mode.git, 
and that it has its own project home site at https://orgmode.org/. 
But to a newcomer or an occasional contributor, this isn't 
obvious: they just see the sources in 'lisp/org/', looking like 
any other Lisp subdir in Emacs.

The fact that Org Mode is special is not clearly documented for 
those whose entry point is the Emacs sources.  While "orgmode.org" 
is mentioned in a header comment in, e.g., 'org.el', it's not 
prominent, and it's especially easy to miss because one often 
lands in the middle of a .el file via etags or grep or some other 
method that doesn't involve entering through the top.

And even for those who do happen to know that Org Mode is special, 
we can't tell from inspection exactly what 'lisp/org/' represents. 
Is it a release of Org Mode?  A regularly-updated clone of the 
upstream development sources?  Something else?

I finally found 
https://orgmode.org/worg/org-maintenance.html#org9ffe058, which 
describes how it works.

So I'd like to add some documentation in our tree, clarifying this 
situation for future searchers.  Most likely, this would be a 
brief mention in CONTRIBUTE, pointing to a new file 
lisp/org/README which would give more details and point to the 
above page.

Any objections / thoughts / suggestions about this, before I start 
on it?
 
Best regards,
-Karl

[2] https://list.orgmode.org/87czmcccrp.fsf@red-bean.com/T/#u

Re: Improving documentation of Org Mode integration into Emacs.

[Karl Fogel]

At the end of my previous message, there was a dangling footnote 
that said

  > [2] https://list.orgmode.org/87czmcccrp.fsf@red-bean.com/T/#u

It's not significant.  It's just left over from an earlier draft 
in which I mentioned the Org Mode question that led me down the 
path of noticing the gap in our documentation.  I decided that 
mention wasn't important, and removed it from the text, but forgot 
about the footnote.

Best regards,
-Karl

RE: [External] : Improving documentation of Org Mode integration into Emacs.

[Drew Adams]

> Any objections / thoughts / suggestions about this, before I start
> on it?

+1.  Please do.

Re: Improving documentation of Org Mode integration into Emacs.

[Stefan Monnier]

> So I'd like to add some documentation in our tree, clarifying this situation
> for future searchers.  Most likely, this would be a brief mention in
> CONTRIBUTE, pointing to a new file lisp/org/README which would give more
> details and point to the above page.

It might generally be a good idea to have a file listing similar other
cases, i.e. packages that are (also) maintained elsewhere, combined with
an explanation of how that "elsewhere" relates to the Emacs sources.
(I'm thinking of CC-mode, vhdl-mode, etc...).


        Stefan

Re: Improving documentation of Org Mode integration into Emacs.

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Date: Sat, 04 Dec 2021 15:14:59 -0600
> 
> I'd like to improve our in-tree documentation of how Org Mode 
> integration into GNU Emacs works.

Thanks.  Can you provide a rough sketch, in the form of list of items,
of what you'd like to document, and who would be the main audience for
that?  I don't have a clear idea of that based on what you posted.

> So I'd like to add some documentation in our tree, clarifying this 
> situation for future searchers.  Most likely, this would be a 
> brief mention in CONTRIBUTE, pointing to a new file 
> lisp/org/README which would give more details and point to the 
> above page.

I'm not sure these places are right for this kind of information, but
maybe it's because I don't have a good understanding of what you'd
like to say there.

Re: Improving documentation of Org Mode integration into Emacs.

[Karl Fogel]

[-- Attachment #1: Type: text/plain, Size: 1639 bytes --]

On 05 Dec 2021, Eli Zaretskii wrote:
>> I'd like to improve our in-tree documentation of how Org Mode 
>> integration into GNU Emacs works.
>
>Thanks.  Can you provide a rough sketch, in the form of list of 
>items,
>of what you'd like to document, and who would be the main 
>audience for
>that?  I don't have a clear idea of that based on what you 
>posted.

Sure.  (The attached patch may answer your questions more 
efficiently, though.)

* Audience:

The audience is Emacs developers and potential developers who are 
tracking down bugs, or trying to contribute enhancements, but who 
don't know that certain packages (such as Org Mode) are externally 
maintained. 

It's better for us all if such a person finds out as soon as 
possible where the active upstream for a package is.  For example, 
if they've found a bug in Org Mode, they should take the bug 
report to the Org Mode forums and discuss it with the Org Mode 
maintainers.  (Maybe sometimes Emacs Devel should also be 
involved, but the person should at least start out in the proper 
upstream forum and then be guided appropriately from there.)

* List of items:

All the packages that ship as part of Emacs but that are 
externally maintained.  Right now, Org Mode is the only one I know 
of, but I suspect there are others.  If people would like to point 
others out to me, I'll update the attached patch to include them 
too.

The attached patch is an attempt to document this situation in 
such a way that developers are likely to encounter the 
documentation when they most need it.  Comments welcome; I'm 
certainly open to other strategies.

Best regards,
-Karl


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: [PATCH] Document external maintenance of some packages --]
[-- Type: text/x-diff, Size: 2490 bytes --]

From b709de6a9ffb4cbb30b480f97f1de009a00769a9 Mon Sep 17 00:00:00 2001
From: Karl Fogel <kfogel@red-bean.com>
Date: Sun, 2 Jan 2022 22:49:35 -0600
Subject: [PATCH] Document external maintenance of some packages

Document the fact that some packages in Emacs are externally
maintained, and document Org Mode's provenance specifically.

For more context, see the thread that starts here:

  https://lists.gnu.org/archive/html/emacs-devel/2021-12/msg00366.html
  From: Karl Fogel
  To: Emacs Devel
  Subject: Improving documentation of Org Mode integration into Emacs.
  Date: Sat, 04 Dec 2021 15:14:59 -0600
  Message-ID: <87zgpgax7w.fsf@red-bean.com>
---
 CONTRIBUTE      | 17 +++++++++++++++++
 lisp/org/README |  6 ++++++
 2 files changed, 23 insertions(+)
 create mode 100644 lisp/org/README

diff --git CONTRIBUTE CONTRIBUTE
index 7c3421ed75..6f3bb9bf32 100644
--- CONTRIBUTE
+++ CONTRIBUTE
@@ -366,6 +366,23 @@ reasons.  These should be marked by including something like "Do not
 merge to master" or anything that matches gitmerge-skip-regexp (see
 admin/gitmerge.el) in the commit message.
 
+** Some packages in Emacs are maintained externally
+
+Sometimes a package that ships as part of GNU Emacs is actually
+maintained as a separate project, with its own upstream repository,
+its own maintainer group, its own development conventions, etc.  The
+upstream project's code is periodically merged into Emacs -- exactly
+when and how such merges happen depends on the package.
+
+Each externally maintained package lives in its own subdirectory in
+the Emacs tree, and that subdirectory should have a "README" file that
+describes the upstream origin.  For example, "lisp/org/README" says
+where the Org Mode project lives and how it's synchronized into Emacs.
+
+Note that not all the independently-maintained packages in Emacs have
+such a "README" yet.  If you know of one that's missing its "README",
+please send in a patch or just let us know.
+
 ** GNU ELPA
 
 This repository does not contain the Emacs Lisp package archive
diff --git lisp/org/README lisp/org/README
new file mode 100644
index 0000000000..f9fe1e6edc
--- /dev/null
+++ lisp/org/README
@@ -0,0 +1,6 @@
+Org Mode is maintained as a separate project, and is periodically
+merged into Emacs.
+
+See https://orgmode.org/ for more information, and see specifically
+https://orgmode.org/worg/org-maintenance.html for information about
+the process of synchronization between upstream Org Mode and Emacs.
-- 
2.34.1

Re: Improving documentation of Org Mode integration into Emacs.

[Stefan Monnier]

> All the packages that ship as part of Emacs but that are externally
> maintained.  Right now, Org Mode is the only one I know of, but I suspect
> there are others.  If people would like to point others out to me, I'll
> update the attached patch to include them too.

"Externally maintained" might not be quite right for the following
packages, but they do fit the description to some extent in the sense
that the maintainer also distributes the code elsewhere (e.g. for XEmacs):
- CC-mode
- VHDL-mode
- Verilog-mode


        Stefan

Re: Improving documentation of Org Mode integration into Emacs.

[Karl Fogel]

[-- Attachment #1: Type: text/plain, Size: 7782 bytes --]

Replying to several emails together here -- from Stefan, Michael, 
Protesilaos, Rudolf, and Eli -- as they're related.   TL;DR: Just 
see the attached revised patch, which incorporates information and 
suggestions from those folks.

On 03 Jan 2022, Stefan Monnier wrote: 
>"Externally maintained" might not be quite right for the 
>following packages, but they do fit the description to some 
>extent in the sense that the maintainer also distributes the code 
>elsewhere (e.g. for XEmacs): - CC-mode - VHDL-mode - Verilog-mode 
 
Well, the question isn't about whether the code is also 
distributed elsewhere.  Rather, the question is "Where should a 
developer go first with bug reports, bug patches, enhancement 
suggestions, etc?"   If she should go to Emacs Devel (or our 
debbugs tracker) first, then no special note is needed.   If she 
should go to some other upstream origin first, or should at least 
consider doing that, then a note is needed.   (For the above three 
packages specifically, I don't know the answer to that question, 
as I haven't looked at them in depth.)   On 03 Jan 2022, Michael 
Albinus wrote: 
>> +Each externally maintained package lives in its own 
>> subdirectory in +the Emacs tree, and that subdirectory should 
>> have a "README" file that +describes the upstream origin.  For 
>> example, "lisp/org/README" says +where the Org Mode project 
>> lives and how it's synchronized into Emacs. 
> That's not the case now. Tramp, for example, does not live in an 
>own subdirectory. Should it? 
 
Ah, thank you for pointing this out.  Okay, there are two 
different ways we could handle that:  1) Document the current 
messy reality (described in detail below).   2) Clean up reality, 
then document that new reality :-).   I think (1) is better right 
now, as it's a smaller change.  The "messy reality" I'm referring 
to is this:  *Some* multi-file externally maintained packages, 
like Tramp, are not contained in a dedicated subdirectory.  All 
the Tramp files live in "lisp/net/", alongside other non-Tramp 
file (some of which are also multi-file packages, like EUDC, 
although unlike Tramp they are not externally maintained packages 
as far as I know).   Meanwhile, *other* multi-file externally 
maintained packages, such as Org Mode, do have their own dedicated 
subdirectory (e.g., "lisp/org/").  In these cases, there is as 
easy solution for documenting the fact of external maintenance: 
put a README in that subdir.  The attached revised patch shows 
this.   Doing (2) would mean putting every multi-file package into 
its own dedicated subdirectory (e.g., "lisp/net/tramp/") -- for 
both externally and internally maintained packages.  I think this 
would be a good organizational improvement for us to make anyway, 
not only because it would simplify our strategy for documenting 
external maintenance for multi-file packages.  But (1) seems like 
the right option here, as it's a smaller step.  (2) would be a 
separate decision that goes beyond the scope of my original 
proposal. 

On 03 Jan 2022, Protesilaos Stavrou wrote: 
>The modus-themes are maintained externally.  I sync them with 
>emacs.git every month or so when I release a new version.  Their 
>.el files are in etc/themes/ while the manual is in doc/misc/. 
>I am happy to provide a README, though neither of those two 
>directories is specific to the modus-themes.   Maybe I should 
>include the relevant information in the themes' manual and/or the 
>"Commentary:" of each .el file? 
 
Thanks for mentioning these.  Please see the attached patch, which 
I think might answer your question (in the diff to CONTRIBUTE). 
On 03 Jan 2022, Rudolf Adamkovič wrote:
>I have a small comment. :)
>I think we can drop the word "actually" from the first sentence:

I agree -- thank you.  Your suggestion is incorporated into the 
new patch attached here.

On 03 Jan 2022, Eli Zaretskii wrote: 
>Thanks.   However, IMO the text as written doesn't belong to 
>CONTRIBUTE.  That file includes actionable instructions to 
>contributors regarding our development procedures, conventions, 
>and requirements.  The purpose of those instructions is to make 
>the contributed changes acceptable and matching our practices. 
>By contrast, the text that you propose is just information that 
>is not actionable.  So if this text is to stay in its present 
>form, it should be somewhere else, perhaps in README.   If you do 
>want it to be in CONTRIBUTE, it mostly be comprised of some 
>specific instructions what to do or what not to do.  The purely 
>informational part should ideally be shorter and more focused on 
>what contributors need to know in order to, well, contribute. 
 
The attached (revised) patch gives some specific instructions as 
examples.  But CONTRIBUTE's job is just to explain the general 
situation about externally maintained packages and give a few 
examples, so that developers understand why they need to be watch 
out for this situation.  That's what the revised text in the 
attached patch does.   Let me explain more clearly why I think 
CONTRIBUTE needs this section:  The CONTRIBUTE file is where 
developers go to learn how to contribute to Emacs.  But right now, 
when a developer finds a bug in (or wants to make an improvement 
to) a package that is distributed with Emacs but is externally 
maintained, she could easily miss the fact that the package is 
externally maintained!  That means she'll often bring her bug 
report or enhancement suggestion to the wrong place first: Emacs's 
own forums, instead of to the true upstream forums.   In other 
words, CONTRIBUTE needs to document this phenomenon because 
CONTRIBUTE is where we point developers to to learn how to 
contribute to Emacs -- and this information is about routing 
contributions correctly.   But CONTRIBUTE shouldn't give the 
specific instructions for each package.  It should just show the 
developer how to figure out if a package is externally maintained, 
and then that package can point the developer in the right 
direction in whatever way is most appropriate.

>Please note that none of what you say here is in the text, not 
>even as a hint.  If the idea is to tell contributors to direct 
>bug reports for those packages elsewhere, why not tell that 
>explicitly, and why not include the relevant URLs where those 
>bugs should be reported? 
 
This is made more explicit now in CONTRIBUTE.

Again, I don't include all the specific upstream URLs in 
CONTRIBUTE because we're not listing all the externally maintained 
packages in CONTRIBUTE.  CONTRIBUTE just explains how to detect 
such a package, and then lets the package's files give the proper 
pointers. 
 
>> +See https://orgmode.org/ for more information, and see 
>> specifically +https://orgmode.org/worg/org-maintenance.html for 
>> information about +the process of synchronization between 
>> upstream Org Mode and Emacs. 
> The beginning of this is promising, but then it strays: instead 
>of telling people to send patches and bug reports about Org to 
>those addresses and not to Emacs's tracker, it invites them to 
>learn about the process of synchronization between Org and Emacs, 
>something that I think is of secondary importance (to say the 
>least) for causal contributors. 
 
Thanks; good idea.  I've improved the proposed lisp/org/README as 
you indicate above.  The new language also makes clearer why -- in 
the specific case of Org Mode -- one might want to learn about the 
process of synchronization, since with Org Mode changes flow 
bidirectionally (which may not be the case for all externally 
maintained packages).

Best regards,
-Karl


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: [PATCH] Document external maintenance of some packages --]
[-- Type: text/x-diff, Size: 5104 bytes --]

From 79527535f74b8299fa2d1612d0458d4972e05ac1 Mon Sep 17 00:00:00 2001
From: Karl Fogel <kfogel@red-bean.com>
Date: Sun, 2 Jan 2022 22:49:35 -0600
Subject: [PATCH] Document external maintenance of some packages

Document the fact that some packages in Emacs are externally
maintained and document how to discover their upstream origin.

For more context, see the thread that starts here:

  https://lists.gnu.org/archive/html/emacs-devel/2021-12/msg00366.html
  From: Karl Fogel
  To: Emacs Devel
  Subject: Improving documentation of Org Mode integration into Emacs.
  Date: Sat, 04 Dec 2021 15:14:59 -0600
  Message-ID: <87zgpgax7w.fsf@red-bean.com>
---
 CONTRIBUTE      | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 lisp/org/README | 19 +++++++++++++++++++
 2 files changed, 68 insertions(+)
 create mode 100644 lisp/org/README

diff --git CONTRIBUTE CONTRIBUTE
index 7c3421ed75..527ad1ed35 100644
--- CONTRIBUTE
+++ CONTRIBUTE
@@ -366,6 +366,55 @@ reasons.  These should be marked by including something like "Do not
 merge to master" or anything that matches gitmerge-skip-regexp (see
 admin/gitmerge.el) in the commit message.
 
+** Some packages in Emacs are maintained externally
+
+Sometimes a package that ships as part of GNU Emacs is maintained as a
+separate project, with its own upstream repository, its own maintainer
+group, its own development conventions, etc.  The upstream project's
+code is periodically merged into Emacs (exactly when and how such
+merges happen depends on the package).
+
+So when you are making a contribution -- such as fixing a bug or
+proposing an enhancement -- to one of these externally maintained
+packages, you often need to deal with that package at its upstream
+source.  For example, if you're trying to fix a bug, you should obtain
+the latest version of the packages's code from that upstream source,
+since that code will likely be ahead of whatever version is in Emacs
+and you'll want to test the latest code to see if the bug is still
+present.  Likewise, if you're starting a development discussion, you
+should generally do so in the package's own forums rather than in
+Emacs's forums; the upstream developers can provide guidance about
+whether Emacs's developers need to be brought in to the discussion.
+
+*** How to tell which packages are externally maintained
+
+When an externally maintained package is just one file, then a comment
+near the top of the file should indicate the upstream origin.
+
+When an externally maintained package involves multiple files, then
+there are a couple of possibilities:
+
+1. The files all live in one dedicated subdirectory that is specific
+   to that package.  In this case, there should be a README in that
+   subdirectory indicating the upstream origin.  For example,
+   "lisp/org/README" indicates where the Org Mode project lives and
+   what that provenance implies for contributors.
+
+2. The package's files all live in the same directory, and share a
+   common filename prefix, but there are also other files in that
+   directory that are unrelated to the package in question.
+
+   In this case, there should be a comment near the top of the
+   package's main entry-point file -- i.e., the file that one loads to
+   `provide' the package -- giving the provenance indication.  For
+   example, the Tramp package is made up of multiple files
+   ("lisp/net/tramp*") and the package's upstream origin is described
+   in a comment near the top of "lisp/net/tramp.el".
+
+Note that there may be some independently-maintained packages in Emacs
+that still lack a clear provenance indicator.  If you find one, please
+send in a patch or just let us know.
+
 ** GNU ELPA
 
 This repository does not contain the Emacs Lisp package archive
diff --git lisp/org/README lisp/org/README
new file mode 100644
index 0000000000..6c8022f56a
--- /dev/null
+++ lisp/org/README
@@ -0,0 +1,19 @@
+Org Mode is maintained as a separate project, and is periodically
+merged into Emacs.  To view or participate in Org Mode development,
+please go to https://orgmode.org/ and follow the instructions there.
+
+The source code from the upstream Org Mode project is usually not
+identical to the version of Org Mode in Emacs.  The upstream project
+often has recent changes that have not yet been merged into Emacs, and
+Emacs sometimes has local changes to Org Mode that have not yet been
+backported to upstream (https://orgmode.org/worg/org-maintenance.html
+documents how the Org Mode project synchronizes changes with Emacs).
+
+Thus, if you're investigating a bug you encountered in Org Mode in
+Emacs, you should obtain the latest upstream code and see if the bug
+is present there.  If the bug is present, then the upstream Org Mode
+project is the proper place to fix it.  If the bug is not present
+there, that could be because it has already been fixed upstream, or it
+could be because the bug was only introduced on the Emacs side and has
+not yet been backported upstream.  Either way, this is something you
+will need to know in order to know where to contribute your fix.
-- 
2.34.1

Re: Improving documentation of Org Mode integration into Emacs.

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Date: Tue, 04 Jan 2022 21:09:47 -0600
> 
> On 03 Jan 2022, Eli Zaretskii wrote: 
> >Thanks.   However, IMO the text as written doesn't belong to 
> >CONTRIBUTE.  That file includes actionable instructions to 
> >contributors regarding our development procedures, conventions, 
> >and requirements.  The purpose of those instructions is to make 
> >the contributed changes acceptable and matching our practices. 
> >By contrast, the text that you propose is just information that 
> >is not actionable.  So if this text is to stay in its present 
> >form, it should be somewhere else, perhaps in README.   If you do 
> >want it to be in CONTRIBUTE, it mostly be comprised of some 
> >specific instructions what to do or what not to do.  The purely 
> >informational part should ideally be shorter and more focused on 
> >what contributors need to know in order to, well, contribute. 
>  
> The attached (revised) patch gives some specific instructions as 
> examples.

I don't think providing just examples is the best idea.  See below.

> But CONTRIBUTE's job is just to explain the general 
> situation about externally maintained packages and give a few 
> examples, so that developers understand why they need to be watch 
> out for this situation.

I think we should explain the issue concisely, and then provide
specific instructions.  This text is IMO enough for the first part:

  +Sometimes a package that ships as part of GNU Emacs is maintained as a
  +separate project, with its own upstream repository, its own maintainer
  +group, its own development conventions, etc.  The upstream project's
  +code is periodically merged into Emacs (exactly when and how such
  +merges happen depends on the package).
  +
  +So when you are making a contribution -- such as fixing a bug or
  +proposing an enhancement -- to one of these externally maintained
  +packages, you often need to deal with that package at its upstream
  +source.

Following that, we should provide a full, exhaustive list of all such
packages in Emacs core, each one with a corresponding URL (mailing
list, upstream repository with an issue tracker, etc.).  It might be a
good idea to have this list on a separate file, perhaps in etc/, and
only have CONTRIBUTE point to that file.

> But CONTRIBUTE shouldn't give the specific instructions for each
> package.  It should just show the developer how to figure out if a
> package is externally maintained, and then that package can point
> the developer in the right direction in whatever way is most
> appropriate.

I disagree.  If we don't provide a full list of such packages with
precise instructions, we will not make any significant progress:
people would still need to ask us about the details, when they aren't
"smart" enough (or patient enough) to read the instructions that teach
them how to deduce that by themselves.  Moreover, as you have found
out already, there's no standard way such packages use to "plug"
themselves into Emacs, so it's likely that any general instructions we
provide will be inaccurate in some cases.  An exhaustive "cookbook" is
much better, IMO, and is easier to maintain in the long run.  It will
also be shorter, which is a nice bonus, given today's "TL;DR"
attitude.

Thanks.

Re: Improving documentation of Org Mode integration into Emacs.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

> I think we should explain the issue concisely, and then provide
> specific instructions.  This text is IMO enough for the first part:
>
>   +Sometimes a package that ships as part of GNU Emacs is maintained as a
>   +separate project, with its own upstream repository, its own maintainer
>   +group, its own development conventions, etc.  The upstream project's
>   +code is periodically merged into Emacs (exactly when and how such
>   +merges happen depends on the package).
>   +
>   +So when you are making a contribution -- such as fixing a bug or
>   +proposing an enhancement -- to one of these externally maintained
>   +packages, you often need to deal with that package at its upstream
>   +source.
>
> Following that, we should provide a full, exhaustive list of all such
> packages in Emacs core, each one with a corresponding URL (mailing
> list, upstream repository with an issue tracker, etc.).  It might be a
> good idea to have this list on a separate file, perhaps in etc/, and
> only have CONTRIBUTE point to that file.

We have already admin/MAINTAINERS, which could easily be extended for
this. I, for example, am mentioned there as responsible for
Tramp. Adding the Tramp relevant URLs would be easy, and so for all
other external maintained packages.

> Thanks.

Best regards, Michael.

Re: Improving documentation of Org Mode integration into Emacs.

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Karl Fogel <kfogel@red-bean.com>,  emacs-devel@gnu.org
> Date: Wed, 05 Jan 2022 15:17:30 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Following that, we should provide a full, exhaustive list of all such
> > packages in Emacs core, each one with a corresponding URL (mailing
> > list, upstream repository with an issue tracker, etc.).  It might be a
> > good idea to have this list on a separate file, perhaps in etc/, and
> > only have CONTRIBUTE point to that file.
> 
> We have already admin/MAINTAINERS, which could easily be extended for
> this.

Yes, that'd be fine, I think.

Re: Improving documentation of Org Mode integration into Emacs.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> > Following that, we should provide a full, exhaustive list of all such
>> > packages in Emacs core, each one with a corresponding URL (mailing
>> > list, upstream repository with an issue tracker, etc.).  It might be a
>> > good idea to have this list on a separate file, perhaps in etc/, and
>> > only have CONTRIBUTE point to that file.
>>
>> We have already admin/MAINTAINERS, which could easily be extended for
>> this.
>
> Yes, that'd be fine, I think.

I've pushed this for Tramp, comments welcome.

Best regards, Michael.

Re: Improving documentation of Org Mode integration into Emacs.

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: kfogel@red-bean.com,  emacs-devel@gnu.org
> Date: Thu, 06 Jan 2022 13:40:47 +0100
> 
> >> We have already admin/MAINTAINERS, which could easily be extended for
> >> this.
> >
> > Yes, that'd be fine, I think.
> 
> I've pushed this for Tramp, comments welcome.

LGTM, thanks.

Re: Improving documentation of Org Mode integration into Emacs.

[Protesilaos Stavrou]

On 2022-01-06, 15:58 +0200, Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Michael Albinus <michael.albinus@gmx.de>
>> Cc: kfogel@red-bean.com,  emacs-devel@gnu.org
>> Date: Thu, 06 Jan 2022 13:40:47 +0100
>> 
>> >> We have already admin/MAINTAINERS, which could easily be extended for
>> >> this.
>> >
>> > Yes, that'd be fine, I think.
>> 
>> I've pushed this for Tramp, comments welcome.
>
> LGTM, thanks.

I followed Michael's example to add information about the Modus themes.

Thank you!

-- 
Protesilaos Stavrou
https://protesilaos.com

Re: Improving documentation of Org Mode integration into Emacs.

[Karl Fogel]

[-- Attachment #1: Type: text/plain, Size: 2430 bytes --]

On 05 Jan 2022, Eli Zaretskii wrote:
>I think we should explain the issue concisely, and then provide
>specific instructions.  This text is IMO enough for the first 
>part:
>
>[...]
>
>Following that, we should provide a full, exhaustive list of all 
>such
>packages in Emacs core, each one with a corresponding URL 
>(mailing
>list, upstream repository with an issue tracker, etc.).  It might 
>be a
>good idea to have this list on a separate file, perhaps in etc/, 
>and
>only have CONTRIBUTE point to that file.

The attached patch puts a concise text in CONTRIBUTE as you 
described,
and adds an entry for Org Mode to the new "Externally maintained 
packages" section of admin/MAINTAINERS -- following Michael 
Albinius's and Protesilaos Stavrou's lead re Tramp and Modus 
Themes respectively.

The Org Mode entry is larger than the Tramp and Modus Themes 
entries, because the situation with Org Mode is more complex. 
That's why I've attached this patch for review instead of just 
committing it.

>I disagree.  If we don't provide a full list of such packages 
>with
>precise instructions, we will not make any significant progress:
>people would still need to ask us about the details, when they 
>aren't
>"smart" enough (or patient enough) to read the instructions that 
>teach
>them how to deduce that by themselves.  Moreover, as you have 
>found
>out already, there's no standard way such packages use to "plug"
>themselves into Emacs, so it's likely that any general 
>instructions we
>provide will be inaccurate in some cases.  An exhaustive 
>"cookbook" is
>much better, IMO, and is easier to maintain in the long run.  It 
>will
>also be shorter, which is a nice bonus, given today's "TL;DR"
>attitude.

That's persuasive; I think it's a good solution.  Feedback on the 
attached patch is welcome.  A couple of notes about it:

* Although the "Branches" section in CONTRIBUTE implies that these 
  changes should go on the "emacs-28" branch, this patch is 
  against "master" because Michael and Protesilaos committed their 
  changes to "master".  (I think it's all fine on "master" too, 
  and the guidance in CONTRIBUTE is a bit ambiguous about this 
  kind of change anyway.)

* In admin/MAINTAINERS, I did not list 
  "test/lisp/org/org-tests.el" as a 
  file maintained by the Org Mode project, because it looks like 
  that file exists only in Emacs and is not shipped with Org Mode. 

Best regards,
-Karl


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: [PATCH] Document external maintenance of some packages --]
[-- Type: text/x-diff, Size: 4022 bytes --]

From 6a9a0132b7313a16539b2ec8b3fb9cac390b5915 Mon Sep 17 00:00:00 2001
From: Karl Fogel <kfogel@red-bean.com>
Date: Sun, 9 Jan 2022 16:07:08 -0600
Subject: [PATCH] Document external maintenance of some packages

Document the fact that some packages in Emacs are externally
maintained, and specifically document Org Mode's external maintenance.

For more context, see the thread that starts here:

  https://lists.gnu.org/archive/html/emacs-devel/2021-12/msg00366.html
  From: Karl Fogel
  To: Emacs Devel
  Subject: Improving documentation of Org Mode integration into Emacs.
  Date: Sat, 04 Dec 2021 15:14:59 -0600
  Message-ID: <87zgpgax7w.fsf@red-bean.com>
---
 CONTRIBUTE        | 18 ++++++++++++++++++
 admin/MAINTAINERS | 37 +++++++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)

diff --git CONTRIBUTE CONTRIBUTE
index 7c3421ed75..04d0ab00d2 100644
--- CONTRIBUTE
+++ CONTRIBUTE
@@ -366,6 +366,24 @@ reasons.  These should be marked by including something like "Do not
 merge to master" or anything that matches gitmerge-skip-regexp (see
 admin/gitmerge.el) in the commit message.
 
+** Some packages in Emacs are maintained externally
+
+Sometimes a package that ships as part of GNU Emacs is maintained as a
+separate project, with its own upstream repository, its own maintainer
+group, its own development conventions, etc.  The upstream project's
+code is periodically merged into Emacs (exactly when and how such
+merges happen depends on the package).
+
+So when you are making a contribution -- such as fixing a bug or
+proposing an enhancement -- to one of these externally maintained
+packages, you often need to deal with that package at its upstream
+source.
+
+See section "Externally maintained packages" in "admin/MAINTAINERS"
+for a list of such packages.  If you discover an externally maintained
+package in Emacs that is not yet listed there, please send in a patch
+or just let us know.
+
 ** GNU ELPA
 
 This repository does not contain the Emacs Lisp package archive
diff --git admin/MAINTAINERS admin/MAINTAINERS
index e87c3e0204..5b8d73cad7 100644
--- admin/MAINTAINERS
+++ admin/MAINTAINERS
@@ -316,6 +316,43 @@ Modus themes
         doc/misc/modus-themes.org
         etc/themes/modus*.el
 
+Org Mode
+	Home Page: https://orgmode.org/
+	Maintainer: Org Mode developers
+	Repository: git://git.sv.gnu.org/emacs/org-mode.git
+	Mailing list: emacs-orgmode@gnu.org
+	Bug Reports: M-x org-submit-bug-report
+	Notes: Org Mode is maintained as a separate project that is
+          periodically merged into Emacs.  To view or participate in
+          Org Mode development, please go to https://orgmode.org/ and
+          follow the instructions there.
+
+          The source code from the upstream Org Mode project is
+          usually not identical to the version of Org Mode in Emacs.
+          The upstream project often has recent changes that have not
+          yet been merged into Emacs, and Emacs sometimes has local
+          changes to Org Mode that have not yet been backported to
+          upstream.  https://orgmode.org/worg/org-maintenance.html
+          documents how the Org Mode project synchronizes changes with
+          Emacs.
+
+          If you're investigating a bug you encountered in Org Mode in
+          Emacs, you should obtain the latest upstream code and see if
+          the bug is present there.  If the bug is present, then the
+          upstream Org Mode project is the proper place to fix it.  If
+          the bug is not present there, that could be because it has
+          already been fixed upstream, or it could be because the bug
+          was only introduced on the Emacs side and has not yet been
+          backported upstream.  You will need to figure out what the
+          situation is in order to know where to contribute your fix.
+
+	lisp/org/*.el
+	etc/org/*
+	etc/refcards/orgcard.tex
+	doc/misc/org.org
+	doc/misc/org-setup.org
+	doc/misc/org.texi
+
 \f
 ;;; Local Variables:
 ;;;   coding: utf-8
-- 
2.34.1

Re: Improving documentation of Org Mode integration into Emacs.

[Michael Albinus]

Karl Fogel <kfogel@red-bean.com> writes:

Hi Karl,

> * In admin/MAINTAINERS, I did not list   "test/lisp/org/org-tests.el"
>   as a   file maintained by the Org Mode project, because it looks
>   like   that file exists only in Emacs and is not shipped with Org
>   Mode.

That's OK. Org-mode has a lot of own test files not integrated into
Emacs repo; I hope we can get them too.

> +So when you are making a contribution -- such as fixing a bug or
> +proposing an enhancement -- to one of these externally maintained
> +packages, you often need to deal with that package at its upstream
> +source.

I'm not sure that this is always the case. For Tramp, I'm happy if
people refer to the Emacs repo files; sync with the upstream package is
something contributors don't need to worry about. We shall keep the
barrier low.

More important are compatibility restrictions. All of these externally
maintained packages have their policy, we shall advice potential
contributors to respect them. Refer to the respective Package-Requires:
header line.

> +Org Mode
> +	Home Page: https://orgmode.org/
> +	Maintainer: Org Mode developers

The sync between org-mode and Emacs is performed by Kyle Meyer
<kyle@kyleam.com>, shall we mention him as the guy to be contacted in
case of?

> Best regards,
> -Karl

Best regards, Michael.

Re: Improving documentation of Org Mode integration into Emacs.

[Michael Albinus]

PS: doc/misc/org.texi is a generated file, no need to mention it.

Best regards, Michael.

Re: Improving documentation of Org Mode integration into Emacs.

[Karl Fogel]

On 10 Jan 2022, Michael Albinus wrote:
>PS: doc/misc/org.texi is a generated file, no need to mention it.

Interesting -- it is version-controlled, in both the Emacs and the 
Org Mode repositories:

  Emacs:     doc/misc/org.texi
  Org Mode:  doc/org.texi

(Currently, in the former it is the Org Mode 9.5 manual and in the 
latter it is the Org Mode 9.6 manual.)

Since it exists in both repositories, and is presumably part of 
the synchronization (even if only implicitly, by being regenerated 
independently on both sides after its sources are synchronized), 
maybe it should still be mentioned?

Best regards,
-Karl

Re: Improving documentation of Org Mode integration into Emacs.

[Michael Albinus]

Karl Fogel <kfogel@red-bean.com> writes:

> Since it exists in both repositories, and is presumably part of the
> synchronization (even if only implicitly, by being regenerated
> independently on both sides after its sources are synchronized), maybe
> it should still be mentioned?

Well, let the org people decide.

> Best regards,
> -Karl

Best regards, Michael.

Re: Improving documentation of Org Mode integration into Emacs.

[Robert Pluim]

>>>>> On Mon, 10 Jan 2022 03:13:22 -0600, Karl Fogel <kfogel@red-bean.com> said:

    Karl> On 10 Jan 2022, Michael Albinus wrote:
    >> PS: doc/misc/org.texi is a generated file, no need to mention it.

    Karl> Interesting -- it is version-controlled, in both the Emacs and the Org
    Karl> Mode repositories:

    Karl>  Emacs:     doc/misc/org.texi
    Karl>  Org Mode:  doc/org.texi

It was version controlled in Emacs, but was deleted by fddd63f8. Of
course itʼs still in the git history.

Robert
-- 

Re: Improving documentation of Org Mode integration into Emacs.

[Karl Fogel]

On 10 Jan 2022, Robert Pluim wrote:
>>>>>> On Mon, 10 Jan 2022 03:13:22 -0600, Karl Fogel 
>>>>>> <kfogel@red-bean.com> said:
>
>    Karl> On 10 Jan 2022, Michael Albinus wrote:
>    >> PS: doc/misc/org.texi is a generated file, no need to 
>    >> mention it.
>
>    Karl> Interesting -- it is version-controlled, in both the 
>    Emacs and the Org
>    Karl> Mode repositories:
>
>    Karl>  Emacs:     doc/misc/org.texi
>    Karl>  Org Mode:  doc/org.texi
>
>It was version controlled in Emacs, but was deleted by 
>fddd63f8. Of
>course itʼs still in the git history.

Oh, sigh, thanks.  Today I learned that using 'git log' is not the 
way to check if a file is *currently* versioned at a certain path. 
One should instead use

  $ git ls-files doc/misc/org.texi

or better yet

  $ git ls-files --error-unmatch doc/misc/org.texi

I'll take 'doc/misc/org.texi out of the change.

Best regards,
-Karl

Re: Improving documentation of Org Mode integration into Emacs.

[Karl Fogel]

On 10 Jan 2022, Michael Albinus wrote:
>> * In admin/MAINTAINERS, I did not list 
>> "test/lisp/org/org-tests.el"
>>   as a   file maintained by the Org Mode project, because it 
>>   looks
>>   like   that file exists only in Emacs and is not shipped with 
>>   Org
>>   Mode.
>
>That's OK. Org-mode has a lot of own test files not integrated 
>into
>Emacs repo; I hope we can get them too.

I hope so too.  However, for our purposes here, we don't care 
about files on the Org Mode side that aren't currently integrated 
into Emacs's repository; we only care about what's present in 
Emacs.

>> +So when you are making a contribution -- such as fixing a bug 
>> or
>> +proposing an enhancement -- to one of these externally 
>> maintained
>> +packages, you often need to deal with that package at its 
>> upstream
>> +source.
>
>I'm not sure that this is always the case. For Tramp, I'm happy 
>if
>people refer to the Emacs repo files; sync with the upstream 
>package is
>something contributors don't need to worry about. We shall keep 
>the
>barrier low.

Hmmm.  In that case, should Tramp be listed in the "Externally 
maintained packages" section of admin/MAINTAINERS at all?

The purpose of that section, and of the new material in 
CONTRIBUTE, is to make contributors be aware of the situations in 
which they *do* need to pay attention to the fact of external 
maintenance -- that is, situations in which contributors might 
need to do something differently from how one would normally do 
it.

In situations where they can just send their contribution to Emacs 
in the usual Emacs-y way, then there is no need for special 
documentation in the first place.

We could change "often" to "sometimes" in the above-quoted text, 
but I think it's worth asking if Tramp should even be listed, if 
Tramp is happy to receive contributions via the usual Emacs 
project route anyway.

>More important are compatibility restrictions. All of these 
>externally
>maintained packages have their policy, we shall advice potential
>contributors to respect them. Refer to the respective 
>Package-Requires:
>header line.
>
>> +Org Mode
>> +	Home Page: https://orgmode.org/
>> +	Maintainer: Org Mode developers
>
>The sync between org-mode and Emacs is performed by Kyle Meyer
><kyle@kyleam.com>, shall we mention him as the guy to be 
>contacted in
>case of?

IMHO Emacs should avoid duplicating documentation that's available 
from the upstream projects themselves.  We should just send people 
upstream to get the latest information, whether about 
compatibility guidelines or anything else.

In fact, I wasn't even sure about listing the Org Mode version 
control repository explicitly in the section I added; I only did 
it for consistency with the other similar sections.  Ideally, the 
contributor should just go look at orgmode.org (or whatever the 
appropriate upstream landing page is, for other packages) and 
follow the pointers there.

The reason I added the long note in the Org Mode section is that 
Org Mode's situation is unusually complex, and AFAIK it's not 
summarized like that -- i.e., from the GNU Emacs development point 
of view -- on Org Mode's own site.

Best regards,
-Karl

Re: Improving documentation of Org Mode integration into Emacs.

[Michael Albinus]

Karl Fogel <kfogel@red-bean.com> writes:

>> I'm not sure that this is always the case. For Tramp, I'm happy if
>> people refer to the Emacs repo files; sync with the upstream package
>> is
>> something contributors don't need to worry about. We shall keep the
>>barrier low.
>
> Hmmm.  In that case, should Tramp be listed in the "Externally
> maintained packages" section of admin/MAINTAINERS at all?

It is OK to be listed there. I just don't want to *urge* people to use
the Tramp repo, unless it is needed. And the information about Tramp's
bug tracker and backward compatibility (just pushed to the repo) makes
it worthwile.

> Best regards,
> -Karl

Best regards, Michael.

Re: Improving documentation of Org Mode integration into Emacs.

[Karl Fogel]

On 10 Jan 2022, Michael Albinus wrote:
>>> I'm not sure that this is always the case. For Tramp, I'm 
>>> happy if
>>> people refer to the Emacs repo files; sync with the upstream 
>>> package
>>> is something contributors don't need to worry about. We shall 
>>> keep
>>> the barrier low.
>>
>> Hmmm.  In that case, should Tramp be listed in the "Externally
>> maintained packages" section of admin/MAINTAINERS at all?
>
>It is OK to be listed there. I just don't want to *urge* people 
>to use
>the Tramp repo, unless it is needed.

Hmm, but how will they know whether it is needed?  Perhaps the new 
"Notes" field in the Tramp entry could give some guidance on that.

The original goal of this change was to provide guidance to 
contributors who might otherwise waste time doing the wrong thing. 
For example, they might fail to check upstream to see if a bug has 
already been fixed there, or is being discussed there; or they 
might send a contribution to Emacs when really it should be sent 
somewhere else; etc.

So if there are circumstances in which contributors should worry 
about these possibilities for Tramp, then the Tramp entry in that 
file should explain what those circumstances are -- this is, after 
all, what the entry is for :-).

>And the information about Tramp's bug tracker and backward 
>compatibility (just pushed to the repo) makes it worthwile.

Got it.  So it sounds like the "Notes" field should give some 
advice about checking the upstream sources and bug tracker first, 
when trying to find out if a bug is already known/addressed.  And 
if you're happy to accept patches against the sources shipped in 
Emacs, even when those sources are slightly out-of-date with 
respect to what you have upstream, then "Notes" can say that too.

Thoughts?

Best regards,
-Karl

Re: Improving documentation of Org Mode integration into Emacs.

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Date: Mon, 10 Jan 2022 10:20:27 -0600
> Cc: Kyle Meyer <kyle@kyleam.com>, emacs-devel@gnu.org
> 
> >And the information about Tramp's bug tracker and backward 
> >compatibility (just pushed to the repo) makes it worthwile.
> 
> Got it.  So it sounds like the "Notes" field should give some 
> advice about checking the upstream sources and bug tracker first, 
> when trying to find out if a bug is already known/addressed.  And 
> if you're happy to accept patches against the sources shipped in 
> Emacs, even when those sources are slightly out-of-date with 
> respect to what you have upstream, then "Notes" can say that too.

IMO, that'd be in the "too much information" department.

We don't need to say everything there, just enough for the people to
know how to handle a bug report without wasting their time and ours.

Re: Improving documentation of Org Mode integration into Emacs.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> IMO, that'd be in the "too much information" department.
>
> We don't need to say everything there, just enough for the people to
> know how to handle a bug report without wasting their time and ours.

1+

Best regards, Michael.

Re: Improving documentation of Org Mode integration into Emacs.

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Date: Sun, 09 Jan 2022 16:34:16 -0600
> 
> --- CONTRIBUTE
> +++ CONTRIBUTE
> @@ -366,6 +366,24 @@ reasons.  These should be marked by including something like "Do not
>  merge to master" or anything that matches gitmerge-skip-regexp (see
>  admin/gitmerge.el) in the commit message.
>  
> +** Some packages in Emacs are maintained externally
> +
> +Sometimes a package that ships as part of GNU Emacs is maintained as a
> +separate project, with its own upstream repository, its own maintainer
> +group, its own development conventions, etc.  The upstream project's
> +code is periodically merged into Emacs (exactly when and how such
> +merges happen depends on the package).
> +
> +So when you are making a contribution -- such as fixing a bug or
> +proposing an enhancement -- to one of these externally maintained
> +packages, you often need to deal with that package at its upstream
> +source.
> +
> +See section "Externally maintained packages" in "admin/MAINTAINERS"
> +for a list of such packages.  If you discover an externally maintained
> +package in Emacs that is not yet listed there, please send in a patch
> +or just let us know.

This is okay, but I'd lose the last sentence: it isn't different from
saying "if you see something wrong in Emacs, please submit a patch".

> +Org Mode
> +	Home Page: https://orgmode.org/
> +	Maintainer: Org Mode developers
> +	Repository: git://git.sv.gnu.org/emacs/org-mode.git
> +	Mailing list: emacs-orgmode@gnu.org
> +	Bug Reports: M-x org-submit-bug-report
> +	Notes: Org Mode is maintained as a separate project that is
> +          periodically merged into Emacs.  To view or participate in
> +          Org Mode development, please go to https://orgmode.org/ and
> +          follow the instructions there.
> +
> +          The source code from the upstream Org Mode project is
> +          usually not identical to the version of Org Mode in Emacs.
> +          The upstream project often has recent changes that have not
> +          yet been merged into Emacs, and Emacs sometimes has local
> +          changes to Org Mode that have not yet been backported to
> +          upstream.  https://orgmode.org/worg/org-maintenance.html
> +          documents how the Org Mode project synchronizes changes with
> +          Emacs.
> +
> +          If you're investigating a bug you encountered in Org Mode in
> +          Emacs, you should obtain the latest upstream code and see if
> +          the bug is present there.  If the bug is present, then the
> +          upstream Org Mode project is the proper place to fix it.  If
> +          the bug is not present there, that could be because it has
> +          already been fixed upstream, or it could be because the bug
> +          was only introduced on the Emacs side and has not yet been
> +          backported upstream.  You will need to figure out what the
> +          situation is in order to know where to contribute your fix.

I'd lose the two last paragraphs.  They are not really needed for
people to report issues with Org.  If/when someone becomes intimately
involved with Org development, they will learn those aspects; but it
is not the job of CONTRIBUTE or MAINTAINERS to teach them that.

Thanks.

Re: Improving documentation of Org Mode integration into Emacs.

[Karl Fogel]

On 10 Jan 2022, Eli Zaretskii wrote:
>> +See section "Externally maintained packages" in 
>> "admin/MAINTAINERS"
>> +for a list of such packages.  If you discover an externally 
>> maintained
>> +package in Emacs that is not yet listed there, please send in 
>> a patch
>> +or just let us know.
>
>This is okay, but I'd lose the last sentence: it isn't different 
>from
>saying "if you see something wrong in Emacs, please submit a 
>patch".

Okay.  I'll commit with changes as noted previously in this thread 
and in this email.  I may change the wording of the previous 
sentence above to somehow express the point that the list in 
admin/MAINTAINERS is not guaranteed to be complete.

>> +Org Mode
>> +	Home Page: https://orgmode.org/
>> +	Maintainer: Org Mode developers
>> +	Repository: git://git.sv.gnu.org/emacs/org-mode.git
>> +	Mailing list: emacs-orgmode@gnu.org
>> +	Bug Reports: M-x org-submit-bug-report
>> +	Notes: Org Mode is maintained as a separate project that 
>> is
>> +          periodically merged into Emacs.  To view or 
>> participate in
>> +          Org Mode development, please go to 
>> https://orgmode.org/ and
>> +          follow the instructions there.
>> +
>> +          The source code from the upstream Org Mode project 
>> is
>> +          usually not identical to the version of Org Mode in 
>> Emacs.
>> +          The upstream project often has recent changes that 
>> have not
>> +          yet been merged into Emacs, and Emacs sometimes has 
>> local
>> +          changes to Org Mode that have not yet been 
>> backported to
>> +          upstream. 
>> https://orgmode.org/worg/org-maintenance.html
>> +          documents how the Org Mode project synchronizes 
>> changes with
>> +          Emacs.
>> +
>> +          If you're investigating a bug you encountered in Org 
>> Mode in
>> +          Emacs, you should obtain the latest upstream code 
>> and see if
>> +          the bug is present there.  If the bug is present, 
>> then the
>> +          upstream Org Mode project is the proper place to fix 
>> it.  If
>> +          the bug is not present there, that could be because 
>> it has
>> +          already been fixed upstream, or it could be because 
>> the bug
>> +          was only introduced on the Emacs side and has not 
>> yet been
>> +          backported upstream.  You will need to figure out 
>> what the
>> +          situation is in order to know where to contribute 
>> your fix.
>
>I'd lose the two last paragraphs.  They are not really needed for
>people to report issues with Org.  If/when someone becomes 
>intimately
>involved with Org development, they will learn those aspects; but 
>it
>is not the job of CONTRIBUTE or MAINTAINERS to teach them that.

Okay.

>> Got it.  So it sounds like the "Notes" field should give some 
>> advice about checking the upstream sources and bug tracker 
>> first, 
>> when trying to find out if a bug is already known/addressed. 
>> And 
>> if you're happy to accept patches against the sources shipped 
>> in 
>> Emacs, even when those sources are slightly out-of-date with 
>> respect to what you have upstream, then "Notes" can say that 
>> too.
>
>IMO, that'd be in the "too much information" department.
>
>We don't need to say everything there, just enough for the people 
>to
>know how to handle a bug report without wasting their time and 
>ours.

We have different judgement about what type of information to 
include in these entries, but at this point the differences are 
small, and probably most contributors will be able to figure 
things out.  I'll eliminate those two paragraphs.

Thanks for the review, Eli.

Best regards,
-Karl

Re: Improving documentation of Org Mode integration into Emacs.

[Michael Albinus]

Karl Fogel <kfogel@red-bean.com> writes:

Hi Karl,

> +Each externally maintained package lives in its own subdirectory in
> +the Emacs tree, and that subdirectory should have a "README" file that
> +describes the upstream origin.  For example, "lisp/org/README" says
> +where the Org Mode project lives and how it's synchronized into Emacs.

That's not the case now. Tramp, for example, does not live in an own
subdirectory. Should it?

Best regards, Michael.

Re: Improving documentation of Org Mode integration into Emacs.

[Protesilaos Stavrou]

On 2022-01-02, 23:05 -0600, Karl Fogel <kfogel@red-bean.com> wrote:

> +Each externally maintained package lives in its own subdirectory in
> +the Emacs tree, and that subdirectory should have a "README" file that
> +describes the upstream origin.  For example, "lisp/org/README" says
> +where the Org Mode project lives and how it's synchronized into Emacs.
> +
> +Note that not all the independently-maintained packages in Emacs have
> +such a "README" yet.  If you know of one that's missing its "README",
> +please send in a patch or just let us know.

The modus-themes are maintained externally.  I sync them with emacs.git
every month or so when I release a new version.  Their .el files are in
etc/themes/ while the manual is in doc/misc/.

I am happy to provide a README, though neither of those two directories
is specific to the modus-themes.

Maybe I should include the relevant information in the themes' manual
and/or the "Commentary:" of each .el file?

-- 
Protesilaos Stavrou
https://protesilaos.com

Re: Improving documentation of Org Mode integration into Emacs.

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Cc: Eli Zaretskii <eliz@gnu.org>
> Date: Sun, 02 Jan 2022 23:05:05 -0600
> 
> >> I'd like to improve our in-tree documentation of how Org Mode 
> >> integration into GNU Emacs works.
> >
> >Thanks.  Can you provide a rough sketch, in the form of list of 
> >items,
> >of what you'd like to document, and who would be the main 
> >audience for
> >that?  I don't have a clear idea of that based on what you 
> >posted.
> 
> Sure.  (The attached patch may answer your questions more 
> efficiently, though.)

Thanks.

However, IMO the text as written doesn't belong to CONTRIBUTE.  That
file includes actionable instructions to contributors regarding our
development procedures, conventions, and requirements.  The purpose of
those instructions is to make the contributed changes acceptable and
matching our practices.  By contrast, the text that you propose is
just information that is not actionable.  So if this text is to stay
in its present form, it should be somewhere else, perhaps in README.

If you do want it to be in CONTRIBUTE, it mostly be comprised of some
specific instructions what to do or what not to do.  The purely
informational part should ideally be shorter and more focused on what
contributors need to know in order to, well, contribute.

> It's better for us all if such a person finds out as soon as 
> possible where the active upstream for a package is.  For example, 
> if they've found a bug in Org Mode, they should take the bug 
> report to the Org Mode forums and discuss it with the Org Mode 
> maintainers.  (Maybe sometimes Emacs Devel should also be 
> involved, but the person should at least start out in the proper 
> upstream forum and then be guided appropriately from there.)

Please note that none of what you say here is in the text, not even as
a hint.  If the idea is to tell contributors to direct bug reports for
those packages elsewhere, why not tell that explicitly, and why not
include the relevant URLs where those bugs should be reported?

> +See https://orgmode.org/ for more information, and see specifically
> +https://orgmode.org/worg/org-maintenance.html for information about
> +the process of synchronization between upstream Org Mode and Emacs.

The beginning of this is promising, but then it strays: instead of
telling people to send patches and bug reports about Org to those
addresses and not to Emacs's tracker, it invites them to learn about
the process of synchronization between Org and Emacs, something that I
think is of secondary importance (to say the least) for causal
contributors.

Re: Improving documentation of Org Mode integration into Emacs.

[Rudolf Adamkovič]

Karl Fogel <kfogel@red-bean.com> writes:

> Comments welcome; I'm certainly open to other strategies.

I have a small comment. :)

I think we can drop the word "actually" from the first sentence:

"… is actually maintained as a separate project, …"

Rudy
-- 
"Logic is a science of the necessary laws of thought, without which no
employment of the understanding and the reason takes place." -- Immanuel
Kant, 1785

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia