* Convert README.org to plain text README while installing package @ 2022-06-04 13:14 Akib Azmain Turja 2022-06-04 13:32 ` Tassilo Horn 2022-06-04 16:36 ` Stefan Monnier 0 siblings, 2 replies; 505+ messages in thread From: Akib Azmain Turja @ 2022-06-04 13:14 UTC (permalink / raw) To: emacs-devel [-- Attachment #1: Type: text/plain, Size: 613 bytes --] Packages on ELPA or NonGNU ELPA with README.org show pretty description both with describe-package and web browser. But after installing a package, describe-package shows the Org source code, which contains many useless things (I mean useless for showing in the *Help* buffer). Isn't it possible to convert README.org to plain text README while compiling the package, possibly optionally? Changing package--compile function should do the job. -- Akib Azmain Turja This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 13:14 Convert README.org to plain text README while installing package Akib Azmain Turja @ 2022-06-04 13:32 ` Tassilo Horn 2022-06-04 14:17 ` Alan Mackenzie 2022-06-04 17:22 ` Akib Azmain Turja 2022-06-04 16:36 ` Stefan Monnier 1 sibling, 2 replies; 505+ messages in thread From: Tassilo Horn @ 2022-06-04 13:32 UTC (permalink / raw) To: Akib Azmain Turja; +Cc: emacs-devel Akib Azmain Turja <akib@disroot.org> writes: Hi Akib, > Packages on ELPA or NonGNU ELPA with README.org show pretty > description both with describe-package and web browser. But after > installing a package, describe-package shows the Org source code, > which contains many useless things (I mean useless for showing in the > *Help* buffer). > > Isn't it possible to convert README.org to plain text README while > compiling the package, possibly optionally? Changing package--compile > function should do the job. An alternative would be to enable (parts of) org-mode in the *Help* buffer. I've just tried `M-x describe-package RET corfu RET` which comes with a README.org and then enabling org-mode in the *Help* buffer. That looked really nice and didn't seem to cause bad effects except that the [back] / [forward] buttons stopped working. Bye, Tassilo ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 13:32 ` Tassilo Horn @ 2022-06-04 14:17 ` Alan Mackenzie 2022-06-04 14:31 ` Tassilo Horn ` (2 more replies) 2022-06-04 17:22 ` Akib Azmain Turja 1 sibling, 3 replies; 505+ messages in thread From: Alan Mackenzie @ 2022-06-04 14:17 UTC (permalink / raw) To: Tassilo Horn; +Cc: Akib Azmain Turja, emacs-devel Hello, Tassilo. On Sat, Jun 04, 2022 at 15:32:10 +0200, Tassilo Horn wrote: > Akib Azmain Turja <akib@disroot.org> writes: > Hi Akib, > > Packages on ELPA or NonGNU ELPA with README.org show pretty > > description both with describe-package and web browser. But after > > installing a package, describe-package shows the Org source code, > > which contains many useless things (I mean useless for showing in the > > *Help* buffer). > > Isn't it possible to convert README.org to plain text README while > > compiling the package, possibly optionally? Changing package--compile > > function should do the job. > An alternative would be to enable (parts of) org-mode in the *Help* > buffer. I've just tried `M-x describe-package RET corfu RET` which > comes with a README.org and then enabling org-mode in the *Help* buffer. > That looked really nice and didn't seem to cause bad effects except that > the [back] / [forward] buttons stopped working. No, no, no, no! Org mode is a highly complicated, obscure mode which is NOT part of core Emacs, and mustn't become so. What you're proposing is a slippery slope, where ever greater portions of org mode would get pushed into the core, causing ever greater problems for those who do not use org mode. You pointed out one such problem yourself, org mode key bindings will take up key binding space currently used by other modes and by users. Surely the solution has got to be to encourage package authors to write plain text (or .texi) documentation, by pointing out the difficulties the non-standard .org format creates. > Bye, > Tassilo -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 14:17 ` Alan Mackenzie @ 2022-06-04 14:31 ` Tassilo Horn 2022-06-04 17:39 ` Akib Azmain Turja 2022-06-04 16:09 ` Stefan Kangas 2022-06-04 17:32 ` Akib Azmain Turja 2 siblings, 1 reply; 505+ messages in thread From: Tassilo Horn @ 2022-06-04 14:31 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Akib Azmain Turja, emacs-devel Don't panic, Alan! 😁 The idea would be to only activate the font-lock rules of org (or markdown-mode, if available) while somehow protecting the package headers and help-mode footer section (i.e., the buttons). Bye, Tassilo 04.06.2022 16:18:21 Alan Mackenzie <acm@muc.de>: > Hello, Tassilo. > > On Sat, Jun 04, 2022 at 15:32:10 +0200, Tassilo Horn wrote: >> Akib Azmain Turja <akib@disroot.org> writes: > >> Hi Akib, > >>> Packages on ELPA or NonGNU ELPA with README.org show pretty >>> description both with describe-package and web browser. But after >>> installing a package, describe-package shows the Org source code, >>> which contains many useless things (I mean useless for showing in the >>> *Help* buffer). > >>> Isn't it possible to convert README.org to plain text README while >>> compiling the package, possibly optionally? Changing package--compile >>> function should do the job. > >> An alternative would be to enable (parts of) org-mode in the *Help* >> buffer. I've just tried `M-x describe-package RET corfu RET` which >> comes with a README.org and then enabling org-mode in the *Help* buffer. >> That looked really nice and didn't seem to cause bad effects except that >> the [back] / [forward] buttons stopped working. > > No, no, no, no! Org mode is a highly complicated, obscure mode which is > NOT part of core Emacs, and mustn't become so. What you're proposing is > a slippery slope, where ever greater portions of org mode would get > pushed into the core, causing ever greater problems for those who do not > use org mode. > > You pointed out one such problem yourself, org mode key bindings will > take up key binding space currently used by other modes and by users. > > Surely the solution has got to be to encourage package authors to write > plain text (or .texi) documentation, by pointing out the difficulties > the non-standard .org format creates. > >> Bye, >> Tassilo > ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 14:31 ` Tassilo Horn @ 2022-06-04 17:39 ` Akib Azmain Turja 0 siblings, 0 replies; 505+ messages in thread From: Akib Azmain Turja @ 2022-06-04 17:39 UTC (permalink / raw) To: Tassilo Horn, Alan Mackenzie; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 2574 bytes --] Tassilo Horn <tsdh@gnu.org> writes: > Don't panic, Alan! 😁 > > The idea would be to only activate the font-lock rules of org (or markdown-mode, if available) while somehow protecting the package headers and help-mode footer section (i.e., the buttons). But I don't like that idea. I don't think that's a really good idea, since the *Help* buffer is not for only describe-package, its for all purposes. I think converting Org (or Markdown or whatever) to plain text should solve most (if not all) of the problems. Then showing that file shouldn't need any special treatment at all. > > Bye, > Tassilo > > 04.06.2022 16:18:21 Alan Mackenzie <acm@muc.de>: > >> Hello, Tassilo. >> >> On Sat, Jun 04, 2022 at 15:32:10 +0200, Tassilo Horn wrote: >>> Akib Azmain Turja <akib@disroot.org> writes: >> >>> Hi Akib, >> >>>> Packages on ELPA or NonGNU ELPA with README.org show pretty >>>> description both with describe-package and web browser. But after >>>> installing a package, describe-package shows the Org source code, >>>> which contains many useless things (I mean useless for showing in the >>>> *Help* buffer). >> >>>> Isn't it possible to convert README.org to plain text README while >>>> compiling the package, possibly optionally? Changing package--compile >>>> function should do the job. >> >>> An alternative would be to enable (parts of) org-mode in the *Help* >>> buffer. I've just tried `M-x describe-package RET corfu RET` which >>> comes with a README.org and then enabling org-mode in the *Help* buffer. >>> That looked really nice and didn't seem to cause bad effects except that >>> the [back] / [forward] buttons stopped working. >> >> No, no, no, no! Org mode is a highly complicated, obscure mode which is >> NOT part of core Emacs, and mustn't become so. What you're proposing is >> a slippery slope, where ever greater portions of org mode would get >> pushed into the core, causing ever greater problems for those who do not >> use org mode. >> >> You pointed out one such problem yourself, org mode key bindings will >> take up key binding space currently used by other modes and by users. >> >> Surely the solution has got to be to encourage package authors to write >> plain text (or .texi) documentation, by pointing out the difficulties >> the non-standard .org format creates. >> >>> Bye, >>> Tassilo >> -- Akib Azmain Turja This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 14:17 ` Alan Mackenzie 2022-06-04 14:31 ` Tassilo Horn @ 2022-06-04 16:09 ` Stefan Kangas 2022-06-04 17:27 ` Alan Mackenzie 2022-06-04 17:32 ` Akib Azmain Turja 2 siblings, 1 reply; 505+ messages in thread From: Stefan Kangas @ 2022-06-04 16:09 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Tassilo Horn, Akib Azmain Turja, Emacs developers Alan Mackenzie <acm@muc.de> writes: > No, no, no, no! Org mode is a highly complicated, obscure mode which is > NOT part of core Emacs, and mustn't become so. AFAICT, Org-mode has been distributed with Emacs for more than five major releases by now (approx. 15 years). > Surely the solution has got to be to encourage package authors to write > plain text (or .texi) documentation, by pointing out the difficulties > the non-standard .org format creates. In the Emacs community, Org-mode is ubiquitous. I think we should actively encourage using it for most types of documentation, certainly for README type files. Texinfo has its strengths of course, but it is unfortunately perceived as arcane and hard to use by many. And Org-mode can be converted to be read in 'M-x info'. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 16:09 ` Stefan Kangas @ 2022-06-04 17:27 ` Alan Mackenzie 2022-06-04 21:10 ` Stefan Kangas 2022-06-05 8:38 ` Michael Albinus 0 siblings, 2 replies; 505+ messages in thread From: Alan Mackenzie @ 2022-06-04 17:27 UTC (permalink / raw) To: Stefan Kangas; +Cc: Tassilo Horn, Akib Azmain Turja, Emacs developers Hello, Stefan. On Sat, Jun 04, 2022 at 18:09:45 +0200, Stefan Kangas wrote: > Alan Mackenzie <acm@muc.de> writes: > > No, no, no, no! Org mode is a highly complicated, obscure mode which > > is NOT part of core Emacs, and mustn't become so. > AFAICT, Org-mode has been distributed with Emacs for more than five > major releases by now (approx. 15 years). That's a strawman. Org mode is not part of the Emacs core. It's not something in the part of Emacs inevitably used by any normal proficient user. Neither is Gnus. Neither is CC Mode, for that matter. By contrast *Help*, Info, font-lock, the package manager, a lot of other things, and a vast selection of commands bound to keys in the global key map, are the core. > > Surely the solution has got to be to encourage package authors to > > write plain text (or .texi) documentation, by pointing out the > > difficulties the non-standard .org format creates. > In the Emacs community, Org-mode is ubiquitous. Only in the sense that it's distributed with each Emacs release. > I think we should actively encourage using it for most types of > documentation, certainly for README type files. I disagree entirely with you. Org mode is highly complicated, obscure (I've never managed to get a feel for what it does), and difficult to learn (I've never managed it). A text file is far easier to read for those not familiar with org. We're talking about READMEs here. They're typically 20 to 100 lines long. A text file is ideal for these. > Texinfo has its strengths of course, but it is unfortunately perceived > as arcane and hard to use by many. It's moderately easy to use, somewhat difficult to learn. > And Org-mode can be converted to be read in 'M-x info'. You mean it _has_ to be converted to be read? -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 17:27 ` Alan Mackenzie @ 2022-06-04 21:10 ` Stefan Kangas 2022-06-05 8:38 ` Michael Albinus 1 sibling, 0 replies; 505+ messages in thread From: Stefan Kangas @ 2022-06-04 21:10 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Tassilo Horn, Akib Azmain Turja, Emacs developers Alan Mackenzie <acm@muc.de> writes: > We're talking about READMEs here. They're > typically 20 to 100 lines long. A text file is ideal for these. A text file is fine of course, but they lack useful features such as including images (e.g. screenshots). You would need some markup for that, and Org-mode happens to have solid built-in support. > > Texinfo has its strengths of course, but it is unfortunately perceived > > as arcane and hard to use by many. > > It's moderately easy to use, somewhat difficult to learn. Agreed. > > And Org-mode can be converted to be read in 'M-x info'. > > You mean it _has_ to be converted to be read? I don't think so, no. I'm just saying that you can do what we do with the org manual, so you get to write Org-mode, but still get to read in 'M-x info'. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 17:27 ` Alan Mackenzie 2022-06-04 21:10 ` Stefan Kangas @ 2022-06-05 8:38 ` Michael Albinus 2022-06-05 8:51 ` Po Lu 1 sibling, 1 reply; 505+ messages in thread From: Michael Albinus @ 2022-06-05 8:38 UTC (permalink / raw) To: Alan Mackenzie Cc: Stefan Kangas, Tassilo Horn, Akib Azmain Turja, Emacs developers Alan Mackenzie <acm@muc.de> writes: > Hello, Stefan. Hi, >> I think we should actively encourage using it for most types of >> documentation, certainly for README type files. > > I disagree entirely with you. Org mode is highly complicated, obscure > (I've never managed to get a feel for what it does), and difficult to > learn (I've never managed it). A text file is far easier to read for > those not familiar with org. We're talking about READMEs here. They're > typically 20 to 100 lines long. A text file is ideal for these. 1+. README is a text file which is consulted in order to get information about the related software. There's no rule to visit it by Emacs only. Best regards, Michael. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 8:38 ` Michael Albinus @ 2022-06-05 8:51 ` Po Lu 2022-06-05 10:26 ` Tassilo Horn 2022-06-05 15:15 ` Ihor Radchenko 0 siblings, 2 replies; 505+ messages in thread From: Po Lu @ 2022-06-05 8:51 UTC (permalink / raw) To: Michael Albinus Cc: Alan Mackenzie, Stefan Kangas, Tassilo Horn, Akib Azmain Turja, Emacs developers Michael Albinus <michael.albinus@gmx.de> writes: > Alan Mackenzie <acm@muc.de> writes: > >> Hello, Stefan. > > Hi, > >>> I think we should actively encourage using it for most types of >>> documentation, certainly for README type files. >> >> I disagree entirely with you. Org mode is highly complicated, obscure >> (I've never managed to get a feel for what it does), and difficult to >> learn (I've never managed it). A text file is far easier to read for >> those not familiar with org. We're talking about READMEs here. They're >> typically 20 to 100 lines long. A text file is ideal for these. > > 1+. README is a text file which is consulted in order to get information > about the related software. There's no rule to visit it by Emacs only. > > Best regards, Michael. Hear, hear! It also takes a relatively long time to load Org mode, so opening a README.org-type file for the first time will always cause an annoying pause. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 8:51 ` Po Lu @ 2022-06-05 10:26 ` Tassilo Horn 2022-06-05 11:15 ` Michael Albinus ` (5 more replies) 2022-06-05 15:15 ` Ihor Radchenko 1 sibling, 6 replies; 505+ messages in thread From: Tassilo Horn @ 2022-06-05 10:26 UTC (permalink / raw) To: Po Lu Cc: Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Po Lu <luangruo@yahoo.com> writes: >>>> I think we should actively encourage using it for most types of >>>> documentation, certainly for README type files. >>> >>> I disagree entirely with you. Org mode is highly complicated, >>> obscure (I've never managed to get a feel for what it does), and >>> difficult to learn (I've never managed it). A text file is far >>> easier to read for those not familiar with org. We're talking about >>> READMEs here. They're typically 20 to 100 lines long. A text file >>> is ideal for these. >> >> 1+. README is a text file which is consulted in order to get >> information about the related software. There's no rule to visit it >> by Emacs only. People nowadays write README.org and README.md files instead of plain-text READMEs because the packages' repositories are hosted on some forge (sr.ht, github, gitlab, gitea, whatever) which supports rendering those formats in a nice way. So the choice of format is natural for package authors. And they frequently contain markup like headings, bullet lists, bold, italics, and code snippets which simply look much better than their text-only alternative (even in their editing/non-rendered versions). > Hear, hear! > > It also takes a relatively long time to load Org mode, so opening a > README.org-type file for the first time will always cause an annoying > pause. That's true but wouldn't it be possible to extract the font-lock/faces related parts in a separate file with no dependencies which can be loaded quickly enough? I mean, the purpose would only be to display org and markdown READMEs with the highlighting intended by the package author but there's no point in enabling the gazillon of other features (from editing to time-tracking to spreadsheet computations) those packages might provide in addition. Bye, Tassilo ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 10:26 ` Tassilo Horn @ 2022-06-05 11:15 ` Michael Albinus 2022-06-05 16:52 ` [External] : " Drew Adams 2022-06-06 0:19 ` Tim Cross 2022-06-05 11:23 ` Ihor Radchenko ` (4 subsequent siblings) 5 siblings, 2 replies; 505+ messages in thread From: Michael Albinus @ 2022-06-05 11:15 UTC (permalink / raw) To: Tassilo Horn Cc: Po Lu, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Tassilo Horn <tsdh@gnu.org> writes: Hi Tassilo, >>>> I disagree entirely with you. Org mode is highly complicated, >>>> obscure (I've never managed to get a feel for what it does), and >>>> difficult to learn (I've never managed it). A text file is far >>>> easier to read for those not familiar with org. We're talking about >>>> READMEs here. They're typically 20 to 100 lines long. A text file >>>> is ideal for these. >>> >>> 1+. README is a text file which is consulted in order to get >>> information about the related software. There's no rule to visit it >>> by Emacs only. > > People nowadays write README.org and README.md files instead of > plain-text READMEs because the packages' repositories are hosted on some > forge (sr.ht, github, gitlab, gitea, whatever) which supports rendering > those formats in a nice way. So the choice of format is natural for > package authors. And they frequently contain markup like headings, > bullet lists, bold, italics, and code snippets which simply look much > better than their text-only alternative (even in their > editing/non-rendered versions). Not everybody uses Github/Gitlab/<you name it> infrastructure exclusively. There are dinosaurs like me, who simply inspect tarballs and alike. I have no problem if there are structured README.org or README.md files in parallel. But a README file should be plain text. > Bye, > Tassilo Best regards, Michael. ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-05 11:15 ` Michael Albinus @ 2022-06-05 16:52 ` Drew Adams 2022-06-06 0:19 ` Tim Cross 1 sibling, 0 replies; 505+ messages in thread From: Drew Adams @ 2022-06-05 16:52 UTC (permalink / raw) To: Michael Albinus, Tassilo Horn Cc: Po Lu, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel@gnu.org > I have no problem if there are structured README.org or README.md files > in parallel. But a README file should be plain text. This. Not just either/or. Plain-text README should be a minimal requirement. Other formats are nice-to-have, for some contexts & some users. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 11:15 ` Michael Albinus 2022-06-05 16:52 ` [External] : " Drew Adams @ 2022-06-06 0:19 ` Tim Cross 2022-06-06 9:59 ` Philip Kaludercic ` (3 more replies) 1 sibling, 4 replies; 505+ messages in thread From: Tim Cross @ 2022-06-06 0:19 UTC (permalink / raw) To: emacs-devel Michael Albinus <michael.albinus@gmx.de> writes: > > I have no problem if there are structured README.org or README.md files > in parallel. But a README file should be plain text. > I've seen this mentioned multiple times in this thread and it doesn't make sense to me. Org files *are* plain text. This is one of (perhaps the biggest) selling points for org mode. They don't use any form of 'binary' data and can be read just fine in any text editor or just using cat/less/more whatever. They may look a little *ugly*, especially with respect to URLs, but are still quite readable - a lot more readable than other plain text formats such as xml or html or json etc. I also find arguments based around org being complex and difficult to learn to be somewhat overstated. Org is powerful and very configurable, which means there can be a lot to learn if you want to leverage off all it has to offer. However, like emacs, the basics are very simple and easy to learn. While I'm not arguing that org should be forced upon everyone and I would agree we need to keep potential load time issues in mind, there seems to be lots in this thread over stating the issues and jumping to extremes. All that seems to really be under consideration is to enable rendering of *org files in help buffers using org font locking and perhaps enabling folding, which could be very beneficial for large readme files and would not matter for small ones. I also suspect this is something which could be disabled with a simple variable setting for those who really don't like it. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 0:19 ` Tim Cross @ 2022-06-06 9:59 ` Philip Kaludercic 2022-06-06 13:47 ` Tim Cross 2022-06-06 11:33 ` Alan Mackenzie ` (2 subsequent siblings) 3 siblings, 1 reply; 505+ messages in thread From: Philip Kaludercic @ 2022-06-06 9:59 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel Tim Cross <theophilusx@gmail.com> writes: > Michael Albinus <michael.albinus@gmx.de> writes: > >> >> I have no problem if there are structured README.org or README.md files >> in parallel. But a README file should be plain text. >> > > I've seen this mentioned multiple times in this thread and it doesn't > make sense to me. > > Org files *are* plain text. This is one of (perhaps the biggest) selling > points for org mode. They don't use any form of 'binary' data and can be > read just fine in any text editor or just using cat/less/more whatever. > They may look a little *ugly*, especially with respect to URLs, but are > still quite readable - a lot more readable than other plain text formats > such as xml or html or json etc. I have consult installed, and if I want to read the package summary generated by C-h P, I half of the buffer is just Org configuration: --8<---------------cut here---------------start------------->8--- #+title: consult.el - Consulting completing-read #+author: Daniel Mendler #+language: en #+export_file_name: consult.texi #+texinfo_dir_category: Emacs misc features #+texinfo_dir_title: Consult: (consult). #+texinfo_dir_desc: Useful commands built on completing-read. #+html: <a href="https://www.gnu.org/software/emacs/"><img alt="GNU Emacs" src="https://github.com/minad/corfu/blob/screenshots/emacs.svg?raw=true"/></a> #+html: <a href="http://elpa.gnu.org/packages/consult.html"><img alt="GNU ELPA" src="https://elpa.gnu.org/packages/consult.svg"/></a> #+html: <a href="http://elpa.gnu.org/devel/consult.html"><img alt="GNU-devel ELPA" src="https://elpa.gnu.org/devel/consult.svg"/></a> #+html: <a href="https://melpa.org/#/consult"><img alt="MELPA" src="https://melpa.org/packages/consult-badge.svg"/></a> #+html: <a href="https://stable.melpa.org/#/consult"><img alt="MELPA Stable" src="https://stable.melpa.org/packages/consult-badge.svg"/></a> * Introduction :properties: :description: Why Consult? :end: #+cindex: introduction --8<---------------cut here---------------end--------------->8--- While plain text, it is not what I am interested in at this point. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 9:59 ` Philip Kaludercic @ 2022-06-06 13:47 ` Tim Cross 2022-06-06 14:15 ` Philip Kaludercic 0 siblings, 1 reply; 505+ messages in thread From: Tim Cross @ 2022-06-06 13:47 UTC (permalink / raw) To: Philip Kaludercic; +Cc: emacs-devel Philip Kaludercic <philipk@posteo.net> writes: > Tim Cross <theophilusx@gmail.com> writes: > >> Michael Albinus <michael.albinus@gmx.de> writes: >> >>> >>> I have no problem if there are structured README.org or README.md files >>> in parallel. But a README file should be plain text. >>> >> >> I've seen this mentioned multiple times in this thread and it doesn't >> make sense to me. >> >> Org files *are* plain text. This is one of (perhaps the biggest) selling >> points for org mode. They don't use any form of 'binary' data and can be >> read just fine in any text editor or just using cat/less/more whatever. >> They may look a little *ugly*, especially with respect to URLs, but are >> still quite readable - a lot more readable than other plain text formats >> such as xml or html or json etc. > > I have consult installed, and if I want to read the package summary > generated by C-h P, I half of the buffer is just Org configuration: > > #+title: consult.el - Consulting completing-read > #+author: Daniel Mendler > #+language: en > #+export_file_name: consult.texi > #+texinfo_dir_category: Emacs misc features > #+texinfo_dir_title: Consult: (consult). > #+texinfo_dir_desc: Useful commands built on completing-read. > > #+html: <a href="https://www.gnu.org/software/emacs/"><img alt="GNU Emacs" src="https://github.com/minad/corfu/blob/screenshots/emacs.svg?raw=true"/></a> > #+html: <a href="http://elpa.gnu.org/packages/consult.html"><img alt="GNU ELPA" src="https://elpa.gnu.org/packages/consult.svg"/></a> > #+html: <a href="http://elpa.gnu.org/devel/consult.html"><img alt="GNU-devel ELPA" src="https://elpa.gnu.org/devel/consult.svg"/></a> > #+html: <a href="https://melpa.org/#/consult"><img alt="MELPA" src="https://melpa.org/packages/consult-badge.svg"/></a> > #+html: <a href="https://stable.melpa.org/#/consult"><img alt="MELPA Stable" src="https://stable.melpa.org/packages/consult-badge.svg"/></a> > > * Introduction > :properties: > :description: Why Consult? > :end: > #+cindex: introduction > > While plain text, it is not what I am interested in at this point. and in what seems to be typical in this thread, you have selected an extreme example to demonstrate your point. This is definitely not a typical README.org file - at least not from my experience. This is also a completely pointless debate as Stefan has already indicated auto translation to ascii will likely be what is done anyway. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 13:47 ` Tim Cross @ 2022-06-06 14:15 ` Philip Kaludercic 0 siblings, 0 replies; 505+ messages in thread From: Philip Kaludercic @ 2022-06-06 14:15 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel Tim Cross <theophilusx@gmail.com> writes: > Philip Kaludercic <philipk@posteo.net> writes: > >> Tim Cross <theophilusx@gmail.com> writes: >> >>> Michael Albinus <michael.albinus@gmx.de> writes: >>> >>>> >>>> I have no problem if there are structured README.org or README.md files >>>> in parallel. But a README file should be plain text. >>>> >>> >>> I've seen this mentioned multiple times in this thread and it doesn't >>> make sense to me. >>> >>> Org files *are* plain text. This is one of (perhaps the biggest) selling >>> points for org mode. They don't use any form of 'binary' data and can be >>> read just fine in any text editor or just using cat/less/more whatever. >>> They may look a little *ugly*, especially with respect to URLs, but are >>> still quite readable - a lot more readable than other plain text formats >>> such as xml or html or json etc. >> >> I have consult installed, and if I want to read the package summary >> generated by C-h P, I half of the buffer is just Org configuration: >> >> #+title: consult.el - Consulting completing-read >> #+author: Daniel Mendler >> #+language: en >> #+export_file_name: consult.texi >> #+texinfo_dir_category: Emacs misc features >> #+texinfo_dir_title: Consult: (consult). >> #+texinfo_dir_desc: Useful commands built on completing-read. >> >> #+html: <a href="https://www.gnu.org/software/emacs/"><img alt="GNU >> Emacs" >> src="https://github.com/minad/corfu/blob/screenshots/emacs.svg?raw=true"/></a> >> #+html: <a href="http://elpa.gnu.org/packages/consult.html"><img >> alt="GNU ELPA" src="https://elpa.gnu.org/packages/consult.svg"/></a> >> #+html: <a href="http://elpa.gnu.org/devel/consult.html"><img >> alt="GNU-devel ELPA" >> src="https://elpa.gnu.org/devel/consult.svg"/></a> >> #+html: <a href="https://melpa.org/#/consult"><img alt="MELPA" >> src="https://melpa.org/packages/consult-badge.svg"/></a> >> #+html: <a href="https://stable.melpa.org/#/consult"><img alt="MELPA >> Stable" >> src="https://stable.melpa.org/packages/consult-badge.svg"/></a> >> >> * Introduction >> :properties: >> :description: Why Consult? >> :end: >> #+cindex: introduction >> >> While plain text, it is not what I am interested in at this point. > > and in what seems to be typical in this thread, you have selected an > extreme example to demonstrate your point. This is definitely not a > typical README.org file - at least not from my experience. While true, I don't get why this is a problem. I don't think making the package developers simplify their READMEs would a better solution, since they are also frequently used to generate manuals. As to whether or not the package description should be the same as the manual, is a different question (I am inclined to think the package description should be a brief summary of a few paragraphs at most). ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 0:19 ` Tim Cross 2022-06-06 9:59 ` Philip Kaludercic @ 2022-06-06 11:33 ` Alan Mackenzie 2022-06-06 12:26 ` Akib Azmain Turja 2022-06-06 13:57 ` Tim Cross 2022-06-06 16:56 ` Michael Albinus 2022-06-07 20:57 ` Jean Louis 3 siblings, 2 replies; 505+ messages in thread From: Alan Mackenzie @ 2022-06-06 11:33 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel Hello, Tim. On Mon, Jun 06, 2022 at 10:19:38 +1000, Tim Cross wrote: > Michael Albinus <michael.albinus@gmx.de> writes: > > I have no problem if there are structured README.org or README.md > > files in parallel. But a README file should be plain text. I agree with this. > I've seen this mentioned multiple times in this thread and it doesn't > make sense to me. > Org files *are* plain text. This is one of (perhaps the biggest) selling > points for org mode. They don't use any form of 'binary' data and can be > read just fine in any text editor or just using cat/less/more whatever. We're reduced to arguing about the meaning of "plain text". The way I see it, plain text means to be read as is by the user. In that sense, the formats you mention below, xml, html, etc. are _not_ plain text - they're designed for machine processing. A typical spam email in HTML has the human readable pieces swamped by the machine readable pieces. I think you're arguing that this isn't the case with org mode files, though Philip Kaludercic pointed out an org file where this wasn't the case. > They may look a little *ugly*, especially with respect to URLs, but are > still quite readable - a lot more readable than other plain text formats > such as xml or html or json etc. And their performance in standard programs like grep, wc, etc. is that much worse than plain text. > I also find arguments based around org being complex and difficult to > learn to be somewhat overstated. There are 784 key bindings in org mode. How can you say that this isn't complex and difficult to learn? > Org is powerful and very configurable, .... That contradicts your previous statement to some extent. Any program which is very configurable _needs_ to be configured. > .... which means there can be a lot to learn if you want to leverage > off all it has to offer. I don't want to do this. I want to avoid org mode being loaded into my Emacs; it fouls up my key bindings to a significant extent. Particularly if I just want to read a 50 line README. > However, like emacs, the basics are very simple and easy to learn. I don't agree that the basics of Emacs are simple and easy to learn at all. That's a strong reason why other editors have become popular. Why should I have to spend time learning a super-complicated mode just to read a 50 line README? > While I'm not arguing that org should be forced upon everyone .... If a README file is README.org, then org mode is forced upon anybody wishing to read it in Emacs. This is why README files should be plain text. > .... and I would agree we need to keep potential load time issues in > mind, there seems to be lots in this thread over stating the issues and > jumping to extremes. I think org mode is an extreme, with its 784 key bindings which seem to violate written and unwritten conventions. > All that seems to really be under consideration is to enable rendering > of *org files in help buffers using org font locking and perhaps > enabling folding, which could be very beneficial for large readme files > and would not matter for small ones. I also suspect this is something > which could be disabled with a simple variable setting for those who > really don't like it. "It" could best be avoided by having plain text README files. -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 11:33 ` Alan Mackenzie @ 2022-06-06 12:26 ` Akib Azmain Turja 2022-06-06 13:57 ` Tim Cross 1 sibling, 0 replies; 505+ messages in thread From: Akib Azmain Turja @ 2022-06-06 12:26 UTC (permalink / raw) To: Alan Mackenzie, Tim Cross; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 3871 bytes --] Alan Mackenzie <acm@muc.de> writes: > Hello, Tim. > > On Mon, Jun 06, 2022 at 10:19:38 +1000, Tim Cross wrote: > >> Michael Albinus <michael.albinus@gmx.de> writes: > > >> > I have no problem if there are structured README.org or README.md >> > files in parallel. But a README file should be plain text. > > I agree with this. > >> I've seen this mentioned multiple times in this thread and it doesn't >> make sense to me. > >> Org files *are* plain text. This is one of (perhaps the biggest) selling >> points for org mode. They don't use any form of 'binary' data and can be >> read just fine in any text editor or just using cat/less/more whatever. > > We're reduced to arguing about the meaning of "plain text". The way I > see it, plain text means to be read as is by the user. In that sense, > the formats you mention below, xml, html, etc. are _not_ plain text - > they're designed for machine processing. A typical spam email in HTML > has the human readable pieces swamped by the machine readable pieces. I > think you're arguing that this isn't the case with org mode files, though > Philip Kaludercic pointed out an org file where this wasn't the case. > >> They may look a little *ugly*, especially with respect to URLs, but are >> still quite readable - a lot more readable than other plain text formats >> such as xml or html or json etc. > > And their performance in standard programs like grep, wc, etc. is that > much worse than plain text. > >> I also find arguments based around org being complex and difficult to >> learn to be somewhat overstated. > > There are 784 key bindings in org mode. How can you say that this isn't > complex and difficult to learn? > >> Org is powerful and very configurable, .... > > That contradicts your previous statement to some extent. Any program > which is very configurable _needs_ to be configured. > >> .... which means there can be a lot to learn if you want to leverage >> off all it has to offer. > > I don't want to do this. I want to avoid org mode being loaded into my > Emacs; it fouls up my key bindings to a significant extent. Particularly > if I just want to read a 50 line README. > >> However, like emacs, the basics are very simple and easy to learn. > > I don't agree that the basics of Emacs are simple and easy to learn at > all. That's a strong reason why other editors have become popular. Why > should I have to spend time learning a super-complicated mode just to > read a 50 line README? > >> While I'm not arguing that org should be forced upon everyone .... > > If a README file is README.org, then org mode is forced upon anybody > wishing to read it in Emacs. This is why README files should be plain > text. > >> .... and I would agree we need to keep potential load time issues in >> mind, there seems to be lots in this thread over stating the issues and >> jumping to extremes. > > I think org mode is an extreme, with its 784 key bindings which seem to > violate written and unwritten conventions. > >> All that seems to really be under consideration is to enable rendering >> of *org files in help buffers using org font locking and perhaps >> enabling folding, which could be very beneficial for large readme files >> and would not matter for small ones. I also suspect this is something >> which could be disabled with a simple variable setting for those who >> really don't like it. > > "It" could best be avoided by having plain text README files. > > -- > Alan Mackenzie (Nuremberg, Germany). > Just to add, many of those 784 keybindings of Org are DWIM or context-dependent, which makes it even harder to learn. -- Akib Azmain Turja This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 11:33 ` Alan Mackenzie 2022-06-06 12:26 ` Akib Azmain Turja @ 2022-06-06 13:57 ` Tim Cross 2022-06-06 16:02 ` Eli Zaretskii ` (2 more replies) 1 sibling, 3 replies; 505+ messages in thread From: Tim Cross @ 2022-06-06 13:57 UTC (permalink / raw) To: Alan Mackenzie; +Cc: emacs-devel Alan Mackenzie <acm@muc.de> writes: > Hello, Tim. > > On Mon, Jun 06, 2022 at 10:19:38 +1000, Tim Cross wrote: > >> Michael Albinus <michael.albinus@gmx.de> writes: > > >> > I have no problem if there are structured README.org or README.md >> > files in parallel. But a README file should be plain text. > > I agree with this. > >> I've seen this mentioned multiple times in this thread and it doesn't >> make sense to me. > >> Org files *are* plain text. This is one of (perhaps the biggest) selling >> points for org mode. They don't use any form of 'binary' data and can be >> read just fine in any text editor or just using cat/less/more whatever. > > We're reduced to arguing about the meaning of "plain text". The way I > see it, plain text means to be read as is by the user. In that sense, > the formats you mention below, xml, html, etc. are _not_ plain text - > they're designed for machine processing. A typical spam email in HTML > has the human readable pieces swamped by the machine readable pieces. I > think you're arguing that this isn't the case with org mode files, though > Philip Kaludercic pointed out an org file where this wasn't the case. > Philip's example is an extreme case and not representative of normal README.org files. >> They may look a little *ugly*, especially with respect to URLs, but are >> still quite readable - a lot more readable than other plain text formats >> such as xml or html or json etc. > > And their performance in standard programs like grep, wc, etc. is that > much worse than plain text. No, I disagree with this completely. Org mode markup is very simple and rarely has any impact with regards to grep, ag or any other 'shell' tool It is precisely why I consider it to be a plain text format - all of these standard plain trext processing tools work in the main just as well as they do on any ASCII file. Again, org mode is not like XML or HTML or other formats which do make such processing difficult. > >> I also find arguments based around org being complex and difficult to >> learn to be somewhat overstated. > > There are 784 key bindings in org mode. How can you say that this isn't > complex and difficult to learn? > Very simply because the vast majority of those key bindings only come into play when yhou use advanced features of org mode, such as the agenda or table editing or noweb mode. If your just using basic org mode features, very very few of those key bindings even come into play. Just go throgh that list and remove any bindings relating to agenda mode, table editing mode, source block modes, clock table mode, list editing mode, etc and you end up with very few key bindings (all of which are udner the C-c prefix, unlike your previous claim). >> Org is powerful and very configurable, .... > > That contradicts your previous statement to some extent. Any program > which is very configurable _needs_ to be configured. > ABsolute nonsense. Just because you can configure somethig doesn't automatically mean you have to configure it. Emacs is extremely configurable, but you can use it jsut fine without doing any configuration at all. >> .... which means there can be a lot to learn if you want to leverage >> off all it has to offer. > > I don't want to do this. I want to avoid org mode being loaded into my > Emacs; it fouls up my key bindings to a significant extent. Particularly > if I just want to read a 50 line README. > Other posts have already explained this is NOT what is being proposed and that it would not affect your key bindings in the slightest. All that is being propsoed here is that a read only buffer will use org mode to format/display the content. Very simple and not the monster you insist. >> However, like emacs, the basics are very simple and easy to learn. > > I don't agree that the basics of Emacs are simple and easy to learn at > all. That's a strong reason why other editors have become popular. Why > should I have to spend time learning a super-complicated mode just to > read a 50 line README? > Well, basically, you don't. That has already bene explained and does not need to be reitterated here. However, to be clear, all that is being proposed is that org formatting is applied to the contgents of this read-only buffer. There will not be huge numbers of unwanted/unexptrected key bindings and there won't be a huge number of other changes. You will likely have some additional key bindings which may make navigating larger README files easier and that is about it. All that is really being proposed here is org font locking and outline mode navigation. All very simple really. >> While I'm not arguing that org should be forced upon everyone .... > > If a README file is README.org, then org mode is forced upon anybody > wishing to read it in Emacs. This is why README files should be plain > text. > >> .... and I would agree we need to keep potential load time issues in >> mind, there seems to be lots in this thread over stating the issues and >> jumping to extremes. > > I think org mode is an extreme, with its 784 key bindings which seem to > violate written and unwritten conventions. That 750 key bindings is an extreme over statement and not what is being proposed. Yet again, people jumping to extremes based on ignorance and paranoia. Spend the time to go through the key bindings listed in the org manual and you will soon realise that the majority of those bindings only come into effect in specialised mdoes - none of which are relevant to a READM.org ile (for example, all the key bindings relating to agenda mode). I find it odd that someone can on one hand argue so hard against using a mode based on the complexity and vast nubmer of key bindings and at the same time admit they have never spent the time to learn the mode. This seems like an arguument based on fear and uncertainty about something you don't know rather thaa one based on fact. Perhaps look more closely at what was actually being proposed (which was not a full org mode situation) and spend enough time to at least udnerstand the basics before condemnation. > >> All that seems to really be under consideration is to enable rendering >> of *org files in help buffers using org font locking and perhaps >> enabling folding, which could be very beneficial for large readme files >> and would not matter for small ones. I also suspect this is something >> which could be disabled with a simple variable setting for those who >> really don't like it. > > "It" could best be avoided by having plain text README files. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 13:57 ` Tim Cross @ 2022-06-06 16:02 ` Eli Zaretskii 2022-06-07 6:14 ` Tim Cross 2022-06-06 19:19 ` Alan Mackenzie 2022-06-07 0:43 ` Po Lu 2 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-06 16:02 UTC (permalink / raw) To: Tim Cross; +Cc: acm, emacs-devel > From: Tim Cross <theophilusx@gmail.com> > Cc: emacs-devel@gnu.org > Date: Mon, 06 Jun 2022 23:57:55 +1000 > > > There are 784 key bindings in org mode. How can you say that this isn't > > complex and difficult to learn? > > > Very simply because the vast majority of those key bindings only come > into play when yhou use advanced features of org mode, such as the > agenda or table editing or noweb mode. If your just using basic org mode > features, very very few of those key bindings even come into play. Just > go throgh that list and remove any bindings relating to agenda mode, > table editing mode, source block modes, clock table mode, list editing > mode, etc and you end up with very few key bindings (all of which are > udner the C-c prefix, unlike your previous claim). How about if you do this exercise and walk us through the results? Visit an Org file, type "C-h b", then show us the list, and tell which bindings you think are "basic" and which are "advanced", and why you think so. I can tell you what I see with my very simple 1570-line Org file, which just has some todo items organized in 4-level hierarchy: . ~100 keys that look "general Org" to me . ~30 keys that Org remaps to its s own commands, like 'yank' and 'backward-sentence' . ~40 keys bound to org-babel-SOMETHING -- no idea why these are bound by default, but they are there . ~60 more keys that I'm not sure whether they are "basic" or not, but they are all bound by default, just because I visited an Org file So "just" 230 key bindings. And this is without any minor/add-on Org mode/feature enabled, at least according to "C-h m". Do you see something different? Are you still saying that it's not a lot, or that it's "based on ignorance and paranoia"? If so, please point out where I'm ignorant and/or paranoiac, because I'd really like to know. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 16:02 ` Eli Zaretskii @ 2022-06-07 6:14 ` Tim Cross 2022-06-07 11:21 ` Eli Zaretskii 2022-06-07 16:02 ` Alan Mackenzie 0 siblings, 2 replies; 505+ messages in thread From: Tim Cross @ 2022-06-07 6:14 UTC (permalink / raw) To: Eli Zaretskii; +Cc: acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Tim Cross <theophilusx@gmail.com> >> Cc: emacs-devel@gnu.org >> Date: Mon, 06 Jun 2022 23:57:55 +1000 >> >> > There are 784 key bindings in org mode. How can you say that this isn't >> > complex and difficult to learn? >> > >> Very simply because the vast majority of those key bindings only come >> into play when yhou use advanced features of org mode, such as the >> agenda or table editing or noweb mode. If your just using basic org mode >> features, very very few of those key bindings even come into play. Just >> go throgh that list and remove any bindings relating to agenda mode, >> table editing mode, source block modes, clock table mode, list editing >> mode, etc and you end up with very few key bindings (all of which are >> udner the C-c prefix, unlike your previous claim). > > How about if you do this exercise and walk us through the results? > Visit an Org file, type "C-h b", then show us the list, and tell which > bindings you think are "basic" and which are "advanced", and why you > think so. > > I can tell you what I see with my very simple 1570-line Org file, > which just has some todo items organized in 4-level hierarchy: > > . ~100 keys that look "general Org" to me > . ~30 keys that Org remaps to its s own commands, like 'yank' and > 'backward-sentence' > . ~40 keys bound to org-babel-SOMETHING -- no idea why these are > bound by default, but they are there > . ~60 more keys that I'm not sure whether they are "basic" or not, > but they are all bound by default, just because I visited an Org > file > > So "just" 230 key bindings. And this is without any minor/add-on Org > mode/feature enabled, at least according to "C-h m". > > Do you see something different? Are you still saying that it's not a > lot, or that it's "based on ignorance and paranoia"? If so, please > point out where I'm ignorant and/or paranoiac, because I'd really like > to know. I think Alan's response has pretty much confirmed what I was saying. Alan has made it fairly clear that his real underlying concern is about a stealthy strategy to get org mode more intrinsically tied into Emacs and in the end, making it so critical to normal operation that everyone will be forced to learn org mode regardless. I don't necessarily agree with such an argument, but think having a debate around that would be more up-front and in some ways honest compared to what appear to be somewhat contrived alternative arguments about excessive key bindings and added complexity. My characterisation of this all being fear and paranoia may have been overstating things, but then again I'm not sure. There certainly does seem to be considerable fear involved. I don't know about paranoia, but I have not seen any evil plan to have org mode assimilate Emacs as if it was the Borg and I've seen no discussions on the org list about ways to get org more ingrained in Emacs or make Emacs more dependent on org. With respect to ignorance, Alan made it quite clear he has not spent the time or effort to learn org mode, so he is ignorant about its operation and lacks any real understanding of its operation. He has made assertions about the number of key bindings which are wildly over inflated and about required levels of complexity which simply would not be necessary with the org mode rendering of a read only buffer. In short, he has made a lot of assumptions about both the key bindings and complexity based on only a fairly cursory scan of the manual and very limited experience. All that appears to have occurred here is that someone noted that packages often had a readme.org file and wondered if this should be translated into just a plain readme file. Someone else suggested we could have package descriptions render the readme.org file using org and not require export into plain readme file and mentioned some possible advantages this could have wrt navigation, rendering of tables, links and images or inclusion of source code snippets, examples etc. From this point, some somewhat extreme claims and arguments have been thrown in. One was Alan's claim that org would introduce 785 new key bindings, many of which not bounmd to C-c and which would presumably either pollute the key space or conflict with existing bindings. All I have done is point out this claim was inaccurate and overstated due to an overly simplistic analysis of what is in the key index of the org manual. it should be noted that if we were to add 'native' rendering of readme.org files, this would likely be done with an org minor mode rather than a full blown org mode as very little of the overall org functionality would be required in this situation. All that would be required is org rendering support and org support for navigation (folding/unfolding, following links and possibly list/table navigation). All the advanced org functionality relating to babel, noweb, todo and time management, agendas, etc have no relevance in this scenario. LIkewise, if this functionality was provided via a minor mode, it would also be possible for people to disable that mode and be left with just the readme.org file, which is just plain text and quite readable. Alan's claim was 785 additional key bindings, many of which not bound to C-c. Running Emacs wiht -q and opeing up an org file, I find the following - 236 org related key bindings - far less than 785 Of these, a number are - actually remapping of existing bindings, so not new ones. This leaves 208. I said that for a read only buffer, many of the org key bindings are not relevant as they relate to features which are not pertinent to a read only org buffer. Any bindings relating to babel, todo management, time management, agendas etc have no relevance when reading a readme.org file. Really, the only key bindings of any relevance are navigation related bindings (including opening/closing folds, following links, etc). Removing all unnecessary org bindings leave us with just 77 new bindings. Of these remaining bindings, quite a few could also be removed, but I've left them in as I only wanted to remove those which would clearly be of no benefit in a read only buffer rendering of a readme.org file. In an org minor mode setup for rendering of readme.org (and possibly other read-only rendering of org files), I estimate at least another 30 bindings could be removed because they have no real benefit or are of only marginal benefit. Creating an org minor mode for this purpose which only introduced 20 or so new bindings would easily be possible. The key point is that claims of 785 new bindings arfe incorrect and misleading. Even being very conservative, we are really talking about 10% of the number being incorrectly claimed. if we introduced a minor mode for this, we are talking likely less than 20 or so. Arguments regarding this being a 'slippery slope' or 'edge of the wedge' in an org takeover are a completely different argument and there may well be some concerns which may need to be considered. However, if that is the real concern, then argue based on that rather than exaggerated claims about excessive key bindings, key binding polution/conflicts and excessive complexity. In a similar manner, lets not jump to conclusions or throw in wild speculation about things which have not been proposed. For example, nobody has suggested everyone would be required to provide readme.org files, nobody has suggested you could not just have a readme or a readme.txt file, nobody has suggested making anyone provide any specific format. All that has been rasied is whether, when a package has a readme.org file, should it be translated into a 'plain text' (ASCII/UTF-8) format or whether it might be useful to have the file formatted and nicely displayed using existing org-mode formatting capabilities. It could even be that the ability to have 'native' rendering of readme.org files could be n optional feature which those who want it can enable. All of this now seems to be irrelevant as Stefan has indicated he will just automatically convert the readme.org to a plain readme. I hope that the original readme.org file will still be included for those who do prefer to use org to render and navigate a well formatted readme file which includes working and concise links, source code syntax highlighting, well formatted examples and fast navigation. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 6:14 ` Tim Cross @ 2022-06-07 11:21 ` Eli Zaretskii 2022-06-08 13:12 ` Ihor Radchenko 2022-06-08 19:34 ` Convert README.org to plain text README while installing package Tim Cross 2022-06-07 16:02 ` Alan Mackenzie 1 sibling, 2 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-07 11:21 UTC (permalink / raw) To: Tim Cross; +Cc: acm, emacs-devel > From: Tim Cross <theophilusx@gmail.com> > Cc: acm@muc.de, emacs-devel@gnu.org > Date: Tue, 07 Jun 2022 16:14:52 +1000 > > > I can tell you what I see with my very simple 1570-line Org file, > > which just has some todo items organized in 4-level hierarchy: > > > > . ~100 keys that look "general Org" to me > > . ~30 keys that Org remaps to its s own commands, like 'yank' and > > 'backward-sentence' > > . ~40 keys bound to org-babel-SOMETHING -- no idea why these are > > bound by default, but they are there > > . ~60 more keys that I'm not sure whether they are "basic" or not, > > but they are all bound by default, just because I visited an Org > > file > > > > So "just" 230 key bindings. And this is without any minor/add-on Org > > mode/feature enabled, at least according to "C-h m". > > > > Do you see something different? Are you still saying that it's not a > > lot, or that it's "based on ignorance and paranoia"? If so, please > > point out where I'm ignorant and/or paranoiac, because I'd really like > > to know. > > I think Alan's response has pretty much confirmed what I was saying. Alan has > made it fairly clear that his real underlying concern is about a stealthy > strategy to get org mode more intrinsically tied into Emacs and in the end, > making it so critical to normal operation that everyone will be forced to learn > org mode regardless. I don't necessarily agree with such an argument, but think > having a debate around that would be more up-front and in some ways honest > compared to what appear to be somewhat contrived alternative arguments about > excessive key bindings and added complexity. My characterisation of this all > being fear and paranoia may have been overstating things, but then again I'm not > sure. There certainly does seem to be considerable fear involved. I don't know > about paranoia, but I have not seen any evil plan to have org mode assimilate > Emacs as if it was the Borg and I've seen no discussions on the org list about > ways to get org more ingrained in Emacs or make Emacs more dependent on org. > > With respect to ignorance, Alan made it quite clear he has not spent > the time or effort to learn org mode, so he is ignorant about its > operation and lacks any real understanding of its operation. He has made > assertions about the number of key bindings which are wildly over > inflated and about required levels of complexity which simply would not > be necessary with the org mode rendering of a read only buffer. In > short, he has made a lot of assumptions about both the key bindings and > complexity based on only a fairly cursory scan of the manual and very > limited experience. > > All that appears to have occurred here is that someone noted that packages often > had a readme.org file and wondered if this should be translated into just a > plain readme file. Someone else suggested we could have package descriptions > render the readme.org file using org and not require export into plain readme > file and mentioned some possible advantages this could have wrt navigation, > rendering of tables, links and images or inclusion of source code snippets, > examples etc. > > >From this point, some somewhat extreme claims and arguments have been thrown in. > One was Alan's claim that org would introduce 785 new key bindings, many of > which not bounmd to C-c and which would presumably either pollute the key space > or conflict with existing bindings. All I have done is point out this claim was > inaccurate and overstated due to an overly simplistic analysis of what is in the > key index of the org manual. > > it should be noted that if we were to add 'native' rendering of readme.org > files, this would likely be done with an org minor mode rather than a full blown > org mode as very little of the overall org functionality would be required in > this situation. All that would be required is org rendering support and org > support for navigation (folding/unfolding, following links and possibly > list/table navigation). All the advanced org functionality relating to babel, > noweb, todo and time management, agendas, etc have no relevance in this > scenario. LIkewise, if this functionality was provided via a minor mode, it > would also be possible for people to disable that mode and be left with just the > readme.org file, which is just plain text and quite readable. > > Alan's claim was 785 additional key bindings, many of which not bound to C-c. The above is completely irrelevant to my questions. > Running Emacs wiht -q and opeing up an org file, I find the following > > - 236 org related key bindings - far less than 785 Of these, a number are > - actually remapping of existing bindings, so not new ones. This leaves 208. ??? How is remapping "not new"? It takes some very common Emacs commands and redirects them to different commands. E.g., 'open-line' now does something quite different. This means the user should either go learn what the Org commands do, or be prepared to be surprised. > I said that for a read only buffer, many of the org key bindings are not > relevant as they relate to features which are not pertinent to a read only org > buffer. Any bindings relating to babel, todo management, time management, > agendas etc have no relevance when reading a readme.org file. Then why does Org define them in that case? > Really, the only > key bindings of any relevance are navigation related bindings (including > opening/closing folds, following links, etc). Removing all unnecessary org > bindings leave us with just 77 new bindings. Of these remaining bindings, quite > a few could also be removed, but I've left them in as I only wanted to remove > those which would clearly be of no benefit in a read only buffer rendering of a > readme.org file. In an org minor mode setup for rendering of readme.org (and > possibly other read-only rendering of org files), I estimate at least another 30 > bindings could be removed because they have no real benefit or are of only > marginal benefit. Creating an org minor mode for this purpose which only > introduced 20 or so new bindings would easily be possible. The key point is that > claims of 785 new bindings arfe incorrect and misleading. Even being very > conservative, we are really talking about 10% of the number being incorrectly claimed. > if we introduced a minor mode for this, we are talking likely less than 20 or so. I'm sorry, but IMO this hand-waves away a real problem by ignoring the problematic UX of suddenly having more than 200 keybindings defined just because I visited a small file. I hoped you will present some serious analysis of the issue, and perhaps even show me where I'm wrong in my conclusions. Sadly, it sounds like all you wanted to do is to prove that you are right, the facts notwithstanding. You accuse Alan in extremism, but your own argumentation is another example of a similar extremism -- just in the opposite direction. This doesn't facilitate useful discussions of what I think is a real problem. Any unbiased observer should agree that adding 200+ key bindings when a small file is visited is a problem that needs to be solved. Org is too large for such simple jobs, and should be refactored to avoid loading everything under the sun when a README is visited. And we should work towards solving it, not sweep it under the carpet. P.S. Btw, here's one more demonstration that Org loads too much by default: C-x C-f ~/org/Plan.org RET => "Package vc-mtn is deprecated" Please believe me that NOTHING in my file refers or requires mtn. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 11:21 ` Eli Zaretskii @ 2022-06-08 13:12 ` Ihor Radchenko 2022-06-08 14:15 ` Eli Zaretskii 2022-06-08 19:34 ` Convert README.org to plain text README while installing package Tim Cross 1 sibling, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-08 13:12 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Tim Cross, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> - 236 org related key bindings - far less than 785 Of these, a number are >> - actually remapping of existing bindings, so not new ones. This leaves 208. > > ??? How is remapping "not new"? It takes some very common Emacs > commands and redirects them to different commands. E.g., 'open-line' > now does something quite different. This means the user should either > go learn what the Org commands do, or be prepared to be surprised. There are 3 main purposes of remappings in Org: 1. To implement some optional functionality. That functionality is not enabled by default and no surprises will be given to user. 2. To implement the original functionality where the existing Emacs customization is not sufficient. For example, org-backward-paragraph has to re-implement backward-paragraph as the original function only supports regexp-bases paragraph boundaries. Regexps are not sufficient in Org syntax. 3. Like (1), but deliberately changing the default behavior to what Org users prefer historically. This is usually not much different from customisations that are usually made in define-derived-mode, except that Emacs does not have enough flexibility to do it there. >> I said that for a read only buffer, many of the org key bindings are not >> relevant as they relate to features which are not pertinent to a read only org >> buffer. Any bindings relating to babel, todo management, time management, >> agendas etc have no relevance when reading a readme.org file. > > Then why does Org define them in that case? I am not very sure what is the problem with the number of bindings. It is not different from Emacs itself. For example, I am sometimes puzzled by <F2> prefix key in Emacs defining bindings for two-column.el. Over years of using Emacs I only used them by accident and always felt like the "break" my Emacs. Yet, I am fine with those hundreds of default Emacs bindings I never use. I do not have to read the full Emacs manual to understand those bindings (they are even not all described in the Emacs manual!). Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 13:12 ` Ihor Radchenko @ 2022-06-08 14:15 ` Eli Zaretskii 2022-06-08 15:15 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-08 14:15 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Tim Cross <theophilusx@gmail.com>, acm@muc.de, emacs-devel@gnu.org > Date: Wed, 08 Jun 2022 21:12:11 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> - 236 org related key bindings - far less than 785 Of these, a number are > >> - actually remapping of existing bindings, so not new ones. This leaves 208. > > > > ??? How is remapping "not new"? It takes some very common Emacs > > commands and redirects them to different commands. E.g., 'open-line' > > now does something quite different. This means the user should either > > go learn what the Org commands do, or be prepared to be surprised. > > There are 3 main purposes of remappings in Org: I'm sure there are good reasons for that. My point is that such remappings effectively force the user to re-learn the commands he/she is very familiar with. So it's a non-trivial burden. > >> I said that for a read only buffer, many of the org key bindings are not > >> relevant as they relate to features which are not pertinent to a read only org > >> buffer. Any bindings relating to babel, todo management, time management, > >> agendas etc have no relevance when reading a readme.org file. > > > > Then why does Org define them in that case? > > I am not very sure what is the problem with the number of bindings. It > is not different from Emacs itself. The difference is that we had years or decades to get used to the Emacs defaults, and once Org is turned on in a buffer, one has a lot of new stuff to get used to. Unless Org is used constantly, you will forget most of those changes till the next time, so this re-learning experience will be repeated every time. It isn't a catastrophe, of course, but we should recognize this as an issue, especially if many of the bindings aren't needed. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 14:15 ` Eli Zaretskii @ 2022-06-08 15:15 ` Ihor Radchenko 2022-06-08 16:16 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-08 15:15 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Ihor Radchenko <yantar92@gmail.com> >> Cc: Tim Cross <theophilusx@gmail.com>, acm@muc.de, emacs-devel@gnu.org >> Date: Wed, 08 Jun 2022 21:12:11 +0800 >> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> >> - 236 org related key bindings - far less than 785 Of these, a number are >> >> - actually remapping of existing bindings, so not new ones. This leaves 208. >> > >> > ??? How is remapping "not new"? It takes some very common Emacs >> > commands and redirects them to different commands. E.g., 'open-line' >> > now does something quite different. This means the user should either >> > go learn what the Org commands do, or be prepared to be surprised. >> >> There are 3 main purposes of remappings in Org: > > I'm sure there are good reasons for that. My point is that such > remappings effectively force the user to re-learn the commands he/she > is very familiar with. So it's a non-trivial burden. Not really. The remappings usually intend to re-implement the usual expected Emacs behavior inside Org. It's just that it is not always possible using the built-in functions. Hence, we implement a layer on top. There should not be anything to learn with regard to remapped commands given that Org defaults are not changed. The difference is that things like, say, paragraph-separate, are not even noticed by users of various major modes. Org had to remap the commands, which is more immediately visible. >> >> I said that for a read only buffer, many of the org key bindings are not >> >> relevant as they relate to features which are not pertinent to a read only org >> >> buffer. Any bindings relating to babel, todo management, time management, >> >> agendas etc have no relevance when reading a readme.org file. >> > >> > Then why does Org define them in that case? >> >> I am not very sure what is the problem with the number of bindings. It >> is not different from Emacs itself. > > The difference is that we had years or decades to get used to the > Emacs defaults, and once Org is turned on in a buffer, one has a lot > of new stuff to get used to. Unless Org is used constantly, you will > forget most of those changes till the next time, so this re-learning > experience will be repeated every time. Isn't it the same for any other major mode? > It isn't a catastrophe, of course, but we should recognize this as an > issue, especially if many of the bindings aren't needed. I am not sure what you mean by aren't needed. They are needed if you use their functionality. They are not needed if you don't. There is no doubt that you do not need most of the bindings just to navigate Org files or do basic editing. You do not need to learn those other bindings either. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 15:15 ` Ihor Radchenko @ 2022-06-08 16:16 ` Eli Zaretskii 2022-06-09 9:15 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-08 16:16 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, acm@muc.de, emacs-devel@gnu.org > Date: Wed, 08 Jun 2022 23:15:16 +0800 > > >> There are 3 main purposes of remappings in Org: > > > > I'm sure there are good reasons for that. My point is that such > > remappings effectively force the user to re-learn the commands he/she > > is very familiar with. So it's a non-trivial burden. > > Not really. The remappings usually intend to re-implement the usual > expected Emacs behavior inside Org. It's just that it is not always > possible using the built-in functions. Hence, we implement a layer on > top. > > There should not be anything to learn with regard to remapped commands > given that Org defaults are not changed. Are you sure this is always true? There are several dozens of remapped commands; did you audit all of them? And anyway, even if what you say is 110% true, how am I as a user to know that up front? I'm used to read the documentation of every command I don't already know by heart, so when faced with such massive remapping, I have quite some reading to do before I can feel myself at ease. And please note that, unlike Alan, I _do_ use Org, just not very often, at least these days. So what I'm sating doesn't come from the POV of an anti-Org user. I _want_ Org to be easier and less demanding to use. > > The difference is that we had years or decades to get used to the > > Emacs defaults, and once Org is turned on in a buffer, one has a lot > > of new stuff to get used to. Unless Org is used constantly, you will > > forget most of those changes till the next time, so this re-learning > > experience will be repeated every time. > > Isn't it the same for any other major mode? No, not IME. Show me another general-purpose editing mode that defines so many key bindings. The only modes that get close are those where text is read-only, so normal editing is impossible anyway. And even those leave alone basic movement commands, like C-S-<up> which Alan mentioned. > > It isn't a catastrophe, of course, but we should recognize this as an > > issue, especially if many of the bindings aren't needed. > > I am not sure what you mean by aren't needed. Ask Tim Cross: the claim that most of the 230 bindings I counted aren't needed comes from him. > There is no doubt that you do not need most of the bindings just to > navigate Org files or do basic editing. You do not need to learn those > other bindings either. Then why bind them by default? why not wait until I actually need that functionality? If that's hard or impossible to detect automatically, let the user say so. For example, I could envision a minor mode that enables Org-Babel and binds the corresponding commands to keys. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 16:16 ` Eli Zaretskii @ 2022-06-09 9:15 ` Ihor Radchenko 2022-06-09 9:38 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-09 9:15 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> Not really. The remappings usually intend to re-implement the usual >> expected Emacs behavior inside Org. It's just that it is not always >> possible using the built-in functions. Hence, we implement a layer on >> top. >> >> There should not be anything to learn with regard to remapped commands >> given that Org defaults are not changed. > > Are you sure this is always true? There are several dozens of > remapped commands; did you audit all of them? I have audited a couple of them and relied on a good faith to other Org devs + absence of bug reports relevant to the raised concern. Rather then re-checking the remapped functions, I'd rather just fix any concrete bug reports about misbehavior, if such reports arise. > And anyway, even if what you say is 110% true, how am I as a user to > know that up front? I'm used to read the documentation of every > command I don't already know by heart, so when faced with such massive > remapping, I have quite some reading to do before I can feel myself at > ease. > > And please note that, unlike Alan, I _do_ use Org, just not very > often, at least these days. So what I'm sating doesn't come from the > POV of an anti-Org user. I _want_ Org to be easier and less demanding > to use. While I understand the possible fear of unexpected or unnatural behavior of the remapped command, I'd like to point out that command behavior can span over quite a significant range even without remapping. Just the fact that some major mode does not use remapping, does not mean that you are guaranteed to get the normal command behavior. Some major modes rely on post-command-hook instead of remapping, which not only modifies the original command behavior in unspecified ways, but is also not directly visible from the mode bindings. Or consider changes in filter-buffer-substring-function and how it can modify killing text. So, I'd say that your concern is not really justified. There is generally no reliable way to tell if an arbitrary major mode modifies standard command behavior just by looking at its key bindings. You have to rely on faith and report bugs if you stumble upon them. >> > The difference is that we had years or decades to get used to the >> > Emacs defaults, and once Org is turned on in a buffer, one has a lot >> > of new stuff to get used to. Unless Org is used constantly, you will >> > forget most of those changes till the next time, so this re-learning >> > experience will be repeated every time. >> >> Isn't it the same for any other major mode? > > No, not IME. Show me another general-purpose editing mode that > defines so many key bindings. The only modes that get close are those > where text is read-only, so normal editing is impossible anyway. I am not sure what you mean by general-purpose. Auctex binds a decent number of key-bindings. VHDL mode binds even more (I just looked into one of the largest mode built-in prog modes). markdown-mode binds over 100 keys not counting prefix maps. > And even those leave alone basic movement commands, like C-S-<up> > which Alan mentioned. C-S-<up> is not a basic movement command. By default, it is activating region. Without shift-select-mode, it is not bound to anything. Org does provide support for shift-select-mode, but we never got overwhelming requests to enable this support by default. C-<up>/<down>, which is the actual basic movement command is not overridden by Org. AFAIK, the only shadowed command is M-<right>/<left> with its other C-<right>/<left> binding being untouched. However, I am not even sure what Org can do about this. Arrow bindings are one of the most core and frequently used commands. Removing them will dissatisfy existing users. Note that Emacs dedicates C-c + letter for user bindings. Anything else is not guaranteed to be not shadowed by major/minor modes. If users absolutely want they personal bindings to be preferred, there is always override-global-map. >> > It isn't a catastrophe, of course, but we should recognize this as an >> > issue, especially if many of the bindings aren't needed. >> >> I am not sure what you mean by aren't needed. > > Ask Tim Cross: the claim that most of the 230 bindings I counted > aren't needed comes from him. He was referring to reading Org files without editing. You can argue the same for reading pretty much any other major-mode files. Most bindings will not be needed. >> There is no doubt that you do not need most of the bindings just to >> navigate Org files or do basic editing. You do not need to learn those >> other bindings either. > > Then why bind them by default? why not wait until I actually need that > functionality? If that's hard or impossible to detect automatically, > let the user say so. For example, I could envision a minor mode that > enables Org-Babel and binds the corresponding commands to keys. Org-babel bindings are needed as soon as you create a source block inside the Org file. Are you proposing to track changes in buffer and autoload babel as soon as a source block appears inside the buffer? I'd say that conventional approach is binding the relevant commands and letting Emacs to do autoloading once those commands are called. Would it help if we hide the babel-related stuff under dedicated prefix map? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-09 9:15 ` Ihor Radchenko @ 2022-06-09 9:38 ` Eli Zaretskii 2022-06-11 3:52 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-09 9:38 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, acm@muc.de, emacs-devel@gnu.org > Date: Thu, 09 Jun 2022 17:15:05 +0800 > > > Are you sure this is always true? There are several dozens of > > remapped commands; did you audit all of them? > > I have audited a couple of them and relied on a good faith to other Org > devs + absence of bug reports relevant to the raised concern. > Rather then re-checking the remapped functions, I'd rather just fix any > concrete bug reports about misbehavior, if such reports arise. Didn't this discussion raise such "bug reports"? Why not consider what several people here said as such a bug report? The difference is that these reports come from people who use Org rarely, so the concerns are different. But if the Org developers want to have Org adopted more widely, I think they should listen to these concerns, not reject them. > While I understand the possible fear of unexpected or unnatural > behavior of the remapped command, I'd like to point out that command > behavior can span over quite a significant range even without remapping. > Just the fact that some major mode does not use remapping, does not mean > that you are guaranteed to get the normal command behavior. Most major modes don't remap global and frequently-used commands. Those that do are problematic, and should be fixed rather than be used as examples of what's "kosher". > Some major modes rely on post-command-hook instead of remapping, which > not only modifies the original command behavior in unspecified ways, but > is also not directly visible from the mode bindings. Or consider changes > in filter-buffer-substring-function and how it can modify killing text. And that is in your opinion a Good Thing? I don't think it is, and therefore don't consider such problematic behavior a good example for development. > So, I'd say that your concern is not really justified. ??? Just because "others do that"? > There is generally no reliable way to tell if an arbitrary major > mode modifies standard command behavior just by looking at its key > bindings. You have to rely on faith and report bugs if you stumble > upon them. I don't see how this solves the concerns. We should still try to minimize such problematic behaviors as much as possible. > > No, not IME. Show me another general-purpose editing mode that > > defines so many key bindings. The only modes that get close are those > > where text is read-only, so normal editing is impossible anyway. > > I am not sure what you mean by general-purpose. Modes for editing mostly human-readable text. > Auctex binds a decent number of key-bindings. VHDL mode binds even > more (I just looked into one of the largest mode built-in prog > modes). markdown-mode binds over 100 keys not counting prefix maps. Two of these aren't part of Emacs, and VHDL is an example of programming language, where text is not really for human consumption. > > And even those leave alone basic movement commands, like C-S-<up> > > which Alan mentioned. > > C-S-<up> is not a basic movement command. Of course, it is: it moves by paragraphs. > By default, it is activating > region. Without shift-select-mode, it is not bound to anything. But shift-select-mode is on by default. > > Then why bind them by default? why not wait until I actually need that > > functionality? If that's hard or impossible to detect automatically, > > let the user say so. For example, I could envision a minor mode that > > enables Org-Babel and binds the corresponding commands to keys. > > Org-babel bindings are needed as soon as you create a source block > inside the Org file. Are you proposing to track changes in buffer and > autoload babel as soon as a source block appears inside the buffer? That's one way. Another one is to have a minor mode that users could turn on when they need it (or turn it on by default in their init files if they need it a lot). > Would it help if we hide the babel-related stuff under dedicated prefix > map? Probably. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-09 9:38 ` Eli Zaretskii @ 2022-06-11 3:52 ` Ihor Radchenko 2022-06-11 6:52 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-11 3:52 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Ihor Radchenko <yantar92@gmail.com> >> Cc: theophilusx@gmail.com, acm@muc.de, emacs-devel@gnu.org >> Date: Thu, 09 Jun 2022 17:15:05 +0800 >> >> > Are you sure this is always true? There are several dozens of >> > remapped commands; did you audit all of them? >> >> I have audited a couple of them and relied on a good faith to other Org >> devs + absence of bug reports relevant to the raised concern. >> Rather then re-checking the remapped functions, I'd rather just fix any >> concrete bug reports about misbehavior, if such reports arise. > > Didn't this discussion raise such "bug reports"? Why not consider > what several people here said as such a bug report? The difference is > that these reports come from people who use Org rarely, so the > concerns are different. But if the Org developers want to have Org > adopted more widely, I think they should listen to these concerns, not > reject them. To clarify, it is metally difficult to process bug reports raised in such discussions because they are often entangled with quite negative attitude and much more broad claims. What I have summarised so far from the discussions: 1. Org mode shadows default C-S-<up>/C-S-<down> bindings. This can be solved in two ways: a. Change org-support-shift-select to t by default + fix the fact that org-support-shift-select is actually ignored by C-S-<up>/<down>. b. Disable Org mode arrow bindings by default and move them to a dedicated minor-mode / setting. (This will make existing Org users angry) Note that I do not see C-c / and C-c C-s as a bug. The first one is not recommended for major-mode but allowed. C-c C-s is dedicated to major modes. 2. Org mode binds too many keys The solution can be disabling those keys and moving them to dedicated minor modes / settings. After thinking, I do not like the idea of enabling some keys depending on the presence absense of tables/source blocks inside the Org file. This will not solve the issue in the files actually containing all those and may surprise users. 3. Some of the context-dependent Org command names are confusing as they do not reveal what the command does. The solution is to rename those commands to something more meaningful. Totally doable. 4. Org remaps some of the default commands. I can see how it can be confusing, but do not see it as a critical issue. See my response below. 5. Org mode is too slow to load. I do not see an easy way to resolve this apart from general refactoring and modularizing. Profiling is not very helpful in this specific case, unfortunately. At least, my profiling skills are not good enough. >> While I understand the possible fear of unexpected or unnatural >> behavior of the remapped command, I'd like to point out that command >> behavior can span over quite a significant range even without remapping. >> Just the fact that some major mode does not use remapping, does not mean >> that you are guaranteed to get the normal command behavior. > > Most major modes don't remap global and frequently-used commands. > Those that do are problematic, and should be fixed rather than be used > as examples of what's "kosher". Let's take one example: org-forward-paragraph that is remapping forward-paragraph. The docstring is: Move forward by a paragraph, or equivalent, unit. ... The function moves point between two structural elements (paragraphs, tables, lists, etc.). It also provides the following special moves for convenience: - on a table or a property drawer, move to its beginning; - on comment, example, export, source and verse blocks, stop at blank lines; - skip consecutive clocks, diary S-exps, and keywords. In Org mode, there is no single definition of a paragraph. The paragraphs or equivalent syntax elements are context-depended and we have to use parser to know where to move. The default forward-paragraph can only be controlled by paragraph-start and paragraph-separate regexps. Both are not sufficient to cater Org mode. Regexps do not cut it for Org mode. Hence, the built-in version of forward-paragraph is not usable in Org mode. If we did not remap it, forward-paragraph would not really move by paragraph, as it is understood by human. It would be more confusing than org version of the command. If you know a better way to resolve the described limitation, please let me know. Similar limitations motivated <remap> <backward-paragraph> org-backward-paragraph <remap> <backward-sentence> org-backward-sentence <remap> <comment-dwim> org-comment-dwim <remap> <forward-paragraph> org-forward-paragraph <remap> <forward-sentence> org-forward-sentence <remap> <move-beginning-of-line> org-beginning-of-line <remap> <move-end-of-line> org-end-of-line <remap> <delete-indentation> org-delete-indentation The following functions have a natural fallback to default behavior and might be moved to minor-modes enabled by default. However, their default behavior is context dependent and motivated by the lack of flexibility of the built-in equivalents. Similar to the above functions. <remap> <open-line> org-open-line <remap> <move-beginning-of-line> org-beginning-of-line <remap> <move-end-of-line> org-end-of-line The following functions might be implemented using built-in Emacs functionality: <remap> <transpose-words> org-transpose-words could use find-word-boundary-function-table <remap> <fill-paragraph> org-fill-paragraph appears to be redundant as we do set fill-paragraph-function <remap> <yank> org-yank may be instead implemented using yank-handler property (it takes care about adjusting level of the inserted headlines), but not completely. We cannot control properties of text from outside of Org mode and thus the functionality cannot be fully implemented using built-in Emacs features The following commands are providing convenience self-alignment functionality and protect from accidental editing of invisible text (AFAIK, this issue is not addressed by Emacs): <remap> <kill-line> org-kill-line <remap> <delete-backward-char> org-delete-backward-char <remap> <delete-char> org-delete-char <remap> <self-insert-command> org-self-insert-command I am not sure how to achieve equivalent functionality using built-in Emacs customisations. The following functions are convenience building for prople coming from outline mode. I do not see any problem here. <remap> <outline-backward-same-level> org-backward-heading-same-level <remap> <outline-demote> org-demote-subtree <remap> <outline-forward-same-level> org-forward-heading-same-level <remap> <outline-insert-heading> org-ctrl-c-ret <remap> <outline-mark-subtree> org-mark-subtree <remap> <outline-next-visible-heading> org-next-visible-heading <remap> <outline-previous-visible-heading> org-previous-visible-heading <remap> <outline-promote> org-promote-subtree <remap> <outline-show-branches> org-kill-note-or-show-branches <remap> <outline-show-children> org-fold-show-children <remap> <outline-show-subtree> org-fold-show-subtree >> Some major modes rely on post-command-hook instead of remapping, which >> not only modifies the original command behavior in unspecified ways, but >> is also not directly visible from the mode bindings. Or consider changes >> in filter-buffer-substring-function and how it can modify killing text. > > And that is in your opinion a Good Thing? I don't think it is, and > therefore don't consider such problematic behavior a good example for > development. Maybe. But then we need a more "appropriate" way to implement the functionality Org provides. See the above. >> So, I'd say that your concern is not really justified. > > ??? Just because "others do that"? >> There is generally no reliable way to tell if an arbitrary major >> mode modifies standard command behavior just by looking at its key >> bindings. You have to rely on faith and report bugs if you stumble >> upon them. > > I don't see how this solves the concerns. We should still try to > minimize such problematic behaviors as much as possible. I'd like you to focus on the last example with filter-buffer-substring-function or on the ability for major mode to customise fill-paragraph-function or paragraph-separate, etc. Those changes can alter the default commands drastically. Yet, they will not be noticeable by users from looking at the M-x describe-bindings. I do not say that major modes should change the default behavior in unexpected ways. Rather the opposite - major modes should adjust the default commands to behave more expectedly within the major mode context. However, remapping is by no means an indication that a major mode is going to change things unexpectedly. Its neither sufficient nor required condition. >> > No, not IME. Show me another general-purpose editing mode that >> > defines so many key bindings. The only modes that get close are those >> > where text is read-only, so normal editing is impossible anyway. >> ... >> Auctex binds a decent number of key-bindings. VHDL mode binds even >> more (I just looked into one of the largest mode built-in prog >> modes). markdown-mode binds over 100 keys not counting prefix maps. > > Two of these aren't part of Emacs, and VHDL is an example of > programming language, where text is not really for human consumption. I get your point on VHDL. However, I am not sure why you reject the non-built-in examples. Are you implying that text modes outside Emacs core are worse than built-in? At least Auctex is largely popular among users and yet I have never seen as many complaints about Auctex complexity as I do for Org (I may be biased). Auctex and markdown-mode deal with similar editing tasks with Org mode. The fact that they also use large number of key bindings is an indication that they are really needed for editing complex documents. >> > And even those leave alone basic movement commands, like C-S-<up> >> > which Alan mentioned. >> >> C-S-<up> is not a basic movement command. > > Of course, it is: it moves by paragraphs. Nope. It is unbound command working because of key translation: C-<down> (translated from C-S-<down>) runs the command forward-paragraph (found in global-map), which is an interactive native compiled Lisp function in ‘paragraphs.el’. >> By default, it is activating >> region. Without shift-select-mode, it is not bound to anything. > > But shift-select-mode is on by default. Fair point. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-11 3:52 ` Ihor Radchenko @ 2022-06-11 6:52 ` Eli Zaretskii 2022-06-12 8:40 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-11 6:52 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, acm@muc.de, emacs-devel@gnu.org > Date: Sat, 11 Jun 2022 11:52:05 +0800 > > > Didn't this discussion raise such "bug reports"? Why not consider > > what several people here said as such a bug report? The difference is > > that these reports come from people who use Org rarely, so the > > concerns are different. But if the Org developers want to have Org > > adopted more widely, I think they should listen to these concerns, not > > reject them. > > To clarify, it is metally difficult to process bug reports raised in > such discussions because they are often entangled with quite negative > attitude and much more broad claims. > > What I have summarised so far from the discussions: > > 1. Org mode shadows default C-S-<up>/C-S-<down> bindings. > > This can be solved in two ways: > a. Change org-support-shift-select to t by default + fix the fact > that org-support-shift-select is actually ignored by > C-S-<up>/<down>. > b. Disable Org mode arrow bindings by default and move them to a > dedicated minor-mode / setting. > (This will make existing Org users angry) > > Note that I do not see C-c / and C-c C-s as a bug. The first one is > not recommended for major-mode but allowed. C-c C-s is dedicated to > major modes. > > 2. Org mode binds too many keys > > The solution can be disabling those keys and moving them to dedicated > minor modes / settings. > > After thinking, I do not like the idea of enabling some keys > depending on the presence absense of tables/source blocks inside the > Org file. This will not solve the issue in the files actually > containing all those and may surprise users. > > 3. Some of the context-dependent Org command names are confusing as they > do not reveal what the command does. > > The solution is to rename those commands to something more > meaningful. Totally doable. > > 4. Org remaps some of the default commands. > I can see how it can be confusing, but do not see it as a critical > issue. See my response below. > > 5. Org mode is too slow to load. > I do not see an easy way to resolve this apart from general > refactoring and modularizing. Profiling is not very helpful in this > specific case, unfortunately. At least, my profiling skills are not > good enough. Thank you, I think this is a very good summary. I hope some of these issues could be worked on and improved in the future. > > Most major modes don't remap global and frequently-used commands. > > Those that do are problematic, and should be fixed rather than be used > > as examples of what's "kosher". > > Let's take one example: org-forward-paragraph that is remapping > forward-paragraph. > > The docstring is: > > Move forward by a paragraph, or equivalent, unit. > ... > The function moves point between two structural > elements (paragraphs, tables, lists, etc.). > > It also provides the following special moves for convenience: > > - on a table or a property drawer, move to its beginning; > - on comment, example, export, source and verse blocks, stop > at blank lines; > - skip consecutive clocks, diary S-exps, and keywords. > > In Org mode, there is no single definition of a paragraph. The > paragraphs or equivalent syntax elements are context-depended and we > have to use parser to know where to move. > > The default forward-paragraph can only be controlled by paragraph-start > and paragraph-separate regexps. Both are not sufficient to cater Org > mode. Regexps do not cut it for Org mode. > > Hence, the built-in version of forward-paragraph is not usable in Org > mode. If we did not remap it, forward-paragraph would not really move by > paragraph, as it is understood by human. It would be more confusing than > org version of the command. For a serious discussion of this example, I'd need more detail about the aspects you mentioned above (it is not trivial to deduce them from looking at the top levels of org-forward-paragraph). Specifically: . how does paragraph definition depend on context in Org, and what are the possible definitions in the existing contexts? . why don't paragraph-start and paragraph-separate fit Org, and can we come up with a small number of additional variables that would augment these two so that the built-in forward-paragraph could be extended to cover Org? > If you know a better way to resolve the described limitation, please let > me know. The better way, IMO, is this: . try to find a way of extending the built-in forward-paragraph to cover the Org use cases, by adding variables in addition to the 2 existing ones (I don't yet understand why "regexps do not cut it for Org mode -- it sounds like a very strong assertion) . if the above doesn't work, make forward-paragraph work through forward-paragraph-function, so that Org could define its own function . in any case, the add-on convenience features of org-forward-paragraph should be provided as options that the user can control My main concern is that forward-paragraph is used when editing the purely textual parts of an Org document, and when used there, casual users of Org will expect it to behave as the command behaves in plain text. Any deviation from the behavior of the built-in command will confuse such users when they edit the plain-text parts, and should therefore be avoided. > The following functions have a natural fallback to default behavior and > might be moved to minor-modes enabled by default. However, their default > behavior is context dependent and motivated by the lack of flexibility > of the built-in equivalents. Similar to the above functions. > > <remap> <open-line> org-open-line > <remap> <move-beginning-of-line> org-beginning-of-line > <remap> <move-end-of-line> org-end-of-line Each of these (and other examples you provided) should require a separate serious discussion. Let's take open-line as an example. It is basically the built-in open-line, unless org-special-ctrl-o is non-nil. A trivial extension of the built-in command could have prevented the need to remap. (And why does the support for tables in org-open-line need to be turned on outside of orgtbl-mode?) > <remap> <yank> org-yank > may be instead implemented using yank-handler property (it takes care > about adjusting level of the inserted headlines), but not completely. We > cannot control properties of text from outside of Org mode and thus the > functionality cannot be fully implemented using built-in Emacs features IMO, org-yank is sufficiently different to justify not taking over the built-in command. Again, consider a user who edits the plain-text portions of an Org document and expects the "usual" behavior of C-y. > >> Some major modes rely on post-command-hook instead of remapping, which > >> not only modifies the original command behavior in unspecified ways, but > >> is also not directly visible from the mode bindings. Or consider changes > >> in filter-buffer-substring-function and how it can modify killing text. > > > > And that is in your opinion a Good Thing? I don't think it is, and > > therefore don't consider such problematic behavior a good example for > > development. > > Maybe. But then we need a more "appropriate" way to implement the > functionality Org provides. See the above. Each of these commands should be discussed separately, and the best solution found and implemented. There's no single answer to that question for all of these remapped commands. But remapping should be avoided as much as possible; see below. > > I don't see how this solves the concerns. We should still try to > > minimize such problematic behaviors as much as possible. > > I'd like you to focus on the last example with > filter-buffer-substring-function or on the ability for major mode to > customise fill-paragraph-function or paragraph-separate, etc. Those > changes can alter the default commands drastically. Yet, they will not > be noticeable by users from looking at the M-x describe-bindings. When fill-paragraph-function and other means of customizing a command to the particular mode make sense, they will not need to be noticed, because they will do what the users expect. Moreover, customizing the behavior of commands through those variables has a couple of important advantages: . it doesn't use remapping, so it doesn't stick out and bother conscientious Emacs users, who are used to study unfamiliar commands before using them . they don't require separate documentation, apart of saying that paragraph-start and paragraph-separate can be modified by modes as appropriate -- all the rest of the command's documentation is still intact and easily grasped in the context of any mode (this is also one more reason why "niceties" like special behavior inside tables should be separately controlled) > I do not say that major modes should change the default behavior in > unexpected ways. Rather the opposite - major modes should adjust the > default commands to behave more expectedly within the major mode > context. However, remapping is by no means an indication that a major > mode is going to change things unexpectedly. Its neither sufficient nor > required condition. My point is that there are better ways of customizing the behavior of a built-in command. Org is now an integral part of Emacs, so it should implement these customizations in ways that are available to a core package; remapping is basically a tool used by unbundled packages that cannot affect the core in any way. > >> Auctex binds a decent number of key-bindings. VHDL mode binds even > >> more (I just looked into one of the largest mode built-in prog > >> modes). markdown-mode binds over 100 keys not counting prefix maps. > > > > Two of these aren't part of Emacs, and VHDL is an example of > > programming language, where text is not really for human consumption. > > I get your point on VHDL. However, I am not sure why you reject the > non-built-in examples. Are you implying that text modes outside Emacs > core are worse than built-in? No, I'm saying that, sadly, we have no real control on what the developers of unbundled packages decide to do. Thus, what you see there is the evidence of our lack of control more than anything else. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-11 6:52 ` Eli Zaretskii @ 2022-06-12 8:40 ` Ihor Radchenko 2022-06-12 8:56 ` Eli Zaretskii ` (2 more replies) 0 siblings, 3 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-12 8:40 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > Thank you, I think this is a very good summary. I hope some of these > issues could be worked on and improved in the future. It is in my todo-list. I may find some time in future to work on it. > For a serious discussion of this example, I'd need more detail about > the aspects you mentioned above (it is not trivial to deduce them from > looking at the top levels of org-forward-paragraph). Specifically: > > . how does paragraph definition depend on context in Org, and what > are the possible definitions in the existing contexts? > . why don't paragraph-start and paragraph-separate fit Org, and can > we come up with a small number of additional variables that would > augment these two so that the built-in forward-paragraph could be > extended to cover Org? I do not want to start a serious discussion on this just yet as I do not plant to work on this specific area in near future, however I would like to answer some of your questions in order to provide some insight for Emacs developers. To answer your question about paragraph definition, we need to clarify what "paragraph" means. (*) Plain text documents, in their simplest forms, usually consist of a series of sections containing a series of paragraphs each. With paragraphs being a series of sentences. The paragraphs are usually separated by blank lines or by indentation. Org mode extends the above simple structure by defining additional syntactic elements, which are equivalent to paragraphs. Apart from paragraphs, there can be tables, lists, other specially treated blocks of text (source code blocks, comments, verse blocks, drawers, etc). These elements may or may not be separated with blank lines. For example, | This table | is considered as a new "paragraph"-like element | | Not separated | from the above paragraph by blank lines | #+begin_src emacs-lisp (message "And here we have yet another paragraph-like element") ;; This element may contain empty lines inside, yet these empty lines ;; are not indicating "paragraph"-like element boundary. (message "They may, in emacs-lisp specifically, from some point of view; but consider other possible programming languages with arbitrary syntax; or even elisp, with the above blank line inside string.") #+end_src When a user calls forward-paragraph on a simple paragraph as defined in (*), built-in forward-paragraph will behave narutally. However, things become tricky when we have various paragraph-like elements. How can one configure forward-paragraph to forward classic paragraph, tables, source blocks, and all other possible elements? To make things more complex, Org mode syntax is not context-free. #+end_src in here does not belong to a source block because it lacks #+begin_src opener. Also, :DRAWER-LIKE-THIS: Cannot intersect with other source-block-looking constructs like #+begin_src emacs-lisp (message "I am not a source block") :END: The #+end_src here is also not a part of source block because it cannot intersect drawer boundary. >> If you know a better way to resolve the described limitation, please let >> me know. > > The better way, IMO, is this: > > . try to find a way of extending the built-in forward-paragraph to > cover the Org use cases, by adding variables in addition to the 2 > existing ones (I don't yet understand why "regexps do not cut it > for Org mode -- it sounds like a very strong assertion) For regexps, I hope that the above example illustrated the problem clearly. Also, note that a number of people attempted to develop BNF/EBNF parsers for Org mode, but failed because of issues tike the above. Org is not context-free and cannot be parsed using LR parsers, let alone simple regexps. For more insight, you may also consult org-element-paragraph-parser code, which has to deal with the problems illustrated above. (note that even this function alone is not sufficient to find paragraphs; it also relies on other parser code). > . if the above doesn't work, make forward-paragraph work through > forward-paragraph-function, so that Org could define its own > function Introducing the required flexibility into Emacs core is certainly an option. Though it is somewhat difficult one because Org also has to support older versions of Emacs. We are somewhat limited in using newly introduced built-in features. > . in any case, the add-on convenience features of > org-forward-paragraph should be provided as options that the user > can control They are not necessarily add-ons. First of all, org-forward-paragraph is dealing with the above complications. Using forward-paragraph directly would behave unexpectedly on various Org elements. > My main concern is that forward-paragraph is used when editing the > purely textual parts of an Org document, and when used there, casual > users of Org will expect it to behave as the command behaves in plain > text. Any deviation from the behavior of the built-in command will > confuse such users when they edit the plain-text parts, and should > therefore be avoided. There is no difference between forward-paragraph and org-forward-paragraph on purely texture parts. The difference is only on the Org-specific constructs, where forward-paragraph would behave unexpectedly. >> The following functions have a natural fallback to default behavior and >> might be moved to minor-modes enabled by default. However, their default >> behavior is context dependent and motivated by the lack of flexibility >> of the built-in equivalents. Similar to the above functions. >> >> <remap> <open-line> org-open-line >> <remap> <move-beginning-of-line> org-beginning-of-line >> <remap> <move-end-of-line> org-end-of-line > > Each of these (and other examples you provided) should require a > separate serious discussion. Let's take open-line as an example. It > is basically the built-in open-line, unless org-special-ctrl-o is > non-nil. A trivial extension of the built-in command could have > prevented the need to remap. (And why does the support for tables in > org-open-line need to be turned on outside of orgtbl-mode?) You misunderstand orgtbl-mode. orgtbl-mode is optional minor-mode provided for users who want to edit tables "like in Org mode", but in _other_ major modes. Inside Org mode, tables are always supported. Also, org-special-ctrl-o is non-nil by default. Using built-in open-line on Org tables can produce incorrect formatting. For example, calling open-line on the following table | this | is | table | | with | <point> 2 | lines | will produce | this | is | table | | with | 2 | lines | while org-open-line will produce | this | is | table | | | | | | with | 2 | lines | The default behavior is not satisfactory and somewhat unexpected. >> <remap> <yank> org-yank >> may be instead implemented using yank-handler property (it takes care >> about adjusting level of the inserted headlines), but not completely. We >> cannot control properties of text from outside of Org mode and thus the >> functionality cannot be fully implemented using built-in Emacs features > > IMO, org-yank is sufficiently different to justify not taking over the > built-in command. Again, consider a user who edits the plain-text > portions of an Org document and expects the "usual" behavior of C-y. I personally tend to agree here. > When fill-paragraph-function and other means of customizing a command > to the particular mode make sense, they will not need to be noticed, > because they will do what the users expect. Moreover, customizing the > behavior of commands through those variables has a couple of important > advantages: > > . it doesn't use remapping, so it doesn't stick out and bother > conscientious Emacs users, who are used to study unfamiliar > commands before using them > . they don't require separate documentation, apart of saying that > paragraph-start and paragraph-separate can be modified by modes as > appropriate -- all the rest of the command's documentation is > still intact and easily grasped in the context of any mode (this > is also one more reason why "niceties" like special behavior > inside tables should be separately controlled) I get your point. As long as default customisations are sufficient to achive the expected and reasonable behavior, they should indeed be preferred. >> I get your point on VHDL. However, I am not sure why you reject the >> non-built-in examples. Are you implying that text modes outside Emacs >> core are worse than built-in? > > No, I'm saying that, sadly, we have no real control on what the > developers of unbundled packages decide to do. Thus, what you see > there is the evidence of our lack of control more than anything else. I do not get your point here. I was referring to the markdown-mode and Auctex to illustrate the _need_ to have numerous key bindings in order to edit complex Org documents. It is not just something introduced into Org out of the blue. Other people solved similar problems and did not come up with significantly less key bindings. If you say that usual 40-50 major mode bindings are sufficient, I am not convinced. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-12 8:40 ` Ihor Radchenko @ 2022-06-12 8:56 ` Eli Zaretskii 2022-06-12 9:46 ` Ihor Radchenko 2022-06-12 13:53 ` Kévin Le Gouguec 2022-06-12 20:07 ` Common keybindings idea for multiple markups Jean Louis 2 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-12 8:56 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, acm@muc.de, emacs-devel@gnu.org > Date: Sun, 12 Jun 2022 16:40:31 +0800 > > > For a serious discussion of this example, I'd need more detail about > > the aspects you mentioned above (it is not trivial to deduce them from > > looking at the top levels of org-forward-paragraph). Specifically: > > > > . how does paragraph definition depend on context in Org, and what > > are the possible definitions in the existing contexts? > > . why don't paragraph-start and paragraph-separate fit Org, and can > > we come up with a small number of additional variables that would > > augment these two so that the built-in forward-paragraph could be > > extended to cover Org? > > I do not want to start a serious discussion on this just yet as I do not > plant to work on this specific area in near future, however I would like > to answer some of your questions in order to provide some insight for > Emacs developers. Thanks, but if we are not going to continue this discussion until we come to some conclusions, I see no reason to keep talking about it. I do understand better the difficulties now (thanks!), but I'm not yet convinced that the existing solution is the best one. > > My main concern is that forward-paragraph is used when editing the > > purely textual parts of an Org document, and when used there, casual > > users of Org will expect it to behave as the command behaves in plain > > text. Any deviation from the behavior of the built-in command will > > confuse such users when they edit the plain-text parts, and should > > therefore be avoided. > > There is no difference between forward-paragraph and > org-forward-paragraph on purely texture parts. The difference is only on > the Org-specific constructs, where forward-paragraph would behave > unexpectedly. How does Org make sure that the different behavior doesn't happen without the user's intent? Isn't it possible that Org perceives something as a sign of an "Org-specific construct" when the user didn't mean that? When that happens, what can the user do to avoid the unintended (from that user's POV) Org-specific behavior? These situations are where user control is necessary, especially for casual, non-seasoned users of Org. > >> <remap> <open-line> org-open-line > >> <remap> <move-beginning-of-line> org-beginning-of-line > >> <remap> <move-end-of-line> org-end-of-line > > > > Each of these (and other examples you provided) should require a > > separate serious discussion. Let's take open-line as an example. It > > is basically the built-in open-line, unless org-special-ctrl-o is > > non-nil. A trivial extension of the built-in command could have > > prevented the need to remap. (And why does the support for tables in > > org-open-line need to be turned on outside of orgtbl-mode?) > > You misunderstand orgtbl-mode. orgtbl-mode is optional minor-mode > provided for users who want to edit tables "like in Org mode", but in > _other_ major modes. Inside Org mode, tables are always supported. What if the user doesn't intend to use Org tables? How can such a user turn off this automatic support, which interprets some patterns of buffer text as an indication of a table, and turns on the special behavior reserved for Org tables? > Also, org-special-ctrl-o is non-nil by default. Using built-in open-line > on Org tables can produce incorrect formatting. For example, calling > open-line on the following table > > | this | is | table | > | with | <point> 2 | lines | > > will produce > > | this | is | table | > | with | > 2 | lines | > > while org-open-line will produce > > | this | is | table | > | | | | > | with | 2 | lines | > > The default behavior is not satisfactory and somewhat unexpected. You assume that users always expect what Org does in that case. This is not a given, if we want to make Org more attractive for editing text that is "less Org-specific", so to speak, like README files, NEWS files, etc. > > No, I'm saying that, sadly, we have no real control on what the > > developers of unbundled packages decide to do. Thus, what you see > > there is the evidence of our lack of control more than anything else. > > I do not get your point here. I was referring to the markdown-mode and > Auctex to illustrate the _need_ to have numerous key bindings in order > to edit complex Org documents. It is not just something introduced into > Org out of the blue. Other people solved similar problems and did not > come up with significantly less key bindings. If you say that usual > 40-50 major mode bindings are sufficient, I am not convinced. I'm not arguing with the need, I'm arguing with the particular implementation that caters to that need. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-12 8:56 ` Eli Zaretskii @ 2022-06-12 9:46 ` Ihor Radchenko 2022-06-12 9:59 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-12 9:46 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> I do not want to start a serious discussion on this just yet as I do not >> plant to work on this specific area in near future, however I would like >> to answer some of your questions in order to provide some insight for >> Emacs developers. > > Thanks, but if we are not going to continue this discussion until we > come to some conclusions, I see no reason to keep talking about it. I > do understand better the difficulties now (thanks!), but I'm not yet > convinced that the existing solution is the best one. There are some interim conclusions for now. I agree that the approach Org takes on remapping may be improved. I will keep in mind to reach to emacs-devel about possibility to extend built-in Emacs functionality if it is required to get rid of the Org remappings. >> There is no difference between forward-paragraph and >> org-forward-paragraph on purely texture parts. The difference is only on >> the Org-specific constructs, where forward-paragraph would behave >> unexpectedly. > > How does Org make sure that the different behavior doesn't happen > without the user's intent? Isn't it possible that Org perceives > something as a sign of an "Org-specific construct" when the user > didn't mean that? When that happens, what can the user do to avoid > the unintended (from that user's POV) Org-specific behavior? > > These situations are where user control is necessary, especially for > casual, non-seasoned users of Org. ... >> You misunderstand orgtbl-mode. orgtbl-mode is optional minor-mode >> provided for users who want to edit tables "like in Org mode", but in >> _other_ major modes. Inside Org mode, tables are always supported. > > What if the user doesn't intend to use Org tables? How can such a > user turn off this automatic support, which interprets some patterns > of buffer text as an indication of a table, and turns on the special > behavior reserved for Org tables? Let me be clear here. Org syntax is fixed as much as possible. One cannot just tell Org to ignore tables in Org buffers. However, users can alter behavior of specific Org commands if they wish to. Using available customization and other usual Elisp tools. One would not expect, say, tex-mode behave correctly if the user arbitrarily changes buffer syntax table. Similarly, tables are the core part of Org syntax. >> Also, org-special-ctrl-o is non-nil by default. Using built-in open-line >> on Org tables can produce incorrect formatting. For example, calling >> open-line on the following table >> >> | this | is | table | >> | with | <point> 2 | lines | >> ... >> The default behavior is not satisfactory and somewhat unexpected. > > You assume that users always expect what Org does in that case. This > is not a given, if we want to make Org more attractive for editing > text that is "less Org-specific", so to speak, like README files, NEWS > files, etc. We do not assume that users always expect specific behavior. That's wy org-special-ctrl-o is customization. It's default value has been agreed upon and has not been questioned by many. The current behavior is what most of Org users expect. The default open-line behavior is not. Otherwise, nobody would bother implementing org-open-line. >> > No, I'm saying that, sadly, we have no real control on what the >> > developers of unbundled packages decide to do. Thus, what you see >> > there is the evidence of our lack of control more than anything else. >> >> I do not get your point here. I was referring to the markdown-mode and >> Auctex to illustrate the _need_ to have numerous key bindings in order >> to edit complex Org documents. It is not just something introduced into >> Org out of the blue. Other people solved similar problems and did not >> come up with significantly less key bindings. If you say that usual >> 40-50 major mode bindings are sufficient, I am not convinced. > > I'm not arguing with the need, I'm arguing with the particular > implementation that caters to that need. Sorry, but I do not understand what you mean here, except that you are dissatisfied with the existing implementation. AFAIU, you objected the number of Org bindings. That many of them are not needed. I think my reply was targeting your objection. I did not recognize any kind of reference to implementation specifics. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-12 9:46 ` Ihor Radchenko @ 2022-06-12 9:59 ` Eli Zaretskii 2022-06-15 5:13 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-12 9:59 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, acm@muc.de, emacs-devel@gnu.org > Date: Sun, 12 Jun 2022 17:46:04 +0800 > > > Thanks, but if we are not going to continue this discussion until we > > come to some conclusions, I see no reason to keep talking about it. I > > do understand better the difficulties now (thanks!), but I'm not yet > > convinced that the existing solution is the best one. > > There are some interim conclusions for now. > I agree that the approach Org takes on remapping may be improved. > I will keep in mind to reach to emacs-devel about possibility to extend > built-in Emacs functionality if it is required to get rid of the Org > remappings. Thanks, I think this will indeed help in the future. > > What if the user doesn't intend to use Org tables? How can such a > > user turn off this automatic support, which interprets some patterns > > of buffer text as an indication of a table, and turns on the special > > behavior reserved for Org tables? > > Let me be clear here. Org syntax is fixed as much as possible. One > cannot just tell Org to ignore tables in Org buffers. However, users can > alter behavior of specific Org commands if they wish to. Using available > customization and other usual Elisp tools. > > One would not expect, say, tex-mode behave correctly if the user > arbitrarily changes buffer syntax table. Similarly, tables are the core > part of Org syntax. The syntax of TeX is dictated by an external tool. By contrast, the syntax of Org is determined by Org itself. Is an Org table always preceded by some directive that makes it impossible to interpret innocent text as a table? If so, perhaps the problems that were bothering me don't actually exist. But if Org interprets some text patterns as meaning that there's a table here, that could be contrary to user expectations. > >> | this | is | table | > >> | with | <point> 2 | lines | > >> ... > >> The default behavior is not satisfactory and somewhat unexpected. > > > > You assume that users always expect what Org does in that case. This > > is not a given, if we want to make Org more attractive for editing > > text that is "less Org-specific", so to speak, like README files, NEWS > > files, etc. > > We do not assume that users always expect specific behavior. That's wy > org-special-ctrl-o is customization. Relying on a user option for something that can need frequent adjustment is not the best UX. defcustom is only a good solution when it expresses a more-or-less constant user preference that change only very rarely. If I may need to change the value before invoking a command, that's inconvenient. > It's default value has been agreed upon and has not been questioned > by many. Maybe because most Org users are frequent Org users and always know what they want well enough? As mentioned, I have other kind of users in mind. > >> I do not get your point here. I was referring to the markdown-mode and > >> Auctex to illustrate the _need_ to have numerous key bindings in order > >> to edit complex Org documents. It is not just something introduced into > >> Org out of the blue. Other people solved similar problems and did not > >> come up with significantly less key bindings. If you say that usual > >> 40-50 major mode bindings are sufficient, I am not convinced. > > > > I'm not arguing with the need, I'm arguing with the particular > > implementation that caters to that need. > > Sorry, but I do not understand what you mean here, except that you are > dissatisfied with the existing implementation. AFAIU, you objected the > number of Org bindings. That many of them are not needed. I think my > reply was targeting your objection. I did not recognize any kind of > reference to implementation specifics. Adding keybindings is a solution to a problem. I'm saying that alternatives to that solution were not seriously explored fro those unbundled packages. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-12 9:59 ` Eli Zaretskii @ 2022-06-15 5:13 ` Ihor Radchenko 2022-06-15 12:49 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-15 5:13 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > The syntax of TeX is dictated by an external tool. By contrast, the > syntax of Org is determined by Org itself. Not exactly. We have to keep backwards compatibility and keep in mind external packages. So, syntax changes in Org must be avoided unless strictly necessary. And they should be backwards compatible even if the changes are necessary. > Is an Org table always preceded by some directive that makes it > impossible to interpret innocent text as a table? If so, perhaps the > problems that were bothering me don't actually exist. > > But if Org interprets some text patterns as meaning that there's a > table here, that could be contrary to user expectations. Org tables are always preceded by ^[ \t]*| I am not sure what is the problem here. One can just not start lines with | or put zero-width space if starting lines from | is absolutely necessary. >> We do not assume that users always expect specific behavior. That's wy >> org-special-ctrl-o is customization. > > Relying on a user option for something that can need frequent > adjustment is not the best UX. defcustom is only a good solution when > it expresses a more-or-less constant user preference that change only > very rarely. If I may need to change the value before invoking a > command, that's inconvenient. I am not sure why you need a frequent adjustment of org-special-ctrl-o. This is the first time I hear about such a need. >> It's default value has been agreed upon and has not been questioned >> by many. > > Maybe because most Org users are frequent Org users and always know > what they want well enough? As mentioned, I have other kind of users > in mind. I understand, but we have to consider the needs to majority of users, right? Customizations are there for other kind of users. Almost any kind of default behavior, be it Org or Emacs, will suffer from some users not liking it. >> > I'm not arguing with the need, I'm arguing with the particular >> > implementation that caters to that need. >> >> Sorry, but I do not understand what you mean here, except that you are >> dissatisfied with the existing implementation. AFAIU, you objected the >> number of Org bindings. That many of them are not needed. I think my >> reply was targeting your objection. I did not recognize any kind of >> reference to implementation specifics. > > Adding keybindings is a solution to a problem. I'm saying that > alternatives to that solution were not seriously explored fro those > unbundled packages. How can you tell it with confidence? I, personally, cannot come up with any better solution and haven't seen any alternative to the existing numerous key bindings, except context-depended dwim bindings (which are also frown upon). If you have an ideas about any better way to deal with editing complex markups, please share it. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-15 5:13 ` Ihor Radchenko @ 2022-06-15 12:49 ` Eli Zaretskii 2022-06-17 5:55 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-15 12:49 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, acm@muc.de, emacs-devel@gnu.org > Date: Wed, 15 Jun 2022 13:13:17 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > > The syntax of TeX is dictated by an external tool. By contrast, the > > syntax of Org is determined by Org itself. > > Not exactly. We have to keep backwards compatibility and keep in mind > external packages. So, syntax changes in Org must be avoided unless > strictly necessary. And they should be backwards compatible even if the > changes are necessary. This is a tangent, entirely unrelated to what I said. My point was that, unlike TeX source, which can only be submitted to TeX, and therefore must follow TeX syntax and is written by people who are familiar with TeX, Org is being advertised as a mode for editing mostly-plain text. And in plain text, what Org decided to take as an indication of a table could very well be something else, because the user just typed plain text, and was oblivious to its meaning in Org. So Org is _unlike_ TeX mode, in that the assumption that every environment or markup are only used where their Org meaning is intended -- that assumption is not necessarily true, while it is in TeX. > > Is an Org table always preceded by some directive that makes it > > impossible to interpret innocent text as a table? If so, perhaps the > > problems that were bothering me don't actually exist. > > > > But if Org interprets some text patterns as meaning that there's a > > table here, that could be contrary to user expectations. > > Org tables are always preceded by ^[ \t]*| That's what I remembered, and that is exactly what bothers me in the context of this discussion: seemingly innocent text sequences are interpreted by Org in some very special way, and the user doesn't always have good means to disable that interpretation when it is not intended. > I am not sure what is the problem here. One can just not start lines > with | or put zero-width space if starting lines from | is absolutely > necessary. Imagine a user who has no idea that a space and a | at the beginning of a line means it's an Org table, and thus won't realize that this magically changes how commands behave in that chunk of text. That is the problem that was in my mind when I said that the special table-related behavior should be better controllable and perhaps off by default. > >> We do not assume that users always expect specific behavior. That's wy > >> org-special-ctrl-o is customization. > > > > Relying on a user option for something that can need frequent > > adjustment is not the best UX. defcustom is only a good solution when > > it expresses a more-or-less constant user preference that change only > > very rarely. If I may need to change the value before invoking a > > command, that's inconvenient. > > I am not sure why you need a frequent adjustment of org-special-ctrl-o. If some instances of the same sequence of characters are indeed meant to be a table, and other instances in the same file aren't, or if I need to edit a file where they have this meaning and soon after a file where they don't, I'd need to flip the option on and off very often. > >> It's default value has been agreed upon and has not been questioned > >> by many. > > > > Maybe because most Org users are frequent Org users and always know > > what they want well enough? As mentioned, I have other kind of users > > in mind. > > I understand, but we have to consider the needs to majority of users, > right? Majority of Org users or majority of Emacs users? If we want to promote Org to be used more widely and frequently, that would inevitably mean it will be used by Emacs users who use Org only infrequently, and those are the ones I'm thinking about. We should make it easier to use Org infrequently, by people who don't do everything in Org. > >> > I'm not arguing with the need, I'm arguing with the particular > >> > implementation that caters to that need. > >> > >> Sorry, but I do not understand what you mean here, except that you are > >> dissatisfied with the existing implementation. AFAIU, you objected the > >> number of Org bindings. That many of them are not needed. I think my > >> reply was targeting your objection. I did not recognize any kind of > >> reference to implementation specifics. > > > > Adding keybindings is a solution to a problem. I'm saying that > > alternatives to that solution were not seriously explored fro those > > unbundled packages. > > How can you tell it with confidence? I can, because I'm talking about what the Emacs core developers did (or, rather, did not do). If you somehow interpreted that to allude to the developers of the respective packages, that was not the intent. IOW, I'm saying that when you compare the ELPA packages to Org in these aspects, the comparison is not really valid, because Org gets much more attention from the core developers than the ELPA packages, especially since there's a tendency to use Org in more situations. > If you have an ideas about any better way to deal with editing > complex markups, please share it. As I said before, these issues should be discussed one by one. There's no reason to believe there will be a single solution for all of them. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-15 12:49 ` Eli Zaretskii @ 2022-06-17 5:55 ` Ihor Radchenko 2022-06-17 6:40 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-17 5:55 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> Not exactly. We have to keep backwards compatibility and keep in mind >> external packages. So, syntax changes in Org must be avoided unless >> strictly necessary. And they should be backwards compatible even if the >> changes are necessary. > > This is a tangent, entirely unrelated to what I said. My point was > that, unlike TeX source, which can only be submitted to TeX, and > therefore must follow TeX syntax and is written by people who are > familiar with TeX, Org is being advertised as a mode for editing > mostly-plain text. And in plain text, what Org decided to take as an > indication of a table could very well be something else, because the > user just typed plain text, and was oblivious to its meaning in Org. > > So Org is _unlike_ TeX mode, in that the assumption that every > environment or markup are only used where their Org meaning is > intended -- that assumption is not necessarily true, while it is in > TeX. I think we are mis-communicating. Org mode is not just a major mode, it is a markup language. org-mode in Emacs is a mode for editing documents written using Org markup. The markup is fixed with tables being a part of it. Altering the markup rules is not currently supported for technical reasons (specific Org markup is not parameterized in various places in the code-base; though efforts to modulate everything, including some markup semantics, are ongoing). Note that Org markup has facilities to escape text constructs using verbatim/code or #+begin_quote/#+begin_example constructs. >> Org tables are always preceded by ^[ \t]*| > > That's what I remembered, and that is exactly what bothers me in the > context of this discussion: seemingly innocent text sequences are > interpreted by Org in some very special way, and the user doesn't > always have good means to disable that interpretation when it is not > intended. I do not think that Org will support major user changes in Org syntax any time soon or in future. At least, there is no intention to guarantee such support. I am not sure why you consider using pre-defined markup constructs as "innocent". It is problematic in any kind of markup language, be it Org, markdown, texinfo, or anything else. >> I am not sure what is the problem here. One can just not start lines >> with | or put zero-width space if starting lines from | is absolutely >> necessary. > > Imagine a user who has no idea that a space and a | at the beginning > of a line means it's an Org table, and thus won't realize that this > magically changes how commands behave in that chunk of text. That is > the problem that was in my mind when I said that the special > table-related behavior should be better controllable and perhaps off > by default. Fontification will hint about different semantic meaning. FYI, I can blame comment-dwim for having the same problem - if user does not know about comment syntax, comment-dwim will magically change its meaning in different chunks of text. The same goes for open-line when there is fill-prefix or even not-easily-visible 'left-margin text property. >> > Relying on a user option for something that can need frequent >> > adjustment is not the best UX. defcustom is only a good solution when >> > it expresses a more-or-less constant user preference that change only >> > very rarely. If I may need to change the value before invoking a >> > command, that's inconvenient. >> >> I am not sure why you need a frequent adjustment of org-special-ctrl-o. > > If some instances of the same sequence of characters are indeed meant > to be a table, and other instances in the same file aren't, or if I > need to edit a file where they have this meaning and soon after a file > where they don't, I'd need to flip the option on and off very often. I'd say that you should not use Org on files that do not use Org syntax. If you do what to do it, you should not be surprised that the same mode behaves the same way despite you having different idea in mind. More practically, you can always use file-local variables. Or, if you think that flipping between some Emacs-global default behavior and major-mode-specific behavior is frequent or important use-case, why would you not introduce global Emacs functionality or at least convention to do exactly this? I do not understand why this very general problem is blamed on Org specifically and not on other major modes. >> > Maybe because most Org users are frequent Org users and always know >> > what they want well enough? As mentioned, I have other kind of users >> > in mind. >> >> I understand, but we have to consider the needs to majority of users, >> right? > > Majority of Org users or majority of Emacs users? > > If we want to promote Org to be used more widely and frequently, that > would inevitably mean it will be used by Emacs users who use Org only > infrequently, and those are the ones I'm thinking about. We should > make it easier to use Org infrequently, by people who don't do > everything in Org. I agree here. However, I am not sure how to achieve this. We can limit Org functionality to lower the entry barrier for infrequent Org users, yes. However, it will necessarily mean that most of frequent Org users have to adjust their Org mode configurations - not good. Should we mark some of the Org commands disabled by default? Though it would make sense to mark the whole groups of commands. Is it possible? Is it also possible to disable commands only for users who never customized Org? >> > Adding keybindings is a solution to a problem. I'm saying that >> > alternatives to that solution were not seriously explored fro those >> > unbundled packages. >> >> How can you tell it with confidence? > > I can, because I'm talking about what the Emacs core developers did > (or, rather, did not do). If you somehow interpreted that to allude > to the developers of the respective packages, that was not the intent. > > IOW, I'm saying that when you compare the ELPA packages to Org in > these aspects, the comparison is not really valid, because Org gets > much more attention from the core developers than the ELPA packages, > especially since there's a tendency to use Org in more situations. I get your point. However, I do not see a good solution here. Suggestions are welcome. (I'd prefer if the discussion about the principles of key binding / editing design were in a separate thread, CCing both emacs-devel and Org mode ML). This thread is raising so many issues of various levels of criticality, generality, and validity that it is very hard to stay on topic. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-17 5:55 ` Ihor Radchenko @ 2022-06-17 6:40 ` Eli Zaretskii 2022-06-18 4:40 ` Limitations on using Org mode in buffers mixing Org markup with non-Org markup (was: Convert README.org to plain text README while installing package) Ihor Radchenko ` (2 more replies) 0 siblings, 3 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-17 6:40 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, acm@muc.de, emacs-devel@gnu.org > Date: Fri, 17 Jun 2022 13:55:41 +0800 > > > So Org is _unlike_ TeX mode, in that the assumption that every > > environment or markup are only used where their Org meaning is > > intended -- that assumption is not necessarily true, while it is in > > TeX. > > I think we are mis-communicating. Most probably, see below. > >> Org tables are always preceded by ^[ \t]*| > > > > That's what I remembered, and that is exactly what bothers me in the > > context of this discussion: seemingly innocent text sequences are > > interpreted by Org in some very special way, and the user doesn't > > always have good means to disable that interpretation when it is not > > intended. > > I do not think that Org will support major user changes in Org syntax > any time soon or in future. At least, there is no intention to guarantee > such support. > > I am not sure why you consider using pre-defined markup constructs as > "innocent". It is problematic in any kind of markup language, be it Org, > markdown, texinfo, or anything else. Neither of the other markup modes is being proposed for viewing and editing documents that were not originally edited under those modes. By contrast, there's a fraction of Emacs contributors and developers who repeatedly suggest to use Org for documents that were not originally written in Org. A notable example (not the only one) is recent discussions of turning on Org when visiting NEWS files. If you think these ideas are problematic from the POV of Org developers, please voice this opinion whenever such proposals are brought up. Those proposals, and in general the proposals to use Org widely in unrelated contexts, is what I had in mind all the time in this discussion. Perhaps now you can better understand some of my comments and responses. For example, what is your opinion of using Org markup in email messages? There are a lot of examples of that, both here and on the bug tracker. People use Org markup and Org-style code blocks quite a lot, and reading that is always jarring to me. For some reason, people assume that I read my email in Org mode or some derivative of Org. > I'd say that you should not use Org on files that do not use Org syntax. See above: there are suggestions to do just that. People are proposing to use Org in many situations where Org was not originally used, and therefore the "usual" Org assumptions don't hold. > > If we want to promote Org to be used more widely and frequently, that > > would inevitably mean it will be used by Emacs users who use Org only > > infrequently, and those are the ones I'm thinking about. We should > > make it easier to use Org infrequently, by people who don't do > > everything in Org. > > I agree here. However, I am not sure how to achieve this. We can limit > Org functionality to lower the entry barrier for infrequent Org users, > yes. However, it will necessarily mean that most of frequent Org users > have to adjust their Org mode configurations - not good. > > Should we mark some of the Org commands disabled by default? Though it > would make sense to mark the whole groups of commands. Is it possible? > Is it also possible to disable commands only for users who never > customized Org? I don't know the answers. But I'd appreciate if these consequences and prerequisites of using Org in places where it wasn't used originally could be voiced when such suggestions are brought up, and by Org developers on top of that. I quite frequently opposition to such proposals is regarded as pure conservatism, so if there are valid technical reasons and problems to solve before such proposals can be considered practical, it would be good to have them part of those discussions. > This thread is raising so many issues of various levels of > criticality, generality, and validity that it is very hard to stay on > topic. Agreed. There should be a separate thread for each specific topic. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Limitations on using Org mode in buffers mixing Org markup with non-Org markup (was: Convert README.org to plain text README while installing package) 2022-06-17 6:40 ` Eli Zaretskii @ 2022-06-18 4:40 ` Ihor Radchenko 2022-06-18 7:10 ` Eli Zaretskii 2022-06-18 5:10 ` Convert README.org to plain text README while installing package Po Lu 2022-06-20 6:05 ` Kévin Le Gouguec 2 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-18 4:40 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> I do not think that Org will support major user changes in Org syntax >> any time soon or in future. At least, there is no intention to guarantee >> such support. > > Neither of the other markup modes is being proposed for viewing and > editing documents that were not originally edited under those modes. > By contrast, there's a fraction of Emacs contributors and developers > who repeatedly suggest to use Org for documents that were not > originally written in Org. A notable example (not the only one) is > recent discussions of turning on Org when visiting NEWS files. Do I understand correctly that users would like to have some Org features (like fontification) in text that partially follows Org markup, but not fully? If so, Org mode cannot be used there as a major mode. non-Org parts of syntax will cause undefined behavior. Which is to be expected as major mode (every major mode) uses certain assumptions about the text in buffer. Even text-mode will behave weirdly if binary data is mixed with plain text. Org major mode expects text in buffers to be using Org markup. On the other hand, some parts of Org functionality are available in a form of minor modes or individual commands. As usual, minor modes are less demanding on the text in buffer. AFAIK, we have the following functionality exposed to non-Org buffers: - org-open-at-point-global/org-insert-link-global commands - orgtbl-mode to edit tables, which use Org table syntax outside Org mode - (ELPA) orgalist mode to edit lists written using Org list syntax outside Org mode - (MELPA) org-msg mode to edit and send messages using Org markup (it exports to html mime parts) - outline minor mode share a lot of functionality with Org without relying on Org markup. AFAIK, some Org features have been ported back to outline mode as well. This dedicated functionality is designed to be used outside Org markup buffers. People who request Org mode to be used in non-Org buffers probably have a specific subset of useful Org features in mind. I'd ask them which features they want to and then consider exposing them out from Org mode into minor mode (if not yet covered by the above). > If you think these ideas are problematic from the POV of Org > developers, please voice this opinion whenever such proposals are > brought up. Those proposals, and in general the proposals to use Org > widely in unrelated contexts, is what I had in mind all the time in > this discussion. Perhaps now you can better understand some of my > comments and responses. I do understand you comments now. However, I do not follow emacs-devel closely. So, it would be more efficient to CC Org ML in such discussions. > For example, what is your opinion of using Org markup in email > messages? There are a lot of examples of that, both here and on the > bug tracker. People use Org markup and Org-style code blocks quite a > lot, and reading that is always jarring to me. For some reason, > people assume that I read my email in Org mode or some derivative of > Org. I also noticed that. AFAIU it is sometimes the text part sent by org-msg and sometimes people just using Org syntax, because why not. I personally do not see a problem with using Org markup to indicate code blocks. At least, it is not much different from markdown some other people are using in emails. Even without Org mode, such markup is perfectly readable. AFAIK, there is no convention on indicating "special" parts of plain text email (e.g. code snippets) in mailing lists. The convention for indicating "markup" in emails is associating the correct mime-type to the message part. If it is html, the mail agent should render html. If it is Org, Org can be rendered. If it is plain text, the behavior is undefined in principle - people sending plain text should make the text readable as plain text in text-mode (Org can be made readable usually, but same can be said for e.g. Markdown). Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Limitations on using Org mode in buffers mixing Org markup with non-Org markup (was: Convert README.org to plain text README while installing package) 2022-06-18 4:40 ` Limitations on using Org mode in buffers mixing Org markup with non-Org markup (was: Convert README.org to plain text README while installing package) Ihor Radchenko @ 2022-06-18 7:10 ` Eli Zaretskii 2022-06-19 5:15 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-18 7:10 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, acm@muc.de, emacs-devel@gnu.org > Date: Sat, 18 Jun 2022 12:40:10 +0800 > > > Neither of the other markup modes is being proposed for viewing and > > editing documents that were not originally edited under those modes. > > By contrast, there's a fraction of Emacs contributors and developers > > who repeatedly suggest to use Org for documents that were not > > originally written in Org. A notable example (not the only one) is > > recent discussions of turning on Org when visiting NEWS files. > > Do I understand correctly that users would like to have some Org > features (like fontification) in text that partially follows Org markup, > but not fully? It's more than that. People suggest to start using Org for files that were not written under Org mode, and the reason is not just fontifications, but also Org outline-related and other commands. IIUC what you mean by "Org markup", it basically isn't used at all in those files (but would probably "seep in" given enough time, if we indeed begin using Org there). > If so, Org mode cannot be used there as a major mode. non-Org parts of > syntax will cause undefined behavior. Which is to be expected as > major mode (every major mode) uses certain assumptions about the text in > buffer. Even text-mode will behave weirdly if binary data is mixed with > plain text. Org major mode expects text in buffers to be using Org > markup. Yup. > On the other hand, some parts of Org functionality are available in a > form of minor modes or individual commands. As usual, minor modes are > less demanding on the text in buffer. > > AFAIK, we have the following functionality exposed to non-Org buffers: > - org-open-at-point-global/org-insert-link-global commands > - orgtbl-mode to edit tables, which use Org table syntax outside Org mode > - (ELPA) orgalist mode to edit lists written using Org list syntax outside Org mode > - (MELPA) org-msg mode to edit and send messages using Org markup (it > exports to html mime parts) > - outline minor mode share a lot of functionality with Org without > relying on Org markup. AFAIK, some Org features have been ported back > to outline mode as well. Indeed, files that currently use Outline mode are the ones where the above proposals are voiced most frequently. The other minor modes are not necessarily relevant; at least I don't recall those parts of Org's functionality being used as the reasons for using Org where it wasn't before. > People who request Org mode to be used in non-Org buffers probably have > a specific subset of useful Org features in mind. I'd ask them which > features they want to and then consider exposing them out from Org mode > into minor mode (if not yet covered by the above). Thanks, will keep this in mind next time such an issue comes up. > > If you think these ideas are problematic from the POV of Org > > developers, please voice this opinion whenever such proposals are > > brought up. Those proposals, and in general the proposals to use Org > > widely in unrelated contexts, is what I had in mind all the time in > > this discussion. Perhaps now you can better understand some of my > > comments and responses. > > I do understand you comments now. However, I do not follow emacs-devel > closely. So, it would be more efficient to CC Org ML in such > discussions. Fair enough. > The convention for indicating "markup" in emails is associating the > correct mime-type to the message part. If it is html, the mail agent > should render html. If it is Org, Org can be rendered. If it is plain > text, the behavior is undefined in principle - people sending plain text > should make the text readable as plain text in text-mode (Org can be > made readable usually, but same can be said for e.g. Markdown). The difference between HTML and Org is that all MUAs support the former, whereas Org can only be supported by Emacs-based MUA (and is not supported by all Emacs-based MUA: e.g., Rmail doesn't currently support it), and then only if the mime-type has been set correctly. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Limitations on using Org mode in buffers mixing Org markup with non-Org markup (was: Convert README.org to plain text README while installing package) 2022-06-18 7:10 ` Eli Zaretskii @ 2022-06-19 5:15 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-19 5:15 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, acm, emacs-devel I will reply to the last paragraph only. Other questions appear to be settled. Eli Zaretskii <eliz@gnu.org> writes: >> The convention for indicating "markup" in emails is associating the >> correct mime-type to the message part. If it is html, the mail agent >> should render html. If it is Org, Org can be rendered. If it is plain >> text, the behavior is undefined in principle - people sending plain text >> should make the text readable as plain text in text-mode (Org can be >> made readable usually, but same can be said for e.g. Markdown). > > The difference between HTML and Org is that all MUAs support the > former, whereas Org can only be supported by Emacs-based MUA (and is > not supported by all Emacs-based MUA: e.g., Rmail doesn't currently > support it), and then only if the mime-type has been set correctly. I think I was not very clear in the above. I do not advocate that MUA should support Org out of the box (it would be nice if it could, especially if people use Org instead of html, but that is a bit tangent to what I was talking about). I'd like to focus on plain text. If we forget about Org _major mode_, there is Org markup. Some people may disagree, but Org markup is perfectly readable as a non-rendered plain text. Similarly, Markdown is perfectly readable. Or any other ad-hoc self-consistent plain text convention. Admittedly, Org may be a bit too verbose in some cases (for some people, not for me; and it does not mean that alternative opinions should be ignored). (I do not say that abusing Org markup cannot make plain text unreadable, it can be said even for the purest plain text email; it was discussed in another branch). On top of being readable Org/Markdown can also be exported to html adding a benefit to users who prefer html rendering. Unlike Markdown (AFAIK), Org can also be exported to plain text, removing all the "unnecessary markup clutter" some people are so triggered about. Ideally, if some user prefers to use Org markup when writing emails, MUA should take care about supplying appropriately converted plain text version and html version of the mime-parts. That will let people read whatever format they prefer. Not all MUA support this, unfortunately. So, we have what we have and some people prefer to *write* emails using Org markup and others do not like to *read* such emails. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-17 6:40 ` Eli Zaretskii 2022-06-18 4:40 ` Limitations on using Org mode in buffers mixing Org markup with non-Org markup (was: Convert README.org to plain text README while installing package) Ihor Radchenko @ 2022-06-18 5:10 ` Po Lu 2022-06-18 11:28 ` Lars Ingebrigtsen 2022-06-20 6:05 ` Kévin Le Gouguec 2 siblings, 1 reply; 505+ messages in thread From: Po Lu @ 2022-06-18 5:10 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Ihor Radchenko, theophilusx, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > For example, what is your opinion of using Org markup in email > messages? There are a lot of examples of that, both here and on the > bug tracker. People use Org markup and Org-style code blocks quite a > lot, and reading that is always jarring to me. For some reason, > people assume that I read my email in Org mode or some derivative of > Org. +1. I hope people will stop doing that. It is the reason I can't use my work email to talk to those people, since the filter thinks the Org syntax is a virus. It's also visually jarring. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-18 5:10 ` Convert README.org to plain text README while installing package Po Lu @ 2022-06-18 11:28 ` Lars Ingebrigtsen 2022-06-18 13:33 ` Stefan Monnier 0 siblings, 1 reply; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-18 11:28 UTC (permalink / raw) To: Po Lu; +Cc: Eli Zaretskii, Ihor Radchenko, theophilusx, acm, emacs-devel Po Lu <luangruo@yahoo.com> writes: > +1. I hope people will stop doing that. It is the reason I can't use > my work email to talk to those people, since the filter thinks the Org > syntax is a virus. > > It's also visually jarring. Most Emacs mail readers understand the syntax and format it in a readable and pretty way, so I hope people won't stop doing that. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-18 11:28 ` Lars Ingebrigtsen @ 2022-06-18 13:33 ` Stefan Monnier 2022-06-18 15:50 ` tomas 2022-06-19 11:08 ` Lars Ingebrigtsen 0 siblings, 2 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-18 13:33 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Po Lu, Eli Zaretskii, Ihor Radchenko, theophilusx, acm, emacs-devel > Most Emacs mail readers understand the syntax and format it in a > readable and pretty way, so I hope people won't stop doing that. Really? I don't mind the Org syntax in email, FWIW, but I can't remember ever seeing a MUA that recognizes it and displays it in a pretty way. Does Gnus does that? How/when? Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-18 13:33 ` Stefan Monnier @ 2022-06-18 15:50 ` tomas 2022-06-18 16:00 ` Visuwesh 2022-06-19 11:08 ` Lars Ingebrigtsen 1 sibling, 1 reply; 505+ messages in thread From: tomas @ 2022-06-18 15:50 UTC (permalink / raw) To: emacs-devel [-- Attachment #1: Type: text/plain, Size: 765 bytes --] On Sat, Jun 18, 2022 at 09:33:16AM -0400, Stefan Monnier wrote: > > Most Emacs mail readers understand the syntax and format it in a > > readable and pretty way, so I hope people won't stop doing that. > > Really? I don't mind the Org syntax in email, FWIW, but I can't > remember ever seeing a MUA that recognizes it and displays it in > a pretty way. Does Gnus does that? How/when? Same with me: I actually use mutt and my reader/writer is vim (yes, really: I do most of my stuff with Emacs, but mail is one of the last surviving realms vor vim). I even catch myself formatting things in an Orgish way with vim (no special plugin: I know there are). But I can totally understand that some people don't like it. What to do? Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-18 15:50 ` tomas @ 2022-06-18 16:00 ` Visuwesh 2022-06-18 17:13 ` tomas 2022-06-18 23:22 ` [External] : " Drew Adams 0 siblings, 2 replies; 505+ messages in thread From: Visuwesh @ 2022-06-18 16:00 UTC (permalink / raw) To: tomas; +Cc: emacs-devel [சனி ஜூன் 18, 2022 17:50] tomas@tuxteam.de wrote: > On Sat, Jun 18, 2022 at 09:33:16AM -0400, Stefan Monnier wrote: >> > Most Emacs mail readers understand the syntax and format it in a >> > readable and pretty way, so I hope people won't stop doing that. >> >> Really? I don't mind the Org syntax in email, FWIW, but I can't >> remember ever seeing a MUA that recognizes it and displays it in >> a pretty way. Does Gnus does that? How/when? > > Same with me: I actually use mutt and my reader/writer is vim (yes, > really: I do most of my stuff with Emacs, but mail is one of the > last surviving realms vor vim). I even catch myself formatting > things in an Orgish way with vim (no special plugin: I know there > are). > > But I can totally understand that some people don't like it. What > to do? > I find it jarring when people throw in the very verbose org format for source blocks. Other org syntax is just fine IMO. > Cheers ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-18 16:00 ` Visuwesh @ 2022-06-18 17:13 ` tomas 2022-06-18 17:33 ` Yuri Khan ` (4 more replies) 2022-06-18 23:22 ` [External] : " Drew Adams 1 sibling, 5 replies; 505+ messages in thread From: tomas @ 2022-06-18 17:13 UTC (permalink / raw) To: emacs-devel [-- Attachment #1: Type: text/plain, Size: 497 bytes --] On Sat, Jun 18, 2022 at 09:30:47PM +0530, Visuwesh wrote: > [சனி ஜூன் 18, 2022 17:50] tomas@tuxteam.de wrote: [...] > > But I can totally understand that some people don't like it. What > > to do? > > > > I find it jarring when people throw in the very verbose org format for > source blocks. Other org syntax is just fine IMO. How would you delimit a source block? Perhaps there are some ideas for the Org folks (I'm a happy user, no more :) Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-18 17:13 ` tomas @ 2022-06-18 17:33 ` Yuri Khan 2022-06-18 23:27 ` [External] : " Drew Adams 2022-06-18 17:45 ` Eli Zaretskii ` (3 subsequent siblings) 4 siblings, 1 reply; 505+ messages in thread From: Yuri Khan @ 2022-06-18 17:33 UTC (permalink / raw) To: tomas; +Cc: Emacs developers On Sun, 19 Jun 2022 at 00:14, <tomas@tuxteam.de> wrote: > How would you delimit a source block? Classic Markdown does it by indenting with +4 spaces, but this has no way to declare the language of the block. (defun greet () (message "Hello World!")) GitHub-flavored Markdown does it with three backticks, optionally followed by the language name on the same line, followed by lines of the source block, followed by three backticks on a separate line. ```elisp (defun greet() (message "Hello World!")) ``` Either is much less verbose/visually noisy than the equivalent Org syntax. ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-18 17:33 ` Yuri Khan @ 2022-06-18 23:27 ` Drew Adams 0 siblings, 0 replies; 505+ messages in thread From: Drew Adams @ 2022-06-18 23:27 UTC (permalink / raw) To: Yuri Khan, tomas@tuxteam.de; +Cc: Emacs developers > GitHub-flavored Markdown does it with three backticks, optionally > followed by the language name on the same line, followed by lines of > the source block, followed by three backticks on a separate line. > > ```elisp > (defun greet() > (message "Hello World!")) > ``` StackExchange/Overflow uses the same. > Either is much less verbose/visually noisy than the equivalent Org > syntax. Yes. (There might be good reasons / a need for Org's noisier delimiting. Dunno.) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-18 17:13 ` tomas 2022-06-18 17:33 ` Yuri Khan @ 2022-06-18 17:45 ` Eli Zaretskii 2022-06-18 17:55 ` Visuwesh ` (2 subsequent siblings) 4 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-18 17:45 UTC (permalink / raw) To: tomas; +Cc: emacs-devel > Date: Sat, 18 Jun 2022 19:13:39 +0200 > From: tomas@tuxteam.de > > > I find it jarring when people throw in the very verbose org format for > > source blocks. Other org syntax is just fine IMO. > > How would you delimit a source block? Why do you need to delimit it at all? A code block is usually preceded by some text that makes it quite clear that what follows is a code fragment. And if you indent the code block by 2 or more spaces, that makes it stand out enough to understand where it ends. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-18 17:13 ` tomas 2022-06-18 17:33 ` Yuri Khan 2022-06-18 17:45 ` Eli Zaretskii @ 2022-06-18 17:55 ` Visuwesh 2022-06-18 18:39 ` tomas 2022-06-18 23:25 ` [External] : " Drew Adams 2022-06-19 1:48 ` Po Lu 4 siblings, 1 reply; 505+ messages in thread From: Visuwesh @ 2022-06-18 17:55 UTC (permalink / raw) To: tomas; +Cc: emacs-devel [சனி ஜூன் 18, 2022 19:13] tomas@tuxteam.de wrote: > On Sat, Jun 18, 2022 at 09:30:47PM +0530, Visuwesh wrote: >> [சனி ஜூன் 18, 2022 17:50] tomas@tuxteam.de wrote: > > [...] > >> > But I can totally understand that some people don't like it. What >> > to do? >> > >> >> I find it jarring when people throw in the very verbose org format for >> source blocks. Other org syntax is just fine IMO. > > How would you delimit a source block? > I see no reason for fancy markup in emails when the code snippets are going to be small. If it is long, then it is better to attach it as a file anyway. Personally, I indent source blocks by four spaces. > Perhaps there are some ideas for the Org folks (I'm a happy user, no > more :) > > Cheers ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-18 17:55 ` Visuwesh @ 2022-06-18 18:39 ` tomas 0 siblings, 0 replies; 505+ messages in thread From: tomas @ 2022-06-18 18:39 UTC (permalink / raw) To: emacs-devel [-- Attachment #1: Type: text/plain, Size: 853 bytes --] On Sat, Jun 18, 2022 at 11:25:13PM +0530, Visuwesh wrote: > [சனி ஜூன் 18, 2022 19:13] tomas@tuxteam.de wrote: > > > On Sat, Jun 18, 2022 at 09:30:47PM +0530, Visuwesh wrote: > >> [சனி ஜூன் 18, 2022 17:50] tomas@tuxteam.de wrote: > > > > [...] > > > >> > But I can totally understand that some people don't like it. What > >> > to do? > >> > > >> > >> I find it jarring when people throw in the very verbose org format for > >> source blocks. Other org syntax is just fine IMO. > > > > How would you delimit a source block? > > > > I see no reason for fancy markup in emails when the code snippets are > going to be small. If it is long, then it is better to attach it as a > file anyway. Personally, I indent source blocks by four spaces. [...] Thanks (also Eli and Yuri :) Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-18 17:13 ` tomas ` (2 preceding siblings ...) 2022-06-18 17:55 ` Visuwesh @ 2022-06-18 23:25 ` Drew Adams 2022-06-19 0:24 ` Tim Cross 2022-06-19 1:48 ` Po Lu 4 siblings, 1 reply; 505+ messages in thread From: Drew Adams @ 2022-06-18 23:25 UTC (permalink / raw) To: tomas@tuxteam.de, emacs-devel@gnu.org > > I find it jarring when people throw in the very verbose org format > > for source blocks. > > How would you delimit a source block? Dunno what OP had in mind, but if, as for the context of my comment, there's no need to delimit it, then, well ... no need to delimit it. Maybe it's a habit. Dunno. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: [External] : Re: Convert README.org to plain text README while installing package 2022-06-18 23:25 ` [External] : " Drew Adams @ 2022-06-19 0:24 ` Tim Cross 2022-06-19 4:35 ` tomas 0 siblings, 1 reply; 505+ messages in thread From: Tim Cross @ 2022-06-19 0:24 UTC (permalink / raw) To: emacs-devel Drew Adams <drew.adams@oracle.com> writes: >> > I find it jarring when people throw in the very verbose org format >> > for source blocks. >> >> How would you delimit a source block? > > Dunno what OP had in mind, but if, as for > the context of my comment, there's no need > to delimit it, then, well ... no need to > delimit it. > > Maybe it's a habit. Dunno. I suspect there are likely a number of contributing factors, including - Increase in number of people using org's literate programming facilities to manage their config files. When showing configs as part of a question or providing an answer, it is very easy to select and cut the src block for pasting - A number of Emacs MUAs can do some nice things with source blocks delimited with #+begin_src/#+end_src. For example, mu4e will apply syntax highlighting/font-locking for the specified language. As it uses Gnus under the hood, I expect Gnus does this as well. You don't get such highlighting if you only 'indent'. I get this behaviour, but did not enable any special feature - came 'out of the box'. - Some org users will use the org-mime package. This package makes it trivial to convert a buffer which uses org mode syntax to a multi-part mail message with plain text and html where the html is rendered with formatted source blocks and the org markup, embedded images, different headings etc. Of course, if you setup your MUA to prefer plain text over html, you will see the org markup text (though I think org-mime can be configured to export to plain text for the text/plain part). I rarely use this as I prefer plain text email. However, I have had some clients who depend on HTML email messages and using org mode and org-mime makes it very easy to work with them and still remain in my Emacs environment. Not sure this would be possible without org mode. Really nice thing is that I can decide to use org mode for composing a message on a message by message basis. I can even start composing the message and when I realise it will probably look better formatted as HTML, switch to composing using org markup and then send it as a multi-part message. I still think the worst thing that ever happened to email was moving away from plain text and allowing HTML. Unfortunately, HTML has now become so ubiquitous in email, people now see plain text as 'old' or 'primitive'. There is also the issue that most people tend to use proportional fonts these days, so the old days of being able to create OK formatted tabular data in plain text are gone - you need HTML now. As to the verbosity or jarring nature of #+begin_src/#+end_src as delimiters - I really just don't notice them. I guess that after years of reading config files and using programming languages with too much boiler plate code, I've developed the skill to easily ignore such trivial constructs. Of course, if your using emacs, it is reasonably trivial to make them less intrusive with various font locking techniques. If your not using Emacs for your email, well I guess that is what happens when you use an inferior solutions ;-) If you are using emacs and haven't bothered doing anything to reduce the jarring nature, I guess it isn't that bad an itch after all. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: [External] : Re: Convert README.org to plain text README while installing package 2022-06-19 0:24 ` Tim Cross @ 2022-06-19 4:35 ` tomas 2022-06-19 5:36 ` Tim Cross 0 siblings, 1 reply; 505+ messages in thread From: tomas @ 2022-06-19 4:35 UTC (permalink / raw) To: emacs-devel [-- Attachment #1: Type: text/plain, Size: 2615 bytes --] On Sun, Jun 19, 2022 at 10:24:10AM +1000, Tim Cross wrote: > > Drew Adams <drew.adams@oracle.com> writes: > > >> > I find it jarring when people throw in the very verbose org format > >> > for source blocks. > >> > >> How would you delimit a source block? > > > > Dunno what OP had in mind, but if, as for > > the context of my comment, there's no need > > to delimit it, then, well ... no need to > > delimit it. > > > > Maybe it's a habit. Dunno. > > I suspect there are likely a number of contributing factors, including [good insights] > I still think the worst thing that ever happened to email was moving > away from plain text and allowing HTML. Unfortunately, HTML has now > become so ubiquitous in email, people now see plain text as 'old' or > 'primitive'. There is also the issue that most people tend to use > proportional fonts these days, so the old days of being able to create OK > formatted tabular data in plain text are gone - you need HTML now. Definitely. > As to the verbosity or jarring nature of #+begin_src/#+end_src as > delimiters - I really just don't notice them. I guess that after years > of reading config files and using programming languages with too much > boiler plate code, I've developed the skill to easily ignore such > trivial constructs. But that's the point. To some people, obviously, #+begin_foo is at least as jarring as HTML markup seems to us (I'm not talking about sick Microsoft markup, where 3/4 of the file is fonts and CSS, that's even worse. I'm talking about somewhat "readable" HTML, some renderers try to do this). > Of course, if your using emacs, it is reasonably trivial to make them > less intrusive with various font locking techniques. If your not using > Emacs for your email, well I guess that is what happens when you use an > inferior solutions ;-) If you are using emacs and haven't bothered doing > anything to reduce the jarring nature, I guess it isn't that bad an itch > after all. I think here we are at the core of the conflict. "Well, duh, use Emacs" (and implicitly: "use Org mode") is akin to "well, duh, use a browser". As fond as I am of Org myself (I admitted in this thread to writing it by hand without mode support), I think expecting people to follow suit is bound to create this conflict. No, to me, Org markup isn't jarring. But I can perfectly well relate to this feeling. I think, at this point it would be much wiser to take a step back and think about a solution instead of just repeating "but Org is better" :-) Cheers -- t > > > [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: [External] : Re: Convert README.org to plain text README while installing package 2022-06-19 4:35 ` tomas @ 2022-06-19 5:36 ` Tim Cross 2022-06-19 6:43 ` tomas 0 siblings, 1 reply; 505+ messages in thread From: Tim Cross @ 2022-06-19 5:36 UTC (permalink / raw) To: tomas; +Cc: emacs-devel <tomas@tuxteam.de> writes: > [[PGP Signed Part:Undecided]] > On Sun, Jun 19, 2022 at 10:24:10AM +1000, Tim Cross wrote: >> >> Drew Adams <drew.adams@oracle.com> writes: >> >> >> > I find it jarring when people throw in the very verbose org format >> >> > for source blocks. >> >> >> >> How would you delimit a source block? >> > >> > Dunno what OP had in mind, but if, as for >> > the context of my comment, there's no need >> > to delimit it, then, well ... no need to >> > delimit it. >> > >> > Maybe it's a habit. Dunno. >> >> I suspect there are likely a number of contributing factors, including > > [good insights] > >> I still think the worst thing that ever happened to email was moving >> away from plain text and allowing HTML. Unfortunately, HTML has now >> become so ubiquitous in email, people now see plain text as 'old' or >> 'primitive'. There is also the issue that most people tend to use >> proportional fonts these days, so the old days of being able to create OK >> formatted tabular data in plain text are gone - you need HTML now. > > Definitely. > >> As to the verbosity or jarring nature of #+begin_src/#+end_src as >> delimiters - I really just don't notice them. I guess that after years >> of reading config files and using programming languages with too much >> boiler plate code, I've developed the skill to easily ignore such >> trivial constructs. > > But that's the point. To some people, obviously, #+begin_foo is at > least as jarring as HTML markup seems to us (I'm not talking about > sick Microsoft markup, where 3/4 of the file is fonts and CSS, that's > even worse. I'm talking about somewhat "readable" HTML, some renderers > try to do this). > >> Of course, if your using emacs, it is reasonably trivial to make them >> less intrusive with various font locking techniques. If your not using >> Emacs for your email, well I guess that is what happens when you use an >> inferior solutions ;-) If you are using emacs and haven't bothered doing >> anything to reduce the jarring nature, I guess it isn't that bad an itch >> after all. > > I think here we are at the core of the conflict. "Well, duh, use Emacs" > (and implicitly: "use Org mode") is akin to "well, duh, use a browser". > > As fond as I am of Org myself (I admitted in this thread to writing > it by hand without mode support), I think expecting people to follow > suit is bound to create this conflict. > > No, to me, Org markup isn't jarring. But I can perfectly well relate > to this feeling. > > I think, at this point it would be much wiser to take a step back > and think about a solution instead of just repeating "but Org is > better" :-) > > Cheers I think you missed my point. For the record, I was not implying "Just use org". I was suggesting that if your using Emacs, there are numerous things you can do to reduce the 'jarring' nature of org block syntax in email messages (such as using font-lock to alter the appearance and reduce that impact. However, if your not using something as powerful as Emacs to read your email, your likely screwed and if you are, but have done nothing to reduce the visual jarring/offence of org block syntax, then it obviously isn't as common or offensive to motivate you using that powerful editor to change the situation. at no point did I say or imply the only solution was to use org mode. Instead, I was simply pointing out there are some good reasons some people will use it and there are some advantages with Emacs mail readers (such as font-lock source code snippets). The implication is that it isn't going away, so either you have to put up with it or you have to find a way to reduce its impact and Emacs provides a wealth of functionality to do the latter. of course there is also the option to just complain about it, which is easy, but doesn't change a lot. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: [External] : Re: Convert README.org to plain text README while installing package 2022-06-19 5:36 ` Tim Cross @ 2022-06-19 6:43 ` tomas 2022-06-19 17:03 ` Drew Adams 0 siblings, 1 reply; 505+ messages in thread From: tomas @ 2022-06-19 6:43 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1808 bytes --] On Sun, Jun 19, 2022 at 03:36:43PM +1000, Tim Cross wrote: [...] > I think you missed my point. For the record, I was not implying "Just > use org". I didn't intend to convey that. I was just inviting to squint a bit and to realise that the org side arguments are as valid as those for any other markup. I think we are touching a fundamental problem: either you have a sufficiently unequivocal markup as to be machine readable, then it automatically becomes somewhat heavy-handed, or you accept that the machine has to apply some sort of mushy heuristics to try to extract part of the markup. And don't get me started on the "semantic" part (watch the Texinfo branch of this thread :). The one's semantic is the other's syntax. Semantics is like an onion: once you peel off all layers, you are left with... nothing. My point is rater that we'll possibly have to accept that to some, even seasoned Emacs users, Org might be too heavy handed, and the mitigations you propose might be of no help. This shouldn't preclude us of exploring what is in-between. But we need a very open mind, and if Po Lu, Alan, Drew et al say "it feels jarring" we better take it seriously instead of saying "nah, you'll get used to it" (I'm exaggerating a bit, I know). Would I like a unified markup to have for Texinfo, docstrings, quick notes, comments? Sure. Would I like for it to be machine readable? You bet. Would I like for it to be easy on the eyes, even as raw text? No less. Would I like for it to have links my Emacs can automatically follow? Yes, please. Would I like for it to be extensible, so I can come up with an "this is a FOO span"? Exactly. But with that wishlist, it could well be ony Alan Perlis's lollipop will do. Still worth a try :-) Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-19 6:43 ` tomas @ 2022-06-19 17:03 ` Drew Adams 0 siblings, 0 replies; 505+ messages in thread From: Drew Adams @ 2022-06-19 17:03 UTC (permalink / raw) To: tomas@tuxteam.de, Tim Cross; +Cc: emacs-devel@gnu.org > My point is rater that we'll possibly have to accept that to some, > even seasoned Emacs users, Org might be too heavy handed, and the > mitigations you propose might be of no help. > > This shouldn't preclude us of exploring what is in-between. But > we need a very open mind, and if Po Lu, Alan, Drew et al say > "it feels jarring" Just to be clear, I never said if feels jarring. I said that IF it doesn't add anything (and at least for me it doesn't) then it's just noise (for me it is). Occam says don't add noise uselessly. I said, "IF, as for the context of my comment [which was emacs.StackExchange], there's no need to delimit it, THEN, well ... no need to delimit it." [Emphasis added.] And I said: "(There might be good reasons / a need for Org's noisier delimiting. Dunno.)" My comments were not particularly about email, which the thread seems now to be focusing on. ___ As for email: I don't use Emacs to read email. So for me, at least, Org delimiting offers no positive; it's just noise. It's noise I can live with. This isn't about me. It's about what's good for Emacs. My point was that in contexts where it doesn't really add anything, maybe people just use it out of habit (dunno). And IF it doesn't add anything THEN.... FWIW, I use Outlook for email. I have no problem with HTML mail. There's HTML markup under the surface, of course, but I don't see its noise. Same with using a web browser. And HTML markup and its rendering does have a purpose/use. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-18 17:13 ` tomas ` (3 preceding siblings ...) 2022-06-18 23:25 ` [External] : " Drew Adams @ 2022-06-19 1:48 ` Po Lu 2022-06-19 2:08 ` Tim Cross 4 siblings, 1 reply; 505+ messages in thread From: Po Lu @ 2022-06-19 1:48 UTC (permalink / raw) To: tomas; +Cc: emacs-devel tomas@tuxteam.de writes: > How would you delimit a source block? > > Perhaps there are some ideas for the Org folks (I'm a happy user, no > more :) 4 spaces at the start. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-19 1:48 ` Po Lu @ 2022-06-19 2:08 ` Tim Cross 2022-06-19 3:00 ` Po Lu 2022-06-19 6:07 ` Eli Zaretskii 0 siblings, 2 replies; 505+ messages in thread From: Tim Cross @ 2022-06-19 2:08 UTC (permalink / raw) To: emacs-devel Po Lu <luangruo@yahoo.com> writes: > tomas@tuxteam.de writes: > >> How would you delimit a source block? >> >> Perhaps there are some ideas for the Org folks (I'm a happy user, no >> more :) > > 4 spaces at the start. which is fine for basic markdown where all you need is to delineate the block. However, org has much more functionality than that for source blocks and many options and/or arguments that can be passed into such blocks. All of which makes using just indentation inadequate. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-19 2:08 ` Tim Cross @ 2022-06-19 3:00 ` Po Lu 2022-06-19 6:07 ` Eli Zaretskii 1 sibling, 0 replies; 505+ messages in thread From: Po Lu @ 2022-06-19 3:00 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel Tim Cross <theophilusx@gmail.com> writes: > which is fine for basic markdown where all you need is to delineate the > block. However, org has much more functionality than that for source > blocks and many options and/or arguments that can be passed into such > blocks. All of which makes using just indentation inadequate. And please explain those options are necessary in mail, where the only meaning intended to be communicated is that the block of text is code, and not something else? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-19 2:08 ` Tim Cross 2022-06-19 3:00 ` Po Lu @ 2022-06-19 6:07 ` Eli Zaretskii 2022-06-19 6:41 ` Tim Cross 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-19 6:07 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel > From: Tim Cross <theophilusx@gmail.com> > Date: Sun, 19 Jun 2022 12:08:56 +1000 > > > 4 spaces at the start. > > which is fine for basic markdown where all you need is to delineate the > block. However, org has much more functionality than that for source > blocks and many options and/or arguments that can be passed into such > blocks. All of which makes using just indentation inadequate. Doesn't HTML express all of those adequately? Then sending HTML email is a much better alternative, IMO, as it will allow much more MUAs to produce a readable and clear display of the email. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-19 6:07 ` Eli Zaretskii @ 2022-06-19 6:41 ` Tim Cross 0 siblings, 0 replies; 505+ messages in thread From: Tim Cross @ 2022-06-19 6:41 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Tim Cross <theophilusx@gmail.com> >> Date: Sun, 19 Jun 2022 12:08:56 +1000 >> >> > 4 spaces at the start. >> >> which is fine for basic markdown where all you need is to delineate the >> block. However, org has much more functionality than that for source >> blocks and many options and/or arguments that can be passed into such >> blocks. All of which makes using just indentation inadequate. > > Doesn't HTML express all of those adequately? Then sending HTML email > is a much better alternative, IMO, as it will allow much more MUAs to > produce a readable and clear display of the email. Agreed. That is why I mentioned org-mime as that is what it can help do. ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-18 16:00 ` Visuwesh 2022-06-18 17:13 ` tomas @ 2022-06-18 23:22 ` Drew Adams 1 sibling, 0 replies; 505+ messages in thread From: Drew Adams @ 2022-06-18 23:22 UTC (permalink / raw) To: Visuwesh, tomas@tuxteam.de; +Cc: emacs-devel@gnu.org > I find it jarring when people throw in the very verbose org format for > source blocks. Other org syntax is just fine IMO. Yeah, it can be noisy. I see people sometimes adding it around a code snippet posted on emacs.StackExchange. Not only useless but almost obnoxious. The markup highlights Lisp text, but it of course does nothing useful with the Org noise around the Lisp. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-18 13:33 ` Stefan Monnier 2022-06-18 15:50 ` tomas @ 2022-06-19 11:08 ` Lars Ingebrigtsen 2022-06-19 17:10 ` [External] : " Drew Adams 1 sibling, 1 reply; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-19 11:08 UTC (permalink / raw) To: Stefan Monnier Cc: Po Lu, Eli Zaretskii, Ihor Radchenko, theophilusx, acm, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > Really? I don't mind the Org syntax in email, FWIW, but I can't > remember ever seeing a MUA that recognizes it and displays it in > a pretty way. Does Gnus does that? How/when? Isn't this fontified like Emacs Lisp for you? #+BEGIN_SRC emacs-lisp (defun org-xor (a b) "Exclusive or." (if a (not b) b)) #+END_SRC -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-19 11:08 ` Lars Ingebrigtsen @ 2022-06-19 17:10 ` Drew Adams 0 siblings, 0 replies; 505+ messages in thread From: Drew Adams @ 2022-06-19 17:10 UTC (permalink / raw) To: Lars Ingebrigtsen, Stefan Monnier Cc: Po Lu, Eli Zaretskii, Ihor Radchenko, theophilusx@gmail.com, acm@muc.de, emacs-devel@gnu.org > Isn't this fontified like Emacs Lisp for you? > > #+BEGIN_SRC emacs-lisp > (defun org-xor (a b) > "Exclusive or." > (if a (not b) b)) > #+END_SRC (I realize you're asking Stefan, and I'm not he.) But out of curiosity, about what proportion of Emacs users do you suppose use Emacs for email? (Not to mention the proportion of those that use Org mode with email.) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-17 6:40 ` Eli Zaretskii 2022-06-18 4:40 ` Limitations on using Org mode in buffers mixing Org markup with non-Org markup (was: Convert README.org to plain text README while installing package) Ihor Radchenko 2022-06-18 5:10 ` Convert README.org to plain text README while installing package Po Lu @ 2022-06-20 6:05 ` Kévin Le Gouguec 2022-06-20 8:49 ` Tim Cross 2 siblings, 1 reply; 505+ messages in thread From: Kévin Le Gouguec @ 2022-06-20 6:05 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Ihor Radchenko, theophilusx, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > For example, what is your opinion of using Org markup in email > messages? There are a lot of examples of that, both here and on the > bug tracker. People use Org markup and Org-style code blocks quite a > lot, and reading that is always jarring to me. For some reason, > people assume that I read my email in Org mode or some derivative of > Org. (There's been a whole subthread about this particular point, but I don't think I saw anyone spell this out clearly so here we go:) lisp/gnus/mm-uu.el has picked up on Org-style code blocks since 2011-01-20 "mm-decode.el (mm-inline-media-tests): Add text/x-sh. […]" (6e3165fb24). That's around the time I discovered Emacs; I used Gnus as my daily mail driver for years before I opened my first Org file. I've probably sent a couple #+begin_src's before even knowing what Org was, assuming it was one of those conventions Emacs picked up on (like "> " citations, "--" signature blocks and "--- 8< ---" scissor lines). (I don't even remember when I realized that Emacs shipped more than one MUA, and that not every maintainer used Gnus) So apologies for partaking in that practice. FWIW, if my story is any indication, not everyone who sent those blocks was inherently advocating for using Org syntax pervasively: they might just have assumed that it made life better for anyone reading their mail with Emacs (because they automatically got syntax highlighting out of it), and not realize that instead they were making it worse (by sandwiching perfectly good code between noisy sigils). ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-20 6:05 ` Kévin Le Gouguec @ 2022-06-20 8:49 ` Tim Cross 0 siblings, 0 replies; 505+ messages in thread From: Tim Cross @ 2022-06-20 8:49 UTC (permalink / raw) To: Kévin Le Gouguec; +Cc: Eli Zaretskii, Ihor Radchenko, acm, emacs-devel Kévin Le Gouguec <kevin.legouguec@gmail.com> writes: > > So apologies for partaking in that practice. FWIW, if my story is any > indication, not everyone who sent those blocks was inherently advocating > for using Org syntax pervasively: they might just have assumed that it > made life better for anyone reading their mail with Emacs (because they > automatically got syntax highlighting out of it), and not realize that > instead they were making it worse (by sandwiching perfectly good code > between noisy sigils). I don't believe there is anything to apologise for. Some people find the #+begin_src/#+end_src delimiting of blocks irritating and others don't. To be honest, I had never even given it a thought until this thread. Given it isn't something seen as a common complaint, I wouldn't worry about it too much. I actually quite like getting code snippets which are syntax highlighted, but I do use Emacs for my MUA. The fact we see few complaints about it means we don't really have any idea how many people find it a problem. The low numbers complaining could indicate few care or few find it an issue or it could mean the practice is not common enough to motivate complaining. Either way, just a storm in a tea cup. For those who do use Emacs as there MUA and who also use org-mode, the good news is you can avoid the ire of some users by converting your org content to fairly good looking HTML formatted messages. Everyone gets to see syntax highlighted code - at least everyone who has a mail reader capable of rendering HTML mail, which is probably the vast majority these days. Bad news is, this further encourages HTML for email (I would have preferred #+begin_src/#+end_src), but that argument is pretty much dead in the water having been won by HTML. For those who do prefer HTML mail, org-mode is probably your only real solution when using Emacs as your MUA. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-12 8:40 ` Ihor Radchenko 2022-06-12 8:56 ` Eli Zaretskii @ 2022-06-12 13:53 ` Kévin Le Gouguec 2022-06-12 20:07 ` Common keybindings idea for multiple markups Jean Louis 2 siblings, 0 replies; 505+ messages in thread From: Kévin Le Gouguec @ 2022-06-12 13:53 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Eli Zaretskii, theophilusx, acm, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: >> . if the above doesn't work, make forward-paragraph work through >> forward-paragraph-function, so that Org could define its own >> function > > Introducing the required flexibility into Emacs core is certainly an > option. Though it is somewhat difficult one because Org also has to > support older versions of Emacs. We are somewhat limited in using newly > introduced built-in features. A tiny note on that one very specific issue: it is possible to expose bleeding-edge core Emacs features to older Emacsen using ELPA's ":core" package type. Eglot, for example, makes use of improvements in eldoc.el, flymake.el, project.el, xref.el that live exclusively on emacs's master branch, and have not made it into a formal release yet. As long as those core libraries are protected from changes which make them reliant on newer features that cannot be released on ELPA[1], they allow their dependents to keep targeting older Emacs releases. E.g. the master branch's project.el still aims to support Emacs 26.1. So to some extent, "the technology exists" to allow Org to add hooks to core Emacs features, and use those hooks through GNU ELPA, *and* support older Emacsen. [1] See e.g. 2022-02-03 "Don't use `string-search` in soap-client (bug#53744)" (1b0342628e). ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Common keybindings idea for multiple markups 2022-06-12 8:40 ` Ihor Radchenko 2022-06-12 8:56 ` Eli Zaretskii 2022-06-12 13:53 ` Kévin Le Gouguec @ 2022-06-12 20:07 ` Jean Louis 2 siblings, 0 replies; 505+ messages in thread From: Jean Louis @ 2022-06-12 20:07 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Eli Zaretskii, theophilusx, acm, emacs-devel * Ihor Radchenko <yantar92@gmail.com> [2022-06-12 20:52]: > I was referring to the markdown-mode and > Auctex to illustrate the _need_ to have numerous key bindings in order > to edit complex Org documents. It is not just something introduced into > Org out of the blue. Other people solved similar problems and did not > come up with significantly less key bindings. There are many markups and using various of them is very beneficial due to various outputs and qualities. Example is Asciidoc and Asciidoctor implementation AsciiDoc Writer’s Guide | Asciidoctor https://asciidoctor.org/docs/asciidoc-writers-guide/# and if I may say, there may be reasons why is Asciidoctor so much financially supported: Asciidoctor - Open Collective https://opencollective.com/asciidoctor or txt2tags: txt2tags https://txt2tags.org/ and many others like reStructuredText and Wiki and what else, too many. The way to go to minimize at least some key bindings is to find the union on what is common to each of those markups. Common markup tags are headers, italic, bold, underlined, monospaced, lists, links, tables, separators, etc. etc. Once it is found what is common, then Emacs and markup modes may define how those markups look like. Then we can use functions similar to facemenu-set-bold to get that bold markup in a way how a markup mode specifies it. Then if key binding is for example "C-c b" for "bold" that bold would appear different in HTML mode, rather like <strong>word</strong> but in Markdown it would appear as **strong** and in Org as *strong* and so on. Markup mode would define how those common markups are to be applied, while Emacs could have its standardized key bindings. That would make life easier for us who use multiple markups. -- Jean Take action in Free Software Foundation campaigns: https://www.fsf.org/campaigns In support of Richard M. Stallman https://stallmansupport.org/ ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 11:21 ` Eli Zaretskii 2022-06-08 13:12 ` Ihor Radchenko @ 2022-06-08 19:34 ` Tim Cross 2022-06-09 5:18 ` Eli Zaretskii 2022-06-09 19:48 ` Alan Mackenzie 1 sibling, 2 replies; 505+ messages in thread From: Tim Cross @ 2022-06-08 19:34 UTC (permalink / raw) To: Eli Zaretskii; +Cc: acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Tim Cross <theophilusx@gmail.com> >> Cc: acm@muc.de, emacs-devel@gnu.org >> Date: Tue, 07 Jun 2022 16:14:52 +1000 >> >> > I can tell you what I see with my very simple 1570-line Org file, >> > which just has some todo items organized in 4-level hierarchy: >> > >> > . ~100 keys that look "general Org" to me >> > . ~30 keys that Org remaps to its s own commands, like 'yank' and >> > 'backward-sentence' >> > . ~40 keys bound to org-babel-SOMETHING -- no idea why these are >> > bound by default, but they are there >> > . ~60 more keys that I'm not sure whether they are "basic" or not, >> > but they are all bound by default, just because I visited an Org >> > file >> > >> > So "just" 230 key bindings. And this is without any minor/add-on Org >> > mode/feature enabled, at least according to "C-h m". >> > >> > Do you see something different? Are you still saying that it's not a >> > lot, or that it's "based on ignorance and paranoia"? If so, please >> > point out where I'm ignorant and/or paranoiac, because I'd really like >> > to know. >> >> I think Alan's response has pretty much confirmed what I was saying. Alan has >> made it fairly clear that his real underlying concern is about a stealthy >> strategy to get org mode more intrinsically tied into Emacs and in the end, >> making it so critical to normal operation that everyone will be forced to learn >> org mode regardless. I don't necessarily agree with such an argument, but think >> having a debate around that would be more up-front and in some ways honest >> compared to what appear to be somewhat contrived alternative arguments about >> excessive key bindings and added complexity. My characterisation of this all >> being fear and paranoia may have been overstating things, but then again I'm not >> sure. There certainly does seem to be considerable fear involved. I don't know >> about paranoia, but I have not seen any evil plan to have org mode assimilate >> Emacs as if it was the Borg and I've seen no discussions on the org list about >> ways to get org more ingrained in Emacs or make Emacs more dependent on org. >> >> With respect to ignorance, Alan made it quite clear he has not spent >> the time or effort to learn org mode, so he is ignorant about its >> operation and lacks any real understanding of its operation. He has made >> assertions about the number of key bindings which are wildly over >> inflated and about required levels of complexity which simply would not >> be necessary with the org mode rendering of a read only buffer. In >> short, he has made a lot of assumptions about both the key bindings and >> complexity based on only a fairly cursory scan of the manual and very >> limited experience. >> >> All that appears to have occurred here is that someone noted that packages often >> had a readme.org file and wondered if this should be translated into just a >> plain readme file. Someone else suggested we could have package descriptions >> render the readme.org file using org and not require export into plain readme >> file and mentioned some possible advantages this could have wrt navigation, >> rendering of tables, links and images or inclusion of source code snippets, >> examples etc. >> >> >From this point, some somewhat extreme claims and arguments have been thrown in. >> One was Alan's claim that org would introduce 785 new key bindings, many of >> which not bounmd to C-c and which would presumably either pollute the key space >> or conflict with existing bindings. All I have done is point out this claim was >> inaccurate and overstated due to an overly simplistic analysis of what is in the >> key index of the org manual. >> >> it should be noted that if we were to add 'native' rendering of readme.org >> files, this would likely be done with an org minor mode rather than a full blown >> org mode as very little of the overall org functionality would be required in >> this situation. All that would be required is org rendering support and org >> support for navigation (folding/unfolding, following links and possibly >> list/table navigation). All the advanced org functionality relating to babel, >> noweb, todo and time management, agendas, etc have no relevance in this >> scenario. LIkewise, if this functionality was provided via a minor mode, it >> would also be possible for people to disable that mode and be left with just the >> readme.org file, which is just plain text and quite readable. >> >> Alan's claim was 785 additional key bindings, many of which not bound to C-c. > > The above is completely irrelevant to my questions. > >> Running Emacs wiht -q and opeing up an org file, I find the following >> >> - 236 org related key bindings - far less than 785 Of these, a number are >> - actually remapping of existing bindings, so not new ones. This leaves 208. > > ??? How is remapping "not new"? It takes some very common Emacs > commands and redirects them to different commands. E.g., 'open-line' > now does something quite different. This means the user should either > go learn what the Org commands do, or be prepared to be surprised. This are not new key bindings, they are redefinitions of existing key bindings to make them work in an org mode context in a wayu which is consistent with user expectations. They are not new in the sense of them being a key binding which did not exist prior to loading org mode. > >> I said that for a read only buffer, many of the org key bindings are not >> relevant as they relate to features which are not pertinent to a read only org >> buffer. Any bindings relating to babel, todo management, time management, >> agendas etc have no relevance when reading a readme.org file. > > Then why does Org define them in that case? > because standard org mode is not typically used in read only buffers. How org is defined now and how it would be defined/loaded when it comes to a specialised use case as discussed here are completely different. As I said in previous posts, I would expect that you would define an org minor mode which would only include the key bindings which would be appropriate for the use case of viewing a read-only org file. >> Really, the only >> key bindings of any relevance are navigation related bindings (including >> opening/closing folds, following links, etc). Removing all unnecessary org >> bindings leave us with just 77 new bindings. Of these remaining bindings, quite >> a few could also be removed, but I've left them in as I only wanted to remove >> those which would clearly be of no benefit in a read only buffer rendering of a >> readme.org file. In an org minor mode setup for rendering of readme.org (and >> possibly other read-only rendering of org files), I estimate at least another 30 >> bindings could be removed because they have no real benefit or are of only >> marginal benefit. Creating an org minor mode for this purpose which only >> introduced 20 or so new bindings would easily be possible. The key point is that >> claims of 785 new bindings arfe incorrect and misleading. Even being very >> conservative, we are really talking about 10% of the number being incorrectly claimed. >> if we introduced a minor mode for this, we are talking likely less than 20 or so. > > I'm sorry, but IMO this hand-waves away a real problem by ignoring the > problematic UX of suddenly having more than 200 keybindings defined > just because I visited a small file. I hoped you will present some > serious analysis of the issue, and perhaps even show me where I'm > wrong in my conclusions. Sadly, it sounds like all you wanted to do > is to prove that you are right, the facts notwithstanding. Sorry, but it seems you are the one trying to drag this off into a different argument about whether org mode is too large or defines too much or loads too much. As stated multiple times in my previous posts in this thread, should org mode be used in the context being discussed, you would define a new org minor mode whic only included the functionality and key bindings relevant for this use case. You would not just use a full org mode for this situation. > > You accuse Alan in extremism, but your own argumentation is another > example of a similar extremism -- just in the opposite direction. > I have never said that 200 key bindings wasn't a lot. All I've said is that 200 key bindings is a LOT less than the 784 being claimed. I went further to state that in a specialised read-only org mode, you would be able to reduce that number to around 50 or possibly even fewer. I also reject what was stated about the org mode key bindings polluting the global key space. I find no evidence of such pollution and in fact believe org mode is very good at not doing so. I find very few (if any) examples of org doing global key binding and in fact have to define new personal bindings when I want global access to org functionality (such as org capture or the org agenda). > This doesn't facilitate useful discussions of what I think is a real > problem. Any unbiased observer should agree that adding 200+ key > bindings when a small file is visited is a problem that needs to be > solved. Org is too large for such simple jobs, and should be > refactored to avoid loading everything under the sun when a README is > visited. And we should work towards solving it, not sweep it under > the carpet. > which is exactly why I was saying that should you want org rendering of small read only files, you would create an org minor mode which only loaded the functionality and key bindings that would be relevant in that context. It is only you who are assuming that a full org mode would be used in this context. Nobody has tried to sweep anything under the carpet. Right back in my very 1st post to this thread, I stated that should org be used to render content in things like a help buffer or package description whhich would presumably be a read only presentation, you would want an org minor mode which only loads the functionality, including appropriate key bindings which made sense in that context. This sentiment was also mirrored by others in the thread. . Questions about general org mode and whether it defines too many key bindings or too many features when loaded is a completely separate question. While there could be some important discussion which needs to happen in that context, it has little to do with what was being discussed here and even less relevance to what I was saying, whihc was that claiming opening an org file would define 784 new key bindings and would pollute the key space was an exaggeration and over statement of what happens when you open an org file. Even your own example confirmed this. As to whether defining 200 key bindings is too many - well I don't think you can possibly say either way without significantly more analysis. It also isn't at all given that adding 200 new key bindings in a new mode is of itself problematic. There could be very good reasons to add that many bindings. Besides, I'm not sure it even is a problem. Emacs has lots of key bindings - the vast majority of which I never use. I don't find this a problem and I'm not convinced just citing absolute binding numbers in itself is evidence of a problem. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 19:34 ` Convert README.org to plain text README while installing package Tim Cross @ 2022-06-09 5:18 ` Eli Zaretskii 2022-06-09 19:48 ` Alan Mackenzie 1 sibling, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-09 5:18 UTC (permalink / raw) To: Tim Cross; +Cc: acm, emacs-devel > From: Tim Cross <theophilusx@gmail.com> > Cc: acm@muc.de, emacs-devel@gnu.org > Date: Thu, 09 Jun 2022 05:34:09 +1000 > > > ??? How is remapping "not new"? It takes some very common Emacs > > commands and redirects them to different commands. E.g., 'open-line' > > now does something quite different. This means the user should either > > go learn what the Org commands do, or be prepared to be surprised. > > This are not new key bindings, they are redefinitions of existing key > bindings to make them work in an org mode context in a wayu which is > consistent with user expectations. That's exactly my point: suddenly commands that I'm familiar with are working in ways that are different, even if slightly different. A conscientious Emacs user will want/need to study the differences, before using those remapped commands. > They are not new in the sense of them being a key binding which did > not exist prior to loading org mode. This completely misses the point I explain above. The _commands_ behave differently, and that is all that counts. > >> Any bindings relating to babel, todo management, time management, > >> agendas etc have no relevance when reading a readme.org file. > > > > Then why does Org define them in that case? > > because standard org mode is not typically used in read only buffers. Which is one more argument towards more modularity in Org, so that only the relevant bindings and commands are defined for each use case. > As to whether defining 200 key bindings is too many - well I don't think > you can possibly say either way without significantly more analysis. Which is why I suggested you do this kind of analysis. > It also isn't at all given that adding 200 new key bindings in a new > mode is of itself problematic. There could be very good reasons to > add that many bindings. You have just explained that many of them are not relevant, even for editing simple enough Org files. For example, everything related to org-babel -- why do these bindings have to be available by default? Many Org files have no code blocks at all. > Besides, I'm not sure it even is a problem. Emacs has lots of key > bindings - the vast majority of which I never use. I don't find this > a problem and I'm not convinced just citing absolute binding numbers > in itself is evidence of a problem. Well, I _am_ convinced, looking at this from the POV of the Emacs maintainer. I also think almost everyone agrees that too many unneeded keybindings are a problem in general -- witness the removal of some veteran global bindings in recent Emacs version, which is evidence that we consider even half a dozen less-then-useful bindings not a good thing. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 19:34 ` Convert README.org to plain text README while installing package Tim Cross 2022-06-09 5:18 ` Eli Zaretskii @ 2022-06-09 19:48 ` Alan Mackenzie 2022-06-11 4:09 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Alan Mackenzie @ 2022-06-09 19:48 UTC (permalink / raw) To: Tim Cross; +Cc: Eli Zaretskii, emacs-devel Hello, Tim. On Thu, Jun 09, 2022 at 05:34:09 +1000, Tim Cross wrote: > Eli Zaretskii <eliz@gnu.org> writes: [ .... ] > >> Alan's claim was 785 additional key bindings, many of which not > >> bound to C-c. It wasn't a "claim" it was a statement of fact. These 784 bindings are DOCUMENTED in org.info. You attempted to explain it away by saying, at first vaguely, that not all of them "come into play". Only later did you make that meaningful by saying that not all of them get loaded at any time. Many of these binding are not bound to C-c C-<letter> or other major mode sequences. > > The above is completely irrelevant to my questions. > >> Running Emacs wiht -q and opeing up an org file, I find the following > >> - 236 org related key bindings - far less than 785. 236 and 784 are both good approximations to infinity when it comes to key binding numbers. Even 236 is barely manageable for anybody wanting to learn the mode in the way I would. [ .... ] > > Then why does Org define them [lots of key bindings] in that case? > because standard org mode is not typically used in read only buffers. > How org is defined now and how it would be defined/loaded when it comes > to a specialised use case as discussed here are completely different. > As I said in previous posts, I would expect that you would define an > org minor mode which would only include the key bindings which would be > appropriate for the use case of viewing a read-only org file. I think the use case is that of a small org file rather than a read-only one. For some value of "small". [ .... ] > > I'm sorry, but IMO this hand-waves away a real problem by ignoring the > > problematic UX of suddenly having more than 200 keybindings defined > > just because I visited a small file. I hoped you will present some > > serious analysis of the issue, and perhaps even show me where I'm > > wrong in my conclusions. Sadly, it sounds like all you wanted to do > > is to prove that you are right, the facts notwithstanding. > Sorry, but it seems you are the one trying to drag this off into a > different argument about whether org mode is too large or defines too > much or loads too much. That has been the main topic of this discussion for several rounds in the thread. I've found your responses very frustrating, feeling like you've been broad brushing precise topics with vagueness, and like I'm dealing with a politician rather than a hacker. In a typical thread in this mailing list, pin-point accuracy is rare, but participants attempt to answer what the other person means rather than the precise literal meaning of what was said (sometimes getting it wrong). I don't feel this has been happening in this part of this thread. > As stated multiple times in my previous posts in this thread, .... And overbearing with it. Why? > .... should org mode be used in the context being discussed, you would > define a new org minor mode whic only included the functionality and > key bindings relevant for this use case. You would not just use a full > org mode for this situation. The context being discussed is, I think, "small" org files rather than read-only ones, whatever small means here. I think we're all agreed that full org mode is inappropriate for these. > > You accuse Alan in extremism, but your own argumentation is another > > example of a similar extremism -- just in the opposite direction. > I have never said that 200 key bindings wasn't a lot. All I've said is > that 200 key bindings is a LOT less than the 784 being claimed. It's not a LOT less, both being unmanageably large. > I went further to state that in a specialised read-only org mode, you > would be able to reduce that number to around 50 or possibly even > fewer. I also reject what was stated about the org mode key bindings > polluting the global key space. I find no evidence of such pollution > .... I find plenty of evidence in org.info. > and in fact believe org mode is very good at not doing so. Go into org.info and count up all the "new" (as you have defined it above) bindings which don't begin C-c C-<letter> or other major mode sequences. There are more of these than the total number of bindings a typical mode has. Examples: * C-c /: Sparse Trees. * M-DOWN: Structure Editing. > I find very few (if any) examples of org doing global key binding and > in fact have to define new personal bindings when I want global access > to org functionality (such as org capture or the org agenda). > > This doesn't facilitate useful discussions of what I think is a real > > problem. Any unbiased observer should agree that adding 200+ key > > bindings when a small file is visited is a problem that needs to be > > solved. Org is too large for such simple jobs, and should be > > refactored to avoid loading everything under the sun when a README is > > visited. And we should work towards solving it, not sweep it under > > the carpet. > which is exactly why I was saying that should you want org rendering of > small read only files, you would create an org minor mode which only > loaded the functionality and key bindings that would be relevant in that > context. It is only you who are assuming that a full org mode would be > used in this context. I think Eli has experienced this. He is not "assuming" it at all. > Nobody has tried to sweep anything under the carpet. Right back in > my very 1st post to this thread, I stated that should org be used to > render content in things like a help buffer or package description > whhich would presumably be a read only presentation, you would want an > org minor mode which only loads the functionality, including appropriate > key bindings which made sense in that context. This sentiment was also > mirrored by others in the thread. . > Questions about general org mode and whether it defines too many key > bindings or too many features when loaded is a completely separate > question. Not at all. Without org mode's massive numbers of bindings (several hundred), the question of refactoring it wouldn't even have been an issue. > While there could be some important discussion which needs to happen in > that context, it has little to do with what was being discussed here > and even less relevance to what I was saying, which was that claiming > opening an org file would define 784 new key bindings .... That was not claimed. The thing remarked upon was that there are 784 key bindings in org mode. [ .... ] > As to whether defining 200 key bindings is too many - well I don't think > you can possibly say either way without significantly more analysis. It > also isn't at all given that adding 200 new key bindings in a new mode > is of itself problematic. Problems with it have been identified in this thread. > There could be very good reasons to add that many bindings. Besides, > I'm not sure it even is a problem. Emacs has lots of key bindings - the > vast majority of which I never use. I don't find this a problem and I'm > not convinced just citing absolute binding numbers in itself is > evidence of a problem. Maybe not, but the problems are there nonetheless. -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-09 19:48 ` Alan Mackenzie @ 2022-06-11 4:09 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-11 4:09 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Tim Cross, Eli Zaretskii, emacs-devel Alan Mackenzie <acm@muc.de> writes: >> >> Alan's claim was 785 additional key bindings, many of which not >> >> bound to C-c. > > It wasn't a "claim" it was a statement of fact. These 784 bindings are > DOCUMENTED in org.info. You attempted to explain it away by saying, at > first vaguely, that not all of them "come into play". Only later did you > make that meaningful by saying that not all of them get loaded at any > time. > > Many of these binding are not bound to C-c C-<letter> or other major mode > sequences. Please note that not only C-c C-<letter> are dedicated to major modes. Also, C-c <non-letter> (though not all C-c <non-letter> are recommended). I vent through all the bindings we define in the manual and the only bindings that do not use reserved major-mode bindings space are: C-' C-, <control character/s>-<arrow> <control character/s>-<RET> Out of the above bindings only C-S-<up>/<down> override the default bindings. But only when shift-select-mode is enabled (which had not been the default when these Org bindings were introduced). Other bindings might cause unexpected behavior when user has personal bindings. However, please note that Emacs recommends binding C-c <letter> for user space. Anything else is a "gray" zone. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 6:14 ` Tim Cross 2022-06-07 11:21 ` Eli Zaretskii @ 2022-06-07 16:02 ` Alan Mackenzie 2022-06-07 18:14 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Stefan Monnier 2022-06-07 21:51 ` Convert README.org to plain text README while installing package Tim Cross 1 sibling, 2 replies; 505+ messages in thread From: Alan Mackenzie @ 2022-06-07 16:02 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel Hello, Tim. On Tue, Jun 07, 2022 at 16:14:52 +1000, Tim Cross wrote: > Eli Zaretskii <eliz@gnu.org> writes: > >> From: Tim Cross <theophilusx@gmail.com> > >> Cc: emacs-devel@gnu.org > >> Date: Mon, 06 Jun 2022 23:57:55 +1000 > >> > There are 784 key bindings in org mode. How can you say that this isn't > >> > complex and difficult to learn? > >> Very simply because the vast majority of those key bindings only come > >> into play when you use advanced features of org mode, such as the > >> agenda or table editing or noweb mode. If your just using basic org mode > >> features, very very few of those key bindings even come into play. Just > >> go throgh that list and remove any bindings relating to agenda mode, > >> table editing mode, source block modes, clock table mode, list editing > >> mode, etc and you end up with very few key bindings (all of which are > >> udner the C-c prefix, unlike your previous claim). > > How about if you do this exercise and walk us through the results? > > Visit an Org file, type "C-h b", then show us the list, and tell which > > bindings you think are "basic" and which are "advanced", and why you > > think so. > > I can tell you what I see with my very simple 1570-line Org file, > > which just has some todo items organized in 4-level hierarchy: > > . ~100 keys that look "general Org" to me > > . ~30 keys that Org remaps to its s own commands, like 'yank' and > > 'backward-sentence' > > . ~40 keys bound to org-babel-SOMETHING -- no idea why these are > > bound by default, but they are there > > . ~60 more keys that I'm not sure whether they are "basic" or not, > > but they are all bound by default, just because I visited an Org > > file > > So "just" 230 key bindings. And this is without any minor/add-on Org > > mode/feature enabled, at least according to "C-h m". > > Do you see something different? Are you still saying that it's not a > > lot, or that it's "based on ignorance and paranoia"? If so, please > > point out where I'm ignorant and/or paranoiac, because I'd really like > > to know. > I think Alan's response has pretty much confirmed what I was saying. Alan has > made it fairly clear that his real underlying concern is about a stealthy > strategy to get org mode more intrinsically tied into Emacs and in the end, > making it so critical to normal operation that everyone will be forced to learn > org mode regardless. I don't necessarily agree with such an argument, but think > having a debate around that would be more up-front and in some ways honest > compared to what appear to be somewhat contrived alternative arguments about > excessive key bindings and added complexity. You've misread the situation. My objections to org mode are almost entirely due to its excessive complexity as measured by the number of key bindings. [ .... ] > With respect to ignorance, Alan made it quite clear he has not spent > the time or effort to learn org mode, so he is ignorant about its > operation and lacks any real understanding of its operation. This is indeed the case. > He has made assertions about the number of key bindings which are > wildly over inflated ..... You have said this several times in your post without any justification. There are 784 entries in the key index in org.info. You're saying it's "wildly over inflated" to use this number, at least as a first estimate. Why are you making such a counter intuitive claim without any justification? Eli counted around 230 bindings from C-h m. It thus seems likely that on average, an org mode key sequence is used for 3½ distinct functions. This is not a good thing. > .... and about required levels of complexity which simply would not be > necessary with the org mode rendering of a read only buffer. You're not saying that loading a RO readme.org would somehow only load a small part of org mode, surely? > In short, he has made a lot of assumptions about both the key bindings > and complexity based on only a fairly cursory scan of the manual and > very limited experience. And now you're trying to be patronising, but failing. ;-) Why do you make no attempt to explain why my assumptions are invalid? > All that appears to have occurred here is that someone noted that > packages often had a readme.org file and wondered if this should be > translated into just a plain readme file. Someone else suggested we > could have package descriptions render the readme.org file using org > and not require export into plain readme file and mentioned some > possible advantages this could have wrt navigation, rendering of > tables, links and images or inclusion of source code snippets, examples > etc. > From this point, some somewhat extreme claims and arguments have been > thrown in. One was Alan's claim that org would introduce 785 new key > bindings, many of which not bounmd to C-c and which would presumably > either pollute the key space or conflict with existing bindings. All I > have done is point out this claim was inaccurate and overstated due to > an overly simplistic analysis of what is in the key index of the org > manual. You have made an unfounded assertion of my inaccuracy. Are you suggesting, perhaps, that of these 784 entries in org.info's key index, the vast bulk are dummies? Some of these bindings (I don't think I used the word "many", but may have done) are not in the standard major mode space C-c C-<letter>. Some of these clash with existing bindings, examples of which I gave last night. > it should be noted that if we were to add 'native' rendering of readme.org > files, this would likely be done with an org minor mode rather than a full blown > org mode as very little of the overall org functionality would be required in > this situation. In other words, an attempt would be made to fix the problems which I am drawing to people's attention. > All that would be required is org rendering support and org support for > navigation (folding/unfolding, following links and possibly list/table > navigation). All the advanced org functionality relating to babel, > noweb, todo and time management, agendas, etc have no relevance in this > scenario. LIkewise, if this functionality was provided via a minor > mode, it would also be possible for people to disable that mode and be > left with just the readme.org file, which is just plain text and quite > readable. > Alan's claim was 785 additional key bindings, many of which not bound to C-c. Actually it was 784, 28². Again, why is this number invalid? > Running Emacs wiht -q and opeing up an org file, I find the following > - 236 org related key bindings - far less than 785 Of these, a number are > - actually remapping of existing bindings, so not new ones. This leaves 208. > I said that for a read only buffer, many of the org key bindings are not > relevant as they relate to features which are not pertinent to a read only org > buffer. Any bindings relating to babel, todo management, time management, > agendas etc have no relevance when reading a readme.org file. Really, the only > key bindings of any relevance are navigation related bindings (including > opening/closing folds, following links, etc). Removing all unnecessary org > bindings leave us with just 77 new bindings. Of these remaining bindings, quite > a few could also be removed, but I've left them in as I only wanted to remove > those which would clearly be of no benefit in a read only buffer rendering of a > readme.org file. In an org minor mode setup for rendering of readme.org (and > possibly other read-only rendering of org files), I estimate at least another 30 > bindings could be removed because they have no real benefit or are of only > marginal benefit. Creating an org minor mode for this purpose which only > introduced 20 or so new bindings would easily be possible. You're describing what might be rather than what is. > The key point is that claims of 785 new bindings are incorrect and > misleading. I resent you continally repeating this unfounded assertion. There are 784 entries in org.info's key index. That is the current state. That is the truth. How can that be "incorrect"? Even if 750 of the 784 bindings somehow "don't come into play", they still exist, and are still problematic. Are you saying that only every tenth entry actually signifies an actual binding, or something like that? > Even being very conservative, we are really talking about 10% of the > number being incorrectly claimed. if we introduced a minor mode for > this, we are talking likely less than 20 or so. > Arguments regarding this being a 'slippery slope' or 'edge of the > wedge' in an org takeover are a completely different argument and there > may well be some concerns which may need to be considered. However, if > that is the real concern, then argue based on that rather than > exaggerated claims about excessive key bindings, key binding > polution/conflicts and excessive complexity. If it weren't for org mode's excessive key bindings and complexity, the "takeover scenario" would scarcely be a worry. [ .... ] -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 505+ messages in thread
* Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-07 16:02 ` Alan Mackenzie @ 2022-06-07 18:14 ` Stefan Monnier 2022-06-07 18:26 ` Org mode and Emacs Lars Ingebrigtsen ` (2 more replies) 2022-06-07 21:51 ` Convert README.org to plain text README while installing package Tim Cross 1 sibling, 3 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-07 18:14 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Tim Cross, emacs-devel Rather than try and figure out who's right or wrong, I hope we can take advantage of this discussion to get some code out that try to solve the problem. AFAICT the problem seen from Emacs, is that Org is large (even for a basic uses, which occasionally leads to high load times) and that it doesn't follow all the usual Emacs conventions, such as overriding/remapping standard key-bindings (the resulting behaviors may sometimes be similar to the standard one, but even when that's the case the redefinition can easily bite the user). The problem seen from Org is that Emacs doesn't use Org enough :-) [ This paragraph's shorter length probably reflects the fact that I'm less familiar with Org than with Emacs. ] I think the way forward is to define a "basic org-mode". This one could be used at many more places where there's currently an occasional desire to use Org that's resisted because of the above problems. Ideally, org-mode would then be defined as an extension of this "basic org-mode". Also ideally some of those extensions would be reworked so they can be used "Emacs-wide" rather than only in Org (obviously, that can only make sense for some of them). We could start this "basic org-mode" as a trivial copycat of `outline-mode` and then start adding Org features to it. The driving constraint is to keep it lightweight and rules-abiding enough that there won't be any resistance to using it, while at the same time making sure that the features added to it can be removed from the "org-mode extension", so that org-mode and "basic org-mode" don't diverge. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-07 18:14 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Stefan Monnier @ 2022-06-07 18:26 ` Lars Ingebrigtsen 2022-06-07 18:48 ` Stefan Monnier 2022-06-07 22:10 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Tim Cross 2022-06-08 13:22 ` Ihor Radchenko 2 siblings, 1 reply; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-07 18:26 UTC (permalink / raw) To: Stefan Monnier; +Cc: Alan Mackenzie, Tim Cross, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > I think the way forward is to define a "basic org-mode". This one could > be used at many more places where there's currently an occasional desire > to use Org that's resisted because of the above problems. I think that people that use Org mode wouldn't be very interested in that, and people that don't use Org can just continue editing those files with `M-x find-file-literally', really. Doing so is fine. But I don't understand the discussion this thread warped out of. When we display a README from Package, we shouldn't be showing the raw text of README.org or README.md or README.html, lightly fontified -- we should be showing a rendering of it. People that read the README aren't supposed to edit it, after all. Which would make the 40K key bindings that Org apparently has completely irrelevant for non-Org users. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-07 18:26 ` Org mode and Emacs Lars Ingebrigtsen @ 2022-06-07 18:48 ` Stefan Monnier 2022-06-07 18:54 ` Eli Zaretskii 2022-06-07 20:54 ` Lars Ingebrigtsen 0 siblings, 2 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-07 18:48 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Alan Mackenzie, Tim Cross, emacs-devel > I think that people that use Org mode wouldn't be very interested in > that, and people that don't use Org can just continue editing those > files with `M-x find-file-literally', really. Doing so is fine. I was thinking of files like etc/NEWS where having "a bit more than outline-mode" would be welcome. > But I don't understand the discussion this thread warped out of. When > we display a README from Package, we shouldn't be showing the raw text > of README.org or README.md Why not? If it can be prettified on the fly by font-lock? Separating "viewing" from "editing" is un-Emacsish, because it puts a barrier between the "consumers" and the "producers" whereas Emacs likes to try and get end-users exposed to the source as mush as possible so it takes little effort for them to change role. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-07 18:48 ` Stefan Monnier @ 2022-06-07 18:54 ` Eli Zaretskii 2022-06-07 19:38 ` Stefan Monnier 2022-06-07 20:54 ` Lars Ingebrigtsen 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-07 18:54 UTC (permalink / raw) To: Stefan Monnier; +Cc: larsi, acm, theophilusx, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Alan Mackenzie <acm@muc.de>, Tim Cross <theophilusx@gmail.com>, > emacs-devel@gnu.org > Date: Tue, 07 Jun 2022 14:48:31 -0400 > > > But I don't understand the discussion this thread warped out of. When > > we display a README from Package, we shouldn't be showing the raw text > > of README.org or README.md > > Why not? Because it's butt-ugly! ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-07 18:54 ` Eli Zaretskii @ 2022-06-07 19:38 ` Stefan Monnier 0 siblings, 0 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-07 19:38 UTC (permalink / raw) To: Eli Zaretskii; +Cc: larsi, acm, theophilusx, emacs-devel >> > But I don't understand the discussion this thread warped out of. When >> > we display a README from Package, we shouldn't be showing the raw text >> > of README.org or README.md >> Why not? > Because it's butt-ugly! I don't think it has to be. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-07 18:48 ` Stefan Monnier 2022-06-07 18:54 ` Eli Zaretskii @ 2022-06-07 20:54 ` Lars Ingebrigtsen 1 sibling, 0 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-07 20:54 UTC (permalink / raw) To: Stefan Monnier; +Cc: Alan Mackenzie, Tim Cross, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >> But I don't understand the discussion this thread warped out of. When >> we display a README from Package, we shouldn't be showing the raw text >> of README.org or README.md > > Why not? If it can be prettified on the fly by font-lock? > > Separating "viewing" from "editing" is un-Emacsish, because it puts > a barrier between the "consumers" and the "producers" whereas Emacs > likes to try and get end-users exposed to the source as mush as possible > so it takes little effort for them to change role. That's true, but on the other hand, we don't show texinfo files to people for a good reason. That's about as "plain text" as normal Org files, or as some Markdown files get. (Most Markdown files are kinda pleasant, but there's plenty of scope for unreadability -- it drops down into HTML for tables, for instance.) Or take doc strings as an example -- it's always been a markup language, too (\\ and \\[] and \\<>), but it's moving more clearly in that direction -- because we want to make the documentation we present the user pretty and easy to read. And we have one-key commands to take us to the source code, which we could have in the Package README rendering, too. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-07 18:14 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Stefan Monnier 2022-06-07 18:26 ` Org mode and Emacs Lars Ingebrigtsen @ 2022-06-07 22:10 ` Tim Cross 2022-06-08 6:06 ` Org mode and Emacs Visuwesh 2022-06-09 22:31 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Richard Stallman 2022-06-08 13:22 ` Ihor Radchenko 2 siblings, 2 replies; 505+ messages in thread From: Tim Cross @ 2022-06-07 22:10 UTC (permalink / raw) To: Stefan Monnier; +Cc: Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > Rather than try and figure out who's right or wrong, I hope we can take > advantage of this discussion to get some code out that try to solve > the problem. > > AFAICT the problem seen from Emacs, is that Org is large (even for > a basic uses, which occasionally leads to high load times) and that it > doesn't follow all the usual Emacs conventions, such as > overriding/remapping standard key-bindings (the resulting behaviors may > sometimes be similar to the standard one, but even when that's the case > the redefinition can easily bite the user). > > The problem seen from Org is that Emacs doesn't use Org enough :-) > [ This paragraph's shorter length probably reflects the fact that I'm > less familiar with Org than with Emacs. ] > > I think the way forward is to define a "basic org-mode". This one could > be used at many more places where there's currently an occasional desire > to use Org that's resisted because of the above problems. > Ideally, org-mode would then be defined as an extension of this "basic > org-mode". Also ideally some of those extensions would be reworked so > they can be used "Emacs-wide" rather than only in Org (obviously, that > can only make sense for some of them). > > We could start this "basic org-mode" as a trivial copycat of > `outline-mode` and then start adding Org features to it. The driving > constraint is to keep it lightweight and rules-abiding enough that there > won't be any resistance to using it, while at the same time making sure > that the features added to it can be removed from the "org-mode > extension", so that org-mode and "basic org-mode" don't diverge. > > Thanks Stefan and I agree. I also think this is in general exactly the direction current org maintainers/developers are taking. At this point, considerable work is being done (by Ihor and others) to - Implement a concise and efficient parser. This has also involved refining and clarifying org syntax, which has had some holes and ambiguities. - Replace org's largely regexp based font locking mechanism with one based on the parsing and tagging made possible by the org parser. This should improve both performance and accuracy in parsing. - Improving efficiency of folding etc - Improving efficiency with respect to larger org files through the use of caching etc. - Increasing org's use of built-in Emacs capabilities rather than using org specific implementations. For example, adopting transient instead of an org implemented module which replicates similar functionality. - Increase modularity an enable loading of the functionality that you want. In particular, the important work being done by Ihor wrt folding, parsing and caching will provide a core of functionality which can be used to define something like an org minor mode which could be used in those situations where we want some basic org functionality, like folding, navigation and formatting/presentation, but we don't need all the additional advanced features, such as babel, todo and time management, spreadsheets, tables and table formulas etc. Of course, to what extent Emacs will/should leverage off org mode is a completely different debate. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-07 22:10 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Tim Cross @ 2022-06-08 6:06 ` Visuwesh 2022-06-08 6:58 ` Tim Cross 2022-06-09 22:31 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Visuwesh @ 2022-06-08 6:06 UTC (permalink / raw) To: Tim Cross; +Cc: Stefan Monnier, Alan Mackenzie, emacs-devel [புதன் ஜூன் 08, 2022] Tim Cross wrote: > [...] > - Increasing org's use of built-in Emacs capabilities rather than using > org specific implementations. For example, adopting transient instead > of an org implemented module which replicates similar functionality. Where can I read more about this? I see it being mentioned a few times in the org-mode mailing list, and in the matrix room. Is the plan to completely remove the org-mks interface and replace it with transient without having an option to use the former? I find org-mks perfectly fine to use and would be sad if it was replaced with transient. Transient needs time to get used to, and the default settings is quite un-Emacsy; I'm not too excited about configuring yet another package that has a hard-to-understand manual TBH. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-08 6:06 ` Org mode and Emacs Visuwesh @ 2022-06-08 6:58 ` Tim Cross 0 siblings, 0 replies; 505+ messages in thread From: Tim Cross @ 2022-06-08 6:58 UTC (permalink / raw) To: Visuwesh; +Cc: Stefan Monnier, Alan Mackenzie, emacs-devel Visuwesh <visuweshm@gmail.com> writes: > [புதன் ஜூன் 08, 2022] Tim Cross wrote: > >> [...] >> - Increasing org's use of built-in Emacs capabilities rather than using >> org specific implementations. For example, adopting transient instead >> of an org implemented module which replicates similar functionality. > > Where can I read more about this? I see it being mentioned a few times > in the org-mode mailing list, and in the matrix room. Is the plan to > completely remove the org-mks interface and replace it with transient > without having an option to use the former? I find org-mks perfectly > fine to use and would be sad if it was replaced with transient. > Transient needs time to get used to, and the default settings is quite > un-Emacsy; I'm not too excited about configuring yet another package > that has a hard-to-understand manual TBH. The specifics of what is planned are still being worked out. Initially, the likely initial candidate for change will be to the export menu. Unless you have extensive low level customisation, it should not be a change which has significant impact on users. In fact, maintaining backwards compatibility and consistency for end users is important to the org developers. It is also quite possible that after an initial investigation, it may be decided transient is not a good option for org mode or perhaps it will be a good option once additonal functionality or enhancements are added. Right now, all that has been agreed is that it would be worthwhile looking at it to see if it can be of benefit in helping to reduce org maintenance overheads and/or increase org's consistency with other emacs packages. What will determine what remains and what choices are available will depend on what involvement people have in the development and what will be maintainable. There are a number of areas in org mode where functionality has been implemented that is 80+% equivalent to functionality which exists or has been added to core emacs. Having this duplication of effort is adding to the burden of maintenance, which is already significant. In general, org maintainers keep an eye on what is being added/expanded in Emacs core and when things are added, like transient mode, they are assessed to see if adopting that functionality would reduce the org maintenance load and improve org's consistency withi the rest of Emacs. At this stage, transient has been added as something to look at in the backlog. When this task makes if off the backlog, the typical process would be for it to be discussed on the org devel list, for initial implementations to be done either in its own branch (if considered a significantly large enough change) or on the development branch otherwise and people will be asked to try it out. So, if your interested in this area, the first thing would be to get on the org devel mail list. Anyone who is particularly keen to see such things added might also initiate discussions and development in their own branch, which could then be added in as a PR. However, like Emacs, any significant development work on org mode also requires FSF copyright assignment. At this specific point in time, the main focus with org has been on stability, performance improvements, especially for large org files with many babel blocks and sections and improvements to the org syntax and provision of an org parser, In particular, I think the work being done by Ihor on folding and the org parser will have huge benefits. In addition to making the code base easier to maintain and improving performance, it will help reduce org's current dependency on complicated and difficult to maintain regular expressions in the font-locking layer. This should improve performance and reduce errors or unexpected results that sometimes occur because of regexp errors etc and reduce time spent by maintainers in tracking down such errors and maintenance of those complex regexps. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-07 22:10 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Tim Cross 2022-06-08 6:06 ` Org mode and Emacs Visuwesh @ 2022-06-09 22:31 ` Richard Stallman 2022-06-09 23:10 ` Tim Cross ` (2 more replies) 1 sibling, 3 replies; 505+ messages in thread From: Richard Stallman @ 2022-06-09 22:31 UTC (permalink / raw) To: Tim Cross; +Cc: monnier, acm, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] If we are going to redesign Org mode syntax, can we please add room for extensibility so that an extension of Org mode syntax could have all the features of Texinfo? If the new syntax can express all the distinctions that Texinfo can already express, that would make it possible to switch to it for our documentation sources. This needs to include conversion into some variant of TeX syntax so that we can generate high-quality printed manuals through TeX. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-09 22:31 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Richard Stallman @ 2022-06-09 23:10 ` Tim Cross 2022-06-10 5:59 ` Eli Zaretskii 2022-06-12 0:42 ` Richard Stallman 2022-06-10 13:56 ` Ihor Radchenko 2022-06-10 19:32 ` Akib Azmain Turja 2 siblings, 2 replies; 505+ messages in thread From: Tim Cross @ 2022-06-09 23:10 UTC (permalink / raw) To: rms; +Cc: monnier, acm, emacs-devel Richard Stallman <rms@gnu.org> writes: > [[[ To any NSA and FBI agents reading my email: please consider ]]] > [[[ whether defending the US Constitution against all enemies, ]]] > [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > If we are going to redesign Org mode syntax, > can we please add room for extensibility > so that an extension of Org mode syntax > could have all the features of Texinfo? > > If the new syntax can express all the distinctions that Texinfo > can already express, that would make it possible to switch to it > for our documentation sources. > > This needs to include conversion into some variant of TeX syntax so > that we can generate high-quality printed manuals through TeX. Nobody has said that org mode syntax was getting redesigned. What I said was that we were working to clarify the existing syntax and remove ambiguities and provide a standard parser. Making org mode syntax equivalent to texinfo syntax seems like a mistake to me. The original idea was to have a light weight syntax which is easyh to learn, not create a clone of texinfo. Besides, I suspect it would be very difficult to maintain backwards compatibility. I"m also not convinced that org mode would be the right solution for Emacs documentation. That is a completely different conversation and I'm not convinced that org mode is a better answer wrt documentation than texinfo. While there are certainly some areas where texinfo is showing its age, it still works well for providing Emacs documentation within the editor. There may be some role for org mode to augment what is provided by tgexinfo, but again, that is a whole different conversation. Note also that the previous thread was about handling of existing org files within Emacs package ecosystem, not about replacing org documentation with org mode. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-09 23:10 ` Tim Cross @ 2022-06-10 5:59 ` Eli Zaretskii 2022-06-10 6:20 ` Ihor Radchenko 2022-06-12 0:42 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-10 5:59 UTC (permalink / raw) To: Tim Cross; +Cc: rms, monnier, acm, emacs-devel > From: Tim Cross <theophilusx@gmail.com> > Cc: monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Date: Fri, 10 Jun 2022 09:10:50 +1000 > > I"m also not convinced that org mode would be the right solution for > Emacs documentation. Then why Org's own manual is written in Org instead of Texinfo? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-10 5:59 ` Eli Zaretskii @ 2022-06-10 6:20 ` Ihor Radchenko 2022-06-10 6:44 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-10 6:20 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Tim Cross, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> I"m also not convinced that org mode would be the right solution for >> Emacs documentation. > > Then why Org's own manual is written in Org instead of Texinfo? I do not know if Org mode would be a good solution for Emacs documentation. However, compared to texinfo, our manual is lacking index entries in all the exported versions but texinfo. The indexes in pdf version of the manual are actually generated by org -> texinfo -> tex -> pdf. AFAIK. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-10 6:20 ` Ihor Radchenko @ 2022-06-10 6:44 ` Eli Zaretskii 2022-06-11 4:49 ` Tim Cross 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-10 6:44 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Tim Cross <theophilusx@gmail.com>, rms@gnu.org, > monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Date: Fri, 10 Jun 2022 14:20:37 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> I"m also not convinced that org mode would be the right solution for > >> Emacs documentation. > > > > Then why Org's own manual is written in Org instead of Texinfo? > > I do not know if Org mode would be a good solution for Emacs > documentation. However, compared to texinfo, our manual is lacking index > entries in all the exported versions but texinfo. Lack of an index makes for a less useful manual, IME. Anyway, is the lack of the index the _only_ missing feature from what the Org markup provides as compared to Texinfo? Tim's message sounded like there are many more omissions. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-10 6:44 ` Eli Zaretskii @ 2022-06-11 4:49 ` Tim Cross 2022-06-11 6:58 ` Eli Zaretskii 2022-06-12 0:42 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Tim Cross @ 2022-06-11 4:49 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Ihor Radchenko, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Ihor Radchenko <yantar92@gmail.com> >> Cc: Tim Cross <theophilusx@gmail.com>, rms@gnu.org, >> monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org >> Date: Fri, 10 Jun 2022 14:20:37 +0800 >> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> >> I"m also not convinced that org mode would be the right solution for >> >> Emacs documentation. >> > >> > Then why Org's own manual is written in Org instead of Texinfo? >> >> I do not know if Org mode would be a good solution for Emacs >> documentation. However, compared to texinfo, our manual is lacking index >> entries in all the exported versions but texinfo. > > Lack of an index makes for a less useful manual, IME. > > Anyway, is the lack of the index the _only_ missing feature from what > the Org markup provides as compared to Texinfo? Tim's message sounded > like there are many more omissions. My statement is simply waht it says. I am not convinced org mode would be the right solution for Emacs documentation. This is for many reasons, some relating to org, some relating to the fact I'm not convinced there is a need to replace texinfo, especially given that in some areas, I think texinfo is more feature rich and consistent and some relating to the feeling I have that the result is hard to justify given the amount of work it would require. At the end of the day, proposing org mode as a replacement for texinfo feels too much like a solution looking for a problem and one that doesn't appear to have clear enough advantages to justify the work needed to make such a transition, especially given work being done in the texinfo area to improve its support for new formats, like epub. Org mode is a fantastic tool and I use it every day. There are ways org mode could be used to augment the Emacs environment, but I don't think it is an all or nothing situation. I'm also not convinced that embedding org into Emacs more and more is necessarily a good thing. Tighter coupling comes with both pros and cons. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-11 4:49 ` Tim Cross @ 2022-06-11 6:58 ` Eli Zaretskii 2022-06-11 7:59 ` Tim Cross ` (2 more replies) 2022-06-12 0:42 ` Richard Stallman 1 sibling, 3 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-11 6:58 UTC (permalink / raw) To: Tim Cross; +Cc: yantar92, rms, monnier, acm, emacs-devel > From: Tim Cross <theophilusx@gmail.com> > Cc: Ihor Radchenko <yantar92@gmail.com>, rms@gnu.org, > monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Date: Sat, 11 Jun 2022 14:49:21 +1000 > > >> >> I"m also not convinced that org mode would be the right solution for > >> >> Emacs documentation. > >> > > >> > Then why Org's own manual is written in Org instead of Texinfo? > >> > >> I do not know if Org mode would be a good solution for Emacs > >> documentation. However, compared to texinfo, our manual is lacking index > >> entries in all the exported versions but texinfo. > > > > Lack of an index makes for a less useful manual, IME. > > > > Anyway, is the lack of the index the _only_ missing feature from what > > the Org markup provides as compared to Texinfo? Tim's message sounded > > like there are many more omissions. > > My statement is simply waht it says. I am not convinced org mode would > be the right solution for Emacs documentation. This is for many reasons, > some relating to org, some relating to the fact I'm not convinced there > is a need to replace texinfo, especially given that in some areas, I > think texinfo is more feature rich and consistent and some relating to the feeling I have > that the result is hard to justify given the amount of work it would > require. Well, it is hard to say anything intelligent without knowing those "many reasons". And if your POV is shared by the Org developers, then I ask again: why is the Org manual, which weighs in at 23,000 lines, is being written in Org? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-11 6:58 ` Eli Zaretskii @ 2022-06-11 7:59 ` Tim Cross 2022-06-11 8:29 ` Eli Zaretskii 2022-06-12 9:05 ` Ihor Radchenko 2022-06-12 22:38 ` Richard Stallman 2 siblings, 1 reply; 505+ messages in thread From: Tim Cross @ 2022-06-11 7:59 UTC (permalink / raw) To: Eli Zaretskii; +Cc: yantar92, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Tim Cross <theophilusx@gmail.com> >> Cc: Ihor Radchenko <yantar92@gmail.com>, rms@gnu.org, >> monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org >> Date: Sat, 11 Jun 2022 14:49:21 +1000 >> >> >> >> I"m also not convinced that org mode would be the right solution for >> >> >> Emacs documentation. >> >> > >> >> > Then why Org's own manual is written in Org instead of Texinfo? >> >> >> >> I do not know if Org mode would be a good solution for Emacs >> >> documentation. However, compared to texinfo, our manual is lacking index >> >> entries in all the exported versions but texinfo. >> > >> > Lack of an index makes for a less useful manual, IME. >> > >> > Anyway, is the lack of the index the _only_ missing feature from what >> > the Org markup provides as compared to Texinfo? Tim's message sounded >> > like there are many more omissions. >> >> My statement is simply waht it says. I am not convinced org mode would >> be the right solution for Emacs documentation. This is for many reasons, >> some relating to org, some relating to the fact I'm not convinced there >> is a need to replace texinfo, especially given that in some areas, I >> think texinfo is more feature rich and consistent and some relating to the feeling I have >> that the result is hard to justify given the amount of work it would >> require. > > Well, it is hard to say anything intelligent without knowing those > "many reasons". > > And if your POV is shared by the Org developers, then I ask again: why > is the Org manual, which weighs in at 23,000 lines, is being written > in Org? I have never claimed my POV was that of the org developers, I've never claimed to be an org developer or to speak in any way for org developers. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-11 7:59 ` Tim Cross @ 2022-06-11 8:29 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-11 8:29 UTC (permalink / raw) To: Tim Cross; +Cc: yantar92, rms, monnier, acm, emacs-devel > From: Tim Cross <theophilusx@gmail.com> > Cc: yantar92@gmail.com, rms@gnu.org, monnier@iro.umontreal.ca, acm@muc.de, > emacs-devel@gnu.org > Date: Sat, 11 Jun 2022 17:59:46 +1000 > > > Eli Zaretskii <eliz@gnu.org> writes: > > >> My statement is simply waht it says. I am not convinced org mode would > >> be the right solution for Emacs documentation. This is for many reasons, > >> some relating to org, some relating to the fact I'm not convinced there > >> is a need to replace texinfo, especially given that in some areas, I > >> think texinfo is more feature rich and consistent and some relating to the feeling I have > >> that the result is hard to justify given the amount of work it would > >> require. > > > > Well, it is hard to say anything intelligent without knowing those > > "many reasons". > > > > And if your POV is shared by the Org developers, then I ask again: why > > is the Org manual, which weighs in at 23,000 lines, is being written > > in Org? > > I have never claimed my POV was that of the org developers, I've never > claimed to be an org developer or to speak in any way for org > developers. I didn't say you have. The above question was aimed at the Org developers, as should be obvious from the context; apologies if that wasn't clear enough. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-11 6:58 ` Eli Zaretskii 2022-06-11 7:59 ` Tim Cross @ 2022-06-12 9:05 ` Ihor Radchenko 2022-06-12 9:18 ` Eli Zaretskii 2022-06-13 22:37 ` Richard Stallman 2022-06-12 22:38 ` Richard Stallman 2 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-12 9:05 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Tim Cross, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > And if your POV is shared by the Org developers, then I ask again: why > is the Org manual, which weighs in at 23,000 lines, is being written > in Org? The original reason is simply https://list.orgmode.org/AANLkTin=gUscE4znd4pYY85X9d2wLkC+JADVcDN8AeoJ@mail.gmail.com/ >>> I was just wondering... Is the manual written in texinfo markup, or is there >>> some obscure .org file behind the manual still? >>> >>> If it really is written in texinfo, is this not a shortcoming? Org mode is >>> capable of generating html and pdf etc. Why not use it for the manual then >>> to set the example and show its powers!? and then https://list.orgmode.org/m2fu8942pr.fsf@tsdye.com/ >>> It's a real pleasure to know that Org can now eat its own dog food (and >>> to see Carsten's memorable phrase on the Org mode list again). Many >>> thanks for picking up this orphaned project. I look forward to the day >>> it serves as the source for org.texi. So, now we have our manual written in Org mode and we never had reasons to come back to texi. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 9:05 ` Ihor Radchenko @ 2022-06-12 9:18 ` Eli Zaretskii 2022-06-12 10:04 ` Ihor Radchenko 2022-06-13 22:37 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-12 9:18 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Tim Cross <theophilusx@gmail.com>, rms@gnu.org, > monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Date: Sun, 12 Jun 2022 17:05:02 +0800 > > https://list.orgmode.org/m2fu8942pr.fsf@tsdye.com/ > >>> It's a real pleasure to know that Org can now eat its own dog food (and > >>> to see Carsten's memorable phrase on the Org mode list again). Many > >>> thanks for picking up this orphaned project. I look forward to the day > >>> it serves as the source for org.texi. > > So, now we have our manual written in Org mode and we never had reasons > to come back to texi. And I get to wait for 2 minutes for the build to finish each time org.org is touched. Right. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 9:18 ` Eli Zaretskii @ 2022-06-12 10:04 ` Ihor Radchenko 2022-06-12 10:15 ` Eli Zaretskii 2022-06-12 19:25 ` Jean Louis 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-12 10:04 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> So, now we have our manual written in Org mode and we never had reasons >> to come back to texi. > > And I get to wait for 2 minutes for the build to finish each time > org.org is touched. Right. This is not very polite. The change did not aim to make your life harder. You are free to open discussion in Org ML about turning back to texi format, but it will, for example, make things harder for me and other people who are not familiar with texinfo format. I am not sure if build time justifies extra maintenance burden. I just pushed several improvements to ox.el. They reduce manual generation time 2x on my system (using main branch). Feel free to try it on your side. AFAIU, the effect should be more noticeable on slower systems. It may also help if you try to profile org-make-manuals from mk/org-fixup.el and share the results. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 10:04 ` Ihor Radchenko @ 2022-06-12 10:15 ` Eli Zaretskii 2022-06-12 10:38 ` Ihor Radchenko 2022-06-12 19:25 ` Jean Louis 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-12 10:15 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, rms@gnu.org, monnier@iro.umontreal.ca, > acm@muc.de, emacs-devel@gnu.org > Date: Sun, 12 Jun 2022 18:04:56 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> So, now we have our manual written in Org mode and we never had reasons > >> to come back to texi. > > > > And I get to wait for 2 minutes for the build to finish each time > > org.org is touched. Right. > > This is not very polite. The change did not aim to make your life > harder. It is an expression of frustration I feel each time the build needs to rebuild the Org manual. I wrote it to contrast that frustration with what I perceived as satisfaction with this solution expressed in your description of the history of this. I really wonder how come no one on the Org list paid any attention to the 10-fold to 40-fold slowdown in the time it takes to build the manual, as result of that change. But that's water under the bridge. > You are free to open discussion in Org ML about turning back to texi > format, but it will, for example, make things harder for me and other > people who are not familiar with texinfo format. I am not sure if build > time justifies extra maintenance burden. I don't see any chance I will convince Org developers to go back to Texinfo at this point. > I just pushed several improvements to ox.el. They reduce manual > generation time 2x on my system (using main branch). Feel free to try it > on your side. AFAIU, the effect should be more noticeable on slower > systems. Thanks, I hope to see this soon in the Emacs repository. > It may also help if you try to profile org-make-manuals from > mk/org-fixup.el and share the results. If profiling can help, wouldn't it be simpler to invoke the same commands from an interactive Emacs session, then show the profile? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 10:15 ` Eli Zaretskii @ 2022-06-12 10:38 ` Ihor Radchenko 2022-06-12 14:36 ` Eli Zaretskii 2022-06-12 14:58 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-12 10:38 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > It is an expression of frustration I feel each time the build needs to > rebuild the Org manual. I wrote it to contrast that frustration with > what I perceived as satisfaction with this solution expressed in your > description of the history of this. That's not my satisfaction. I quoted the emails. In any case, while I understand your frustration, you managed to spread it to me at least. It did not make parsing this thread any easier. > I really wonder how come no one on the Org list paid any attention to > the 10-fold to 40-fold slowdown in the time it takes to build the > manual, as result of that change. But that's water under the bridge. We rarely have bugs related to manual builds. I recall two in many years. Usually documentation is built automatically on ELPA and by our publishing scripts on orgmode.org. >> I just pushed several improvements to ox.el. They reduce manual >> generation time 2x on my system (using main branch). Feel free to try it >> on your side. AFAIU, the effect should be more noticeable on slower >> systems. > > Thanks, I hope to see this soon in the Emacs repository. Not soon. Unless you want major changes for Emacs 28.2. We restricted stable Org branch to critical-only bugfixes until Emacs 28.2 is out. >> It may also help if you try to profile org-make-manuals from >> mk/org-fixup.el and share the results. > > If profiling can help, wouldn't it be simpler to invoke the same > commands from an interactive Emacs session, then show the profile? This is exactly what I meant. To run org-make-manuals from interactive Emacs session. org-make-manuals takes about 20 seconds on my system (for combined Org export and texinfo invocation). Your system is clearly different and might have different bottlenecks. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 10:38 ` Ihor Radchenko @ 2022-06-12 14:36 ` Eli Zaretskii 2022-06-12 15:31 ` Org mode and Emacs Colin Baxter ` (2 more replies) 2022-06-12 14:58 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 1 sibling, 3 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-12 14:36 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, rms@gnu.org, monnier@iro.umontreal.ca, > acm@muc.de, emacs-devel@gnu.org > Date: Sun, 12 Jun 2022 18:38:45 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > > I really wonder how come no one on the Org list paid any attention to > > the 10-fold to 40-fold slowdown in the time it takes to build the > > manual, as result of that change. But that's water under the bridge. > > We rarely have bugs related to manual builds. I recall two in many > years. > > Usually documentation is built automatically on ELPA and by our > publishing scripts on orgmode.org. So basically no one builds Org, including the manual, on their own system? Even not the Org developers? > >> I just pushed several improvements to ox.el. They reduce manual > >> generation time 2x on my system (using main branch). Feel free to try it > >> on your side. AFAIU, the effect should be more noticeable on slower > >> systems. > > > > Thanks, I hope to see this soon in the Emacs repository. > > Not soon. Unless you want major changes for Emacs 28.2. We restricted > stable Org branch to critical-only bugfixes until Emacs 28.2 is out. This is not needed for the emacs-28 branch, so I meant master. > >> It may also help if you try to profile org-make-manuals from > >> mk/org-fixup.el and share the results. > > > > If profiling can help, wouldn't it be simpler to invoke the same > > commands from an interactive Emacs session, then show the profile? > > This is exactly what I meant. To run org-make-manuals from interactive > Emacs session. Then why would I need org-fixup.el? > org-make-manuals takes about 20 seconds on my system (for combined Org > export and texinfo invocation). Your system is clearly different and > might have different bottlenecks. Your Emacs is probably built with optimizations, whereas mine isn't. The optimized version build org.info in about 30 sec, as I said, which is not very different from your timing. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 14:36 ` Eli Zaretskii @ 2022-06-12 15:31 ` Colin Baxter 2022-06-15 5:19 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Ihor Radchenko 2022-09-25 2:14 ` Bastien 2 siblings, 0 replies; 505+ messages in thread From: Colin Baxter @ 2022-06-12 15:31 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Ihor Radchenko, theophilusx, rms, monnier, acm, emacs-devel >>>>> Eli Zaretskii <eliz@gnu.org> writes: >> From: Ihor Radchenko <yantar92@gmail.com> Cc: >> theophilusx@gmail.com, rms@gnu.org, monnier@iro.umontreal.ca, >> acm@muc.de, emacs-devel@gnu.org Date: Sun, 12 Jun 2022 18:38:45 >> +0800 >> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> > I really wonder how come no one on the Org list paid any >> attention to > the 10-fold to 40-fold slowdown in the time it >> takes to build the > manual, as result of that change. But >> that's water under the bridge. >> >> We rarely have bugs related to manual builds. I recall two in >> many years. >> >> Usually documentation is built automatically on ELPA and by our >> publishing scripts on orgmode.org. > So basically no one builds Org, including the manual, on their own > system? Even not the Org developers? I build org-mode from git and have noticed the slowness in building the documentation. Weekly building from git is my way of avoiding having to deal immediately with a myriad of changes in past Versions of org-mode. Best wishes. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 14:36 ` Eli Zaretskii 2022-06-12 15:31 ` Org mode and Emacs Colin Baxter @ 2022-06-15 5:19 ` Ihor Radchenko 2022-06-15 6:46 ` Org mode and Emacs David Engster ` (2 more replies) 2022-09-25 2:14 ` Bastien 2 siblings, 3 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-15 5:19 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> We rarely have bugs related to manual builds. I recall two in many >> years. >> >> Usually documentation is built automatically on ELPA and by our >> publishing scripts on orgmode.org. > > So basically no one builds Org, including the manual, on their own > system? Even not the Org developers? Org releases are not frequent for most users and occasional build time should not matter too much. Nobody complained on the list. Org developers (at least, me) usually run make test. It does not touch documentation. Full re-build is rarely needed because the documentation does not change often. >> > Thanks, I hope to see this soon in the Emacs repository. >> >> Not soon. Unless you want major changes for Emacs 28.2. We restricted >> stable Org branch to critical-only bugfixes until Emacs 28.2 is out. > > This is not needed for the emacs-28 branch, so I meant master. I am not too familiar with how Emacs handles Org updates. AFAIK, Emacs master only tracks our stable tagged releases. Not the working branches (both bugfix and main). I do not know the motivation behind it and I do not know if it is a good idea to change it. Do you think that Emacs master should instead track our Org dev branch? >> >> It may also help if you try to profile org-make-manuals from >> >> mk/org-fixup.el and share the results. >> > >> > If profiling can help, wouldn't it be simpler to invoke the same >> > commands from an interactive Emacs session, then show the profile? >> >> This is exactly what I meant. To run org-make-manuals from interactive >> Emacs session. > > Then why would I need org-fixup.el? I assume that you figured that org-fixup.el contains functions to build manual. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-15 5:19 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Ihor Radchenko @ 2022-06-15 6:46 ` David Engster 2022-06-15 7:36 ` Ihor Radchenko 2022-06-15 12:50 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 2022-06-16 3:19 ` Org mode and Emacs Pankaj Jangid 2 siblings, 1 reply; 505+ messages in thread From: David Engster @ 2022-06-15 6:46 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Eli Zaretskii, theophilusx, rms, monnier, acm, emacs-devel > Eli Zaretskii <eliz@gnu.org> writes: > >>> We rarely have bugs related to manual builds. I recall two in many >>> years. >>> >>> Usually documentation is built automatically on ELPA and by our >>> publishing scripts on orgmode.org. >> >> So basically no one builds Org, including the manual, on their own >> system? Even not the Org developers? > > Org releases are not frequent for most users and occasional build time > should not matter too much. Nobody complained on the list. Well, for the record, I did complain, but that was over 11 years ago. :-) https://lists.gnu.org/archive/html/emacs-devel/2014-12/msg01356.html I still think Org is way too slow to be a viable documentation system for a project like Emacs. -David ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-15 6:46 ` Org mode and Emacs David Engster @ 2022-06-15 7:36 ` Ihor Radchenko 2022-06-15 13:01 ` Eli Zaretskii 2022-06-15 13:34 ` David Engster 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-15 7:36 UTC (permalink / raw) To: David Engster; +Cc: Eli Zaretskii, theophilusx, rms, monnier, acm, emacs-devel David Engster <deng@randomsample.de> writes: >>> So basically no one builds Org, including the manual, on their own >>> system? Even not the Org developers? >> >> Org releases are not frequent for most users and occasional build time >> should not matter too much. Nobody complained on the list. > > Well, for the record, I did complain, but that was over 11 years ago. :-) > > https://lists.gnu.org/archive/html/emacs-devel/2014-12/msg01356.html Point taken. > I still think Org is way too slow to be a viable documentation system > for a project like Emacs. Things have changed since 2014. On my system, using Emacs 28 and latest Org, exporting org-manual + org-guide takes 18.8 sec. This time includes genering the .texi files and running texinfo. From that 18.8 sec, texinfo takes 3.4 sec to run. org export takes: 15.4 sec, which is 4.5 x texinfo time. Considering that Org export is more sofisticated compared to texinfo and that Org export is written in Elisp, I see this performance as acceptable for a documentation system. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-15 7:36 ` Ihor Radchenko @ 2022-06-15 13:01 ` Eli Zaretskii 2022-06-16 5:36 ` Ihor Radchenko 2022-06-15 13:34 ` David Engster 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-15 13:01 UTC (permalink / raw) To: Ihor Radchenko; +Cc: deng, theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Eli Zaretskii <eliz@gnu.org>, theophilusx@gmail.com, rms@gnu.org, > monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Date: Wed, 15 Jun 2022 15:36:16 +0800 > > On my system, using Emacs 28 and latest Org, exporting org-manual + > org-guide takes 18.8 sec. This time includes genering the .texi files > and running texinfo. From that 18.8 sec, texinfo takes 3.4 sec to run. > > org export takes: 15.4 sec, which is 4.5 x texinfo time. > Considering that Org export is more sofisticated compared to texinfo and > that Org export is written in Elisp, I see this performance as > acceptable for a documentation system. I don't know about "acceptable", sorry. It is definitely "endurable", but 18.8 seconds is annoyingly long for an optimized build of Emacs to do anything this simple. Can you find any other single command that's part of building Emacs which takes a comparable amount of time in an optimized build? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-15 13:01 ` Eli Zaretskii @ 2022-06-16 5:36 ` Ihor Radchenko 2022-06-16 5:58 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-16 5:36 UTC (permalink / raw) To: Eli Zaretskii; +Cc: deng, theophilusx, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> On my system, using Emacs 28 and latest Org, exporting org-manual + >> org-guide takes 18.8 sec. This time includes genering the .texi files >> and running texinfo. From that 18.8 sec, texinfo takes 3.4 sec to run. >> >> org export takes: 15.4 sec, which is 4.5 x texinfo time. >> Considering that Org export is more sofisticated compared to texinfo and >> that Org export is written in Elisp, I see this performance as >> acceptable for a documentation system. > > I don't know about "acceptable", sorry. It is definitely "endurable", > but 18.8 seconds is annoyingly long for an optimized build of Emacs to > do anything this simple. Ok. I pushed things a bit further. The latest Org main takes ~8sec to generate Org manual. This _includes_ makeinfo call. Now, Org export takes the same time with texinfo - ~4sec on my system. Further improvements are not trivial, given my familiarity with Org babel and Org export code. > Can you find any other single command that's part of building Emacs > which takes a comparable amount of time in an optimized build? Optimized Emacs build takes >30 minutes on my system. Mainly because of native compilation. 40, 20, or 10 seconds spent generating Org documentation are negligible for me. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-16 5:36 ` Ihor Radchenko @ 2022-06-16 5:58 ` Eli Zaretskii 2022-06-16 9:55 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-16 5:58 UTC (permalink / raw) To: Ihor Radchenko; +Cc: deng, theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: deng@randomsample.de, theophilusx@gmail.com, rms@gnu.org, > monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Date: Thu, 16 Jun 2022 13:36:35 +0800 > > > I don't know about "acceptable", sorry. It is definitely "endurable", > > but 18.8 seconds is annoyingly long for an optimized build of Emacs to > > do anything this simple. > > Ok. I pushed things a bit further. The latest Org main takes ~8sec to > generate Org manual. This _includes_ makeinfo call. Now, Org export > takes the same time with texinfo - ~4sec on my system. Thanks, this is awesome. Hope to see this speedup in the Emacs repository soon. > > Can you find any other single command that's part of building Emacs > > which takes a comparable amount of time in an optimized build? > > Optimized Emacs build takes >30 minutes on my system. Mainly because of > native compilation. The slowness of native-compilation is a long-standing bug report, so I'd rather not use that as a reference for acceptable speed. I meant to compare with other commands run by the build. > 40, 20, or 10 seconds spent generating Org documentation are > negligible for me. They aren't negligible, even in a native-compilation build, because they happen at the end, when all the rest was already done, and no large N in "-j N" of the Make command can help with that last long wait. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-16 5:58 ` Eli Zaretskii @ 2022-06-16 9:55 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-16 9:55 UTC (permalink / raw) To: Eli Zaretskii; +Cc: deng, theophilusx, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> Optimized Emacs build takes >30 minutes on my system. Mainly because of >> native compilation. > > The slowness of native-compilation is a long-standing bug report, so > I'd rather not use that as a reference for acceptable speed. I meant > to compare with other commands run by the build. The other relatively slow part of the built process without native-comp is Scraping autoloads. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-15 7:36 ` Ihor Radchenko 2022-06-15 13:01 ` Eli Zaretskii @ 2022-06-15 13:34 ` David Engster 2022-06-16 6:50 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: David Engster @ 2022-06-15 13:34 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Eli Zaretskii, theophilusx, rms, monnier, acm, emacs-devel > On my system, using Emacs 28 and latest Org, exporting org-manual + > org-guide takes 18.8 sec. This time includes genering the .texi files > and running texinfo. From that 18.8 sec, texinfo takes 3.4 sec to run. On my system, generating org.texi from org.org takes 21s, makeinfo org.texi takes 2.7s. But it's nice to see that it has become much faster compared to 2014. > org export takes: 15.4 sec, which is 4.5 x texinfo time. > Considering that Org export is more sofisticated compared to texinfo and > that Org export is written in Elisp, I see this performance as > acceptable for a documentation system. Why does the implementation language matter whether a documentation system is acceptable? And while sophisticated, Org is actually still missing features (like a proper index) to be a suitable replacement. Org is about 1/16th of the whole Emacs documentation, so we're looking at over 5min if everything was written in Org, give or take. People were already up in arms when the switch from Texinfo v4 to v5 was done (which switched to a Perl implementation). When documentation generation takes a long time, writing it becomes more painful, as you cannot quickly check the resulting output. And I think you underestimate how important a quick build process is. Apart from developer annoyance, you need less resources for CI, for instance. -David ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-15 13:34 ` David Engster @ 2022-06-16 6:50 ` Ihor Radchenko 2022-06-16 10:21 ` David Engster 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-16 6:50 UTC (permalink / raw) To: David Engster; +Cc: Eli Zaretskii, theophilusx, rms, monnier, acm, emacs-devel David Engster <deng@randomsample.de> writes: >> On my system, using Emacs 28 and latest Org, exporting org-manual + >> org-guide takes 18.8 sec. This time includes genering the .texi files >> and running texinfo. From that 18.8 sec, texinfo takes 3.4 sec to run. > > On my system, generating org.texi from org.org takes 21s, makeinfo > org.texi takes 2.7s. But it's nice to see that it has become much faster > compared to 2014. Can you test the latest Org main? >> org export takes: 15.4 sec, which is 4.5 x texinfo time. >> Considering that Org export is more sofisticated compared to texinfo and >> that Org export is written in Elisp, I see this performance as >> acceptable for a documentation system. > > Why does the implementation language matter whether a documentation > system is acceptable? And while sophisticated, Org is actually still > missing features (like a proper index) to be a suitable replacement. Well. It was a rather hand-waving argument: Elisp is not as fast of many other programming languages. So software written in Elisp may be slow despite the best effort. etc etc In any case, that argument is futile now. I managed to get Org export work as fast as texinfo. As for proper index, we do support index for texinfo export specifically and for publishing websites. A more general-purpose index support is in the works. See https://github.com/tecosaur/org-glossary > Org is about 1/16th of the whole Emacs documentation, so we're looking > at over 5min if everything was written in Org, give or take. People were > already up in arms when the switch from Texinfo v4 to v5 was done (which > switched to a Perl implementation). When documentation generation takes > a long time, writing it becomes more painful, as you cannot quickly > check the resulting output. And I think you underestimate how important > a quick build process is. Apart from developer annoyance, you need less > resources for CI, for instance. Fair point. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-16 6:50 ` Ihor Radchenko @ 2022-06-16 10:21 ` David Engster 0 siblings, 0 replies; 505+ messages in thread From: David Engster @ 2022-06-16 10:21 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Eli Zaretskii, theophilusx, rms, monnier, acm, emacs-devel > David Engster <deng@randomsample.de> writes: > >>> On my system, using Emacs 28 and latest Org, exporting org-manual + >>> org-guide takes 18.8 sec. This time includes genering the .texi files >>> and running texinfo. From that 18.8 sec, texinfo takes 3.4 sec to run. >> >> On my system, generating org.texi from org.org takes 21s, makeinfo >> org.texi takes 2.7s. But it's nice to see that it has become much faster >> compared to 2014. > > Can you test the latest Org main? I tested with latest Emacs master and with increased gc-cons-threshold it takes ~5 seconds to generate org.texi. Great job! -David ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-15 5:19 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Ihor Radchenko 2022-06-15 6:46 ` Org mode and Emacs David Engster @ 2022-06-15 12:50 ` Eli Zaretskii 2022-06-16 7:03 ` Ihor Radchenko 2022-06-16 3:19 ` Org mode and Emacs Pankaj Jangid 2 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-15 12:50 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, rms@gnu.org, monnier@iro.umontreal.ca, > acm@muc.de, emacs-devel@gnu.org > Date: Wed, 15 Jun 2022 13:19:10 +0800 > > >> > Thanks, I hope to see this soon in the Emacs repository. > >> > >> Not soon. Unless you want major changes for Emacs 28.2. We restricted > >> stable Org branch to critical-only bugfixes until Emacs 28.2 is out. > > > > This is not needed for the emacs-28 branch, so I meant master. > > I am not too familiar with how Emacs handles Org updates. AFAIK, Emacs > master only tracks our stable tagged releases. Not the working branches > (both bugfix and main). I do not know the motivation behind it and I do > not know if it is a good idea to change it. Do you think that Emacs > master should instead track our Org dev branch? I see no need for such significant changes in our merging practices. Rather, I hoped that the speedup you achieved will be important enough to make an exception and install just it in the Emacs master branch. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-15 12:50 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii @ 2022-06-16 7:03 ` Ihor Radchenko 2022-06-16 8:13 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-16 7:03 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, rms, monnier, acm, emacs-devel [-- Attachment #1: Type: text/plain, Size: 293 bytes --] Eli Zaretskii <eliz@gnu.org> writes: > I see no need for such significant changes in our merging practices. > Rather, I hoped that the speedup you achieved will be important enough > to make an exception and install just it in the Emacs master branch. See the attached patches. Best, Ihor [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-org-export-get-footnote-definition-Pre-cache-referen.patch --] [-- Type: text/x-patch, Size: 3096 bytes --] From 3342137359f122ed7168dc75096c6a5d3839a0c2 Mon Sep 17 00:00:00 2001 Message-Id: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Sun, 12 Jun 2022 13:05:16 +0800 Subject: [PATCH 1/8] org-export-get-footnote-definition: Pre-cache references in parse tree * lisp/ox.el (org-export-get-footnote-definition): Pre-process parse tree once to filter out all non-footnote elements. This speeds up subsequent footnote definition searches. --- lisp/ox.el | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/lisp/ox.el b/lisp/ox.el index 2a3edaa50..7f90dc36f 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -3748,28 +3748,33 @@ (defun org-export-get-footnote-definition (footnote-reference info) (if (not label) (org-element-contents footnote-reference) (let ((cache (or (plist-get info :footnote-definition-cache) (let ((hash (make-hash-table :test #'equal))) + ;; Cache all the footnotes in document for + ;; later search. + (org-element-map (plist-get info :parse-tree) + '(footnote-definition footnote-reference) + (lambda (f) + ;; Skip any standard footnote reference + ;; since those cannot contain a + ;; definition. + (unless (eq (org-element-property :type f) 'standard) + (puthash + (cons :element (org-element-property :label f)) + f + hash))) + info) (plist-put info :footnote-definition-cache hash) hash)))) (or (gethash label cache) (puthash label - (org-element-map (plist-get info :parse-tree) - '(footnote-definition footnote-reference) - (lambda (f) - (cond - ;; Skip any footnote with a different label. - ;; Also skip any standard footnote reference - ;; with the same label since those cannot - ;; contain a definition. - ((not (equal (org-element-property :label f) label)) nil) - ((eq (org-element-property :type f) 'standard) nil) - ((org-element-contents f)) - ;; Even if the contents are empty, we can not - ;; return nil since that would eventually raise - ;; the error. Instead, return the equivalent - ;; empty string. - (t ""))) - info t) + (let ((hashed (gethash (cons :element label) cache))) + (when hashed + (or (org-element-contents hashed) + ;; Even if the contents are empty, we can not + ;; return nil since that would eventually raise + ;; the error. Instead, return the equivalent + ;; empty string. + ""))) cache) (error "Definition not found for footnote %s" label)))))) -- 2.35.1 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-org-export-resolve-fuzyy-link-Pre-cache-all-possible.patch --] [-- Type: text/x-patch, Size: 2598 bytes --] From cff9ed2f7660abaa5bd67e5de87416cb393b2cb8 Mon Sep 17 00:00:00 2001 Message-Id: <cff9ed2f7660abaa5bd67e5de87416cb393b2cb8.1655362876.git.yantar92@gmail.com> In-Reply-To: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> References: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Sun, 12 Jun 2022 13:06:47 +0800 Subject: [PATCH 2/8] org-export-resolve-fuzyy-link: Pre-cache all possible search cells * lisp/ox.el (org-export-resolve-fuzzy-link): Before matching LINK, pre-process and cache all the non-nil search cells in the parse tree. When matching, use the pre-processed info. Fix the :test function for the cache hash table. --- lisp/ox.el | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/lisp/ox.el b/lisp/ox.el index 7f90dc36f..4a9387519 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -4346,17 +4346,27 @@ (defun org-export-resolve-fuzzy-link (link info &rest pseudo-types) (let* ((search-cells (org-export-string-to-search-cell (org-element-property :path link))) (link-cache (or (plist-get info :resolve-fuzzy-link-cache) - (let ((table (make-hash-table :test #'eq))) + (let ((table (make-hash-table :test #'equal))) + ;; Cache all the element search cells. + (org-element-map (plist-get info :parse-tree) + (append pseudo-types '(target) org-element-all-elements) + (lambda (datum) + (dolist (cell (org-export-search-cells datum)) + (if (gethash cell table) + (push datum (gethash cell table)) + (puthash cell (list datum) table))))) (plist-put info :resolve-fuzzy-link-cache table) table))) (cached (gethash search-cells link-cache 'not-found))) (if (not (eq cached 'not-found)) cached (let ((matches - (org-element-map (plist-get info :parse-tree) - (append pseudo-types '(target) org-element-all-elements) - (lambda (datum) - (and (org-export-match-search-cell-p datum search-cells) - datum))))) + (let (result) + (dolist (search-cell search-cells) + (setq result + (nconc + result + (gethash search-cell link-cache)))) + (delq nil result)))) (unless matches (signal 'org-link-broken (list (org-element-property :path link)))) (puthash -- 2.35.1 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: 0003-org-export-resolve-id-link-Pre-cache-all-the-ids-in-.patch --] [-- Type: text/x-patch, Size: 2832 bytes --] From eca7a1d4dea1e8980aee6a02967d4c596114f759 Mon Sep 17 00:00:00 2001 Message-Id: <eca7a1d4dea1e8980aee6a02967d4c596114f759.1655362876.git.yantar92@gmail.com> In-Reply-To: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> References: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Sun, 12 Jun 2022 13:32:35 +0800 Subject: [PATCH 3/8] org-export-resolve-id-link: Pre-cache all the ids in the parse tree * lisp/ox.el (org-export-resolve-id-link): Pre-cache all the ids in the parse tree for faster lookup. --- lisp/ox.el | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/lisp/ox.el b/lisp/ox.el index 4a9387519..b431d7119 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -4393,15 +4393,27 @@ (defun org-export-resolve-id-link (link info) \"custom-id\". Throw an error if no match is found." (let ((id (org-element-property :path link))) ;; First check if id is within the current parse tree. - (or (org-element-map (plist-get info :parse-tree) 'headline - (lambda (headline) - (when (or (equal (org-element-property :ID headline) id) - (equal (org-element-property :CUSTOM_ID headline) id)) - headline)) - info 'first-match) - ;; Otherwise, look for external files. - (cdr (assoc id (plist-get info :id-alist))) - (signal 'org-link-broken (list id))))) + (or (let ((local-ids (or (plist-get info :id-local-cache) + (let ((table (make-hash-table :test #'equal))) + (org-element-map + (plist-get info :parse-tree) + 'headline + (lambda (headline) + (let ((id (org-element-property :ID headline)) + (custom-id (org-element-property :CUSTOM_ID headline))) + (when id + (unless (gethash id table) + (puthash id headline table))) + (when custom-id + (unless (gethash custom-id table) + (puthash custom-id headline table))))) + info) + (plist-put info :id-local-cache table) + table)))) + (gethash id local-ids)) + ;; Otherwise, look for external files. + (cdr (assoc id (plist-get info :id-alist))) + (signal 'org-link-broken (list id))))) (defun org-export-resolve-radio-link (link info) "Return radio-target object referenced as LINK destination. -- 2.35.1 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #5: 0004-org-export-as-Do-not-update-buffer-settings-when-not.patch --] [-- Type: text/x-patch, Size: 3174 bytes --] From fb8b921066ca84aaa7034c7f13e2cc62da40d7ad Mon Sep 17 00:00:00 2001 Message-Id: <fb8b921066ca84aaa7034c7f13e2cc62da40d7ad.1655362876.git.yantar92@gmail.com> In-Reply-To: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> References: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Thu, 16 Jun 2022 01:03:18 +0800 Subject: [PATCH 4/8] org-export-as: Do not update buffer settings when not modified * lisp/ox.el (org-export-as): Use `buffer-chars-modified-tick' and avoid extra invocations of `org-set-regexps-and-options' and `org-update-radio-target-regexp' when the buffer is not changed. Also, disable folding checks. Folding is irrelevant inside export buffer. --- lisp/ox.el | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/lisp/ox.el b/lisp/ox.el index b431d7119..a4512270c 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -2956,11 +2956,12 @@ (defun org-export-as (mapcar (lambda (o) (and (eq (nth 4 o) 'parse) (nth 1 o))) (append (org-export-get-all-options backend) org-export-options-alist)))) - tree) + tree modified-tick) ;; Update communication channel and get parse tree. Buffer ;; isn't parsed directly. Instead, all buffer modifications ;; and consequent parsing are undertaken in a temporary copy. (org-export-with-buffer-copy + (font-lock-mode -1) ;; Run first hook with current back-end's name as argument. (run-hook-with-args 'org-export-before-processing-hook (org-export-backend-name backend)) @@ -2972,6 +2973,7 @@ (defun org-export-as ;; potentially invasive changes. (org-set-regexps-and-options) (org-update-radio-target-regexp) + (setq modified-tick (buffer-chars-modified-tick)) ;; Possibly execute Babel code. Re-run a macro expansion ;; specifically for {{{results}}} since inline source blocks ;; may have generated some more. Refresh buffer properties @@ -2979,8 +2981,10 @@ (defun org-export-as (when org-export-use-babel (org-babel-exp-process-buffer) (org-macro-replace-all '(("results" . "$1")) parsed-keywords) - (org-set-regexps-and-options) - (org-update-radio-target-regexp)) + (unless (eq modified-tick (buffer-chars-modified-tick)) + (org-set-regexps-and-options) + (org-update-radio-target-regexp)) + (setq modified-tick (buffer-chars-modified-tick))) ;; Run last hook with current back-end's name as argument. ;; Update buffer properties and radio targets one last time ;; before parsing. @@ -2988,8 +2992,10 @@ (defun org-export-as (save-excursion (run-hook-with-args 'org-export-before-parsing-hook (org-export-backend-name backend))) - (org-set-regexps-and-options) - (org-update-radio-target-regexp) + (unless (eq modified-tick (buffer-chars-modified-tick)) + (org-set-regexps-and-options) + (org-update-radio-target-regexp)) + (setq modified-tick (buffer-chars-modified-tick)) ;; Update communication channel with environment. (setq info (org-combine-plists -- 2.35.1 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #6: 0005-doc-Makefile-Disable-GC-during-export.patch --] [-- Type: text/x-patch, Size: 1396 bytes --] From 7f3ee582c1105c66a66fb61786c4e086e30460b4 Mon Sep 17 00:00:00 2001 Message-Id: <7f3ee582c1105c66a66fb61786c4e086e30460b4.1655362876.git.yantar92@gmail.com> In-Reply-To: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> References: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Thu, 16 Jun 2022 07:29:06 +0800 Subject: [PATCH 5/8] doc/Makefile: Disable GC during export * doc/Makefile (org.texi): (orgguide.texi): Set `gc-cons-threshold` to `most-positive=fixnum' and thus disable garbage collection while exporting manuals. This reduces the manual generation time. --- doc/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/Makefile b/doc/Makefile index cb6d72bdc..5911bd08a 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -31,12 +31,14 @@ org.texi: org-manual.org $(BATCH) \ --eval '(add-to-list `load-path "../lisp")' \ --eval '(load "../mk/org-fixup.el")' \ + --eval '(setq gc-cons-threshold most-positive-fixnum)' \ --eval '(org-make-manual)' orgguide.texi: org-guide.org $(BATCH) \ --eval '(add-to-list `load-path "../lisp")' \ --eval '(load "../mk/org-fixup.el")' \ + --eval '(setq gc-cons-threshold most-positive-fixnum)' \ --eval '(org-make-guide)' org-version.inc: org.texi -- 2.35.1 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #7: 0006-org-export-data-Concatenate-strings-in-temporary-buf.patch --] [-- Type: text/x-patch, Size: 3492 bytes --] From 4a466e28999bbaa5169b6fad414cf08c6d23cd7b Mon Sep 17 00:00:00 2001 Message-Id: <4a466e28999bbaa5169b6fad414cf08c6d23cd7b.1655362876.git.yantar92@gmail.com> In-Reply-To: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> References: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Thu, 16 Jun 2022 01:01:53 +0800 Subject: [PATCH 6/8] org-export-data: Concatenate strings in temporary buffer for performance * lisp/ox.el (org-export-data): Use temporary buffer to collect export data instead of `mapconcat'. Using buffer puts less load on garbage collector. --- lisp/ox.el | 50 ++++++++++++++++++++++++++++---------------------- 1 file changed, 28 insertions(+), 22 deletions(-) diff --git a/lisp/ox.el b/lisp/ox.el index a4512270c..ae7e41e57 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -1923,28 +1923,34 @@ (defun org-export-data (data info) (and (not greaterp) (memq type org-element-recursive-objects))) (contents - (mapconcat - (lambda (element) (org-export-data element info)) - (org-element-contents - (if (or greaterp objectp) data - ;; Elements directly containing - ;; objects must have their indentation - ;; normalized first. - (org-element-normalize-contents - data - ;; When normalizing first paragraph - ;; of an item or - ;; a footnote-definition, ignore - ;; first line's indentation. - (and - (eq type 'paragraph) - (memq (org-element-type parent) - '(footnote-definition item)) - (eq (car (org-element-contents parent)) - data) - (eq (org-element-property :pre-blank parent) - 0))))) - ""))) + (let ((export-buffer (current-buffer))) + (with-temp-buffer + (dolist (element (org-element-contents + (if (or greaterp objectp) data + ;; Elements directly containing + ;; objects must have their indentation + ;; normalized first. + (org-element-normalize-contents + data + ;; When normalizing first paragraph + ;; of an item or + ;; a footnote-definition, ignore + ;; first line's indentation. + (and + (eq type 'paragraph) + (memq (org-element-type parent) + '(footnote-definition item)) + (eq (car (org-element-contents parent)) + data) + (eq (org-element-property :pre-blank parent) + 0)))))) + (insert + ;; Use right local variable + ;; environment if there are, for + ;; example, #+BIND variables. + (with-current-buffer export-buffer + (org-export-data element info)))) + (buffer-string))))) (broken-link-handler (funcall transcoder data (if (not greaterp) contents -- 2.35.1 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #8: 0007-org-element-map-Avoid-repetitive-plist-get-call.patch --] [-- Type: text/x-patch, Size: 1486 bytes --] From efd91d5f654cc0e734cd3f5cfd1f39a6219fd0dc Mon Sep 17 00:00:00 2001 Message-Id: <efd91d5f654cc0e734cd3f5cfd1f39a6219fd0dc.1655362876.git.yantar92@gmail.com> In-Reply-To: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> References: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Thu, 16 Jun 2022 09:28:27 +0800 Subject: [PATCH 7/8] org-element-map: Avoid repetitive `plist-get' call * lisp/org-element.el (org-element-map): Do not call `(plist-get info :ignore-list)' on every iteration. --- lisp/org-element.el | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lisp/org-element.el b/lisp/org-element.el index 9db1406b3..20b5b0303 100644 --- a/lisp/org-element.el +++ b/lisp/org-element.el @@ -4391,6 +4391,7 @@ (defun org-element-map ;; every element it encounters. (and (not (eq category 'elements)) (setq category 'elements)))))))) + (--ignore-list (plist-get info :ignore-list)) --acc) (letrec ((--walk-tree (lambda (--data) @@ -4400,7 +4401,7 @@ (defun org-element-map (cond ((not --data)) ;; Ignored element in an export context. - ((and info (memq --data (plist-get info :ignore-list)))) + ((and info (memq --data --ignore-list))) ;; List of elements or objects. ((not --type) (mapc --walk-tree --data)) ;; Unconditionally enter parse trees. -- 2.35.1 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #9: 0008-org-cite-list-citations-Cache-footnote-definition-se.patch --] [-- Type: text/x-patch, Size: 3309 bytes --] From 56830c1792b548e9157cdd95f4bb7c093e390a05 Mon Sep 17 00:00:00 2001 Message-Id: <56830c1792b548e9157cdd95f4bb7c093e390a05.1655362876.git.yantar92@gmail.com> In-Reply-To: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> References: <3342137359f122ed7168dc75096c6a5d3839a0c2.1655362876.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Thu, 16 Jun 2022 10:43:29 +0800 Subject: [PATCH 8/8] org-cite-list-citations: Cache footnote-definition searches * lisp/oc.el (org-cite-list-citations): Avoid quadratic complexity. Pre-calculate list of all footnote definitions and cache the footnote label search hits. Do not make `org-element-map' accumulate unused result. --- lisp/oc.el | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/lisp/oc.el b/lisp/oc.el index eb5f519cb..c4cd0268c 100644 --- a/lisp/oc.el +++ b/lisp/oc.el @@ -808,6 +808,8 @@ (defun org-cite-list-citations (info) (or (plist-get info :citations) (letrec ((cites nil) (tree (plist-get info :parse-tree)) + (definition-cache (make-hash-table :test #'equal)) + (definition-list nil) (find-definition ;; Find definition for standard reference LABEL. At ;; this point, it is impossible to rely on @@ -816,11 +818,21 @@ (defun org-cite-list-citations (info) ;; un-processed citation objects. So we use ;; a simplified version of the function above. (lambda (label) - (org-element-map tree 'footnote-definition - (lambda (d) - (and (equal label (org-element-property :label d)) - (or (org-element-contents d) ""))) - info t))) + (or (gethash label definition-cache) + (org-element-map + (or definition-list + (setq definition-list + (org-element-map + tree + 'footnote-definition + #'identity info))) + 'footnote-definition + (lambda (d) + (and (equal label (org-element-property :label d)) + (puthash label + (or (org-element-contents d) "") + definition-cache))) + info t)))) (search-cites (lambda (data) (org-element-map data '(citation footnote-reference) @@ -834,7 +846,8 @@ (defun org-cite-list-citations (info) (_ (let ((label (org-element-property :label datum))) (funcall search-cites - (funcall find-definition label)))))) + (funcall find-definition label))))) + nil) info nil 'footnote-definition t)))) (funcall search-cites tree) (let ((result (nreverse cites))) -- 2.35.1 ^ permalink raw reply related [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-16 7:03 ` Ihor Radchenko @ 2022-06-16 8:13 ` Eli Zaretskii 2022-06-16 11:12 ` Mattias Engdegård 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-16 8:13 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, rms@gnu.org, monnier@iro.umontreal.ca, > acm@muc.de, emacs-devel@gnu.org > Date: Thu, 16 Jun 2022 15:03:19 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > > I see no need for such significant changes in our merging practices. > > Rather, I hoped that the speedup you achieved will be important enough > > to make an exception and install just it in the Emacs master branch. > > See the attached patches. Thanks, I installed these, with one exception: I limited gc-cons-threshold in doc/misc/Makefile.in to 50,000,000, not most-positive-fixnum. The slowdown in producing the Org manual as result of that is negligible, and I didn't feel comfortable with disabling GC altogether, as it caused the Emacs memory footprint go up to 0.5GB. These changes slash the conversion from 2.5 min to just 35 sec on my system, with an unoptimized build of Emacs, which is a terrific speedup. Thanks! ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-16 8:13 ` Eli Zaretskii @ 2022-06-16 11:12 ` Mattias Engdegård 2022-06-16 12:58 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Mattias Engdegård @ 2022-06-16 11:12 UTC (permalink / raw) To: Eli Zaretskii Cc: Ihor Radchenko, Tim Cross, rms, Stefan Monnier, Alan Mackenzie, emacs-devel 16 juni 2022 kl. 10.13 skrev Eli Zaretskii <eliz@gnu.org>: > I limited > gc-cons-threshold in doc/misc/Makefile.in to 50,000,000, not > most-positive-fixnum. Some timings for the export to .texi (old machine, optimised build, bytecode only): | gc-cons-threshold | time | |----------------------+------| | 800000 | 14.1 | (default value) | 25000000 | 6.2 | | 50000000 | 5.7 | | 100000000 | 5.7 | | most-positive-fixnum | 5.1 | thus Eli's choice is very good, and we really should do something about that GC of ours. > a terrific > speedup. Thanks! Echoed! ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-16 11:12 ` Mattias Engdegård @ 2022-06-16 12:58 ` Ihor Radchenko 2022-06-16 16:59 ` Org mode and Emacs Stefan Monnier 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-16 12:58 UTC (permalink / raw) To: Mattias Engdegård Cc: Eli Zaretskii, Tim Cross, rms, Stefan Monnier, Alan Mackenzie, emacs-devel Mattias Engdegård <mattiase@acm.org> writes: >> I limited >> gc-cons-threshold in doc/misc/Makefile.in to 50,000,000, not >> most-positive-fixnum. > > Some timings for the export to .texi (old machine, optimised build, bytecode only): > > | gc-cons-threshold | time | > |----------------------+------| > | 800000 | 14.1 | (default value) > | 25000000 | 6.2 | > | 50000000 | 5.7 | > | 100000000 | 5.7 | > | most-positive-fixnum | 5.1 | > > thus Eli's choice is very good, and we really should do something about that GC of ours. I am wondering if there is a "universal" value suitable for one-off Emacs batch invocations. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-16 12:58 ` Ihor Radchenko @ 2022-06-16 16:59 ` Stefan Monnier 2022-06-17 7:18 ` Larger GC thresholds for non-interactive Emacs (was: Org mode and Emacs) Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-16 16:59 UTC (permalink / raw) To: Ihor Radchenko Cc: Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Ihor Radchenko [2022-06-16 20:58:07] wrote: > Mattias Engdegård <mattiase@acm.org> writes: >>> I limited >>> gc-cons-threshold in doc/misc/Makefile.in to 50,000,000, not >>> most-positive-fixnum. >> >> Some timings for the export to .texi (old machine, optimised build, bytecode only): >> >> | gc-cons-threshold | time | >> |----------------------+------| >> | 800000 | 14.1 | (default value) >> | 25000000 | 6.2 | >> | 50000000 | 5.7 | >> | 100000000 | 5.7 | >> | most-positive-fixnum | 5.1 | >> >> thus Eli's choice is very good, and we really should do something about that GC of ours. > > I am wondering if there is a "universal" value suitable for one-off Emacs > batch invocations. I doubt that's the case, but of course we should try and use values that work "overall best" on average. Maybe we should revisit the current knobs and their default values. Currently the two important knobs we have are: gc-cons-threshold gc-cons-percentage Maybe it's time for some benchmarks with various values for these knobs to see if the values we're using now are "good enough" or might need to be changed. We could also try and be bit more discerning. E.g. currently when the program is in a phase where it builds a lot of new data-structures, we run the GC everytime some amount of new memory has been allocated, but presumably almost none of those objects have died yet (we're still building the overall datastructure) so the GC will be a pure waste of time. OTOH in other phases the code allocates objects which are used only temporarily, in which case it's beneficial to run the GC somewhat frequently to avoid growing our heap unnecessarily (and suffering fragmentation as a result). It's hard to know beforehand whether a GC will be useful or not, tho. But maybe we can find good heuristics. E.g. have something like a `gc-cons-percentage` which depends on how much garbage we collected in the last GC: if a GC doesn't collect any garbage (or very little of it), it's a sign that we're in a phase where running the GC is not very useful so we should wait a bit more until the next GC, whereas if the GC collected a fair bit of garbage, it's a sign that we're in a phase where running the GC is beneficial and we can run it a bit more often. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Larger GC thresholds for non-interactive Emacs (was: Org mode and Emacs) 2022-06-16 16:59 ` Org mode and Emacs Stefan Monnier @ 2022-06-17 7:18 ` Ihor Radchenko 2022-06-17 13:49 ` Larger GC thresholds for non-interactive Emacs Stefan Monnier 2022-06-21 13:04 ` Larger GC thresholds for non-interactive Emacs Lars Ingebrigtsen 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-17 7:18 UTC (permalink / raw) To: Stefan Monnier Cc: Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >>> thus Eli's choice is very good, and we really should do something about that GC of ours. >> >> I am wondering if there is a "universal" value suitable for one-off Emacs >> batch invocations. > > I doubt that's the case, but of course we should try and use values that > work "overall best" on average. Maybe we should revisit the current > knobs and their default values. > > Currently the two important knobs we have are: > > gc-cons-threshold > gc-cons-percentage > > Maybe it's time for some benchmarks with various values for these knobs > to see if the values we're using now are "good enough" or might need to > be changed. Note that I did not initially ask to change the GC defaults for normal Emacs session. What I am trying to suggest is: 1. Have some conservative GC settings in interactive Emacs session (as we do now; I do _not_ suggest changing it) 2. Increase GC thresholds in non-interactive --batch sessions, which tend to not run for a long time. I am a bit afraid to touch the topic of changing the interactive GC defaults. People tend to have contradicting opinions on this. I'd like to ask one thing from the very beginning here: Please spin a separate thread if you have objections on actual GC defaults applicable to normal Emacs sessions. > We could also try and be bit more discerning. E.g. currently when the > program is in a phase where it builds a lot of new data-structures, we > run the GC everytime some amount of new memory has been allocated, but > presumably almost none of those objects have died yet (we're still > building the overall datastructure) so the GC will be a pure waste of > time. OTOH in other phases the code allocates objects which are used > only temporarily, in which case it's beneficial to run the GC somewhat > frequently to avoid growing our heap unnecessarily (and suffering > fragmentation as a result). AFAIK, there is no way to determine such stage automatically. It's usually up to the program. For example, org-element-parse-buffer (function that creates buffer AST in Org) does the following: (let (... (gc-cons-threshold #x40000000)) ... (org-element--parse-elements ...)) A possible useful thing Emacs could help with would be a macro dedicated to let-binds like the above. Something like: (with-reduced-gc ;; Do staff) with-reduced-gc could take care about determining the inner specifics on what alternative gc-cons-threshold value should be used (possibly depending on the system memory information). > It's hard to know beforehand whether a GC will be useful or not, tho. > But maybe we can find good heuristics. E.g. have something like > a `gc-cons-percentage` which depends on how much garbage we collected in > the last GC: if a GC doesn't collect any garbage (or very little of it), > it's a sign that we're in a phase where running the GC is not very > useful so we should wait a bit more until the next GC, whereas if the GC > collected a fair bit of garbage, it's a sign that we're in a phase where > running the GC is beneficial and we can run it a bit more often. FYI, I have played with this approach making use of https://gitlab.com/koral/gcmh In practice, relying on recent GC history is often misleading. GC is normally not very frequent - tens of seconds to minutes. And the practical Emacs memory usage patterns are often irregular: long periods of "idleness"/low intensity usage vs. short bursts of high memory allocations when some "heavy" commands are called. The adjustment you propose will suffer from the following: 1. There is long "idle" period and GC gets less frequent 2. User runs "heavy" command, GC frequency not yet adjusted 3. Command allocates a lots of memory - GC is finally triggered and has to go through a lot of garbage; It often happens _after_ the command finishes. 4. GC re-adjusts the frequency, but Emacs is again "idle" with no "heavy" commands being used. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 7:18 ` Larger GC thresholds for non-interactive Emacs (was: Org mode and Emacs) Ihor Radchenko @ 2022-06-17 13:49 ` Stefan Monnier 2022-06-17 15:11 ` Lars Ingebrigtsen ` (2 more replies) 2022-06-21 13:04 ` Larger GC thresholds for non-interactive Emacs Lars Ingebrigtsen 1 sibling, 3 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-17 13:49 UTC (permalink / raw) To: Ihor Radchenko Cc: Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel >> Maybe it's time for some benchmarks with various values for these knobs >> to see if the values we're using now are "good enough" or might need to >> be changed. > > Note that I did not initially ask to change the GC defaults for normal > Emacs session. You're absolutely right, but we've had various reports of late that indicate that we're probably spending too much time in the GC. So I think we're due for some re-evaluation. > What I am trying to suggest is: > 1. Have some conservative GC settings in interactive Emacs session (as > we do now; I do _not_ suggest changing it) > 2. Increase GC thresholds in non-interactive --batch sessions, which > tend to not run for a long time. Agreed. Then again, we occasionally also use long-running commands interactively so it might be worthwhile to tune the settings for interactive use as well. > I am a bit afraid to touch the topic of changing the interactive GC > defaults. People tend to have contradicting opinions on this. :-) The issue is that it's easy to see "oh god, we're spending so much time in the GC, let's disable it" but much harder to see the "long term" impact of increasing the thresholds (which tend to increase the fragmentation and the total heap size both of which tend to slow down execution but the second additionally makes every GC cycle take longer, again increasing the pressure from naive onlookers to disable the GC). > I'd like to ask one thing from the very beginning here: Please spin a > separate thread if you have objections on actual GC defaults applicable > to normal Emacs sessions. No, I think focusing on batch use is a good idea. Also because it makes it much easier to actually measure the impact. > A possible useful thing Emacs could help with would be a macro dedicated > to let-binds like the above. Something like: > > (with-reduced-gc > ;; Do staff) Sounds about right, tho I think the name of the macro should be related to what the code does rather than to what the author thinks the GC should do about it. Something like `this-code-does-not-generate-garbage`. > with-reduced-gc could take care about determining the inner specifics > on what alternative gc-cons-threshold value should be used (possibly > depending on the system memory information). Sounds good. >> It's hard to know beforehand whether a GC will be useful or not, tho. >> But maybe we can find good heuristics. E.g. have something like >> a `gc-cons-percentage` which depends on how much garbage we collected in >> the last GC: if a GC doesn't collect any garbage (or very little of it), >> it's a sign that we're in a phase where running the GC is not very >> useful so we should wait a bit more until the next GC, whereas if the GC >> collected a fair bit of garbage, it's a sign that we're in a phase where >> running the GC is beneficial and we can run it a bit more often. > > FYI, I have played with this approach making use of > https://gitlab.com/koral/gcmh But this one focuses on interactive use. The kind of heuristic I'm proposing above would only make sense within a single (long-running) command, or in batch mode, I think. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 13:49 ` Larger GC thresholds for non-interactive Emacs Stefan Monnier @ 2022-06-17 15:11 ` Lars Ingebrigtsen 2022-06-17 15:30 ` Lars Ingebrigtsen 2022-06-17 17:48 ` Stefan Monnier 2022-06-18 5:28 ` Ihor Radchenko 2022-07-01 2:34 ` Lisp-level macro to avoid excessive GC in memory-allocating code (was: Larger GC thresholds for non-interactive Emacs) Ihor Radchenko 2 siblings, 2 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-17 15:11 UTC (permalink / raw) To: Stefan Monnier Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel I tried this very advanced patch: diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el index d28ec0be16..1bd0793097 100644 --- a/lisp/emacs-lisp/bytecomp.el +++ b/lisp/emacs-lisp/bytecomp.el @@ -5366,6 +5366,7 @@ batch-byte-compile ;; behavior. (setq attempt-stack-overflow-recovery nil attempt-orderly-shutdown-on-fatal-signal nil) + (setq gc-cons-threshold (* 100 gc-cons-threshold)) (let ((error nil)) (while command-line-args-left (if (file-directory-p (expand-file-name (car command-line-args-left))) A "make -j32 bootstrap" went from 1m25s to 1m6s. 😀 Most of this time is spent doing other things than building lisp files, though, so the impact on byte compilation is huge. rm lisp/*/*.elc time make -j32 goes from 29s to 14s. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply related [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 15:11 ` Lars Ingebrigtsen @ 2022-06-17 15:30 ` Lars Ingebrigtsen 2022-06-17 16:05 ` Stefan Monnier 2022-06-17 17:48 ` Stefan Monnier 1 sibling, 1 reply; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-17 15:30 UTC (permalink / raw) To: Stefan Monnier Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > A "make -j32 bootstrap" went from 1m25s to 1m6s. 😀 I wonder whether we should just have a command line option for this -- i.e., --no-garbage-collection to be used in combination with --batch for jobs that aren't long-running (i.e., will exit before growing huge). -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 15:30 ` Lars Ingebrigtsen @ 2022-06-17 16:05 ` Stefan Monnier 2022-06-17 16:11 ` Lars Ingebrigtsen 2022-06-18 5:35 ` Ihor Radchenko 0 siblings, 2 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-17 16:05 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel >> A "make -j32 bootstrap" went from 1m25s to 1m6s. 😀 > I wonder whether we should just have a command line option for this -- > i.e., --no-garbage-collection to be used in combination with --batch for > jobs that aren't long-running (i.e., will exit before growing huge). "no-garbage-collection" would be either a bug (we'd crash or be killed[*] on long-enough-running jobs) or a lie (we'd still perform GC sometimes). Also I hope we can first try to simply provide better defaults before requiring manual intervention from the user. I haven't seen any evidence yet that the current defaults are good for batch jobs. BTW, I seen everyone's focused on `gc-cons-threshold` but that variable does not adapt to the heap size, so maybe we'd be better off playing with `gc-cons-percentage`. Stefan [*] Like the Org manual's build on elpa.gnu.org yesterday because its gc-threshold was set to most-positive-fixnum and `elpa.gnu.org` only has 1GB of RAM. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 16:05 ` Stefan Monnier @ 2022-06-17 16:11 ` Lars Ingebrigtsen 2022-06-18 5:35 ` Ihor Radchenko 1 sibling, 0 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-17 16:11 UTC (permalink / raw) To: Stefan Monnier Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > "no-garbage-collection" would be either a bug (we'd crash or be killed[*] > on long-enough-running jobs) or a lie (we'd still perform GC sometimes). It's not a bug to do what the user says. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 16:05 ` Stefan Monnier 2022-06-17 16:11 ` Lars Ingebrigtsen @ 2022-06-18 5:35 ` Ihor Radchenko 2022-06-18 13:16 ` Stefan Monnier 1 sibling, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-18 5:35 UTC (permalink / raw) To: Stefan Monnier Cc: Lars Ingebrigtsen, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > BTW, I seen everyone's focused on `gc-cons-threshold` but that variable > does not adapt to the heap size, so maybe we'd be better off > playing with `gc-cons-percentage`. I am not sure if `gc-cons-percentage' is reliable. When we have a small heap (at the beginning of the batch command run), the GC will be triggered frequently. Later, after initial memory allocations for data structures is complete, the same `gc-cons-percentage' will make GC fire less frequently and will have nothing to do with actually allocated/deallocated memory objects in the subsequent parts of the batch process. Consider the following example: (defun init () (allocate-huge-hash-map-object)) (defun run () (dolist (...) (let (this and that) (query the huge hash-map)))) (init) (run) The (init) part will trigger frequent GCs yielding no of very small amount of garbage being collected. The (run) part may use more temporary allocations. Yet, GC will be triggered much less frequently simply because the atomic (let ...) allocations are small compared to the initially allocated hash-map object. I believe that gc-cons-threshold will still be useful as _an upper bound_ to trigger GC. gc-cons-percentage is probably more efficient when it triggers below gc-cons-threshold. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 5:35 ` Ihor Radchenko @ 2022-06-18 13:16 ` Stefan Monnier 0 siblings, 0 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-18 13:16 UTC (permalink / raw) To: Ihor Radchenko Cc: Lars Ingebrigtsen, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Ihor Radchenko [2022-06-18 13:35:04] wrote: > Stefan Monnier <monnier@iro.umontreal.ca> writes: >> BTW, I seen everyone's focused on `gc-cons-threshold` but that variable >> does not adapt to the heap size, so maybe we'd be better off >> playing with `gc-cons-percentage`. > I am not sure if `gc-cons-percentage' is reliable. From where I stand it's the only one that's reliable. It basically controls the percentage of *time* we spend in the GC: if we assume that the time to perform a GC is proportional to the heap size, and that the allocation rate (in MB/s) is more or less constant, then `gc-cons-percentage' directly controls the percentage of time spent in the GC. Of course, those two assumptions don't hold exactly, but they're fairly good approximations of reality. Tweaking `gc-cons-threshold` on the other hand is very coarse and brittle: if the value is significantly smaller than the heap, then we'll spend too much time in the GC and if it's significantly larger then we waste a lot of memory on garbage. There simply can't be any "one good value" for `gc-cons-threshold`. > When we have a small heap (at the beginning of the batch command run), > the GC will be triggered frequently. Yes, but the GC cycles will finish quickly. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 15:11 ` Lars Ingebrigtsen 2022-06-17 15:30 ` Lars Ingebrigtsen @ 2022-06-17 17:48 ` Stefan Monnier 2022-06-17 18:20 ` Lars Ingebrigtsen 2022-06-18 5:58 ` Eli Zaretskii 1 sibling, 2 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-17 17:48 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel > rm lisp/*/*.elc > time make -j32 IIRC this test can give surprising results because it's strongly affected by the time when the byte-compiler files get compiled which is somewhat arbitrary. > goes from 29s to 14s. Wow! Here are the results of a similar test where I tried to avoid the above problem by also removing src/boostrap-emacs (and I also rm'd lisp/loaddefs.el for good measure tho it was probably not a great idea), and I used BYTE_COMPILE_EXTRA_FLAGS="--eval '(setq gc-cons-percentage $FOO)'": make -j24 FOO=0.1 1338.52s user 60.95s system 1256% cpu 1:51.42 total make -j24 FOO=0.1 1338.36s user 60.60s system 1252% cpu 1:51.65 total make -j24 FOO=0.1 1339.53s user 60.08s system 1252% cpu 1:51.76 total make -j24 FOO=0.2 1266.54s user 59.75s system 1264% cpu 1:44.92 total make -j24 FOO=0.2 1269.17s user 60.18s system 1256% cpu 1:45.83 total make -j24 FOO=0.2 1266.10s user 60.42s system 1258% cpu 1:45.43 total make -j24 FOO=0.3 1200.06s user 60.24s system 1251% cpu 1:40.73 total make -j24 FOO=0.3 1198.80s user 60.06s system 1254% cpu 1:40.39 total make -j24 FOO=0.3 1202.49s user 59.60s system 1252% cpu 1:40.80 total make -j24 FOO=0.4 1134.26s user 59.75s system 1233% cpu 1:36.81 total make -j24 FOO=0.4 1133.11s user 60.07s system 1226% cpu 1:37.25 total make -j24 FOO=0.4 1132.18s user 59.94s system 1226% cpu 1:37.18 total make -j24 FOO=0.5 1069.59s user 59.70s system 1211% cpu 1:33.22 total make -j24 FOO=0.5 1070.84s user 60.06s system 1211% cpu 1:33.34 total make -j24 FOO=0.5 1070.93s user 59.86s system 1210% cpu 1:33.43 total make -j24 FOO=1.0 845.92s user 61.24s system 1126% cpu 1:20.55 total make -j24 FOO=1.0 847.92s user 61.82s system 1130% cpu 1:20.46 total make -j24 FOO=1.0 846.59s user 61.46s system 1133% cpu 1:20.10 total make -j24 FOO=2.0 678.14s user 68.64s system 1044% cpu 1:11.52 total make -j24 FOO=2.0 678.04s user 69.19s system 1042% cpu 1:11.64 total make -j24 FOO=2.0 677.66s user 68.24s system 1041% cpu 1:11.59 total make -j24 FOO=3.0 677.50s user 69.38s system 1047% cpu 1:11.28 total make -j24 FOO=3.0 677.42s user 68.43s system 1047% cpu 1:11.19 total make -j24 FOO=3.0 677.98s user 69.10s system 1044% cpu 1:11.50 total make -j24 FOO=4.0 677.43s user 68.58s system 1051% cpu 1:10.96 total make -j24 FOO=4.0 677.09s user 68.49s system 1056% cpu 1:10.57 total make -j24 FOO=4.0 677.60s user 69.23s system 1049% cpu 1:11.17 total make -j24 FOO=5.0 677.22s user 68.90s system 1043% cpu 1:11.52 total make -j24 FOO=5.0 677.47s user 68.91s system 1044% cpu 1:11.49 total make -j24 FOO=5.0 677.13s user 67.98s system 1049% cpu 1:10.98 total make -j24 FOO=7.0 678.21s user 68.57s system 1044% cpu 1:11.52 total make -j24 FOO=7.0 677.48s user 68.50s system 1044% cpu 1:11.42 total make -j24 FOO=7.0 677.95s user 68.25s system 1050% cpu 1:11.00 total make -j24 FOO=10.0 677.30s user 69.52s system 1045% cpu 1:11.45 total make -j24 FOO=10.0 678.18s user 68.55s system 1042% cpu 1:11.62 total make -j24 FOO=10.0 678.97s user 68.45s system 1043% cpu 1:11.64 total make -j24 FOO=20.0 677.99s user 68.42s system 1041% cpu 1:11.67 total make -j24 FOO=20.0 677.69s user 69.31s system 1046% cpu 1:11.36 total make -j24 FOO=20.0 677.89s user 68.70s system 1045% cpu 1:11.39 total Clearly for this test, the default (0.1) results in too low a threshold for best performance and thus too much time spent in the GC. We reach a plateau around percentage=2.0. This suggests that for batch jobs maybe we should bump up `gc-cons-percentage` from 0.1 to something like 1.0 or 2.0. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 17:48 ` Stefan Monnier @ 2022-06-17 18:20 ` Lars Ingebrigtsen 2022-06-17 18:39 ` Alan Mackenzie ` (2 more replies) 2022-06-18 5:58 ` Eli Zaretskii 1 sibling, 3 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-17 18:20 UTC (permalink / raw) To: Stefan Monnier Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > Here are the results of a similar test where I tried to avoid the above > problem by also removing src/boostrap-emacs (and I also rm'd > lisp/loaddefs.el for good measure tho it was probably not a great idea), > and I used > > BYTE_COMPILE_EXTRA_FLAGS="--eval '(setq gc-cons-percentage $FOO)'": > > make -j24 FOO=0.1 1338.52s user 60.95s system 1256% cpu 1:51.42 total [...] > make -j24 FOO=2.0 678.14s user 68.64s system 1044% cpu 1:11.52 total Nice! > This suggests that for batch jobs maybe we should bump up > `gc-cons-percentage` from 0.1 to something like 1.0 or 2.0. Yup. But do you mean in general? I.e., -batch would set that variable to 2.0? Would there be any likely major repercussions -- i.e., jobs that used to run fine would run out of memory? As a quick test, I added a (setq gc-cons-percentage 2.0) to startup.el here, and a git clean -dfx time make -j32 went from 1m29s to 1m4s. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 18:20 ` Lars Ingebrigtsen @ 2022-06-17 18:39 ` Alan Mackenzie 2022-06-17 22:21 ` Stefan Monnier 2022-06-17 22:53 ` Stefan Monnier 2022-06-18 2:32 ` Stefan Monnier 2 siblings, 1 reply; 505+ messages in thread From: Alan Mackenzie @ 2022-06-17 18:39 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Stefan Monnier, Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, emacs-devel Hello, Lars. On Fri, Jun 17, 2022 at 20:20:48 +0200, Lars Ingebrigtsen wrote: > Stefan Monnier <monnier@iro.umontreal.ca> writes: > > Here are the results of a similar test where I tried to avoid the above > > problem by also removing src/boostrap-emacs (and I also rm'd > > lisp/loaddefs.el for good measure tho it was probably not a great idea), > > and I used > > BYTE_COMPILE_EXTRA_FLAGS="--eval '(setq gc-cons-percentage $FOO)'": > > make -j24 FOO=0.1 1338.52s user 60.95s system 1256% cpu 1:51.42 total > [...] > > make -j24 FOO=2.0 678.14s user 68.64s system 1044% cpu 1:11.52 total > Nice! > > This suggests that for batch jobs maybe we should bump up > > `gc-cons-percentage` from 0.1 to something like 1.0 or 2.0. > Yup. But do you mean in general? I.e., -batch would set that variable > to 2.0? Would there be any likely major repercussions -- i.e., jobs > that used to run fine would run out of memory? > As a quick test, I added a (setq gc-cons-percentage 2.0) to startup.el > here, and a > git clean -dfx > time make -j32 > went from 1m29s to 1m4s. Just as another independent test (as if any were needed), I put BYTE_COMPILE_EXTRA_FLAGS = --eval '(setq gc-cons-threshold 50000000)' into lisp/Makefile.in, and my time for $ make -j17 bootstrap dropped from 5m52s to 3m35s, a drop of 39%. Watching the .elc's being built was noticeably faster, even to the naked eye. This was a build with native compilation enabled. > -- > (domestic pets only, the antidote for overdose, milk.) > bloggy blog: http://lars.ingebrigtsen.no -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 18:39 ` Alan Mackenzie @ 2022-06-17 22:21 ` Stefan Monnier 2022-06-17 22:31 ` Lars Ingebrigtsen 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-17 22:21 UTC (permalink / raw) To: Alan Mackenzie Cc: Lars Ingebrigtsen, Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, emacs-devel > Just as another independent test (as if any were needed), I put Thanks anyway, but indeed, it would be good to see tests that do other things than compilation. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 22:21 ` Stefan Monnier @ 2022-06-17 22:31 ` Lars Ingebrigtsen 0 siblings, 0 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-17 22:31 UTC (permalink / raw) To: Stefan Monnier Cc: Alan Mackenzie, Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > Thanks anyway, but indeed, it would be good to see tests that do other > things than compilation. Perhaps rm lisp/leim/ja-dic/ja-dic.el*; time make It goes from around 9 seconds to around 6 seconds (with a cons-percentage of 4.0). "make -j32 check" goes from 18s to 15s (with a value of 2.0). -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 18:20 ` Lars Ingebrigtsen 2022-06-17 18:39 ` Alan Mackenzie @ 2022-06-17 22:53 ` Stefan Monnier 2022-06-18 6:11 ` Eli Zaretskii 2022-06-18 12:45 ` Lars Ingebrigtsen 2022-06-18 2:32 ` Stefan Monnier 2 siblings, 2 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-17 22:53 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel >> This suggests that for batch jobs maybe we should bump up >> `gc-cons-percentage` from 0.1 to something like 1.0 or 2.0. > Yup. But do you mean in general? I.e., -batch would set that variable > to 2.0? Yup. > Would there be any likely major repercussions That's the question we should investigate as well. > -- i.e., jobs that used to run fine would run out of memory? I added the patch below to try and see a bit more what's going on: diff --git a/src/alloc.c b/src/alloc.c index 55e18ecd77e..d954e928eed 100644 --- a/src/alloc.c +++ b/src/alloc.c @@ -6245,6 +6245,11 @@ garbage_collect (void) consing_until_gc = gc_threshold = consing_threshold (gc_cons_threshold, Vgc_cons_percentage, 0); + fprintf (stderr, "GC-%d p=%.1f total=%.1fM thresold=%.1fM\n", + getpid (), XFLOAT_DATA (Vgc_cons_percentage), + total_bytes_of_live_objects () / 1048576.0, + consing_until_gc / 1048576.0); + /* Unblock *after* re-setting `consing_until_gc` in case `unblock_input` signals an error (see bug#43389). */ unblock_input (); and then I ran rm **/*.elc src/bootstrap-emacs lisp/loaddefs.el make -j24 BYTE_COMPILE_EXTRA_FLAGS="--eval '(setq gc-cons-percentage 2.0)' The first thing I see is that with the number of GCs where p=0.1 outweighs those with p=2.0 by a factor 10, so clearly the number of GCs is significantly reduced. I also looked for the number of GCs performed in a given process: % grep 'p=2.0' build-log | sed 's/ .*//' | sort | uniq -c 1 GC-15817 1 GC-15837 1 GC-15841 1 GC-15842 1 GC-15985 1 GC-15990 1 GC-15991 1 GC-15993 1 GC-15995 1 GC-15998 18 GC-16009 18 GC-16021 18 GC-16023 18 GC-16026 18 GC-16028 18 GC-16042 18 GC-16045 18 GC-16048 18 GC-16056 18 GC-16058 1 GC-16062 1 GC-16064 18 GC-16067 17 GC-16079 This says that only 24 processes performed a GC with p=2.0, so the vast majority of the byte-compilation processes finished without performing any GC at all (well, maybe not quite: see below). It's also odd how most performed 0 GC, then 12 performed 1 GC, then 1 performed 17 GCs and 11 performed exactly 18 GCs. What's so magical about 18? Then I looked at those that performed 18 GCs and they all look quite similar: % grep GC-16026 ./+make-2.0.log GC-16026 p=0.1 total=0.5M thresold=0.8M GC-16026 p=2.0 total=3.9M thresold=7.8M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M GC-16026 p=2.0 total=4.1M thresold=9.5M The first GC seems to happen before we set percentage to 2.0 (so apparently all compilation processes performed at least one GC before we set percentage to 2.0 and then the majority of them performed no further GC before exiting). [ So if we set percentage to 2.0 a bit earlier than what happens with `--eval` we may gain yet a bit more time. ] 9.5M is actually exactly 10000000 bytes so I suspect these come from either `cedet/semantic.el` or `international/mule-cmds.el` which both set gc-cons-threshold to (max gc-cons-threshold 10000000). Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 22:53 ` Stefan Monnier @ 2022-06-18 6:11 ` Eli Zaretskii 2022-06-18 12:45 ` Lars Ingebrigtsen 1 sibling, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-18 6:11 UTC (permalink / raw) To: Stefan Monnier Cc: larsi, yantar92, mattiase, theophilusx, rms, acm, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Ihor Radchenko <yantar92@gmail.com>, Mattias Engdegård > <mattiase@acm.org>, Eli Zaretskii <eliz@gnu.org>, Tim Cross > <theophilusx@gmail.com>, rms@gnu.org, Alan Mackenzie <acm@muc.de>, > emacs-devel <emacs-devel@gnu.org> > Date: Fri, 17 Jun 2022 18:53:17 -0400 > > It's also odd how most performed 0 GC, then 12 performed 1 GC, then > 1 performed 17 GCs and 11 performed exactly 18 GCs. What's so magical > about 18? Divine intervention? 18 means "alive" (חי) in gematria. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 22:53 ` Stefan Monnier 2022-06-18 6:11 ` Eli Zaretskii @ 2022-06-18 12:45 ` Lars Ingebrigtsen 2022-06-18 13:26 ` Stefan Monnier 1 sibling, 1 reply; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-18 12:45 UTC (permalink / raw) To: Stefan Monnier Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > The first GC seems to happen before we set percentage to 2.0 (so > apparently all compilation processes performed at least one GC before we > set percentage to 2.0 and then the majority of them performed no further > GC before exiting). Hm, interesting... > [ So if we set percentage to 2.0 a bit earlier than what happens with > `--eval` we may gain yet a bit more time. ] This should probably be early enough, I guess? diff --git a/src/emacs.c b/src/emacs.c index 43b9901e08..33ad9c6b8d 100644 --- a/src/emacs.c +++ b/src/emacs.c @@ -1620,6 +1620,7 @@ main (int argc, char **argv) if (argmatch (argv, argc, "-batch", "--batch", 5, NULL, &skip_args) || only_version) { + Vgc_cons_percentage = make_float (2.0); noninteractive = 1; Vundo_outer_limit = Qnil; } > 9.5M is actually exactly 10000000 bytes so I suspect these come from > either `cedet/semantic.el` or `international/mule-cmds.el` which both > set gc-cons-threshold to (max gc-cons-threshold 10000000). Ah, I see. Perhaps those should increase that even more if it still leads to 18 GCs? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply related [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 12:45 ` Lars Ingebrigtsen @ 2022-06-18 13:26 ` Stefan Monnier 0 siblings, 0 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-18 13:26 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel >> 9.5M is actually exactly 10000000 bytes so I suspect these come from >> either `cedet/semantic.el` or `international/mule-cmds.el` which both >> set gc-cons-threshold to (max gc-cons-threshold 10000000). > > Ah, I see. Perhaps those should increase that even more if it still > leads to 18 GCs? As I just explained in another message, I think messing with `gc-cons-threshold` is a bad idea because it depends on the heap size. We see this here where we end up thinking that maybe we should update this magical number `10000000` with some other magical number. The values of `gc-cons-percentage` aren't nearly as magical and they automatically adjust to changes in heap sizes. We can see that the proposed `gc-cons-percentage` of 2.0 would result in a heap size almost equal to that magical `10000000`, so rather than increase this number we should likely just remove those hacks. As for reducing the number of those 18 GCs, the data we have doesn't tell us that it would bring any benefit. Instead the data says we already use up more than 3 times as much memory as what's truly needed (13.6MB vs 4.1MB), which suggests to me that `10000000` is already rather high. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 18:20 ` Lars Ingebrigtsen 2022-06-17 18:39 ` Alan Mackenzie 2022-06-17 22:53 ` Stefan Monnier @ 2022-06-18 2:32 ` Stefan Monnier 2022-06-18 12:49 ` Lars Ingebrigtsen ` (2 more replies) 2 siblings, 3 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-18 2:32 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel > Yup. But do you mean in general? I.e., -batch would set that variable > to 2.0? Would there be any likely major repercussions -- i.e., jobs > that used to run fine would run out of memory? I've looked a bit further at some of the repercussions. The most obvious immediate one is that a program that needs 10MB of live data will now use up 30MB of heap (10MB of live data at the last GC plus upto 20MB of data allocated since the last GC), so it will increase the process sizes in a non-trivial way. I'd lean towards 1.0 instead of 2.0 for that kind of reason. I also took a look at related data. E.g. comparing p=1.0 to the old p-0.1 on the process that performs the highest number of GCs (188 cycles with p=1.0 and 650 cycles with p=0.1) during an Emacs build. Here are the corresponding last few GC cycles: % grep GC-26227 ./+make-0.1.log | tail -n 10 GC-26227 p=0.1 total=18.8M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.6M free=2.0M thresold=1.9M GC-26227 p=0.1 total=18.7M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.7M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.8M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.8M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.8M free=1.9M thresold=1.9M GC-26227 p=0.1 total=18.8M free=1.8M thresold=1.9M GC-26227 p=0.1 total=145.7M free=32.6M thresold=14.6M GC-26227 p=0.1 total=132.8M free=60.7M thresold=13.3M % grep GC-898 ./+make-1.0.log | tail -n 10 GC-898 p=1.0 total=18.5M free=7.5M thresold=18.5M GC-898 p=1.0 total=18.6M free=7.4M thresold=18.6M GC-898 p=1.0 total=18.6M free=7.4M thresold=18.6M GC-898 p=1.0 total=18.7M free=7.4M thresold=18.7M GC-898 p=1.0 total=18.7M free=7.3M thresold=18.7M GC-898 p=1.0 total=18.8M free=7.3M thresold=18.8M GC-898 p=1.0 total=18.8M free=7.3M thresold=18.8M GC-898 p=1.0 total=18.8M free=7.3M thresold=18.8M GC-898 p=1.0 total=18.8M free=7.3M thresold=18.8M GC-898 p=1.0 total=145.7M free=32.6M thresold=145.7M % Obviously this process ends up with a very large one-step allocation which is arguably interesting in itself (I suspect there's some GC-inhibition going on there), but the more interesting point is that with p=0.1 we get an amount of free space after GC (i.e. blocks we can't release, because of fragmentation) that's about as large as the next threshold, i.e. about 10%, whereas with p=1.0 this amount of unreleasable free space is significantly higher. Most likely this is not wasted space: a lot of it will be re-used for new allocations, but still with 19MB of live data, we end up with 7MB of space we can't release back. I think p=1.0 (and the corresponding implication that we use up about twice as much memory as the minimum we need) might be an acceptable tradeoff (for batch use), but I don't think I'd be comfortable going beyond that. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 2:32 ` Stefan Monnier @ 2022-06-18 12:49 ` Lars Ingebrigtsen 2022-06-18 13:06 ` Stefan Monnier ` (2 more replies) 2022-06-19 7:25 ` Ihor Radchenko 2022-06-19 9:23 ` Ihor Radchenko 2 siblings, 3 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-18 12:49 UTC (permalink / raw) To: Stefan Monnier Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > Obviously this process ends up with a very large one-step allocation > which is arguably interesting in itself (I suspect there's some > GC-inhibition going on there), but the more interesting point > is that with p=0.1 we get an amount of free space after GC > (i.e. blocks we can't release, because of fragmentation) that's about as > large as the next threshold, i.e. about 10%, whereas with p=1.0 this > amount of unreleasable free space is significantly higher. As far as I know, we never actually release back cons space to the OS anyway -- even if we could. (Because glibc's malloc implementation doesn't do that unless we call that trim function, which we don't.) So I'm not sure that makes much difference -- especially in a batch process. > I think p=1.0 (and the corresponding implication that we use up about > twice as much memory as the minimum we need) might be an acceptable > tradeoff (for batch use), but I don't think I'd be comfortable going > beyond that. Right. We can still have the Emacs makefiles increase the threshold for build purposes even if we settle on a lower general level for --batch. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 12:49 ` Lars Ingebrigtsen @ 2022-06-18 13:06 ` Stefan Monnier 2022-06-19 11:03 ` Lars Ingebrigtsen 2022-06-18 13:20 ` Eli Zaretskii 2022-06-20 12:42 ` Lynn Winebarger 2 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-18 13:06 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel >> Obviously this process ends up with a very large one-step allocation >> which is arguably interesting in itself (I suspect there's some >> GC-inhibition going on there), but the more interesting point >> is that with p=0.1 we get an amount of free space after GC >> (i.e. blocks we can't release, because of fragmentation) that's about as >> large as the next threshold, i.e. about 10%, whereas with p=1.0 this >> amount of unreleasable free space is significantly higher. > As far as I know, we never actually release back cons space to the OS > anyway -- even if we could. But we do release to the malloc library, which means it can be reused for other things (instead of being constrained to only contain cons cells, for example). Whether that library in turn releases it back to the OS is out of our control (and apparently current glibc never does so). >> I think p=1.0 (and the corresponding implication that we use up about >> twice as much memory as the minimum we need) might be an acceptable >> tradeoff (for batch use), but I don't think I'd be comfortable going >> beyond that. > > Right. We can still have the Emacs makefiles increase the threshold for > build purposes even if we settle on a lower general level for --batch. We could, I guess, tho the performance difference between p=1.0 and p=2.0 isn't very significant. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 13:06 ` Stefan Monnier @ 2022-06-19 11:03 ` Lars Ingebrigtsen 0 siblings, 0 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-19 11:03 UTC (permalink / raw) To: Stefan Monnier Cc: Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > But we do release to the malloc library, which means it can be reused > for other things (instead of being constrained to only contain cons > cells, for example). Yes, that's true. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 12:49 ` Lars Ingebrigtsen 2022-06-18 13:06 ` Stefan Monnier @ 2022-06-18 13:20 ` Eli Zaretskii 2022-06-19 11:02 ` Lars Ingebrigtsen 2022-06-20 12:42 ` Lynn Winebarger 2 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-18 13:20 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: monnier, yantar92, mattiase, theophilusx, rms, acm, emacs-devel > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Ihor Radchenko <yantar92@gmail.com>, Mattias Engdegård > <mattiase@acm.org>, Eli Zaretskii <eliz@gnu.org>, Tim Cross > <theophilusx@gmail.com>, rms@gnu.org, Alan Mackenzie <acm@muc.de>, > emacs-devel <emacs-devel@gnu.org> > Date: Sat, 18 Jun 2022 14:49:49 +0200 > > Stefan Monnier <monnier@iro.umontreal.ca> writes: > > > Obviously this process ends up with a very large one-step allocation > > which is arguably interesting in itself (I suspect there's some > > GC-inhibition going on there), but the more interesting point > > is that with p=0.1 we get an amount of free space after GC > > (i.e. blocks we can't release, because of fragmentation) that's about as > > large as the next threshold, i.e. about 10%, whereas with p=1.0 this > > amount of unreleasable free space is significantly higher. > > As far as I know, we never actually release back cons space to the OS > anyway -- even if we could. (Because glibc's malloc implementation > doesn't do that unless we call that trim function, which we don't.) AFAIK, that's not really true. We call 'free' and glibc does release to the OS when it can. And don't forget that Emacs calls 'malloc' a lot not just for conses and other Lisp data. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 13:20 ` Eli Zaretskii @ 2022-06-19 11:02 ` Lars Ingebrigtsen 2022-06-19 11:11 ` Eli Zaretskii 2022-06-21 2:01 ` Lynn Winebarger 0 siblings, 2 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-19 11:02 UTC (permalink / raw) To: Eli Zaretskii Cc: monnier, yantar92, mattiase, theophilusx, rms, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > AFAIK, that's not really true. We call 'free' and glibc does release > to the OS when it can. And don't forget that Emacs calls 'malloc' a > lot not just for conses and other Lisp data. Wasn't there a huge discussion about this a couple months ago and the conclusion was that glibc rarely releases memory back to the OS? That's why we added `malloc-trim'. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-19 11:02 ` Lars Ingebrigtsen @ 2022-06-19 11:11 ` Eli Zaretskii 2022-06-21 2:01 ` Lynn Winebarger 1 sibling, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-19 11:11 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: monnier, yantar92, mattiase, theophilusx, rms, acm, emacs-devel > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: monnier@iro.umontreal.ca, yantar92@gmail.com, mattiase@acm.org, > theophilusx@gmail.com, rms@gnu.org, acm@muc.de, emacs-devel@gnu.org > Date: Sun, 19 Jun 2022 13:02:59 +0200 > > Eli Zaretskii <eliz@gnu.org> writes: > > > AFAIK, that's not really true. We call 'free' and glibc does release > > to the OS when it can. And don't forget that Emacs calls 'malloc' a > > lot not just for conses and other Lisp data. > > Wasn't there a huge discussion about this a couple months ago and the > conclusion was that glibc rarely releases memory back to the OS? That's > why we added `malloc-trim'. Well, "rarely" is not the same as "never". How rare is "rarely" really depends on the specific use patterns, and I don't think we have ever established what kinds of uses cause us to release memory more frequently than others. Maybe it's worth our while to collect some data about this? For example, frequent contributors who tend to run Emacs for many days could perhaps take a snapshot of the total VM of the Emacs process and see how it behaves. Then we'd know how frequently the footprint goes down and by how much. I can tell from my own experience that, when I purge my mail INBOX, typically leaving about 2500 email messages in it down from ~4500, the memory footprint of the Emacs process goes down from about 620 MB to about 520-550 MB. But this is not with glibc. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-19 11:02 ` Lars Ingebrigtsen 2022-06-19 11:11 ` Eli Zaretskii @ 2022-06-21 2:01 ` Lynn Winebarger 2022-06-22 0:01 ` Lynn Winebarger 2022-06-22 23:26 ` Stefan Monnier 1 sibling, 2 replies; 505+ messages in thread From: Lynn Winebarger @ 2022-06-21 2:01 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Eli Zaretskii, Stefan Monnier, Ihor Radchenko, Mattias Engdegård, Tim Cross, rms, Alan Mackenzie, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1198 bytes --] On Sun, Jun 19, 2022, 7:05 AM Lars Ingebrigtsen <larsi@gnus.org> wrote: > Eli Zaretskii <eliz@gnu.org> writes: > > > AFAIK, that's not really true. We call 'free' and glibc does release > > to the OS when it can. And don't forget that Emacs calls 'malloc' a > > lot not just for conses and other Lisp data. > > Wasn't there a huge discussion about this a couple months ago and the > conclusion was that glibc rarely releases memory back to the OS? That's > why we added `malloc-trim'. > Was there? I see:. https://debbugs.gnu.org/cgi/bugreport.cgi?bug=38345#71 but it's a little more than a couple of months ago. I'm pretty curious because if I accumulate a large buffer of trace output (running memory up to 100s of MB), killing the buffer doesn't seem to impact gc time substantially. I would think using mmap-based allocation would make release of memory to the system more likely. But mmap is only used for 64+KiB size blocks, and the trim threshold is 128KiB blocks (shouldn't it be lower than the allocation size?) I'm also interested in the improvement in performance reported in that thread from using the jemalloc implementation. Has that been investigated further? Thanks, Lynn [-- Attachment #2: Type: text/html, Size: 2007 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-21 2:01 ` Lynn Winebarger @ 2022-06-22 0:01 ` Lynn Winebarger 2022-06-22 12:59 ` Eli Zaretskii 2022-06-22 23:26 ` Stefan Monnier 1 sibling, 1 reply; 505+ messages in thread From: Lynn Winebarger @ 2022-06-22 0:01 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Eli Zaretskii, Stefan Monnier, Ihor Radchenko, Mattias Engdegård, Tim Cross, rms, Alan Mackenzie, emacs-devel [-- Attachment #1: Type: text/plain, Size: 2552 bytes --] On Mon, Jun 20, 2022 at 10:01 PM Lynn Winebarger <owinebar@gmail.com> wrote: > On Sun, Jun 19, 2022, 7:05 AM Lars Ingebrigtsen <larsi@gnus.org> wrote: > >> Eli Zaretskii <eliz@gnu.org> writes: >> >> > AFAIK, that's not really true. We call 'free' and glibc does release >> > to the OS when it can. And don't forget that Emacs calls 'malloc' a >> > lot not just for conses and other Lisp data. >> >> Wasn't there a huge discussion about this a couple months ago and the >> conclusion was that glibc rarely releases memory back to the OS? That's >> why we added `malloc-trim'. >> > > Was there? I see:. > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=38345#71 > but it's a little more than a couple of months ago. > I'm pretty curious because if I accumulate a large buffer of trace output > (running memory up to 100s of MB), killing the buffer doesn't seem to > impact gc time substantially. > I would think using mmap-based allocation would make release of memory to > the system more likely. But mmap is only used for 64+KiB size blocks, and > the trim threshold is 128KiB blocks (shouldn't it be lower than the > allocation size?) > I'm also interested in the improvement in performance reported in that > thread from using the jemalloc implementation. Has that been investigated > further? > > I tried jemalloc per Ihor's description, using LD_PRELOAD to have it replace the system malloc without having to change or even recompile any emacs code. My test was to start a shell in a buffer, and then cat about 32MB of csv data to the buffer. It seemed like a reasonable approximation of generating large buffers of tracing output for this purpose. I started emacs 28.1 from a build directory with MALLOC_CONF=background_thread:true,retain:false ./src/emacs As expected, the emacs process slowly added about 50MB of memory to RSS. After the cat had completed, I killed the buffer and ran (garbage-collect). The RSS and VSZ of the process dropped about 40MB. Not complete reclamation, but I expect that it would solve the problem of a permanent bump in gc pause time. For some reason, the configure script chooses not to use mmap for malloc. jemalloc is designed to use mmap as much as possible so it isn't limited to freeing only the uppermost regions. I can't tell what (if any) option allows me to overrule the configure scripts decision to use sbrk instead of mmap. That might help make the mallopt knobs more effective. Are there any formal memory-allocation/reclamation benchmarks that I should use instead of this ad hoc task? Lynn [-- Attachment #2: Type: text/html, Size: 3735 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-22 0:01 ` Lynn Winebarger @ 2022-06-22 12:59 ` Eli Zaretskii 2022-06-22 23:30 ` Lynn Winebarger 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-22 12:59 UTC (permalink / raw) To: Lynn Winebarger Cc: larsi, monnier, yantar92, mattiase, theophilusx, rms, acm, emacs-devel > From: Lynn Winebarger <owinebar@gmail.com> > Date: Tue, 21 Jun 2022 20:01:43 -0400 > Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>, > Ihor Radchenko <yantar92@gmail.com>, Mattias Engdegård <mattiase@acm.org>, > Tim Cross <theophilusx@gmail.com>, rms@gnu.org, Alan Mackenzie <acm@muc.de>, > emacs-devel <emacs-devel@gnu.org> > > For some reason, the configure script chooses not to use mmap for malloc. The configure script chooses not to use mmap directly because glibc's malloc already does that internally where it decides that to be beneficial, and because using mmap directly has some disadvantages we'd like to avoid if possible. > jemalloc is designed to use mmap as much as possible So is glibc. > so it isn't limited to freeing only the uppermost regions. Neither is glibc, although it uses different algorithms. > I can't tell what (if any) option allows me to overrule the configure > scripts decision to use sbrk instead of mmap. You don't want that, not on GNU/Linux systems. We switched away of sbrk and mmap for several good reasons. > Are there any formal memory-allocation/reclamation benchmarks that I should > use instead of this ad hoc task? Not that I know of. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-22 12:59 ` Eli Zaretskii @ 2022-06-22 23:30 ` Lynn Winebarger 2022-06-23 7:09 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Lynn Winebarger @ 2022-06-22 23:30 UTC (permalink / raw) To: Eli Zaretskii Cc: Lars Ingebrigtsen, Stefan Monnier, Ihor Radchenko, Mattias Engdegård, Tim Cross, rms, Alan Mackenzie, emacs-devel [-- Attachment #1: Type: text/plain, Size: 2230 bytes --] On Wed, Jun 22, 2022 at 8:59 AM Eli Zaretskii <eliz@gnu.org> wrote: > > From: Lynn Winebarger <owinebar@gmail.com> > > Date: Tue, 21 Jun 2022 20:01:43 -0400 > > Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Monnier < > monnier@iro.umontreal.ca>, > > Ihor Radchenko <yantar92@gmail.com>, Mattias Engdegård < > mattiase@acm.org>, > > Tim Cross <theophilusx@gmail.com>, rms@gnu.org, Alan Mackenzie < > acm@muc.de>, > > emacs-devel <emacs-devel@gnu.org> > > > > For some reason, the configure script chooses not to use mmap for malloc. > > The configure script chooses not to use mmap directly because glibc's > malloc already does that internally where it decides that to be > beneficial, and because using mmap directly has some disadvantages > we'd like to avoid if possible. > > > jemalloc is designed to use mmap as much as possible > > So is glibc. > Wow, so it is - when I tried the same test without jemalloc, both 28.1 and 24.3 gave similar results. I must have misinterpreted the behavior of edebug's trace-buffer as being similar to issues I've had previously (probably due to not using an explicit garbage-collect, or killing rather than deleting text with undo disabled). Maybe I just buried the trace buffer instead of killing it. > so it isn't limited to freeing only the uppermost regions. > > Neither is glibc, although it uses different algorithms. > > > I can't tell what (if any) option allows me to overrule the configure > > scripts decision to use sbrk instead of mmap. > > You don't want that, not on GNU/Linux systems. We switched away of > sbrk and mmap for several good reasons. > Not that I don't enjoy trawling through old threads in the emacs-devel archive (searching for mmap led to a thread from the introduction of the portable dumper), but it would be a lot easier if there were design notes somewhere recording the rationale of the decisions reflected in the current code. The best I can do now is comb through the arguments on the list (and maybe in the bug tracker?) and try to guess which points ended up getting reflected in the code. I will say jemalloc appears to have a lot of heap profiling bells and whistles. [-- Attachment #2: Type: text/html, Size: 3532 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-22 23:30 ` Lynn Winebarger @ 2022-06-23 7:09 ` Eli Zaretskii 2022-06-23 7:18 ` Ihor Radchenko 2022-06-23 17:07 ` Lynn Winebarger 0 siblings, 2 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-23 7:09 UTC (permalink / raw) To: Lynn Winebarger Cc: larsi, monnier, yantar92, mattiase, theophilusx, rms, acm, emacs-devel > From: Lynn Winebarger <owinebar@gmail.com> > Date: Wed, 22 Jun 2022 19:30:20 -0400 > Cc: Lars Ingebrigtsen <larsi@gnus.org>, Stefan Monnier <monnier@iro.umontreal.ca>, > Ihor Radchenko <yantar92@gmail.com>, Mattias Engdegård <mattiase@acm.org>, > Tim Cross <theophilusx@gmail.com>, rms@gnu.org, Alan Mackenzie <acm@muc.de>, > emacs-devel <emacs-devel@gnu.org> > > > I can't tell what (if any) option allows me to overrule the configure > > scripts decision to use sbrk instead of mmap. > > You don't want that, not on GNU/Linux systems. We switched away of > sbrk and mmap for several good reasons. > > Not that I don't enjoy trawling through old threads in the emacs-devel archive > (searching for mmap led to a thread from the introduction of the portable dumper), > but it would be a lot easier if there were design notes somewhere recording > the rationale of the decisions reflected in the current code. We lack someone in the role of the "project historian", who would then summarize and publish the design discussions in the form of such notes. Volunteers are most welcome. > The best I can do now > is comb through the arguments on the list (and maybe in the bug tracker?) and > try to guess which points ended up getting reflected in the code. A better method is to use Git logs to find when the GNU/Linux build stopped using mmap, and then search the archives around that time. > I will say jemalloc appears to have a lot of heap profiling bells and whistles. AFAIU, so does glibc. Some of them where mentioned in the discussions you already unearthed, AFAIR. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 7:09 ` Eli Zaretskii @ 2022-06-23 7:18 ` Ihor Radchenko 2022-06-23 7:37 ` Eli Zaretskii 2022-06-23 17:07 ` Lynn Winebarger 1 sibling, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-23 7:18 UTC (permalink / raw) To: Eli Zaretskii Cc: Lynn Winebarger, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> Not that I don't enjoy trawling through old threads in the emacs-devel archive >> (searching for mmap led to a thread from the introduction of the portable dumper), >> but it would be a lot easier if there were design notes somewhere recording >> the rationale of the decisions reflected in the current code. > > We lack someone in the role of the "project historian", who would then > summarize and publish the design discussions in the form of such > notes. Volunteers are most welcome. What about simply following a convention to link relevant discussion in the commit message? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 7:18 ` Ihor Radchenko @ 2022-06-23 7:37 ` Eli Zaretskii 2022-06-23 8:02 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-23 7:37 UTC (permalink / raw) To: Ihor Radchenko Cc: owinebar, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Lynn Winebarger <owinebar@gmail.com>, larsi@gnus.org, > monnier@iro.umontreal.ca, mattiase@acm.org, theophilusx@gmail.com, > rms@gnu.org, acm@muc.de, emacs-devel@gnu.org > Date: Thu, 23 Jun 2022 15:18:37 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> Not that I don't enjoy trawling through old threads in the emacs-devel archive > >> (searching for mmap led to a thread from the introduction of the portable dumper), > >> but it would be a lot easier if there were design notes somewhere recording > >> the rationale of the decisions reflected in the current code. > > > > We lack someone in the role of the "project historian", who would then > > summarize and publish the design discussions in the form of such > > notes. Volunteers are most welcome. > > What about simply following a convention to link relevant discussion in > the commit message? That convention does exist, and the results of its use are before our eyes ;-). It doesn't help that Git doesn't support retroactive modifications of commit logs, except via cumbersome workarounds that no one seems to like using. And that won't help with past discussions whose commits are in the tree for many years. But yes, using such links is definitely helpful, and I'd encourage people to use that more. (That's one reason why changes should preferably be discussed on the bug tracker: the bug number -- if not forgotten to be mentioned in the log message -- serves as a pointer to the discussions.) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 7:37 ` Eli Zaretskii @ 2022-06-23 8:02 ` Ihor Radchenko 2022-06-23 8:24 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-23 8:02 UTC (permalink / raw) To: Eli Zaretskii Cc: owinebar, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> What about simply following a convention to link relevant discussion in >> the commit message? > > That convention does exist, and the results of its use are before our > eyes ;-). It doesn't help that Git doesn't support retroactive > modifications of commit logs, except via cumbersome workarounds that > no one seems to like using. Good to know. > And that won't help with past discussions whose commits are in the > tree for many years. Can't the past discussions be directly updated by sending an extra email that links back to the new commit? > But yes, using such links is definitely helpful, and I'd encourage > people to use that more. (That's one reason why changes should > preferably be discussed on the bug tracker: the bug number -- if not > forgotten to be mentioned in the log message -- serves as a pointer to > the discussions.) What about the message ID? It should also be unique enough to link to the relevant ML thread. (it is a bit annoying that lists.gnu.org does not support easy search via link id, unlike e.g. https://yhetil.org/emacs-devel/87r13n5muk.fsf@gnus.org/). Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 8:02 ` Ihor Radchenko @ 2022-06-23 8:24 ` Eli Zaretskii 2022-06-23 9:03 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-23 8:24 UTC (permalink / raw) To: Ihor Radchenko Cc: owinebar, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: owinebar@gmail.com, larsi@gnus.org, monnier@iro.umontreal.ca, > mattiase@acm.org, theophilusx@gmail.com, rms@gnu.org, acm@muc.de, > emacs-devel@gnu.org > Date: Thu, 23 Jun 2022 16:02:36 +0800 > > > And that won't help with past discussions whose commits are in the > > tree for many years. > > Can't the past discussions be directly updated by sending an extra email > that links back to the new commit? You mean, rely on search engines to find an addition to a discussion made many years later? My experience with searching the archives is that it many times fails when the discussion crosses the year mark. For starters, display of the archives by thread doesn't work in that case as expected: you are given the illusion that the thread ends. Searching the archives also has the disadvantage that in many cases it's hard to know what are the keywords that would find the discussion efficiently (i.e. without drowning it in thousands of irrelevant messages). Otherwise, I don't see how sending a message to a thread long dead would help in this matter. Am I missing something? > > But yes, using such links is definitely helpful, and I'd encourage > > people to use that more. (That's one reason why changes should > > preferably be discussed on the bug tracker: the bug number -- if not > > forgotten to be mentioned in the log message -- serves as a pointer to > > the discussions.) > > What about the message ID? It should also be unique enough to link to > the relevant ML thread. (it is a bit annoying that lists.gnu.org does > not support easy search via link id, unlike e.g. > https://yhetil.org/emacs-devel/87r13n5muk.fsf@gnus.org/). That's okay, but is it really better than a URL of a message in the archives? It isn't much shorter. The basic problem is to remember to mention a discussion, in any form. Even just a date and the name of the forum are usually enough to quickly find the relevant thread. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 8:24 ` Eli Zaretskii @ 2022-06-23 9:03 ` Ihor Radchenko 2022-06-23 10:08 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-23 9:03 UTC (permalink / raw) To: Eli Zaretskii Cc: owinebar, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> Can't the past discussions be directly updated by sending an extra email >> that links back to the new commit? > > You mean, rely on search engines to find an addition to a discussion > made many years later? My experience with searching the archives is > that it many times fails when the discussion crosses the year mark. > For starters, display of the archives by thread doesn't work in that > case as expected: you are given the illusion that the thread ends. Are you referring to lists.gnu.org thread display interface? If it is as buggy as you are describing, why don't you file a bug report the relevant ML and get the problem fixed? There are alternatives. I personally prefer to search using local maildir copy via notmuch. I believe that gnus should also be able to search across the maildir. Also, there is https://yhetil.org/ with somewhat better search and display interface (IMHO). See https://yhetil.org/emacs-devel/_/text/help/ > Searching the archives also has the disadvantage that in many cases > it's hard to know what are the keywords that would find the discussion > efficiently (i.e. without drowning it in thousands of irrelevant > messages). When I was saying "links back to the new commit", I was referring to the unique commit number. Searching exact match will not give false positives then. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 9:03 ` Ihor Radchenko @ 2022-06-23 10:08 ` Eli Zaretskii 2022-06-25 5:10 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-23 10:08 UTC (permalink / raw) To: Ihor Radchenko Cc: owinebar, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: owinebar@gmail.com, larsi@gnus.org, monnier@iro.umontreal.ca, > mattiase@acm.org, theophilusx@gmail.com, rms@gnu.org, acm@muc.de, > emacs-devel@gnu.org > Date: Thu, 23 Jun 2022 17:03:35 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> Can't the past discussions be directly updated by sending an extra email > >> that links back to the new commit? > > > > You mean, rely on search engines to find an addition to a discussion > > made many years later? My experience with searching the archives is > > that it many times fails when the discussion crosses the year mark. > > For starters, display of the archives by thread doesn't work in that > > case as expected: you are given the illusion that the thread ends. > > Are you referring to lists.gnu.org thread display interface? I mean it, but also every other mailing list software I ever used. They all have these problems, and some more than others. > If it is as buggy as you are describing, why don't you file a bug > report the relevant ML and get the problem fixed? Because I don't believe this is fixable in practice. > There are alternatives. I personally prefer to search using local > maildir copy via notmuch. I believe that gnus should also be able to > search across the maildir. I have local search set up as well, but that only works on messages I decided to archive locally. If you think I'm archiving every forum in which I participate, think again ;-) > Also, there is https://yhetil.org/ with somewhat better search and > display interface (IMHO). See > https://yhetil.org/emacs-devel/_/text/help/ How is it significantly better? > > Searching the archives also has the disadvantage that in many cases > > it's hard to know what are the keywords that would find the discussion > > efficiently (i.e. without drowning it in thousands of irrelevant > > messages). > > When I was saying "links back to the new commit", I was referring to the > unique commit number. Searching exact match will not give false > positives then. What do you do with this when we change the VCS (which happened twice already)? And even if you are only talking about Git SHA signatures, the "no false positives" is not guaranteed, when messages provide only the 7-digit SHA. And in any case, the trigger for this discussion was the situation where you want to answer questions like "why did Emacs stop using sbrk on GNU/Linux", in which case there's no commit ID to search for. Moreover, I don't understand the proposal in general: are you suggesting that every commit related to a discussion would somehow send a message to the thread with the commit ID? If so, how would this work better than our (constantly failing) practice of mentioning the bug number in the commit log messages? These measures don't really work without a gatekeeper. Which we don't have, and probably never will. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 10:08 ` Eli Zaretskii @ 2022-06-25 5:10 ` Ihor Radchenko 2022-06-25 6:03 ` Eli Zaretskii 2022-06-25 7:51 ` Yuri Khan 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-25 5:10 UTC (permalink / raw) To: Eli Zaretskii Cc: owinebar, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> > You mean, rely on search engines to find an addition to a discussion >> > made many years later? My experience with searching the archives is >> > that it many times fails when the discussion crosses the year mark. >> > For starters, display of the archives by thread doesn't work in that >> > case as expected: you are given the illusion that the thread ends. >> >> Are you referring to lists.gnu.org thread display interface? > > I mean it, but also every other mailing list software I ever used. > They all have these problems, and some more than others. > >> If it is as buggy as you are describing, why don't you file a bug >> report the relevant ML and get the problem fixed? > > Because I don't believe this is fixable in practice. Then it raises a bigger question. Do we have anything better than email archives to archive emacs-devel? >> There are alternatives. I personally prefer to search using local >> maildir copy via notmuch. I believe that gnus should also be able to >> search across the maildir. > > I have local search set up as well, but that only works on messages I > decided to archive locally. If you think I'm archiving every forum in > which I participate, think again ;-) I did expect that you at least archive emacs-devel (: >> Also, there is https://yhetil.org/ with somewhat better search and >> display interface (IMHO). See >> https://yhetil.org/emacs-devel/_/text/help/ > > How is it significantly better? Search is more flexible because it at least allows search by date and by some more email fields. Compare search section of https://yhetil.org/emacs-devel/_/text/help/ and https://lists.gnu.org/archive/cgi-bin/namazu.cgi?idxname=bug-gnu-emacs >> > Searching the archives also has the disadvantage that in many cases >> > it's hard to know what are the keywords that would find the discussion >> > efficiently (i.e. without drowning it in thousands of irrelevant >> > messages). >> >> When I was saying "links back to the new commit", I was referring to the >> unique commit number. Searching exact match will not give false >> positives then. > > What do you do with this when we change the VCS (which happened twice > already)? Of course, things will break if we change VCS, but it should not stop us from linking to commits - not linking at all is still worse. Note that I can also ask "what if we change debbugs?". Would it mean that we should not link to bug#? > And even if you are only talking about Git SHA signatures, the "no > false positives" is not guaranteed, when messages provide only the > 7-digit SHA. Sure. But false positives will not be numerous. You can expect a few max - not a big deal to check one by one. > And in any case, the trigger for this discussion was the situation > where you want to answer questions like "why did Emacs stop using sbrk > on GNU/Linux", in which case there's no commit ID to search for. I implied using git log search to identify the relevant commits. > Moreover, I don't understand the proposal in general: are you > suggesting that every commit related to a discussion would somehow > send a message to the thread with the commit ID? If so, how would > this work better than our (constantly failing) practice of mentioning > the bug number in the commit log messages? I do not imply that we should do it _instead_ of mentioning bug number. It's complementary and can be very useful (I know because I did have a hard time finding commits that fixed particular bugs). Also, bugs already need to be closed manually - that closing email directive could as well include the commit hash. > These measures don't really work without a gatekeeper. Which we don't > have, and probably never will. I do not think that it is so much of an issue. Your argument would also apply, e.g. to using double space between sentences in the commit messages or to following log entry format. Yet people do follow these conventions because they are documented in CONTRIBUTE file and because non-compliant commits are being scolded. Not 100%, but frequently enough to make the conventions useful. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-25 5:10 ` Ihor Radchenko @ 2022-06-25 6:03 ` Eli Zaretskii 2022-06-27 9:32 ` Ihor Radchenko 2022-06-25 7:51 ` Yuri Khan 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-25 6:03 UTC (permalink / raw) To: Ihor Radchenko Cc: owinebar, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: owinebar@gmail.com, larsi@gnus.org, monnier@iro.umontreal.ca, > mattiase@acm.org, theophilusx@gmail.com, rms@gnu.org, acm@muc.de, > emacs-devel@gnu.org > Date: Sat, 25 Jun 2022 13:10:19 +0800 > > >> > For starters, display of the archives by thread doesn't work in that > >> > case as expected: you are given the illusion that the thread ends. > >> > >> Are you referring to lists.gnu.org thread display interface? > > > > I mean it, but also every other mailing list software I ever used. > > They all have these problems, and some more than others. > > > >> If it is as buggy as you are describing, why don't you file a bug > >> report the relevant ML and get the problem fixed? > > > > Because I don't believe this is fixable in practice. > > Then it raises a bigger question. Do we have anything better than email > archives to archive emacs-devel? I don't know of any. > >> https://yhetil.org/emacs-devel/_/text/help/ > > > > How is it significantly better? > > Search is more flexible because it at least allows search by date and by > some more email fields. > Compare search section of https://yhetil.org/emacs-devel/_/text/help/ > and https://lists.gnu.org/archive/cgi-bin/namazu.cgi?idxname=bug-gnu-emacs I don't see anything that would make the search more efficient. In particular, all the different method of searching for matches in various parts of the message or in practice of no help at all, since I basically have no idea where the keyword appeared when I start searching. > Note that I can also ask "what if we change debbugs?". Would it mean > that we should not link to bug#? The bug number gives you a quick way of reaching the bug discussion in the email archives of bug-gnu-emacs, and in debbugs itself. These will remain available even if we switch away from debbugs as our issue tracker. So the bug number is definitely a good thing to have in the logs. > > And in any case, the trigger for this discussion was the situation > > where you want to answer questions like "why did Emacs stop using sbrk > > on GNU/Linux", in which case there's no commit ID to search for. > > I implied using git log search to identify the relevant commits. That doesn't work well IME. Would you mind walking me though using that when trying to answer the above question about sbrk? > > These measures don't really work without a gatekeeper. Which we don't > > have, and probably never will. > > I do not think that it is so much of an issue. Your argument would also > apply, e.g. to using double space between sentences in the commit > messages or to following log entry format. Yet people do follow these > conventions because they are documented in CONTRIBUTE file and because > non-compliant commits are being scolded. Not 100%, but frequently enough > to make the conventions useful. Two spaces between sentences is simple enough that we have several eyes looking for that and fixing problems post-mortem. And yet we still have quite a few of them, just try searching in the Emacs tree. Conventions that are harder to spot are followed even less. As a matter of fact, I have hard time making sure each commit that has a bug number mentions that number in the commit log message. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-25 6:03 ` Eli Zaretskii @ 2022-06-27 9:32 ` Ihor Radchenko 2022-06-27 13:20 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-27 9:32 UTC (permalink / raw) To: Eli Zaretskii Cc: owinebar, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> Then it raises a bigger question. Do we have anything better than email >> archives to archive emacs-devel? > > I don't know of any. What about Mailman 3 frontends like Hyperkitty? https://wiki.archlinux.org/title/Hyperkitty#Xapian_search_backend >> Note that I can also ask "what if we change debbugs?". Would it mean >> that we should not link to bug#? > > The bug number gives you a quick way of reaching the bug discussion in > the email archives of bug-gnu-emacs, and in debbugs itself. These > will remain available even if we switch away from debbugs as our issue > tracker. So the bug number is definitely a good thing to have in the > logs. Will the commit hash not be available e.g. on savannah? Do you intend to remove the git history alltogether during the switch? >> > And in any case, the trigger for this discussion was the situation >> > where you want to answer questions like "why did Emacs stop using sbrk >> > on GNU/Linux", in which case there's no commit ID to search for. >> >> I implied using git log search to identify the relevant commits. > > That doesn't work well IME. Would you mind walking me though using > that when trying to answer the above question about sbrk? 1. I listed all the commits that mentioned sbrk in the messages using git log, displaying the statistics. (actually, I used Magit) 2. I noticed that some of the commits changed large number of lines and removed sbrk-related staff. In particular, d12e5d003d Add portable dumper 3. I searched https://yhetil.org/emacs-devel/?q=linux+sbrk and noticed two candidate threads: Re: When should ralloc.c be used?; and Re: Dumper issue, revisited; invalid realloc/free; 4. Looking through the threads; it appears that the pdumper thread had a lot of occurrences of sbrk word: https://yhetil.org/emacs-devel/?q=%22sbrk%22 5. I skimmed through the thread and possibly found relevant message root: https://yhetil.org/emacs-devel/20140625212421.GD179@brightrain.aerifal.cx/ It would have been even easier if I had an idea what sbrk does and where its Linux support is supposed to be in the code. >> > These measures don't really work without a gatekeeper. Which we don't >> > have, and probably never will. >> >> I do not think that it is so much of an issue. Your argument would also >> apply, e.g. to using double space between sentences in the commit >> messages or to following log entry format. Yet people do follow these >> conventions because they are documented in CONTRIBUTE file and because >> non-compliant commits are being scolded. Not 100%, but frequently enough >> to make the conventions useful. > > Two spaces between sentences is simple enough that we have several > eyes looking for that and fixing problems post-mortem. And yet we > still have quite a few of them, just try searching in the Emacs tree. > Conventions that are harder to spot are followed even less. > > As a matter of fact, I have hard time making sure each commit that has > a bug number mentions that number in the commit log message. On Org side, I do not find tracking such things too difficult. Probably due to lesser volume of commits. Note that things like double space, or even bug number checks could be automated. For example, one can write a make target that checks recent commit messages to follow specific rules. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-27 9:32 ` Ihor Radchenko @ 2022-06-27 13:20 ` Eli Zaretskii 2022-06-29 9:35 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-27 13:20 UTC (permalink / raw) To: Ihor Radchenko Cc: owinebar, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: owinebar@gmail.com, larsi@gnus.org, monnier@iro.umontreal.ca, > mattiase@acm.org, theophilusx@gmail.com, rms@gnu.org, acm@muc.de, > emacs-devel@gnu.org > Date: Mon, 27 Jun 2022 17:32:05 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> Then it raises a bigger question. Do we have anything better than email > >> archives to archive emacs-devel? > > > > I don't know of any. > > What about Mailman 3 frontends like Hyperkitty? > https://wiki.archlinux.org/title/Hyperkitty#Xapian_search_backend Not sure what that means in practice. Do you mean I should install that locally? Or what else? You asked, above, do we have a better way of archiving a mailing list, and the solution you propose seems to be yet another way of keeping email archives? I must be missing something. > >> Note that I can also ask "what if we change debbugs?". Would it mean > >> that we should not link to bug#? > > > > The bug number gives you a quick way of reaching the bug discussion in > > the email archives of bug-gnu-emacs, and in debbugs itself. These > > will remain available even if we switch away from debbugs as our issue > > tracker. So the bug number is definitely a good thing to have in the > > logs. > > Will the commit hash not be available e.g. on savannah? Do you intend to > remove the git history alltogether during the switch? No, but I don't want to look in more than one repository, either. > >> > And in any case, the trigger for this discussion was the situation > >> > where you want to answer questions like "why did Emacs stop using sbrk > >> > on GNU/Linux", in which case there's no commit ID to search for. > >> > >> I implied using git log search to identify the relevant commits. > > > > That doesn't work well IME. Would you mind walking me though using > > that when trying to answer the above question about sbrk? > > 1. I listed all the commits that mentioned sbrk in the messages using > git log, displaying the statistics. (actually, I used Magit) > 2. I noticed that some of the commits changed large number of lines and > removed sbrk-related staff. In particular, d12e5d003d Add portable > dumper > 3. I searched https://yhetil.org/emacs-devel/?q=linux+sbrk and noticed > two candidate threads: Re: When should ralloc.c be used?; and Re: > Dumper issue, revisited; invalid realloc/free; How did you decide that these threads are relevant? > 4. Looking through the threads; it appears that the pdumper thread had a > lot of occurrences of sbrk word: > https://yhetil.org/emacs-devel/?q=%22sbrk%22 > 5. I skimmed through the thread and possibly found relevant message > root: > https://yhetil.org/emacs-devel/20140625212421.GD179@brightrain.aerifal.cx/ How much time did this take you? > It would have been even easier if I had an idea what sbrk does and where > its Linux support is supposed to be in the code. sbrk is a system call, so I guess by "its Linux support" you mean where we used to use it Emacs? The answer to that is not always easy to find in practice, unless you happen to know. > > As a matter of fact, I have hard time making sure each commit that has > > a bug number mentions that number in the commit log message. > > On Org side, I do not find tracking such things too difficult. Probably > due to lesser volume of commits. What do you mean by "tracking"? By "making sure" I meant the job of having people who have write access mention the bug number. Detecting they missed after the commit is pushed is too late. > Note that things like double space, or even bug number checks could be > automated. For example, one can write a make target that checks recent > commit messages to follow specific rules. Commit log messages in Git are immutable, so looking up such problems post-factum is not really useful. But that is a tangent; the main point here is that we have quite a few good practices and conventions in place, but have problems living up to them. So any procedure that relies too heavily on these conventions and practices will not be too successful, unless what it relies on is completely automated. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-27 13:20 ` Eli Zaretskii @ 2022-06-29 9:35 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-29 9:35 UTC (permalink / raw) To: Eli Zaretskii Cc: owinebar, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> >> Then it raises a bigger question. Do we have anything better than email >> >> archives to archive emacs-devel? >> > >> > I don't know of any. >> >> What about Mailman 3 frontends like Hyperkitty? >> https://wiki.archlinux.org/title/Hyperkitty#Xapian_search_backend > > Not sure what that means in practice. Do you mean I should install > that locally? Or what else? > > You asked, above, do we have a better way of archiving a mailing > list, and the solution you propose seems to be yet another way of > keeping email archives? I must be missing something. I am trying to challenge your statement that thread display is bad in mailing list web frontends. I fail to believe that the problem with year crossing is impossible to solve. So, I suggested that a newer Mailman frontend might not have the described problem. Can we use it in lists.gnu.org instead of the older web interface? >> Will the commit hash not be available e.g. on savannah? Do you intend to >> remove the git history alltogether during the switch? > > No, but I don't want to look in more than one repository, either. My initial suggestion was bi-directional linking between the commit messages and the relevant ML discussions. There is already a convention to link from commit messages to debbugs, but it would be useful to have the backlinks from debbugs/ML to the relevant commits. Via commit hash or possibly via direct savannah link (whatever is more permanent). I do not think that there is any way to have the commits directly in debbugs. (Or am I wrong?) >> > That doesn't work well IME. Would you mind walking me though using >> > that when trying to answer the above question about sbrk? >> >> 1. I listed all the commits that mentioned sbrk in the messages using >> git log, displaying the statistics. (actually, I used Magit) >> 2. I noticed that some of the commits changed large number of lines and >> removed sbrk-related staff. In particular, d12e5d003d Add portable >> dumper >> 3. I searched https://yhetil.org/emacs-devel/?q=linux+sbrk and noticed >> two candidate threads: Re: When should ralloc.c be used?; and Re: >> Dumper issue, revisited; invalid realloc/free; > > How did you decide that these threads are relevant? ralloc.c and pdumper where the keywords used both in the commits and in the threads. And there were significant number of matching messages in the threads. >> 4. Looking through the threads; it appears that the pdumper thread had a >> lot of occurrences of sbrk word: >> https://yhetil.org/emacs-devel/?q=%22sbrk%22 >> 5. I skimmed through the thread and possibly found relevant message >> root: >> https://yhetil.org/emacs-devel/20140625212421.GD179@brightrain.aerifal.cx/ > > How much time did this take you? The total time I spent replying to your email was 42 minutes (recorded). Out of those 42 minutes I subjectively spent 10-15 minutes trying to answer "Would you mind walking me though using that when trying to answer the above question about sbrk?". Not more than 25 minutes max. >> > As a matter of fact, I have hard time making sure each commit that has >> > a bug number mentions that number in the commit log message. >> >> On Org side, I do not find tracking such things too difficult. Probably >> due to lesser volume of commits. > > What do you mean by "tracking"? By "making sure" I meant the job of > having people who have write access mention the bug number. Detecting > they missed after the commit is pushed is too late. >> Note that things like double space, or even bug number checks could be >> automated. For example, one can write a make target that checks recent >> commit messages to follow specific rules. > > Commit log messages in Git are immutable, so looking up such problems > post-factum is not really useful. But that is a tangent; the main > point here is that we have quite a few good practices and conventions > in place, but have problems living up to them. So any procedure that > relies too heavily on these conventions and practices will not be too > successful, unless what it relies on is completely automated. You are right. I mostly had to deal with people who do not have write access. As for automatic checks for all the committers, what about git pre-push hooks (https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks)? If one can write make commit_check target, it can be configured to run before pushing to Emacs upstream. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-25 5:10 ` Ihor Radchenko 2022-06-25 6:03 ` Eli Zaretskii @ 2022-06-25 7:51 ` Yuri Khan 2022-06-25 8:28 ` Eli Zaretskii 1 sibling, 1 reply; 505+ messages in thread From: Yuri Khan @ 2022-06-25 7:51 UTC (permalink / raw) To: Ihor Radchenko Cc: Eli Zaretskii, owinebar, Lars Magne Ingebrigtsen, Stefan Monnier, Mattias Engdegård, theophilusx, Richard Stallman, Alan Mackenzie, Emacs developers On Sat, 25 Jun 2022 at 12:10, Ihor Radchenko <yantar92@gmail.com> wrote: > > What do you do with this when we change the VCS (which happened twice > > already)? > > Of course, things will break if we change VCS, but it should not stop us > from linking to commits - not linking at all is still worse. Identifying VCS commits in a way that is stable across VCS migration is at least theoretically a solved problem. http://esr.ibiblio.org/?p=3872 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-25 7:51 ` Yuri Khan @ 2022-06-25 8:28 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-25 8:28 UTC (permalink / raw) To: Yuri Khan Cc: yantar92, owinebar, larsi, monnier, mattiase, theophilusx, rms, acm, emacs-devel > From: Yuri Khan <yuri.v.khan@gmail.com> > Date: Sat, 25 Jun 2022 14:51:14 +0700 > Cc: Eli Zaretskii <eliz@gnu.org>, owinebar@gmail.com, > Lars Magne Ingebrigtsen <larsi@gnus.org>, Stefan Monnier <monnier@iro.umontreal.ca>, > Mattias Engdegård <mattiase@acm.org>, theophilusx@gmail.com, > Richard Stallman <rms@gnu.org>, Alan Mackenzie <acm@muc.de>, Emacs developers <emacs-devel@gnu.org> > > On Sat, 25 Jun 2022 at 12:10, Ihor Radchenko <yantar92@gmail.com> wrote: > > > > What do you do with this when we change the VCS (which happened twice > > > already)? > > > > Of course, things will break if we change VCS, but it should not stop us > > from linking to commits - not linking at all is still worse. > > Identifying VCS commits in a way that is stable across VCS migration > is at least theoretically a solved problem. > http://esr.ibiblio.org/?p=3872 We considered that in the past, and decided to use a different method, see CONTRIBUTE. And in any case, the results of following that recommendation are before our eyes... ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 7:09 ` Eli Zaretskii 2022-06-23 7:18 ` Ihor Radchenko @ 2022-06-23 17:07 ` Lynn Winebarger 2022-06-23 18:37 ` Eli Zaretskii 1 sibling, 1 reply; 505+ messages in thread From: Lynn Winebarger @ 2022-06-23 17:07 UTC (permalink / raw) To: Eli Zaretskii Cc: Lars Ingebrigtsen, Stefan Monnier, Ihor Radchenko, Mattias Engdegård, Tim Cross, rms, Alan Mackenzie, emacs-devel [-- Attachment #1: Type: text/plain, Size: 3503 bytes --] On Thu, Jun 23, 2022 at 3:09 AM Eli Zaretskii <eliz@gnu.org> wrote: > > From: Lynn Winebarger <owinebar@gmail.com> > > Not that I don't enjoy trawling through old threads in the emacs-devel > archive > > (searching for mmap led to a thread from the introduction of the > portable dumper), > > but it would be a lot easier if there were design notes somewhere > recording > > the rationale of the decisions reflected in the current code. > > We lack someone in the role of the "project historian", who would then > summarize and publish the design discussions in the form of such > notes. Volunteers are most welcome. > It's not the discussion (or history) per se, but the factors that decided the current design choices. An example would be the #ifdef DOUG_LEA_MALLOC blocks in lisp_align_malloc of alloc.c that prevent allocating via mmap. So far what I've read indicates that step is there to enable dumping by unexec (assuming I've understood correctly). Ideally there would be a design note (possibly generated from comments, possibly free standing) that would explain what the purpose of the restriction is, so you could tell if it's still relevant, and any negative impacts it has. Also, if there are plans to remove it at some point, what (if any) the requirements/test for removing it would be, etc. By "it", I mean any hypothetical design point, not just this particular one. Alternative designs could also be discussed along with the factors that led to their rejection, if it was significant enough. Now, those could be derived from mailing list discussions, but I wouldn't consider them "history". Per subsequent mails in this thread, if a developer or code reviewer made a practice of citing particular mail messages (or even other sources), some extraction process might even be automated, but then there would be the question of copyright assignment. It would help if there were some taxonomy of features/design points for reference. Is the bug database being used for that? Actually, a good example (even though it's more post facto description than prospective design & cost-benefit tradeoff analysis) would be https://github.com/rocky/elisp-bytecode That is really useful documentation that would ideally be in the emacs docs or etc directory. Andrea's development log and paper on the compiler are another example. The development log would benefit (in terms of use as design notes) from de-chronologizing them, figuring out the right taxonomy of the various topics, and gathering the material accordingly. I put in a request to assign@gnu.org a few days ago, but I have not received any response yet. > > > The best I can do now > > is comb through the arguments on the list (and maybe in the bug > tracker?) and > > try to guess which points ended up getting reflected in the code. > > A better method is to use Git logs to find when the GNU/Linux build > stopped using mmap, and then search the archives around that time. > > > I will say jemalloc appears to have a lot of heap profiling bells and > whistles. > > AFAIU, so does glibc. Some of them where mentioned in the discussions > you already unearthed, AFAIR. > I figured it would. jemalloc's maintainers just spend a lot of effort highlighting those features in the documentation. I wouldn't be surprised if it was 3/4 of the material on the package's site is dedicated to how to use jemalloc's profiling for debugging or optimizing memory allocation in the calling program. It makes an impression. Lynn [-- Attachment #2: Type: text/html, Size: 4654 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 17:07 ` Lynn Winebarger @ 2022-06-23 18:37 ` Eli Zaretskii 2022-06-23 22:08 ` Lynn Winebarger 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-23 18:37 UTC (permalink / raw) To: Lynn Winebarger Cc: larsi, monnier, yantar92, mattiase, theophilusx, rms, acm, emacs-devel > From: Lynn Winebarger <owinebar@gmail.com> > Date: Thu, 23 Jun 2022 13:07:44 -0400 > Cc: Lars Ingebrigtsen <larsi@gnus.org>, Stefan Monnier <monnier@iro.umontreal.ca>, > Ihor Radchenko <yantar92@gmail.com>, Mattias Engdegård <mattiase@acm.org>, > Tim Cross <theophilusx@gmail.com>, rms@gnu.org, Alan Mackenzie <acm@muc.de>, > emacs-devel <emacs-devel@gnu.org> > > > We lack someone in the role of the "project historian", who would then > > summarize and publish the design discussions in the form of such > > notes. Volunteers are most welcome. > > > > It's not the discussion (or history) per se, but the factors that decided > the current > design choices. An example would be the #ifdef DOUG_LEA_MALLOC blocks in > lisp_align_malloc > of alloc.c that prevent allocating via mmap. So far what I've read > indicates > that step is there to enable dumping by unexec (assuming I've understood > correctly). > Ideally there would be a design note (possibly generated from comments, > possibly free standing) > that would explain what the purpose of the restriction is, so you could > tell if it's still > relevant, and any negative impacts it has. Also, if there are plans to > remove it at > some point, what (if any) the requirements/test for removing it would be, > etc. By "it", > I mean any hypothetical design point, not just this particular one. > Alternative designs could also be discussed along with the factors that led > to their rejection, if it was significant enough. > Now, those could be derived from mailing list discussions, but I wouldn't > consider them > "history". Per subsequent mails in this thread, if a developer or code > reviewer made > a practice of citing particular mail messages (or even other sources), some > extraction > process might even be automated, but then there would be the question of > copyright > assignment. What you describe is a full-time job, more or less. I'd love to have someone who'd volunteer to record all those points, but we don't have him/her yet. The only places where design is document are the large comments in some source files. > It would help if there were some taxonomy of features/design > points for > reference. Is the bug database being used for that? I don't think so (IIUC what you mean by that), and I don't really see how bugs could serve in that role. Most of the design changes and redesigns in Emacs were developed without any bug report, simply because those who did the job knew that a particular group of problems needs to be taken care of. > Actually, a good example (even though it's more post facto description than > prospective > design & cost-benefit tradeoff analysis) would be > https://github.com/rocky/elisp-bytecode > That is really useful documentation that would ideally be in the emacs docs > or etc directory. That's not design description, though. > I put in a request to assign@gnu.org a few days ago, but I have not > received any response yet. Please ping them and CC me and Lars. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 18:37 ` Eli Zaretskii @ 2022-06-23 22:08 ` Lynn Winebarger 2022-06-24 6:53 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Lynn Winebarger @ 2022-06-23 22:08 UTC (permalink / raw) To: Eli Zaretskii Cc: Lars Ingebrigtsen, Stefan Monnier, Ihor Radchenko, Mattias Engdegård, Tim Cross, rms, Alan Mackenzie, emacs-devel On Thu, Jun 23, 2022 at 2:37 PM Eli Zaretskii <eliz@gnu.org> wrote: > > What you describe is a full-time job, more or less. I'd love to have > someone who'd volunteer to record all those points, but we don't have > him/her yet. > > The only places where design is document are the large comments in > some source files. > > > It would help if there were some taxonomy of features/design > > points for > > reference. Is the bug database being used for that? > > I don't think so (IIUC what you mean by that), and I don't really see > how bugs could serve in that role. I agree, but some of the bug reports are classified as something along the lines of "feature request" or "wishlist". I haven't found any other database tracking anything corresponding to features, proposed or otherwise. In more formal environments, feature implementation would be preceded by some kind of ticket describing the problem to be solved or functionality to be implemented. I don't expect any strictly enforced workflow in this kind of project, but that kind of referent is still useful for the same reasons having an entry in a bug database to reference is useful. A crude taxonomy wouldn't be that hard to come up with, then elaborated with finer points: Value Representation ELisp Abstract Machine Memory Management Intermediate Representations Core Lisp Lisp Assembly LIMPLE Compiler Transformations Serialized Lisp Program Formats ELC ELN PDMP etc Some of these points are already documented in the Elisp manual, of course, and some in the code comments, and others in the emacs-devel archives. Once something is in place, no matter how bare-bones, the key would be maintainers mandating that any substantive introduction or revision of features be tied to appropriate entries for the corresponding design/spec documentation, in the same way and for the same reason coding standards are enforced. That's assuming the maintainers consider such documentation important enough for enabling future potential contributors and maintainers to hold otherwise useful contributions in limbo. And probably that doing so is convenient to do. > Most of the design changes and > redesigns in Emacs were developed without any bug report, simply > because those who did the job knew that a particular group of problems > needs to be taken care of. It's not like there isn't any discussion or justification of features offered prior to code being integrated into the main branch. It's more a challenge of how to weave what's there into a coherent account of what's going on in the code. It would just be easier to automate some sort of design note extraction from the git log if references to mails could be associated with relevant features. I've never used org, but maybe there's some syntax that would be useful? Or maybe some notation from supercite for precise pulling of relevant text from list archives? I feel like if I were a more savvy Emacs user, I would know all of this functionality is already implemented in some combination of CEDET-originated tools, one or more tagging systems, the various interfaces for using the tagging systems, org, etc. > > > Actually, a good example (even though it's more post facto description than > > prospective > > design & cost-benefit tradeoff analysis) would be > > https://github.com/rocky/elisp-bytecode > > That is really useful documentation that would ideally be in the emacs docs > > or etc directory. > > That's not design description, though. You probably have a more nuanced view than me on this. It's true, that document is focused on the specification (the "what") rather than the (detailed) "how" and "why" - is that what you mean? Either way, if you want to understand how the operational semantics of emacs lisp work in practice, a document of that sort is invaluable. Without that, a document explaining the "why" isn't going to be able to be very concrete. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 22:08 ` Lynn Winebarger @ 2022-06-24 6:53 ` Eli Zaretskii 2022-06-25 16:50 ` Lynn Winebarger 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-24 6:53 UTC (permalink / raw) To: Lynn Winebarger Cc: larsi, monnier, yantar92, mattiase, theophilusx, rms, acm, emacs-devel > From: Lynn Winebarger <owinebar@gmail.com> > Date: Thu, 23 Jun 2022 18:08:53 -0400 > Cc: Lars Ingebrigtsen <larsi@gnus.org>, Stefan Monnier <monnier@iro.umontreal.ca>, > Ihor Radchenko <yantar92@gmail.com>, Mattias Engdegård <mattiase@acm.org>, > Tim Cross <theophilusx@gmail.com>, rms@gnu.org, Alan Mackenzie <acm@muc.de>, > emacs-devel <emacs-devel@gnu.org> > > On Thu, Jun 23, 2022 at 2:37 PM Eli Zaretskii <eliz@gnu.org> wrote: > > > > > > It would help if there were some taxonomy of features/design > > > points for > > > reference. Is the bug database being used for that? > > > > I don't think so (IIUC what you mean by that), and I don't really see > > how bugs could serve in that role. > > I agree, but some of the bug reports are classified as something along > the lines of "feature request" or "wishlist". If you look at those, you will see that almost all of them request minor features that have no impact on the design whatsoever. It is very rare to have a feature request on the bug tracker that changes the design or introduces new designs. I would even dare to say it never happened, definitely not on my watch. > I haven't found any other database tracking anything corresponding > to features, proposed or otherwise. We do have etc/TODO. Did you look there? > Once something is in place, no matter how bare-bones, the key would be > maintainers mandating that any substantive introduction or revision of > features be tied to appropriate entries for the corresponding > design/spec documentation, in the same way and for the same reason > coding standards are enforced. I think you are greatly underestimating the efforts required to maintain documentation of this kind, let alone the effort for creating it basically from scratch. Programmers usually don't like writing documentation, and most of them cannot write good documentation even if they did want, in large part because that take years of practice actually doing it. In a project like Emacs, it is nigh impossible to force contributors and developers do what they are reluctant to do in the first place. Without everyone updating the documentation as they change code, any documentation about design and internals will quickly become outdated, and thus not just useless, but downright harmful. We have trouble keeping even our user documentation up-to-date. Are you aware of _any_ Free Software project that has any useful documentation of this kind? Every project I ever knew or was involved with which tried ended up abandoning that. A case in point is GDB, which once had a "GDB Internals" manual -- it was always outdated, and was eventually scrapped because the maintainers decided they could not and didn't want to invest the effort. XEmacs tried in its time to have the internals documented, but that was basically a one-man effort, and even in it whole chapters were never written. Etc. etc. I think there's some conclusion waiting here. > That's assuming the maintainers consider such documentation > important enough for enabling future potential contributors and > maintainers to hold otherwise useful contributions in limbo. And you think we don't? Of course we do! We also want to release Emacs much more frequently, and we want it to have no bugs, and a few other things. But somehow, each such goal cannot be reached for some boring practical reasons, none of them related to their importance in our eyes. We do in general consider it somewhat more important to develop Emacs and accept contributions than to document its internals, that's true. Which is why I said up-thread that without someone who'd volunteer to do this job (thus dedicating his/her time almost exclusively to it), this will probably never happen, given the current state of our resources, which are barely enough to maintain the status quo and move forward at some reasonable pace. > > Most of the design changes and > > redesigns in Emacs were developed without any bug report, simply > > because those who did the job knew that a particular group of problems > > needs to be taken care of. > > It's not like there isn't any discussion or justification of features > offered prior to code being integrated into the main branch. It's more > a challenge of how to weave what's there into a coherent account of > what's going on in the code. IME, you are wrong in that assumption. The significant changes in Emacs design are almost never publicly discussed, not in the way that would allow someone to glean the design from them. E.g., take the bidirectional display example. That one I'm intimately familiar with, so I can tell you with 100% confidence: the current design cannot be possibly gleaned from any discussions. Quite the contrary: the special emacs-bidi mailing list, created back there for such discussions, holds many ideas expressed by people, none of them actually relevant to what we have in Emacs today. At some point I realized that I should stop wasting my time on those discussions, and instead sit down and code this stuff, or the job will never be done. It is no coincidence that the discussions on emacs-bidi died right about the time I started implementing. I did document the main design points in bidi.c, but if you look for design decisions explained there, you won't find such explanations, only the description of the design's end point. And this example is somewhat exceptional, because the amount of documentation of the design and the implementation there is relatively large. In other areas the situation is more bleak. > It would just be easier to automate some sort of design note > extraction from the git log if references to mails could be associated > with relevant features. I've never used org, but maybe there's some > syntax that would be useful? Or maybe some notation from supercite > for precise pulling of relevant text from list archives? I wish this were true. It isn't. The discussions and the commit log messages rarely describe the design, and in many cases barely describe even the particular implementation. In a project where people with write access can commit changes without any review, I don't know how can anything else to be expected. We basically rely on each individual to do the job perfectly and contribute to what you want to see documented. The results are before our eyes, and they shouldn't surprise anyone. > > > https://github.com/rocky/elisp-bytecode > > > That is really useful documentation that would ideally be in the emacs docs > > > or etc directory. > > > > That's not design description, though. > > You probably have a more nuanced view than me on this. It's true, > that document is focused on the specification (the "what") rather than > the (detailed) "how" and "why" - is that what you mean? Of course. > Either way, if you want to understand how the operational semantics > of emacs lisp work in practice, a document of that sort is > invaluable. Without that, a document explaining the "why" isn't > going to be able to be very concrete. I agree, but the "what" is usually already available in the comments to the code, though not everywhere and for every significant feature. The "why" and "how", by contrast are almost completely missing. To summarize: I'm not sure we should continue this discussion, because I don't see where is it going and what could it possibly change in practice. I agree with the value of having all of that documented, I'm just saying it's a large job that needs dedicated individuals, and I don't see how that could be replaced by any semi-automated means. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-24 6:53 ` Eli Zaretskii @ 2022-06-25 16:50 ` Lynn Winebarger 0 siblings, 0 replies; 505+ messages in thread From: Lynn Winebarger @ 2022-06-25 16:50 UTC (permalink / raw) To: Eli Zaretskii Cc: Lars Ingebrigtsen, Stefan Monnier, Ihor Radchenko, Mattias Engdegård, Tim Cross, rms, Alan Mackenzie, emacs-devel [-- Attachment #1: Type: text/plain, Size: 13547 bytes --] On Fri, Jun 24, 2022, 2:54 AM Eli Zaretskii <eliz@gnu.org> wrote: > > From: Lynn Winebarger <owinebar@gmail.com> > > > > > > It would help if there were some taxonomy of features/design > > > > points for > > > > reference. Is the bug database being used for that? > > > > > > I don't think so (IIUC what you mean by that), and I don't really see > > > how bugs could serve in that role. > > > > I agree, but some of the bug reports are classified as something along > > the lines of "feature request" or "wishlist". > > If you look at those, you will see that almost all of them request > minor features that have no impact on the design whatsoever. It is > very rare to have a feature request on the bug tracker that changes > the design or introduces new designs. I would even dare to say it > never happened, definitely not on my watch. > > > I haven't found any other database tracking anything corresponding > > to features, proposed or otherwise. > > We do have etc/TODO. Did you look there? > Yes, but it's not indexed so there's no good way to reference particular items the way bugs are. > > Once something is in place, no matter how bare-bones, the key would be > > maintainers mandating that any substantive introduction or revision of > > features be tied to appropriate entries for the corresponding > > design/spec documentation, in the same way and for the same reason > > coding standards are enforced. > > I think you are greatly underestimating the efforts required to > maintain documentation of this kind, let alone the effort for creating > it basically from scratch. > I just have a lot of sympathy for the points you made in that portable dumper thread, regarding the cost of accepting substantial contributions from transitory contributors. Anyone can create a new feature or improvement and maintain their own variant, but it's a lot easier for them if the project takes over maintenance and breaking it becomes a rejection test for future changes. Everything you say below is sensible. You and the other (current and future) core maintainers are the ones who have to make the trade-off, after all. I'm going to add some comments below, though, because I think my perspective on what is useful is different. If they are obvious or too idiosyncratic to be useful in the context of attracting new contributors/maintainers, feel free to ignore them. > > Programmers usually don't like writing documentation, and most of them > cannot write good documentation even if they did want, in large part > because that take years of practice actually doing it. In a project > like Emacs, it is nigh impossible to force contributors and developers > do what they are reluctant to do in the first place. Without everyone > updating the documentation as they change code, any documentation > about design and internals will quickly become outdated, and thus not > just useless, but downright harmful. We have trouble keeping even our > user documentation up-to-date. > Is it harmful? I suppose it depends on who's relying on it. Diving into the source of a project with Emacs's complexity with a particular design change in mind, personally I prefer to have some guideposts knowing that they may be dated, then ask knowledgeable people if there seems to be a disagreement between the doc and the code. When that happens on a public forum like this list, then there's some reference. It's true, I have a low bar. It might not seem like it, but my preference is to explore a question as much as I can on my own before bothering the experts. In the best case, the question that stumps me also stumps the expert - then I know I've done adequate study. With Emacs (and often other free software projects), it's just hard to tell what I should be responsible for studying prior to asking questions. Beyond the code itself, of course. Are you aware of _any_ Free Software project that has any useful > documentation of this kind? I know some "Open Source" projects (eg chromium) that have useful docs. Also projects with academic roots (Larceny Scheme is still my go-to) tend to have them. I'm assuming that in this context, the OSS projects, at least, are not "Free Software project[s]" in the sense you intended, even if the software produced by those projects happens to be Free. I don't do development for a living, so when I first looked into the code of those corporate projects, the general culture around demanding uniformity of build systems and supported configurations was quite a culture shock after being accustomed to the approach of Free Software projects. So I'm assuming you intend to refer to projects that have freedom as an organizing principle, and hence exhibit a self-selection bias toward diverse and idiosyncratic preferences and goals among their contributors. Or something along those lines? Every project I ever knew or was involved > with which tried ended up abandoning that. A case in point is GDB, > which once had a "GDB Internals" manual -- it was always outdated, and > was eventually scrapped because the maintainers decided they could not > and didn't want to invest the effort. I remember thinking it was a shame. But, my bar for usefulness is fairly low. XEmacs tried in its time to > have the internals documented, but that was basically a one-man > effort, and even in it whole chapters were never written. Etc. etc. > > I think there's some conclusion waiting here. > I think we have different problem scopes in mind. I'd take some kind of indexed wiki as a substantial improvement, particularly if there were links between tags in the code as well as files and directories cross-referencing features and VC logs, with the primary archive of discussion happening there rather than an email list (as with discussion around bugs). I'd just like to be able to determine answers to questions like: why is preventing the allocation of lisp objects by mmap necessary? What purpose does ralloc.c serve? Do they still serve a purpose?, and know whether I've done enough self-study to justify asking the experts. > That's assuming the maintainers consider such documentation > > important enough for enabling future potential contributors and > > maintainers to hold otherwise useful contributions in limbo. > > And you think we don't? Of course we do! We also want to release > Emacs much more frequently, and we want it to have no bugs, and a few > other things. But somehow, each such goal cannot be reached for some > boring practical reasons, none of them related to their importance in > our eyes. > I did not mean to imply you don't consider them important (I can see it might read that way). I meant "important enough [relative to all the competing priorities]". I don't know of any other way of expressing importance except relative to competing priorities subject to resource constraints. > We do in general consider it somewhat more important to develop Emacs > and accept contributions than to document its internals, that's true. > Which is why I said up-thread that without someone who'd volunteer to > do this job (thus dedicating his/her time almost exclusively to it), > this will probably never happen, given the current state of our > resources, which are barely enough to maintain the status quo and move > forward at some reasonable pace. > > > > Most of the design changes and > > > redesigns in Emacs were developed without any bug report, simply > > > because those who did the job knew that a particular group of problems > > > needs to be taken care of. > > > > It's not like there isn't any discussion or justification of features > > offered prior to code being integrated into the main branch. It's more > > a challenge of how to weave what's there into a coherent account of > > what's going on in the code. > > IME, you are wrong in that assumption. The significant changes in > Emacs design are almost never publicly discussed, not in the way that > would allow someone to glean the design from them. > Then, I should understand Daniel Colascione's approach with the portable dumper preview thread and the more recent one on a revised gc are either outliers - or that I'm misreading their scope relative to the discussion of design issues they provide? [ That's a straight question to resolve my ignorance, not intended to be snarky. ] > It would just be easier to automate some sort of design note > > extraction from the git log if references to mails could be associated > > with relevant features. I've never used org, but maybe there's some > > syntax that would be useful? Or maybe some notation from supercite > > for precise pulling of relevant text from list archives? > > I wish this were true. It isn't. The discussions and the commit log > messages rarely describe the design, and in many cases barely describe > even the particular implementation. In a project where people with > write access can commit changes without any review, I don't know how > can anything else to be expected. We basically rely on each > individual to do the job perfectly and contribute to what you want to > see documented. The results are before our eyes, and they shouldn't > surprise anyone. > True enough. > > > > > https://github.com/rocky/elisp-bytecode > > > > That is really useful documentation that would ideally be in the > emacs docs > > > > or etc directory. > > > > > > That's not design description, though. > > > > You probably have a more nuanced view than me on this. It's true, > > that document is focused on the specification (the "what") rather than > > the (detailed) "how" and "why" - is that what you mean? > > Of course. > > > Either way, if you want to understand how the operational semantics > > of emacs lisp work in practice, a document of that sort is > > invaluable. Without that, a document explaining the "why" isn't > > going to be able to be very concrete. > > I agree, but the "what" is usually already available in the comments > to the code, though not everywhere and for every significant feature. > The "why" and "how", by contrast are almost completely missing. > In the particular case of the bytecode spec, at least it gives me the sense of the invariants that are being maintained by the implementation. Trying to reverse engineer what those invariants are from a giant C switch statement is always tricky, because they tend to be expressed with boilerplate code, and then sometimes you have clever use of the fact that cases are non-exclusive and "fall through" without an explicit break, etc. That's where it intersects with "design" for me. OTOH, the same switch statement could be written as a dispatch table of higher-order functions exploiting proper tail-recursion in a way I would feel comfortable enough with to not feel the need to lean on a spec. But that assumes those higher-order functions clearly express the invariants they impose. It's a personal preference, I know. To summarize: I'm not sure we should continue this discussion, because > I don't see where is it going and what could it possibly change in > practice. I agree with the value of having all of that documented, > I'm just saying it's a large job that needs dedicated individuals, and > I don't see how that could be replaced by any semi-automated means. > I was surprised my off-the-cuff remark about trawling the archives generated any response in the first place, to be honest. One thing (I think) some of those OSS IDE projects do well is eat their own dogfood in terms of features for projects/documentation/collaborative development models generally. AFAICT, Emacs has support (either in core or in non-core packages) for a lot of different tools/approaches/etc to these issues, but doesn't seem to take advantage of them for emacs development itself, beyond the integrated bug reporting and version control systems. There's support for all these project formats and tags - why isn't there a standard choice of those imposed on the emacs projects itself, e.g. with a centralized repo of pre-generated tags and auto-generated extraction of documentation/data structure definitions? Maybe I just haven't read that section of the manual or the right document in "etc" yet - I freely admit my ignorance. As it is, I've just assumed that the core maintainer(s) have weighed the trade-offs and decided imposing that kind of constraint, even among the core maintainers, would cost more than it is worth (in attracting contributors and maintainers). If there was an "emacs-devel" mode where the tags (and files would be tags considered as entries in a directory "document") linked to a database cross-referenced with the feature database, the VC logs, and a wiki-ish interface that could record both documentation and "talk". Sort of like "help" mode, but with a wikipedia-ish feel. The contributors to that piece of code and maintainers covering the related features would be auto-subscribed to the entry, and any discussion would at least get recorded in a way linked to the thing being discussed. I'm not sure contributors would want to be auto-subscribed like that, but then maybe their contribution should be reviewed by someone who would be. It's just an idea - whether it's useful or practical for the purpose of facilitating the recruitment of new contributors/maintainers, given that some culture change would likely be required, is not something I have the perspective to judge. I do know I would find something like what I've described very useful. Lynn [-- Attachment #2: Type: text/html, Size: 17766 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-21 2:01 ` Lynn Winebarger 2022-06-22 0:01 ` Lynn Winebarger @ 2022-06-22 23:26 ` Stefan Monnier 2022-06-22 23:48 ` Lynn Winebarger 1 sibling, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-22 23:26 UTC (permalink / raw) To: Lynn Winebarger Cc: Lars Ingebrigtsen, Eli Zaretskii, Ihor Radchenko, Mattias Engdegård, Tim Cross, rms, Alan Mackenzie, emacs-devel > Was there? I see:. https://debbugs.gnu.org/cgi/bugreport.cgi?bug=38345#71 > but it's a little more than a couple of months ago. > I'm pretty curious because if I accumulate a large buffer of trace output > (running memory up to 100s of MB), killing the buffer doesn't seem to > impact gc time substantially. The GC does not really look at a buffer's content, so unless there is a non-trivial amount of text-properties (e.g. because of font-lock and such), the GC will take about the same amount of time handling a large buffer as handling a tiny buffer. Similarly, whether the malloc library returns the memory to the system or not shouldn't affect GC times very much if at all. What does affect GC time is the amount of live data as well as the total amount of data that's under the GC's control (i.e. live data, garbage that still needs to be collected, as well as data that's free but that we haven't been able to return to malloc (via `free`)). Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-22 23:26 ` Stefan Monnier @ 2022-06-22 23:48 ` Lynn Winebarger 0 siblings, 0 replies; 505+ messages in thread From: Lynn Winebarger @ 2022-06-22 23:48 UTC (permalink / raw) To: Stefan Monnier Cc: Lars Ingebrigtsen, Eli Zaretskii, Ihor Radchenko, Mattias Engdegård, Tim Cross, rms, Alan Mackenzie, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1813 bytes --] On Wed, Jun 22, 2022 at 7:26 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote: > > Was there? I see:. > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=38345#71 > > but it's a little more than a couple of months ago. > > I'm pretty curious because if I accumulate a large buffer of trace output > > (running memory up to 100s of MB), killing the buffer doesn't seem to > > impact gc time substantially. > > The GC does not really look at a buffer's content, so unless there is > a non-trivial amount of text-properties (e.g. because of font-lock and > such), the GC will take about the same amount of time handling a large > buffer as handling a tiny buffer. > > Similarly, whether the malloc library returns the memory to the system > or not shouldn't affect GC times very much if at all. What does affect > GC time is the amount of live data as well as the total amount of data > that's under the GC's control (i.e. live data, garbage that still needs > to be collected, as well as data that's free but that we haven't been > able to return to malloc (via `free`)). > > That's why I found it so confusing - as though the gc was traversing malloc's free list. Even if the text were killed and saved in the kill ring, it would be a lisp string, so the GC shouldn't be traversing it. I'm not particularly eager to recreate the problem, as I spent (at least) as much time trying to understand edebug's behavior as the lisp macro I was debugging. It's possible there's something special about the edebug trace buffer, or (I suspect more likely) one of those packages I set up made "kill-buffer" bury them instead. I just remember being very frustrated that the gc time seemed to grow proportional to the size of the trace buffer and not be improved by running garbage-collect after killing that buffer. Lynn [-- Attachment #2: Type: text/html, Size: 2505 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 12:49 ` Lars Ingebrigtsen 2022-06-18 13:06 ` Stefan Monnier 2022-06-18 13:20 ` Eli Zaretskii @ 2022-06-20 12:42 ` Lynn Winebarger 2 siblings, 0 replies; 505+ messages in thread From: Lynn Winebarger @ 2022-06-20 12:42 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Stefan Monnier, Ihor Radchenko, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel [-- Attachment #1: Type: text/plain, Size: 837 bytes --] On Sat, Jun 18, 2022 at 8:52 AM Lars Ingebrigtsen <larsi@gnus.org> wrote: > Stefan Monnier <monnier@iro.umontreal.ca> writes: > > As far as I know, we never actually release back cons space to the OS > anyway -- even if we could. (Because glibc's malloc implementation > doesn't do that unless we call that trim function, which we don't.) So > I'm not sure that makes much difference -- especially in a batch process. > > Does this mean that, once the heap size of the process has been increased by a large gc threshold, every subsequent gc will be proportional to the largest heap size ever required, regardless of whether some of it could be permanently freed? So, if the process terminates before collection, then it will run faster, but if it keeps running, the gc "pause" time will be permanently longer for each subsequent gc? [-- Attachment #2: Type: text/html, Size: 1292 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 2:32 ` Stefan Monnier 2022-06-18 12:49 ` Lars Ingebrigtsen @ 2022-06-19 7:25 ` Ihor Radchenko 2022-06-19 8:51 ` Eli Zaretskii 2022-06-19 9:23 ` Ihor Radchenko 2 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-19 7:25 UTC (permalink / raw) To: Stefan Monnier Cc: Lars Ingebrigtsen, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > I also took a look at related data. E.g. comparing p=1.0 to the old > p-0.1 on the process that performs the highest number of GCs (188 cycles > with p=1.0 and 650 cycles with p=0.1) during an Emacs build. Here are > the corresponding last few GC cycles: > > % grep GC-26227 ./+make-0.1.log | tail -n 10 > GC-26227 p=0.1 total=18.8M free=1.9M thresold=1.9M > ... > GC-26227 p=0.1 total=145.7M free=32.6M thresold=14.6M > GC-26227 p=0.1 total=132.8M free=60.7M thresold=13.3M > % grep GC-898 ./+make-1.0.log | tail -n 10 > GC-898 p=1.0 total=18.5M free=7.5M thresold=18.5M > ... > GC-898 p=1.0 total=145.7M free=32.6M thresold=145.7M > % I feel that I am confused about this data. We have gc-cons-threshold and gc-cons-percentage According to the gc-cons-percentage docstring: >> If this portion is smaller than `gc-cons-threshold', this is ignored. which implies that gc-cons-percentage only takes effect when heap_size * gc-cons-percentage >= gc-cons-threshold (= 0.8M) So, heap_size should be >= 80M to take effect at p=1.0 and heap_size should be >= 800M to take effect at p=0.1 yet, you do see an effect at total=18.5M. What am I missing? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-19 7:25 ` Ihor Radchenko @ 2022-06-19 8:51 ` Eli Zaretskii 2022-06-19 9:23 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-19 8:51 UTC (permalink / raw) To: Ihor Radchenko Cc: monnier, larsi, mattiase, theophilusx, rms, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Lars Ingebrigtsen <larsi@gnus.org>, Mattias Engdegård > <mattiase@acm.org>, Eli Zaretskii <eliz@gnu.org>, Tim Cross > <theophilusx@gmail.com>, rms@gnu.org, Alan Mackenzie <acm@muc.de>, > emacs-devel <emacs-devel@gnu.org> > Date: Sun, 19 Jun 2022 15:25:15 +0800 > > I feel that I am confused about this data. > > We have gc-cons-threshold and gc-cons-percentage > > According to the gc-cons-percentage docstring: > > >> If this portion is smaller than `gc-cons-threshold', this is ignored. > > which implies that gc-cons-percentage only takes effect when > heap_size * gc-cons-percentage >= gc-cons-threshold (= 0.8M) > > So, heap_size should be >= 80M to take effect at p=1.0 > and heap_size should be >= 800M to take effect at p=0.1 > > yet, you do see an effect at total=18.5M. > > What am I missing? gc-cons-percentage is measured in units of 1, not in percents. So the value of 1.0 means that the condition heap_size * gc-cons-percentage >= gc-cons-threshold (= 0.8M) will become true whenever heap_size is 0.8MB, not 80MB. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-19 8:51 ` Eli Zaretskii @ 2022-06-19 9:23 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-19 9:23 UTC (permalink / raw) To: Eli Zaretskii Cc: monnier, larsi, mattiase, theophilusx, rms, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > gc-cons-percentage is measured in units of 1, not in percents. So the > value of 1.0 means that the condition > > heap_size * gc-cons-percentage >= gc-cons-threshold (= 0.8M) > > will become true whenever heap_size is 0.8MB, not 80MB. Thanks! I was confused by "percentage" in the variable name. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 2:32 ` Stefan Monnier 2022-06-18 12:49 ` Lars Ingebrigtsen 2022-06-19 7:25 ` Ihor Radchenko @ 2022-06-19 9:23 ` Ihor Radchenko 2022-06-19 22:11 ` Stefan Monnier 2 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-19 9:23 UTC (permalink / raw) To: Stefan Monnier Cc: Lars Ingebrigtsen, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > > % grep GC-26227 ./+make-0.1.log | tail -n 10 > GC-26227 p=0.1 total=18.8M free=1.9M thresold=1.9M > ... > GC-26227 p=0.1 total=18.8M free=1.8M thresold=1.9M > GC-26227 p=0.1 total=145.7M free=32.6M thresold=14.6M > GC-26227 p=0.1 total=132.8M free=60.7M thresold=13.3M > % grep GC-898 ./+make-1.0.log | tail -n 10 > GC-898 p=1.0 total=18.5M free=7.5M thresold=18.5M > ... > GC-898 p=1.0 total=18.8M free=7.3M thresold=18.8M > GC-898 p=1.0 total=145.7M free=32.6M thresold=145.7M > % Now, when I cleared my confusion about the units of gc-cons-percentage, let me share one thought on the above results. In the first series of benchamarks with p=0.1 and total=18.8M, free is roughly equal to threshold. When we increase the threshold, free increases up to some value, but that value is limited by the underlying code: > GC-26227 p=0.1 total=145.7M free=32.6M thresold=14.6M > GC-898 p=1.0 total=145.7M free=32.6M thresold=145.7M In the above two cases, it appears that 32.6M is required to store all the transient values, regardless of the threshold. Hypothesis: 1. When threshold < equilibrium free, we perform redundant GC passes 2. When threshold > equilibrium free, we allocate extra unnecessary memory 3. The optimum memory usage/GC frequency happens at free=threshold Can we test this hypothesis? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-19 9:23 ` Ihor Radchenko @ 2022-06-19 22:11 ` Stefan Monnier 2022-06-20 12:21 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-19 22:11 UTC (permalink / raw) To: Ihor Radchenko Cc: Lars Ingebrigtsen, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel > When we increase the threshold, free increases up to some value, but > that value is limited by the underlying code: A clarification: when you see [...] GC-898 p=1.0 total=18.7M free=7.3M thresold=18.7M GC-898 p=1.0 total=18.8M free=7.3M thresold=18.8M [...] what it means is that after the first of the two GCs, the threshold until the next GC was set at 18.7M, so when the next GC is called, the heap has grown to at least 18.7M+18.7M=37.4M (possibly more), then GC took place, it found 18.8M of live data and of the remaining ≥18.6M it managed to return ≥11.3M back to the malloc library but it had to keep 7.3M as "free memory" because these are contained in malloc'd blocks which still also contain *some* live data (typically, we malloc blocks of say 16kB and then split them into the corresponding number of cons cells, or floats, or strings, so we can only free a block of floats if all the floats it contains are dead). > In the first series of benchamarks with p=0.1 and total=18.8M, free is > roughly equal to threshold. This likely means that we basically neither called `malloc` in between the two GCs (we instead managed to satisfy all the allocations from the set of free objects we still had around) nor called `free` at the end of the GC (because those 1.9M of free memory are spread around in such a way that we can't return any of that memory to the malloc library). The difference between the two cases is that when you have half of the heap made up of dead objects, there's a higher probability that some of those objects are clustered such that they make up a few complete 16kB blocks, whereas when only the dead objects represent only 10% of the heap, almost all 16kB blocks will contain a mis of dead and live objects and hence can't be returned to the malloc library. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-19 22:11 ` Stefan Monnier @ 2022-06-20 12:21 ` Ihor Radchenko 2022-06-23 22:12 ` Stefan Monnier 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-20 12:21 UTC (permalink / raw) To: Stefan Monnier Cc: Lars Ingebrigtsen, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >> When we increase the threshold, free increases up to some value, but >> that value is limited by the underlying code: > > A clarification: when you see > > [...] > GC-898 p=1.0 total=18.7M free=7.3M thresold=18.7M > GC-898 p=1.0 total=18.8M free=7.3M thresold=18.8M > [...] > ... Thanks! I was totally misreading the results. > The difference between the two cases is that when you have half of the > heap made up of dead objects, there's a higher probability that some of > those objects are clustered such that they make up a few complete > 16kB blocks, whereas when only the dead objects represent only 10% of > the heap, almost all 16kB blocks will contain a mis of dead and live > objects and hence can't be returned to the malloc library. I am again confused. Does what you say imply that frequent GC with small threshold is not good because it will have less dead objects and have higher chance of fragmentation? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-20 12:21 ` Ihor Radchenko @ 2022-06-23 22:12 ` Stefan Monnier 2022-06-25 5:13 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-23 22:12 UTC (permalink / raw) To: Ihor Radchenko Cc: Lars Ingebrigtsen, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel >> The difference between the two cases is that when you have half of the >> heap made up of dead objects, there's a higher probability that some of >> those objects are clustered such that they make up a few complete >> 16kB blocks, whereas when only the dead objects represent only 10% of >> the heap, almost all 16kB blocks will contain a mis of dead and live >> objects and hence can't be returned to the malloc library. > > I am again confused. > Does what you say imply that frequent GC with small threshold is not > good because it will have less dead objects and have higher chance of > fragmentation? Not at all. The general rule of thumb is: the sooner you recover free space, the better you can avoid fragmentation. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-23 22:12 ` Stefan Monnier @ 2022-06-25 5:13 ` Ihor Radchenko 2022-06-25 6:08 ` Eli Zaretskii 2022-06-25 8:33 ` Stefan Monnier 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-25 5:13 UTC (permalink / raw) To: Stefan Monnier Cc: Lars Ingebrigtsen, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >>> The difference between the two cases is that when you have half of the >>> heap made up of dead objects, there's a higher probability that some of >>> those objects are clustered such that they make up a few complete >>> 16kB blocks, whereas when only the dead objects represent only 10% of >>> the heap, almost all 16kB blocks will contain a mis of dead and live >>> objects and hence can't be returned to the malloc library. >> >> I am again confused. >> Does what you say imply that frequent GC with small threshold is not >> good because it will have less dead objects and have higher chance of >> fragmentation? > > Not at all. The general rule of thumb is: the sooner you recover free > space, the better you can avoid fragmentation. Sure. But how does the probability to get fragmentation scale with the GC threshold? Is it linear scaling or is it nonlinear (I suspect that it is non-linear)? Can someone quantify it? If we do, it would help to adjust the thresholds more optimally to balance between GC time and fragmentation. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-25 5:13 ` Ihor Radchenko @ 2022-06-25 6:08 ` Eli Zaretskii 2022-06-25 8:33 ` Stefan Monnier 1 sibling, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-25 6:08 UTC (permalink / raw) To: Ihor Radchenko Cc: monnier, larsi, mattiase, theophilusx, rms, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Lars Ingebrigtsen <larsi@gnus.org>, Mattias Engdegård > <mattiase@acm.org>, Eli Zaretskii <eliz@gnu.org>, Tim Cross > <theophilusx@gmail.com>, rms@gnu.org, Alan Mackenzie <acm@muc.de>, > emacs-devel <emacs-devel@gnu.org> > Date: Sat, 25 Jun 2022 13:13:22 +0800 > > > Not at all. The general rule of thumb is: the sooner you recover free > > space, the better you can avoid fragmentation. > > Sure. But how does the probability to get fragmentation scale with the > GC threshold? Is it linear scaling or is it nonlinear (I suspect that it > is non-linear)? Can someone quantify it? If we do, it would help to > adjust the thresholds more optimally to balance between GC time and > fragmentation. The answer depends on the C library being used. If you are talking about glibc, there's still the question of the statistics of allocation size distribution, because AFAIK glibc's malloc implementation uses several separate lists to manage the arena, each list for allocations between X and Y bytes, where X and Y are different for each list. And for allocations larger than 64KB we instruct glibc to use mmap, where there should be no fragmentation at all. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-25 5:13 ` Ihor Radchenko 2022-06-25 6:08 ` Eli Zaretskii @ 2022-06-25 8:33 ` Stefan Monnier 1 sibling, 0 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-25 8:33 UTC (permalink / raw) To: Ihor Radchenko Cc: Lars Ingebrigtsen, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel > Sure. But how does the probability to get fragmentation scale with the > GC threshold? No, fragmentation is too complex an effect to be characterized this way. It heavily depends on the application's allocation patterns as well. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 17:48 ` Stefan Monnier 2022-06-17 18:20 ` Lars Ingebrigtsen @ 2022-06-18 5:58 ` Eli Zaretskii 2022-06-18 13:46 ` Stefan Monnier 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-18 5:58 UTC (permalink / raw) To: Stefan Monnier Cc: larsi, yantar92, mattiase, theophilusx, rms, acm, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Ihor Radchenko <yantar92@gmail.com>, Mattias Engdegård > <mattiase@acm.org>, Eli Zaretskii <eliz@gnu.org>, Tim Cross > <theophilusx@gmail.com>, rms@gnu.org, Alan Mackenzie <acm@muc.de>, > emacs-devel <emacs-devel@gnu.org> > Date: Fri, 17 Jun 2022 13:48:46 -0400 > > BYTE_COMPILE_EXTRA_FLAGS="--eval '(setq gc-cons-percentage $FOO)'": > > make -j24 FOO=0.1 1338.52s user 60.95s system 1256% cpu 1:51.42 total > make -j24 FOO=0.1 1338.36s user 60.60s system 1252% cpu 1:51.65 total > make -j24 FOO=0.1 1339.53s user 60.08s system 1252% cpu 1:51.76 total > make -j24 FOO=0.2 1266.54s user 59.75s system 1264% cpu 1:44.92 total > make -j24 FOO=0.2 1269.17s user 60.18s system 1256% cpu 1:45.83 total > make -j24 FOO=0.2 1266.10s user 60.42s system 1258% cpu 1:45.43 total > make -j24 FOO=0.3 1200.06s user 60.24s system 1251% cpu 1:40.73 total > make -j24 FOO=0.3 1198.80s user 60.06s system 1254% cpu 1:40.39 total > make -j24 FOO=0.3 1202.49s user 59.60s system 1252% cpu 1:40.80 total > make -j24 FOO=0.4 1134.26s user 59.75s system 1233% cpu 1:36.81 total > make -j24 FOO=0.4 1133.11s user 60.07s system 1226% cpu 1:37.25 total > make -j24 FOO=0.4 1132.18s user 59.94s system 1226% cpu 1:37.18 total > make -j24 FOO=0.5 1069.59s user 59.70s system 1211% cpu 1:33.22 total > make -j24 FOO=0.5 1070.84s user 60.06s system 1211% cpu 1:33.34 total > make -j24 FOO=0.5 1070.93s user 59.86s system 1210% cpu 1:33.43 total > make -j24 FOO=1.0 845.92s user 61.24s system 1126% cpu 1:20.55 total > make -j24 FOO=1.0 847.92s user 61.82s system 1130% cpu 1:20.46 total > make -j24 FOO=1.0 846.59s user 61.46s system 1133% cpu 1:20.10 total > make -j24 FOO=2.0 678.14s user 68.64s system 1044% cpu 1:11.52 total > make -j24 FOO=2.0 678.04s user 69.19s system 1042% cpu 1:11.64 total > make -j24 FOO=2.0 677.66s user 68.24s system 1041% cpu 1:11.59 total > make -j24 FOO=3.0 677.50s user 69.38s system 1047% cpu 1:11.28 total > make -j24 FOO=3.0 677.42s user 68.43s system 1047% cpu 1:11.19 total > make -j24 FOO=3.0 677.98s user 69.10s system 1044% cpu 1:11.50 total > make -j24 FOO=4.0 677.43s user 68.58s system 1051% cpu 1:10.96 total > make -j24 FOO=4.0 677.09s user 68.49s system 1056% cpu 1:10.57 total > make -j24 FOO=4.0 677.60s user 69.23s system 1049% cpu 1:11.17 total > make -j24 FOO=5.0 677.22s user 68.90s system 1043% cpu 1:11.52 total > make -j24 FOO=5.0 677.47s user 68.91s system 1044% cpu 1:11.49 total > make -j24 FOO=5.0 677.13s user 67.98s system 1049% cpu 1:10.98 total > make -j24 FOO=7.0 678.21s user 68.57s system 1044% cpu 1:11.52 total > make -j24 FOO=7.0 677.48s user 68.50s system 1044% cpu 1:11.42 total > make -j24 FOO=7.0 677.95s user 68.25s system 1050% cpu 1:11.00 total > make -j24 FOO=10.0 677.30s user 69.52s system 1045% cpu 1:11.45 total > make -j24 FOO=10.0 678.18s user 68.55s system 1042% cpu 1:11.62 total > make -j24 FOO=10.0 678.97s user 68.45s system 1043% cpu 1:11.64 total > make -j24 FOO=20.0 677.99s user 68.42s system 1041% cpu 1:11.67 total > make -j24 FOO=20.0 677.69s user 69.31s system 1046% cpu 1:11.36 total > make -j24 FOO=20.0 677.89s user 68.70s system 1045% cpu 1:11.39 total > > Clearly for this test, the default (0.1) results in too low a threshold > for best performance and thus too much time spent in the GC. We reach > a plateau around percentage=2.0. > > This suggests that for batch jobs maybe we should bump up > `gc-cons-percentage` from 0.1 to something like 1.0 or 2.0. It is important to track the memory footprint of the Emacs process in each run (like, show the maximum VM of all the compilations, for example), because that's the other side of the tradeoff. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 5:58 ` Eli Zaretskii @ 2022-06-18 13:46 ` Stefan Monnier 2022-06-18 14:13 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-18 13:46 UTC (permalink / raw) To: Eli Zaretskii Cc: larsi, yantar92, mattiase, theophilusx, rms, acm, emacs-devel > It is important to track the memory footprint of the Emacs process in > each run (like, show the maximum VM of all the compilations, for > example), because that's the other side of the tradeoff. I measured the amount of free data in a later measurement, which is not quite the same but gives some of that info. Do you happen to know how to get the current process's RSS or VM size from C so I can try and add this data to my log? Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-18 13:46 ` Stefan Monnier @ 2022-06-18 14:13 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-18 14:13 UTC (permalink / raw) To: emacs-devel, Stefan Monnier Cc: larsi, yantar92, mattiase, theophilusx, rms, acm On June 18, 2022 4:46:14 PM GMT+03:00, Stefan Monnier <monnier@iro.umontreal.ca> wrote > > Do you happen to know how to get the current process's RSS or VM size > from C so I can try and add this data to my log? We do that in process-attributes, so you could just use that. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 13:49 ` Larger GC thresholds for non-interactive Emacs Stefan Monnier 2022-06-17 15:11 ` Lars Ingebrigtsen @ 2022-06-18 5:28 ` Ihor Radchenko 2022-07-01 2:34 ` Lisp-level macro to avoid excessive GC in memory-allocating code (was: Larger GC thresholds for non-interactive Emacs) Ihor Radchenko 2 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-18 5:28 UTC (permalink / raw) To: Stefan Monnier Cc: Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >> FYI, I have played with this approach making use of >> https://gitlab.com/koral/gcmh > > But this one focuses on interactive use. The kind of heuristic I'm > proposing above would only make sense within a single (long-running) > command, or in batch mode, I think. Got it. But are there actually long-running batch mode commands over there? The byte-compilation benchmarks in the other replies revealed that the total number of GCs is not very high over the course of single batch invocation. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Lisp-level macro to avoid excessive GC in memory-allocating code (was: Larger GC thresholds for non-interactive Emacs) 2022-06-17 13:49 ` Larger GC thresholds for non-interactive Emacs Stefan Monnier 2022-06-17 15:11 ` Lars Ingebrigtsen 2022-06-18 5:28 ` Ihor Radchenko @ 2022-07-01 2:34 ` Ihor Radchenko 2022-07-01 6:18 ` Eli Zaretskii 2 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-07-01 2:34 UTC (permalink / raw) To: Stefan Monnier Cc: Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >> A possible useful thing Emacs could help with would be a macro dedicated >> to let-binds like the above. Something like: >> >> (with-reduced-gc >> ;; Do staff) > > Sounds about right, tho I think the name of the macro should be related > to what the code does rather than to what the author thinks the GC should > do about it. Something like `this-code-does-not-generate-garbage`. > >> with-reduced-gc could take care about determining the inner specifics >> on what alternative gc-cons-threshold value should be used (possibly >> depending on the system memory information). > > Sounds good. This part of the discussion appears to be missed from the following replies. So, I would like to bring it up again. To clarify a bit on the usefulness of the proposed macro idea, consider the following code: (defvar lst-data) (benchmark-progn (let (result) (dotimes (i 1000000) (push i result)) (setq lst-data result) nil)) This code does not really generate any garbage. Yet GC is triggered frequently: Elapsed time: 0.619852s (0.426893s in 11 GCs) If I instead do (defvar lst-data) (benchmark-progn (let ((gc-cons-threshold most-positive-fixnum)) ;; the value is just for demo purposes (let (result) (dotimes (i 1000000) (push i result)) (setq lst-data result) nil))) Elapsed time: 0.216380s (0.031766s in 1 GCs) The difference is obvious. More generally, any code that generates/returns large data structures is going to trigger frequent GCs regardless whether such code generates any garbage. On the other hand, we cannot, in general terms, predict if real-life code, which allocates large permanent data structures, also produces a lot of actual valid garbage that should be collected. Having some way to prevent excessive garbage collection that is also smarter than simply setting gc-cons-threshold to large value would be useful. As one idea, a lisp program may mark some of the variables to be skipped by GC and to not contribute to GC threshold checks (that is, allocating memory into the marked variables will not increase the memory counter used by GC). WDYT? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Lisp-level macro to avoid excessive GC in memory-allocating code (was: Larger GC thresholds for non-interactive Emacs) 2022-07-01 2:34 ` Lisp-level macro to avoid excessive GC in memory-allocating code (was: Larger GC thresholds for non-interactive Emacs) Ihor Radchenko @ 2022-07-01 6:18 ` Eli Zaretskii 2022-07-01 7:52 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-07-01 6:18 UTC (permalink / raw) To: Ihor Radchenko; +Cc: monnier, mattiase, theophilusx, rms, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Mattias Engdegård <mattiase@acm.org>, Eli Zaretskii > <eliz@gnu.org>, > Tim Cross <theophilusx@gmail.com>, rms@gnu.org, Alan Mackenzie > <acm@muc.de>, emacs-devel <emacs-devel@gnu.org> > Date: Fri, 01 Jul 2022 10:34:34 +0800 > > (defvar lst-data) > (benchmark-progn > (let (result) > (dotimes (i 1000000) > (push i result)) > (setq lst-data result) > nil)) > > This code does not really generate any garbage. Yet GC is triggered > frequently: > > Elapsed time: 0.619852s (0.426893s in 11 GCs) > > If I instead do > > (defvar lst-data) > (benchmark-progn > (let ((gc-cons-threshold most-positive-fixnum)) ;; the value is just for demo purposes > (let (result) > (dotimes (i 1000000) > (push i result)) > (setq lst-data result) > nil))) > > Elapsed time: 0.216380s (0.031766s in 1 GCs) > > The difference is obvious. Please don't forget that GC doesn't only collects unused Lisp objects, it also does other useful memory-management related tasks. It compacts buffer text and strings, and it frees unused slots in various caches (font cache, image cache, etc.). You can find in the archives discussions where innocently-looking code could cause Emacs run out of memory because it used too many fonts without flushing the font cache (any program that works on the list of fonts returned by the likes of x-list-fonts is in danger of bumping into that). > More generally, any code that generates/returns large data structures is > going to trigger frequent GCs regardless whether such code generates any > garbage. > > On the other hand, we cannot, in general terms, predict if real-life > code, which allocates large permanent data structures, also produces a > lot of actual valid garbage that should be collected. Yes, that's the conundrum. > Having some way to prevent excessive garbage collection that is also > smarter than simply setting gc-cons-threshold to large value would be > useful. > > As one idea, a lisp program may mark some of the variables to be skipped > by GC and to not contribute to GC threshold checks (that is, allocating > memory into the marked variables will not increase the memory counter > used by GC). > > WDYT? I'm not sure I understand how this idea can be implemented. The counting of how much Lisp data since last GC was produced is done _before_ variables are bound to the produced data as values. So by the time we know the data is bound to such "special" variables, it's already too late, and the only way to do what you suggest would be to increase consing_until_gc back after we realize this fact. Which would mean computing how much consing was done for the value of these variables, and that would probably slow down the generation of Lisp data, wouldn't it? Or what am I missing? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Lisp-level macro to avoid excessive GC in memory-allocating code (was: Larger GC thresholds for non-interactive Emacs) 2022-07-01 6:18 ` Eli Zaretskii @ 2022-07-01 7:52 ` Ihor Radchenko 2022-07-01 10:45 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-07-01 7:52 UTC (permalink / raw) To: Eli Zaretskii; +Cc: monnier, mattiase, theophilusx, rms, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > Please don't forget that GC doesn't only collects unused Lisp objects, > it also does other useful memory-management related tasks. It > compacts buffer text and strings, and it frees unused slots in various > caches (font cache, image cache, etc.). You can find in the archives > discussions where innocently-looking code could cause Emacs run out of > memory because it used too many fonts without flushing the font cache > (any program that works on the list of fonts returned by the likes of > x-list-fonts is in danger of bumping into that). Then, if we decide to implement the macro I am suggesting, such macro should not affect memory allocation of such sensitive objects: font cache, image cache, etc. Just "safe" memory allocations. >> As one idea, a lisp program may mark some of the variables to be skipped >> by GC and to not contribute to GC threshold checks (that is, allocating >> memory into the marked variables will not increase the memory counter >> used by GC). >> >> WDYT? > > I'm not sure I understand how this idea can be implemented. The > counting of how much Lisp data since last GC was produced is done > _before_ variables are bound to the produced data as values. So by > the time we know the data is bound to such "special" variables, it's > already too late, and the only way to do what you suggest would be to > increase consing_until_gc back after we realize this fact. Which > would mean computing how much consing was done for the value of these > variables, and that would probably slow down the generation of Lisp > data, wouldn't it? > > Or what am I missing? We can do the following: 1. In addition to directly bumping the TOTAL counter of newly allocated memory, we can introduce a new LOCAL counter holding recent allocations within current sexp. 2. Every time we return from a sexp/self-quoting object into assignment, if we are inside the proposed macro and also assigning value to one of the pre-defined symbols, increase the upper LOCAL counter in the parent sexp. Otherwise, do not change the upper LOCAL counter. 3. Perform GC according to TOTAL-LOCAL threshold value. 4. When exiting the macro, set LOCAL to 0, unless inside another such macro. Example (I intentionally avoid using dolist because macros using temporary symbols complicate things): (defvar lst-value) (with-no-gc '(i return lst-value) (let (return (i 0)) (while (< i 1000000) (setq return (cons i return)) (make-string 1 ?a) (setq i (1+ i))) (setq lst-value return))) Let TOTAL be the global counter and LOCAL be the local counter. In the above code will: 1. Eval "0" to bind initial value of i. LOCAL=size_of_int; GLOBAL+=size_of_int. GC code will not count this allocation when deciding whether to perform GC. 2. Return 0 into (i 0) assignment and bump parent LOCAL because i is declared by the macro: LOCAL(let)+=LOCAL == size_of_int 3. Eval "1000000" in (< i 1000000); LOCAL=size_of_int; GLOBAL+=size_of_int. 4. Now, we are not inside an assignment, so LOCAL(<) == 0 (unchanged), TOTAL is increased, and TOTAL-LOCAL will be bumped by size_of_int. 5. Now, we are inside (while ...) and not inside assignment; LOCAL(while)+=LOCAL(<) == size_of_int 5. Eval (cons i return); LOCAL(cons)=cons_size; GLOBAL+=cons_size; 6. Return the new cons into the assignment to return. Because return is declared by the macro, the outer value of local is bumped: LOCAL(while) == size_of_int+cons_size 7. Eval (make-string ...). This will allocate a new string; LOCAL(make-string)=string_length; GLOBAL+=string_length; 8. Return to (while ...) sexp; Because we are not inside the assignment LOCAL(while) is unchanged and the string allocation will contribute to the GC threshold. 9. Eval (1+ i), which will allocate a new integer; LOCAL(1+)=size_of_int; GLOBAL+=size_of_int; 10. The newly allocated integer is assigned to i symbol, declared by the macro; thus LOCAL(while)+=LOCAL(1+) == size_of_int+cons_size+size_of_int; GLOBAL-LOCAL is unchanged and do not count towards next GC. [second iteration] 11. Eval "1000000" in (< i 1000000), which will not allocate any memory because integers are immutable (AFAIK). GLOBAL and LOCAL remain unchanged. ... (all other iterations) LOCAL(while) == size_of_int * 1000000 + cons_size * (1- 1000000) 12. Return from (while ...). It is not an assignment, so LOCAL(let)+=LOCAL(while) 13. Assign lst_value, which will not allocate any extra memory. 14. Return from let: LOCAL(with-no-gc)+=LOCAL(let) 15. Return from with-no-gc. LOCAL=0 and the whole allocated object will now be able to contribute to the GC threshold, unless the example snippet is not by itself wrapped into parent with-no-gc call. In any case, while inside with-no-gc, only a single 1000000 symbol + all the string allocations can trigger GC. Assignments to i,result, and lst_value will only be counted upon exiting the macro. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Lisp-level macro to avoid excessive GC in memory-allocating code (was: Larger GC thresholds for non-interactive Emacs) 2022-07-01 7:52 ` Ihor Radchenko @ 2022-07-01 10:45 ` Eli Zaretskii 2022-07-01 11:12 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-07-01 10:45 UTC (permalink / raw) To: Ihor Radchenko; +Cc: monnier, mattiase, theophilusx, rms, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: monnier@iro.umontreal.ca, mattiase@acm.org, theophilusx@gmail.com, > rms@gnu.org, acm@muc.de, emacs-devel@gnu.org > Date: Fri, 01 Jul 2022 15:52:53 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Please don't forget that GC doesn't only collects unused Lisp objects, > > it also does other useful memory-management related tasks. It > > compacts buffer text and strings, and it frees unused slots in various > > caches (font cache, image cache, etc.). You can find in the archives > > discussions where innocently-looking code could cause Emacs run out of > > memory because it used too many fonts without flushing the font cache > > (any program that works on the list of fonts returned by the likes of > > x-list-fonts is in danger of bumping into that). > > Then, if we decide to implement the macro I am suggesting, such macro > should not affect memory allocation of such sensitive objects: font > cache, image cache, etc. Why not? The same could happen with those cached objects. And doing these jobs does take CPU time. > 1. In addition to directly bumping the TOTAL counter of newly allocated > memory, we can introduce a new LOCAL counter holding recent > allocations within current sexp. > 2. Every time we return from a sexp/self-quoting object into assignment, > if we are inside the proposed macro and also assigning value to one > of the pre-defined symbols, increase the upper LOCAL counter in the > parent sexp. Otherwise, do not change the upper LOCAL counter. > 3. Perform GC according to TOTAL-LOCAL threshold value. > 4. When exiting the macro, set LOCAL to 0, unless inside another such > macro. Is this aligned with what the implementation of the Lisp interpreter actually does? I'm not sure we know, where we allocate memory for Lisp data, whether we are going to bind some variable to the produced data. Thus "count recent allocations within the current sexp" sounds like non-trivial to implement. And what do you mean by "pre-defined symbols"? what makes a symbol "pre-defined"? I also am not sure we have the "parent sexp" in hand, but I'll let experts on the Lisp interpreter to comment on that. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Lisp-level macro to avoid excessive GC in memory-allocating code (was: Larger GC thresholds for non-interactive Emacs) 2022-07-01 10:45 ` Eli Zaretskii @ 2022-07-01 11:12 ` Ihor Radchenko 2022-07-01 13:56 ` Lisp-level macro to avoid excessive GC in memory-allocating code Stefan Monnier 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-07-01 11:12 UTC (permalink / raw) To: Eli Zaretskii; +Cc: monnier, mattiase, theophilusx, rms, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> > Please don't forget that GC doesn't only collects unused Lisp objects, >> > it also does other useful memory-management related tasks. It >> > compacts buffer text and strings, and it frees unused slots in various >> > caches (font cache, image cache, etc.). You can find in the archives >> > discussions where innocently-looking code could cause Emacs run out of >> > memory because it used too many fonts without flushing the font cache >> > (any program that works on the list of fonts returned by the likes of >> > x-list-fonts is in danger of bumping into that). >> >> Then, if we decide to implement the macro I am suggesting, such macro >> should not affect memory allocation of such sensitive objects: font >> cache, image cache, etc. > > Why not? The same could happen with those cached objects. And doing > these jobs does take CPU time. I do not have enough knowledge on the topic to argue on this. What I wanted to say is that we can make some objects to be not affected by the suggested macro if there are reasons to believe that it could be too dangerous. >> 1. In addition to directly bumping the TOTAL counter of newly allocated >> memory, we can introduce a new LOCAL counter holding recent >> allocations within current sexp. >> 2. Every time we return from a sexp/self-quoting object into assignment, >> if we are inside the proposed macro and also assigning value to one >> of the pre-defined symbols, increase the upper LOCAL counter in the >> parent sexp. Otherwise, do not change the upper LOCAL counter. >> 3. Perform GC according to TOTAL-LOCAL threshold value. >> 4. When exiting the macro, set LOCAL to 0, unless inside another such >> macro. > > Is this aligned with what the implementation of the Lisp interpreter > actually does? I'm not sure we know, where we allocate memory for > Lisp data, whether we are going to bind some variable to the produced > data. Thus "count recent allocations within the current sexp" sounds > like non-trivial to implement. I am not 100% sure, but AFAIK lisp interpreter always knows the current lisp nesting level. The extra C-level variable I suggest may be an array of max-lisp-eval-depth size. Then, at every given sexp nesting level, if we know the current level, we automatically gain access to the appropriate extra in-sexp memory allocation info. It is also trivial to ensure that all the deeper nesting levels in the array are set to 0 (after appropriate update of the value corresponding to the current nesting) > And what do you mean by "pre-defined symbols"? what makes a symbol > "pre-defined"? I was referring to (with-no-gc '(pre-defined-symbol1 symbol2 ...) ...) I am not sure if my description above is 100% accurate, but the general idea is: 1. Introduce LOCAL variable holding total memory used to allocate objects within current lisp nesting. This variable can be represented by an array of max-lisp-eval-depth size. 2. GC is triggered based on TOTAL-LOCAL instead of currently used TOTAL. 3. LOCAL[i]+=LOCAL[i+1]; LOCAL[i+1]=0 should be done after anything potentially involving recursive sexp call, except symbol slot assignments to non-"pre-defined" symbols. 4. If we are assigning symbol value slot and the symbol value is not "pre-defined" in the macro, LOCAL is not incremented in 3, which will make all the memory allocated inside that sexp to be counted towards next GC threshold. The above effectively allows GCs only upon returning from symbol assignments (in let/setq/setf/etc). But I believe that it should not be a big deal. > I also am not sure we have the "parent sexp" in hand, but I'll let > experts on the Lisp interpreter to comment on that. Agree. I am nowhere expert. Mostly tried to look into setq implementation I come up with something. Hopefully people more familiar with the interpreter chime in. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Lisp-level macro to avoid excessive GC in memory-allocating code 2022-07-01 11:12 ` Ihor Radchenko @ 2022-07-01 13:56 ` Stefan Monnier 2022-07-01 14:10 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-07-01 13:56 UTC (permalink / raw) To: Ihor Radchenko Cc: Eli Zaretskii, mattiase, theophilusx, rms, acm, emacs-devel >> Is this aligned with what the implementation of the Lisp interpreter >> actually does? I'm not sure we know, where we allocate memory for >> Lisp data, whether we are going to bind some variable to the produced >> data. Thus "count recent allocations within the current sexp" sounds >> like non-trivial to implement. That's also my impression. > I am not 100% sure, but AFAIK lisp interpreter always knows the current > lisp nesting level. Not really, no. There is some related info, but that's a non-trivial part of the cost of interpretation so any increase to that info will tend to slow us down. A simple `with-mostly-allocating-code` macro that increases `gc-cons-percentage` to 1.0 sounds a lot simpler and won't impose any extra burden on "normal" code. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Lisp-level macro to avoid excessive GC in memory-allocating code 2022-07-01 13:56 ` Lisp-level macro to avoid excessive GC in memory-allocating code Stefan Monnier @ 2022-07-01 14:10 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-07-01 14:10 UTC (permalink / raw) To: Stefan Monnier; +Cc: yantar92, mattiase, theophilusx, rms, acm, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Eli Zaretskii <eliz@gnu.org>, mattiase@acm.org, theophilusx@gmail.com, > rms@gnu.org, acm@muc.de, emacs-devel@gnu.org > Date: Fri, 01 Jul 2022 09:56:52 -0400 > > A simple `with-mostly-allocating-code` macro that increases > `gc-cons-percentage` to 1.0 sounds a lot simpler and won't impose any > extra burden on "normal" code. I agree, but if we are going to introduce such an implementation, I think we should make our handling of memory-full situation smarter. Like perhaps keep somewhat larger memory reserve, so that if we hit the memory limit, we could still call GC and continue, instead of signaling a fatal error. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-17 7:18 ` Larger GC thresholds for non-interactive Emacs (was: Org mode and Emacs) Ihor Radchenko 2022-06-17 13:49 ` Larger GC thresholds for non-interactive Emacs Stefan Monnier @ 2022-06-21 13:04 ` Lars Ingebrigtsen 2022-06-21 13:35 ` Lars Ingebrigtsen 1 sibling, 1 reply; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-21 13:04 UTC (permalink / raw) To: Ihor Radchenko Cc: Stefan Monnier, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel I think we settled on 1.0 being a promising new default value for -batch, so I've now pushed a change that makes -batch default to that. This does change the memory profile (making batch processes use more memory), so please give feedback on any regressions you see in real world usage -- if it turns out to be problematic, we'll have to revert it. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Larger GC thresholds for non-interactive Emacs 2022-06-21 13:04 ` Larger GC thresholds for non-interactive Emacs Lars Ingebrigtsen @ 2022-06-21 13:35 ` Lars Ingebrigtsen 0 siblings, 0 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-21 13:35 UTC (permalink / raw) To: Ihor Radchenko Cc: Stefan Monnier, Mattias Engdegård, Eli Zaretskii, Tim Cross, rms, Alan Mackenzie, emacs-devel I've tried doing builds on various machines. Here's from my M1 Apple laptop. Before: make -j8 bootstrap 717.57s user 64.69s system 482% cpu 2:42.11 total After: make -j8 bootstrap 457.93s user 62.46s system 415% cpu 2:05.25 total Nice reduction in time spent (and impressive lowering of the CPU work done). -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-15 5:19 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Ihor Radchenko 2022-06-15 6:46 ` Org mode and Emacs David Engster 2022-06-15 12:50 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii @ 2022-06-16 3:19 ` Pankaj Jangid 2022-06-16 4:03 ` Visuwesh 2 siblings, 1 reply; 505+ messages in thread From: Pankaj Jangid @ 2022-06-16 3:19 UTC (permalink / raw) To: emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: >> This is not needed for the emacs-28 branch, >> so I meant master. > > I am not too familiar with how Emacs handles > Org updates. AFAIK, Emacs master only tracks > our stable tagged releases. Not the working > branches (both bugfix and main). I do not > know the motivation behind it and I do not > know if it is a good idea to change it. Do > you think that Emacs master should instead > track our Org dev branch? > Emacs master is the development branch of Emacs. I have always wanted Org dev branch to be part of that. But may be that Org’s dev branch is not as stable as Emacs’. Hence it is kept that way. I have complained about this branch confusion once on the emacs-devel mailing list. Only to learn that there is a separate mailing list for org development related things. Even for reporting bugs there was a different mailing list for Org. I am not sure what is the situation now. I use whichever version comes with master branch of Emacs. And I report bugs using ‘M-x report-emacs-bug’ only. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-16 3:19 ` Org mode and Emacs Pankaj Jangid @ 2022-06-16 4:03 ` Visuwesh 0 siblings, 0 replies; 505+ messages in thread From: Visuwesh @ 2022-06-16 4:03 UTC (permalink / raw) To: Pankaj Jangid; +Cc: emacs-devel [வியாழன் ஜூன் 16, 2022 08:49] Pankaj Jangid wrote: > Even for reporting bugs there was a different > mailing list for Org. I am not sure what is the > situation now. I use whichever version comes > with master branch of Emacs. And I report bugs > using ‘M-x report-emacs-bug’ only. The situation is still the same and you're supposed to report org-mode bugs with M-x org-submit-bug-report. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 14:36 ` Eli Zaretskii 2022-06-12 15:31 ` Org mode and Emacs Colin Baxter 2022-06-15 5:19 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Ihor Radchenko @ 2022-09-25 2:14 ` Bastien 2 siblings, 0 replies; 505+ messages in thread From: Bastien @ 2022-09-25 2:14 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Ihor Radchenko, theophilusx, rms, monnier, acm, emacs-devel Here is the email I sent when I accepted to switch to using .org as the native format for Org documentation: https://list.orgmode.org/orgmode/87371gfas7.fsf@bzg.fr/ As you can read, I wanted to make the switch as an experiment to see if we were really solving a problem here. I believe we didn't get more contributions to the manual by switching to .org, so I'd be in favor of switching back to using .texi as the native format for Org's documentation. (Not for 9.6, obviously, more probably for 10.0 -- I'll discuss this with other Org maintainers.) The other topics in this thread (make Org's Texinfo exporter provide good .texi manuals, make Org more modular, etc.) are interesting, but they are really separate questions IMHO. -- Bastien ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 10:38 ` Ihor Radchenko 2022-06-12 14:36 ` Eli Zaretskii @ 2022-06-12 14:58 ` Eli Zaretskii 2022-06-15 5:48 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-12 14:58 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, rms@gnu.org, monnier@iro.umontreal.ca, > acm@muc.de, emacs-devel@gnu.org > Date: Sun, 12 Jun 2022 18:38:45 +0800 > > >> It may also help if you try to profile org-make-manuals from > >> mk/org-fixup.el and share the results. > > > > If profiling can help, wouldn't it be simpler to invoke the same > > commands from an interactive Emacs session, then show the profile? > > This is exactly what I meant. To run org-make-manuals from interactive > Emacs session. I've run org-texinfo-export-to-texinfo instead. Is that OK? Here's the profile, collected after loading ox-texinfo.el (NOT .elc): 35 54% - ... 31 48% Automatic GC 4 6% - minibuffer-complete 4 6% - completion-in-region 4 6% - completion--in-region 4 6% - #<compiled -0x1b56346374fd962> 4 6% - apply 4 6% - #<compiled 0x1145271b5beccb90> 4 6% - completion--in-region-1 4 6% - completion--do-completion 4 6% - completion-try-completion 4 6% - completion--nth-completion 4 6% - completion--some 4 6% - #<compiled -0x1c897b8c342611de> 4 6% - completion-basic-try-completion 4 6% - try-completion 4 6% - #<compiled -0x1144c96014867ed9> 4 6% complete-with-action 29 45% - command-execute 29 45% - call-interactively 25 39% - funcall-interactively 25 39% - execute-extended-command 25 39% - command-execute 25 39% - call-interactively 25 39% - funcall-interactively 25 39% - org-texinfo-export-to-texinfo 25 39% - let 23 35% - org-export-to-file 23 35% - org-export-as 10 15% - org-macro-initialize-templates 10 15% - org-macro--collect-macros 5 7% org-macro--find-keyword-value 3 4% - org-collect-keywords 3 4% - org--collect-keywords-1 1 1% - org--collect-keywords-1 1 1% - org-element-at-point 1 1% - org-element--parse-to 1 1% org-element--current-element 2 3% - org-macro--find-date 2 3% org-macro--find-keyword-value 6 9% org-export--delete-comment-trees 4 6% org-export-expand-include-keyword 3 4% - org-macro-replace-all 3 4% - org-macro-expand 3 4% - apply 3 4% - #<lambda 0xf2f300dd4> 3 4% - save-current-buffer 3 4% - set-buffer 3 4% - find-file-noselect 2 3% - find-file-noselect-1 1 1% - after-find-file 1 1% - normal-mode 1 1% - set-auto-mode 1 1% - set-buffer-major-mode 1 1% - fundamental-mode 1 1% - run-mode-hooks 1 1% - hack-local-variables 1 1% - hack-dir-local-variables 1 1% - hack-dir-local--get-variables 1 1% - dir-locals-find-file 1 1% - locate-dominating-file 1 1% dir-locals--all-files 1 1% find-buffer-visiting 2 3% org-export-output-file-name 4 6% - byte-code 4 6% - read-extended-command 4 6% - completing-read 4 6% - completing-read-default 2 3% read-from-minibuffer ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 14:58 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii @ 2022-06-15 5:48 ` Ihor Radchenko 2022-06-15 16:49 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-15 5:48 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> This is exactly what I meant. To run org-make-manuals from interactive >> Emacs session. > > I've run org-texinfo-export-to-texinfo instead. Is that OK? It is a valid option. org-make-manuals just opens the file and runs org-texinfo-export-to-texinfo. > Here's the profile, collected after loading ox-texinfo.el (NOT .elc): > > 35 54% - ... > 31 48% Automatic GC > 10 15% - org-macro--collect-macros > 5 7% org-macro--find-keyword-value > 2 3% - org-macro--find-date > 2 3% org-macro--find-keyword-value > 6 9% org-export--delete-comment-trees > 4 6% org-export-expand-include-keyword The common feature of all the above functions is that they are structured as: (goto-char 1) (while (re-search-forward regexp) (do-stuff)) The fact that no-sublevels are revealed by the profiler signifies that it is regexp search that is likely taking a long time. Note that I currently experience a significant degradation in re-search-forward performance on some Org file using Emacs 29 master, but not Emacs 28 and earlier. What will happen if you run (progn (goto-char 1) (while (re-search-forward "^\\*+ " nil t))) right after exporting. Does it take siginficant amount of time? What if you run M-: (org-macro--collect-macros) directly in the buffer? Is it also slow? Is the profiler data more detailed in such case? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-15 5:48 ` Ihor Radchenko @ 2022-06-15 16:49 ` Eli Zaretskii 2022-06-17 6:11 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-15 16:49 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, rms@gnu.org, monnier@iro.umontreal.ca, > acm@muc.de, emacs-devel@gnu.org > Date: Wed, 15 Jun 2022 13:48:05 +0800 > > What will happen if you run > (progn (goto-char 1) (while (re-search-forward "^\\*+ " nil t))) > right after exporting. Does it take siginficant amount of time? It's instantaneous. > What if you run M-: (org-macro--collect-macros) directly in the buffer? Also instantaneous. > Is the profiler data more detailed in such case? Sorry, I don't understand what you mean here. What should I do to get that possibly more detailed profiler data? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-15 16:49 ` Eli Zaretskii @ 2022-06-17 6:11 ` Ihor Radchenko 2022-06-17 6:41 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-17 6:11 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> What will happen if you run >> (progn (goto-char 1) (while (re-search-forward "^\\*+ " nil t))) >> right after exporting. Does it take siginficant amount of time? > > It's instantaneous. > >> What if you run M-: (org-macro--collect-macros) directly in the buffer? > > Also instantaneous. And yet it took 15% of CPU time according to your profile. Something fishy is going on here. My guess that it is GC. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-17 6:11 ` Ihor Radchenko @ 2022-06-17 6:41 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-17 6:41 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, rms@gnu.org, monnier@iro.umontreal.ca, > acm@muc.de, emacs-devel@gnu.org > Date: Fri, 17 Jun 2022 14:11:05 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> What will happen if you run > >> (progn (goto-char 1) (while (re-search-forward "^\\*+ " nil t))) > >> right after exporting. Does it take siginficant amount of time? > > > > It's instantaneous. > > > >> What if you run M-: (org-macro--collect-macros) directly in the buffer? > > > > Also instantaneous. > > And yet it took 15% of CPU time according to your profile. Something > fishy is going on here. My guess that it is GC. That'd be strange, because AFAIK the profiler accounts for GC separately. Anyway, I think your latest changes close this issue for good. Thanks! ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 10:04 ` Ihor Radchenko 2022-06-12 10:15 ` Eli Zaretskii @ 2022-06-12 19:25 ` Jean Louis 2022-06-13 12:14 ` Eli Zaretskii 1 sibling, 1 reply; 505+ messages in thread From: Jean Louis @ 2022-06-12 19:25 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Eli Zaretskii, theophilusx, rms, monnier, acm, emacs-devel * Ihor Radchenko <yantar92@gmail.com> [2022-06-12 21:00]: > You are free to open discussion in Org ML about turning back to texi > format, but it will, for example, make things harder for me and other > people who are not familiar with texinfo format. I am not sure if build > time justifies extra maintenance burden. If Org exports to texi, export that file, and how I understand it emacs build process was using maybe Org file to convert it to texi. Why not export texi internally in Org project, then let Emacs build process use texi file only. -- Jean Take action in Free Software Foundation campaigns: https://www.fsf.org/campaigns In support of Richard M. Stallman https://stallmansupport.org/ ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 19:25 ` Jean Louis @ 2022-06-13 12:14 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-13 12:14 UTC (permalink / raw) To: Jean Louis; +Cc: yantar92, theophilusx, rms, monnier, acm, emacs-devel > Date: Sun, 12 Jun 2022 22:25:47 +0300 > From: Jean Louis <bugs@gnu.support> > Cc: Eli Zaretskii <eliz@gnu.org>, theophilusx@gmail.com, rms@gnu.org, > monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > > Why not export texi internally in Org project, then let Emacs build > process use texi file only. Because the real source is in Org, and we in the GNU Project are always providing the real source files, not the products of their conversion. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 9:05 ` Ihor Radchenko 2022-06-12 9:18 ` Eli Zaretskii @ 2022-06-13 22:37 ` Richard Stallman 2022-06-14 0:43 ` Ihor Radchenko 2022-06-14 2:28 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 1 sibling, 2 replies; 505+ messages in thread From: Richard Stallman @ 2022-06-13 22:37 UTC (permalink / raw) To: Ihor Radchenko; +Cc: eliz, theophilusx, monnier, acm, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > So, now we have our manual written in Org mode and we never had reasons > to come back to texi. I suspect that it doesn't fully follow the markup conventions for GNU manuals. That's because we designed Texinfo to have markup commands to make all the proper semantic distinctions. If the manual source is written in a language which doesn't have the full gamut of markup distinctions, there is no way to do the markup correctly. It would be useful for someone who understands these conventions to check the Org manual and see. I don't think the slowness of processing nowadays is the crucial issue here. Computers are much faster now than in the 1990s. It used to take a long time for TeX to process the Emacs Lisp Reference Manual. Now it is perhaps 10 times as fast. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-13 22:37 ` Richard Stallman @ 2022-06-14 0:43 ` Ihor Radchenko [not found] ` <87h74mv56b.fsf@localhost> 2022-06-14 2:28 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 1 sibling, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-14 0:43 UTC (permalink / raw) To: rms; +Cc: eliz, theophilusx, monnier, acm, emacs-devel, emacs-orgmode I am CCing Org ML from now signifying that this branch of the thread is directly relevent to Org mode and might be of interest for other Org contributors. Richard Stallman <rms@gnu.org> writes: > > So, now we have our manual written in Org mode and we never had reasons > > to come back to texi. > > I suspect that it doesn't fully follow the markup conventions > for GNU manuals. That's because we designed Texinfo to have markup commands > to make all the proper semantic distinctions. If the manual source > is written in a language which doesn't have the full gamut of markup > distinctions, there is no way to do the markup correctly. > > It would be useful for someone who understands these conventions > to check the Org manual and see. Yes, it would certainly help! The work on manual has been done a long time ago and we also extended our texinfo exported to suit the manual at that time. Assumingly, just enough to handle the Org manual use-cases. Note that we have doc/Documentation_Standards.org explaining some of the conventions. Here is a possibly relevant note inside it: - Only two of the standard Texinfo indexes are used; those for concepts and keys. This has some implications: + The preference is to document commands by key rather than by name + Texinfo commands such as @var and @defoption are not used. The preference for this type of thing is that the user browses the customize groups. If you want or need to refer to, say, a variable then document it as "the variable @code{org-startup-folded}" + Entries in the concept index are normally all lower case unless some other rule dictates otherwise. Without knowing texinfo, the above paragraphs do not make a whole lot of sense for me. So, if someone points out any omissions, it would be helpful for future Org contributors. > I don't think the slowness of processing nowadays is the crucial issue > here. Computers are much faster now than in the 1990s. It used > to take a long time for TeX to process the Emacs Lisp Reference Manual. > Now it is perhaps 10 times as fast. Generally, there is no way Org export to .info gets any faster than texinfo. Org is only able to export to other text formats: org->texi; org->tex; org->html; etc. Convertion to more low-level formats is left to the external tools like texinfo and pdflatex. As for reports on the slow performance, they are still useful as long as they reveal some bottlenecks in Org exporter. Org is being used to export large documents and whole websites. Hence, improving performance in this area is generally helpful, even if it is not strictly a critical blocking issue. What I want to say is: do not expect Org export to be faster than native binary tools, but do not hesitate to report performance issues either. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
[parent not found: <87h74mv56b.fsf@localhost>]
* Org syntax compatibility with texinfo syntax (was: Org mode and Emacs (was: Convert README.org to plain text README while installing package)) [not found] ` <87h74mv56b.fsf@localhost> @ 2022-06-17 6:42 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-17 6:42 UTC (permalink / raw) To: rms; +Cc: eliz, theophilusx, monnier, acm, emacs-orgmode, emacs-devel I am forwarding yet another email from the emacs-devel thread to here. This thread includes both emacs-devel and Org ML. I will still keep emacs-devel in the loop because people from there can provide a useful insight about texinfo capabilities. Richard Stallman wrote: > I don't know for certain that every possible nesting "does the right > thing". I do know that @var{} is used inside many other constructs. > By contrast, @dfn{} would not be nested inside or around other > contructs very much. @key can be nested inside @kbd, and it behaves > a little differently when nested. signifying (in addition to previous message), that 1. Texinfo has some software documentation-specific markup elements 2. Those elements can be nested in non-trivial ways, similar to Org's org-element-object-restrictions, the that nesting can affect formatted export (again, similar to Org). With regards to Org syntax, I do not think that we must include all the possible texinfo elements into Org standard. @dfn, @key, and @var constructs are extremely specific to software documentation and have no sense to maintain as a part of general-purpose markup syntax. However, what we can do is to allow extending Org syntax to support such elements. We do support custom syntax elements already (or are discussing such support): We have the following, potentially customizeable syntax elements: - Org links (extensible via org-link-set-parameters) - Special blocks (de-facto extensible in export backends; also, https://github.com/alhassy/org-special-block-extras/); https://list.orgmode.org/87edzqv4ha.fsf@localhost/T/#m6b95119faa65645fd1c32b0e17b6440f73ecd3af - [FR] Inline special blocks idea being discussed in https://orgmode.org/list/87a6b8pbhg.fsf@posteo.net The links are somewhat limited wrt nesting: links cannot be nested inside links thus limiting their usefulness as custom markup. Hence, we may need to look more closely into the idea of inline special blocks (discuss it in https://orgmode.org/list/87a6b8pbhg.fsf@posteo.net). Special blocks already support custom :type, but lack convenient Elisp interface like org-link-set-parameters. (let's keep this part in https://list.orgmode.org/87edzqv4ha.fsf@localhost/T/#m6b95119faa65645fd1c32b0e17b6440f73ecd3af) I'd like to keep discussion on the specifics of customizeable Org syntax in the relevant threads. In here, I'd like to hear feedback on possible additional incompatibilities between texinfo and Org. Is there anything (not covered by the above list) that is present in texinfo, but impossible in Org? For starters, Org does not have a full support for generating index (apart from ox-texinfo and ox-publish). Though see https://github.com/tecosaur/org-glossary Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-13 22:37 ` Richard Stallman 2022-06-14 0:43 ` Ihor Radchenko @ 2022-06-14 2:28 ` Eli Zaretskii 2022-06-14 2:45 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-14 2:28 UTC (permalink / raw) To: rms; +Cc: yantar92, theophilusx, monnier, acm, emacs-devel > From: Richard Stallman <rms@gnu.org> > Cc: eliz@gnu.org, theophilusx@gmail.com, monnier@iro.umontreal.ca, > acm@muc.de, emacs-devel@gnu.org > Date: Mon, 13 Jun 2022 18:37:04 -0400 > > I don't think the slowness of processing nowadays is the crucial issue > here. Computers are much faster now than in the 1990s. It used > to take a long time for TeX to process the Emacs Lisp Reference Manual. > Now it is perhaps 10 times as fast. The Emacs Lisp Reference manual gets processed in about 15 seconds here. So 2.5 minutes required to process org.org into org.texi is quite annoying. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-14 2:28 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii @ 2022-06-14 2:45 ` Ihor Radchenko 2022-06-14 11:04 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-14 2:45 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rms, theophilusx, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> I don't think the slowness of processing nowadays is the crucial issue >> here. Computers are much faster now than in the 1990s. It used >> to take a long time for TeX to process the Emacs Lisp Reference Manual. >> Now it is perhaps 10 times as fast. > > The Emacs Lisp Reference manual gets processed in about 15 seconds > here. So 2.5 minutes required to process org.org into org.texi is > quite annoying. To be fair, 2.5 minutes are likely caused by unoptimized Emacs build because Org has to use the same unoptimized Emacs executable. In contrast, texinfo is an external tool with release build. So, while some generic improvements can likely be made on Org side, do not expect orders of magnitude improvements. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-14 2:45 ` Ihor Radchenko @ 2022-06-14 11:04 ` Eli Zaretskii 2022-06-14 11:18 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-14 11:04 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, theophilusx, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: rms@gnu.org, theophilusx@gmail.com, monnier@iro.umontreal.ca, > acm@muc.de, emacs-devel@gnu.org > Date: Tue, 14 Jun 2022 10:45:02 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> I don't think the slowness of processing nowadays is the crucial issue > >> here. Computers are much faster now than in the 1990s. It used > >> to take a long time for TeX to process the Emacs Lisp Reference Manual. > >> Now it is perhaps 10 times as fast. > > > > The Emacs Lisp Reference manual gets processed in about 15 seconds > > here. So 2.5 minutes required to process org.org into org.texi is > > quite annoying. > > To be fair, 2.5 minutes are likely caused by unoptimized Emacs build > because Org has to use the same unoptimized Emacs executable. As an Emacs maintainer, I must have the development version built without optimizations in at least one branch, because I need to be able to debug the latest Emacs code at all times. So for me (and I presume other frequent contributors to the development) building and using an unoptimized binary is a must, part of my daily routine. An optimized build of Emacs produces org.texi in about 30 to 40 seconds here, and that is also quite annoying for a production build. Lars recently made significant changes in our build scripts and support code because he didn't like much shorter delays. We build Emacs many times a week, so these delays add up. > So, while some generic improvements can likely be made on Org side, do > not expect orders of magnitude improvements. Any improvement will be welcome, thank you. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-14 11:04 ` Eli Zaretskii @ 2022-06-14 11:18 ` Ihor Radchenko 2022-06-14 11:44 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-14 11:18 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rms, theophilusx, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > An optimized build of Emacs produces org.texi in about 30 to 40 > seconds here, and that is also quite annoying for a production build. > Lars recently made significant changes in our build scripts and > support code because he didn't like much shorter delays. We build > Emacs many times a week, so these delays add up. Is there any practical reason to re-build Org manual every time you rebuild Emacs, even if the source did not change? I though that make would skip building unchanged sources. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-14 11:18 ` Ihor Radchenko @ 2022-06-14 11:44 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-14 11:44 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, theophilusx, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: rms@gnu.org, theophilusx@gmail.com, monnier@iro.umontreal.ca, > acm@muc.de, emacs-devel@gnu.org > Date: Tue, 14 Jun 2022 19:18:57 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > > An optimized build of Emacs produces org.texi in about 30 to 40 > > seconds here, and that is also quite annoying for a production build. > > Lars recently made significant changes in our build scripts and > > support code because he didn't like much shorter delays. We build > > Emacs many times a week, so these delays add up. > > Is there any practical reason to re-build Org manual every time you > rebuild Emacs, even if the source did not change? No, and it doesn't happen unless I bootstrap. > I though that make would skip building unchanged sources. Indeed, it does (except as part of "make bootstrap", which rebuilds everything). But even without a bootstrap, when org.org changes, building several branches needs to process it once per branch. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-11 6:58 ` Eli Zaretskii 2022-06-11 7:59 ` Tim Cross 2022-06-12 9:05 ` Ihor Radchenko @ 2022-06-12 22:38 ` Richard Stallman 2022-06-13 4:38 ` Org mode and Emacs Werner LEMBERG 2022-06-13 16:28 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Christopher Dimech 2 siblings, 2 replies; 505+ messages in thread From: Richard Stallman @ 2022-06-12 22:38 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, yantar92, monnier, acm, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > some relating to org, some relating to the fact I'm not convinced there > > is a need to replace texinfo, especially given that in some areas, I > > think texinfo is more feature rich and consistent and some relating to the feeling I have > > that the result is hard to justify given the amount of work it would > > require. Why replace Texinfo? Because it has some tricky syntax which it inherits directly from TeX. And some syntax which is cumbersome, again imposed directly by TeX. Nowadays nobody can maintain the TeX-based part which is used to generate printed manuals. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 22:38 ` Richard Stallman @ 2022-06-13 4:38 ` Werner LEMBERG 2022-06-13 16:28 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Christopher Dimech 1 sibling, 0 replies; 505+ messages in thread From: Werner LEMBERG @ 2022-06-13 4:38 UTC (permalink / raw) To: rms; +Cc: eliz, theophilusx, yantar92, monnier, acm, emacs-devel > Why replace Texinfo? Because it has some tricky syntax which it > inherits directly from TeX. And some syntax which is cumbersome, > again imposed directly by TeX. Well, the thing is backward compatibility – for the sake of the GNU project. If Texinfo was allowed to drop that (in particular, the many unfortunate restrictions of the `.info` file format), everything would be much easier. > Nowadays nobody can maintain the TeX-based part which is used to > generate printed manuals. How do you come to this conclusion? `texinfo.tex` is actively maintained, and bug reports (which I submitted a lot) are usually fixed within days. Of course, there are some areas where more development would be desirable (in particular, a better font interface based on ideas from LaTeX). On the other hand, the next Texinfo version will contain a converter from Texinfo source to LaTeX source, which should help overcome (at least some of) the font problems. > (In the Info version of the file, the title of the node for @code is > actually formatted wrong. It has single quotes around `@code' in > the title itself. Texinfo does have some problems that need fixing, > and it is hard to fix anything in it.) I strongly suggest that you write to bug-texinfo, discussing this issue and your other concerns. Werner ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 22:38 ` Richard Stallman 2022-06-13 4:38 ` Org mode and Emacs Werner LEMBERG @ 2022-06-13 16:28 ` Christopher Dimech 1 sibling, 0 replies; 505+ messages in thread From: Christopher Dimech @ 2022-06-13 16:28 UTC (permalink / raw) To: rms; +Cc: Eli Zaretskii, theophilusx, yantar92, monnier, acm, emacs-devel > Sent: Monday, June 13, 2022 at 10:38 AM > From: "Richard Stallman" <rms@gnu.org> > To: "Eli Zaretskii" <eliz@gnu.org> > Cc: theophilusx@gmail.com, yantar92@gmail.com, monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Subject: Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) > > [[[ To any NSA and FBI agents reading my email: please consider ]]] > [[[ whether defending the US Constitution against all enemies, ]]] > [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > > > some relating to org, some relating to the fact I'm not convinced there > > > is a need to replace texinfo, especially given that in some areas, I > > > think texinfo is more feature rich and consistent and some relating to the feeling I have > > > that the result is hard to justify given the amount of work it would > > > require. > > Why replace Texinfo? Because it has some tricky syntax which it > inherits directly from TeX. And some syntax which is cumbersome, > again imposed directly by TeX. > > Nowadays nobody can maintain the TeX-based part which is used to > generate printed manuals. > Correct. You got to move on beyond tex no matter the amount of work. > > -- > Dr Richard Stallman (https://stallman.org) > Chief GNUisance of the GNU Project (https://gnu.org) > Founder, Free Software Foundation (https://fsf.org) > Internet Hall-of-Famer (https://internethalloffame.org) > > > > ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-11 4:49 ` Tim Cross 2022-06-11 6:58 ` Eli Zaretskii @ 2022-06-12 0:42 ` Richard Stallman 2022-06-12 1:27 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Richard Stallman @ 2022-06-12 0:42 UTC (permalink / raw) To: Tim Cross; +Cc: eliz, yantar92, monnier, acm, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] One of Texinfo's crucial features is a wide range of semantic markup constructs, each of which can generate different output depending on the output format. For instance, Texinfo has @var, @emph and @dfn, all of which generate italics in printed output, but they differ in what they generate for other output formats. There are probably 15 other such constructs. Does Org format have the ability to make all these distinctions? If not, I suspect that the Org mode documentation isn't following all our style conventions for documentation. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 0:42 ` Richard Stallman @ 2022-06-12 1:27 ` Ihor Radchenko 2022-06-12 5:45 ` Eli Zaretskii 2022-06-12 22:38 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-12 1:27 UTC (permalink / raw) To: rms; +Cc: Tim Cross, eliz, monnier, acm, emacs-devel Richard Stallman <rms@gnu.org> writes: > One of Texinfo's crucial features is a wide range of semantic markup > constructs, each of which can generate different output depending on > the output format. For instance, Texinfo has @var, @emph and @dfn, > all of which generate italics in printed output, but they differ in > what they generate for other output formats. There are probably 15 > other such constructs. Could you please point to the place where I can read about the different generated output? > Does Org format have the ability to make all these distinctions? > If not, I suspect that the Org mode documentation isn't following > all our style conventions for documentation. Org manual does not AFAIK, other than various index entries. However, I am not sure what exactly you are referring to. Could you please provide a link describing the conventions you are referring to? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 1:27 ` Ihor Radchenko @ 2022-06-12 5:45 ` Eli Zaretskii 2022-06-12 22:38 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-12 5:45 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, theophilusx, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Tim Cross <theophilusx@gmail.com>, eliz@gnu.org, > monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Date: Sun, 12 Jun 2022 09:27:57 +0800 > > Richard Stallman <rms@gnu.org> writes: > > > One of Texinfo's crucial features is a wide range of semantic markup > > constructs, each of which can generate different output depending on > > the output format. For instance, Texinfo has @var, @emph and @dfn, > > all of which generate italics in printed output, but they differ in > > what they generate for other output formats. There are probably 15 > > other such constructs. > > Could you please point to the place where I can read about the different > generated output? In the Texinfo manual. It describes, for each markup, what that produces in every major output format. > > Does Org format have the ability to make all these distinctions? > > If not, I suspect that the Org mode documentation isn't following > > all our style conventions for documentation. > > Org manual does not AFAIK, other than various index entries. That is strange to hear. I was under the impression that Org can export to various format, and each such output format has its own export support. So at least in principle, the distinction Richard is talking about should be not only possible, but quite easy to implement, if not already implemented. > However, I am not sure what exactly you are referring to. > Could you please provide a link describing the conventions you are > referring to? Most of them are in the Texinfo manual, and some are in the GNU Coding Standards manual. To take one example, @dfn produces italics in HTML and in printed formats (PDF etc.), and quoted text in Info. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 1:27 ` Ihor Radchenko 2022-06-12 5:45 ` Eli Zaretskii @ 2022-06-12 22:38 ` Richard Stallman 2022-06-12 22:56 ` Tim Cross 2022-06-14 13:18 ` Ihor Radchenko 1 sibling, 2 replies; 505+ messages in thread From: Richard Stallman @ 2022-06-12 22:38 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, eliz, monnier, acm, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > For instance, Texinfo has @var, @emph and @dfn, > > all of which generate italics in printed output, but they differ in > > what they generate for other output formats. There are probably 15 > > other such constructs. > Could you please point to the place where I can read about the different > generated output? They are in the Texinfo manual. Look at node "Indicating", which is section 7.1 in the version I have in Info. (In the Info version of the file, the title of the node for @code is actually formatted wrong. It has single quotes around `@code' in the title itself. Texinfo does have some problems that need fixing, and it is hard to fix anything in it.) > One key reason I worry about going down that road is that I suspect it > would complicate org's syntax. Two key benefits of org mode is that the > basic syntax is simple and it maps reasonably consistently acorss > different output formats. However, this flexibility does come at a cost. > To provide consistency across export formats, the basic formatting > 'concepts' need to be somewhat 'generalised', which means at times you > will loose some of the more advanced or sophisticated formatting power > of some export back-ends. I suspect we are slightly miscommunicating, because Texinfo already generates several different formats of output, and each markup construct is carefully defined about how it should appear in each output format. So I'm sure it is possible to define additional markup constucts and make each one do, in each output format, what Texinfo would (or does) do with it. The only hard part is finding syntax for them. > As pointed out elswhere in this thread, org could support missing > texinfo syntax using texinfo specific blocks. However, that isn't a > great experience from an authoring perspective. It is fine if you only > need to do it occasionally, but if you have to constantly add such > blocks in order to get really well formatted texinfo output, This is rather vague. Why do you thik this would be difficult to use Can we find a solution to make it easy to use? We don't have to be limited to whichever escape syntaxes Org already satisfies. This use is _important_. For this, it is worth making new ones. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 22:38 ` Richard Stallman @ 2022-06-12 22:56 ` Tim Cross 2022-06-13 0:39 ` Ihor Radchenko 2022-06-14 13:18 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Tim Cross @ 2022-06-12 22:56 UTC (permalink / raw) To: rms; +Cc: Ihor Radchenko, eliz, monnier, acm, emacs-devel Richard Stallman <rms@gnu.org> writes: > > One key reason I worry about going down that road is that I suspect it > > would complicate org's syntax. Two key benefits of org mode is that the > > basic syntax is simple and it maps reasonably consistently acorss > > different output formats. However, this flexibility does come at a cost. > > To provide consistency across export formats, the basic formatting > > 'concepts' need to be somewhat 'generalised', which means at times you > > will loose some of the more advanced or sophisticated formatting power > > of some export back-ends. > > I suspect we are slightly miscommunicating, because Texinfo already > generates several different formats of output, and each markup construct > is carefully defined about how it should appear in each output format. > > So I'm sure it is possible to define additional markup constucts and > make each one do, in each output format, what Texinfo would (or does) > do with it. > > The only hard part is finding syntax for them. > Perhaps, though I tend to feel you may have misconceptions regarding the purpose of org and its main design goals. The org mode syntax, especially the syntax for markup (markdown), is very small and concise. This is one of its big strengths. It is also one of the main reasons various markdown dialects have become so popular and why we have so many different markdown dialects - people don't want a rich syntax supporting numerous different sematic elements. They just want a very few. In fact, despite the strong arguments in favour of semantic markup, people tend to prefer typographical markup - italics, bold, underline, etc and they want a markup where they don't have to constantly wonder if they have selected the correct semantic tag for a certain bit of text. They want something which is easy to write in, which does not require referring to a manual to lookup different semantic tags and something which is easier than HTML, LaTeX, TexInfo, odt, xml, etc. I also think the statement "The only hard part is finding syntax for them." is a huge understatement. There are already a couple of requested syntax changes for markup which IMO have far greater merit than supporting more texinfo sematnic markup, which have not been implemented primarily due to the difficulty of ding so in a consistent manner which will not also break the large amount of existing org data out there. You mentioned in another thread that one of the big issues with texinfo was that maintenance of the tex part, used primarily for the printed representation, was difficult to maintain these days (assuming lack of people with expertise in tex). I just wanted to point out that much of the output formatting org supports is also achieved via tex and latex. Chances are that if we extended org syntax to support the broader set of texinfo markup, we would also inherit those same issues. At the very least, we would not be escaping from tex simply because we moved to org mode. Where I think there is a misunderstanding of org is with the emphasis which appears to be occurring on the markup and formatting aspects of org. Org's ability to support a small markdown dialect and generate output in various formats is not org's main emphasis. Org's primary role is not that of being a documentation authoring tool. Org is about a lot more, it is about information gathering and management, task and time management, information capture, 'executable' notes and literate programming and lab books with convenient outline editing support. As the name suggests, it is about helping to organise your life. It isn't about writing documentation. The fact it can support multiple output formats and produce reasonable looking documents is certainly part of it, but in some ways, that was the low hanging fruit which was relatively easy to achieve once all the other parts were in place. If a replacement for texinfo is required, then lets take the lessons we have learned from texinfo, latex/tex, xml, html, markdown and org and implement something appropriate. This might seem like too much of a task, but that is probably what people said when Carsten started developing org mode. There are many ways org mode can be improved. Current work to clarify and refine the syntax, provide a solid and efficient parser, improved font-locking and formatting and more consistent and rich API for both extensions and new export backends are far more critical than adding additional markup/markdown syntax to enable org mode to replace texinfo. IMO such changes are not only misguided, they have the real potential to adversely impact org by making things more complicated and harder to maintain, destroying the advantage of a very small and easy to use markup and moving the focus towards becoming an authoring tool rather than a personal data organisation aid. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 22:56 ` Tim Cross @ 2022-06-13 0:39 ` Ihor Radchenko 2022-06-13 1:42 ` Tim Cross ` (2 more replies) 0 siblings, 3 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-13 0:39 UTC (permalink / raw) To: Tim Cross; +Cc: rms, eliz, monnier, acm, emacs-devel Tim Cross <theophilusx@gmail.com> writes: > There are many ways org mode can be improved. Current work to clarify > and refine the syntax, provide a solid and efficient parser, improved > font-locking and formatting and more consistent and rich API for both > extensions and new export backends are far more critical than adding > additional markup/markdown syntax to enable org mode to replace texinfo. > IMO such changes are not only misguided, they have the real potential to > adversely impact org by making things more complicated and harder to > maintain, destroying the advantage of a very small and easy to use > markup and moving the focus towards becoming an authoring tool rather > than a personal data organisation aid. Tim, I afraid that we a slipping to overgeneralisation again. We may or may not need to make changes to Org syntax for the purpose of writing documentation using Org. Can someone please prepare a small example texi file containing examples of various texi features that are commonly needed and possibly also demonstrating some concrete instances of problems that exist in texinfo? If we have such an example and know the expected outputs, someone can then create an equivalent Org file. Then, the discussion will hopefuly become more constructive. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-13 0:39 ` Ihor Radchenko @ 2022-06-13 1:42 ` Tim Cross 2022-06-13 2:40 ` Ihor Radchenko 2022-06-13 1:47 ` Christopher Dimech 2022-06-13 11:54 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 2 siblings, 1 reply; 505+ messages in thread From: Tim Cross @ 2022-06-13 1:42 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, eliz, monnier, acm, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > Tim Cross <theophilusx@gmail.com> writes: > >> There are many ways org mode can be improved. Current work to clarify >> and refine the syntax, provide a solid and efficient parser, improved >> font-locking and formatting and more consistent and rich API for both >> extensions and new export backends are far more critical than adding >> additional markup/markdown syntax to enable org mode to replace texinfo. >> IMO such changes are not only misguided, they have the real potential to >> adversely impact org by making things more complicated and harder to >> maintain, destroying the advantage of a very small and easy to use >> markup and moving the focus towards becoming an authoring tool rather >> than a personal data organisation aid. > > Tim, I afraid that we a slipping to overgeneralisation again. > We may or may not need to make changes to Org syntax for the purpose of > writing documentation using Org. > > Can someone please prepare a small example texi file containing examples > of various texi features that are commonly needed and possibly also > demonstrating some concrete instances of problems that exist in texinfo? > > If we have such an example and know the expected outputs, someone can > then create an equivalent Org file. Then, the discussion will hopefuly > become more constructive. > Well, I think Richard was pretty clear when he asked for the additional semantic tags listed in the Indicating Definitions, Commands & etc section of the texinfo manual (6.1 in my copy). Looking at just that section and it is clear current org markup syntax can handle less than half. Some of those not handled are the types of semantic elements you would tend to encounter in software manuals and that is just one section. However, I think I've made my position clear and have little more to add, so I will bow out of any further discussion. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-13 1:42 ` Tim Cross @ 2022-06-13 2:40 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-13 2:40 UTC (permalink / raw) To: Tim Cross; +Cc: rms, eliz, monnier, acm, emacs-devel Tim Cross <theophilusx@gmail.com> writes: > Well, I think Richard was pretty clear when he asked for the additional > semantic tags listed in the Indicating Definitions, Commands & etc > section of the texinfo manual (6.1 in my copy). Looking at just that > section and it is clear current org markup syntax can handle less than > half. Some of those not handled are the types of semantic elements you > would tend to encounter in software manuals and that is just one section. To clarify, all those extra semantics boil down to either quoting the text inside in ASCII export or transforming them to /italics/ or ~fixed-width~ in LaTeX/HTML export. Moreover, the majority of extra semantics is simply aliases (for example, @command is an alias to @code). Two exceptions are @acronym and @abbr, which can be easily added as special link types. In general, I so nothing in the 6.1 Indicating Definitions, Commands, etc. that cannot be replicated using Org links or existing emphasis objects: @code{sample} = ~sample~ @kbd{M-a} = [[kbd:M-a]] (with @kbdinputstyle being document keyword) @key{RET} = \RET (new entity type) @samp{text} = [[samp:text]] @verb{text} = =text= @var{name} = [[var:name]] @env{variable}= [[env:variable]] @file{path} = [[file:path]] @command{cmd} = [[command:cmd]] @option{opt} = [[option:opt]] @dfn{def} = [[dfn:def]] @abbr{abbrev, meaning} = [[abbr:meaning][abbrev]] same for @acronym. @email{address,text} = [[mailto:address][text]] @indicateurl{url} = [[url]] @emph{text} = /text/ @strong{text} = *text* @sc{text} = [[sc:][text]] Note that for abbrevs, we have WIP https://github.com/tecosaur/org-glossary 7 Quotations and Examples @quotation: Block Quotations #+begin_verbatim @indentedblock: Indented text blocks N/A but can be a special block @example: Example Text #+begin_example @verbatim: Literal Text #+begin_verse @lisp: Marking a Lisp Example #+begin_src @display: Examples Using the Text Font N/A (not sure here) but can also be a special block @format: Examples Using the Full Line Width I am a bit lost here, but again, anything can be a special block basically @exdent: Undoing a Line’s Indentation ... @flushleft and @flushright ... @raggedright: Ragged Right Text ... @noindent: Omitting Indentation \\ I think @indent: Forcing Indentation Not sure here @cartouche: Rounded Rectangles Can be a block option (I am not sure if we can make it universal for all possible backends) @small… Block Commands Can be a block option Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-13 0:39 ` Ihor Radchenko 2022-06-13 1:42 ` Tim Cross @ 2022-06-13 1:47 ` Christopher Dimech 2022-06-13 2:47 ` Ihor Radchenko 2022-06-13 11:54 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 2 siblings, 1 reply; 505+ messages in thread From: Christopher Dimech @ 2022-06-13 1:47 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Tim Cross, rms, eliz, monnier, acm, emacs-devel > Sent: Monday, June 13, 2022 at 12:39 PM > From: "Ihor Radchenko" <yantar92@gmail.com> > To: "Tim Cross" <theophilusx@gmail.com> > Cc: rms@gnu.org, eliz@gnu.org, monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Subject: Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) > > Tim Cross <theophilusx@gmail.com> writes: > > > There are many ways org mode can be improved. Current work to clarify > > and refine the syntax, provide a solid and efficient parser, improved > > font-locking and formatting and more consistent and rich API for both > > extensions and new export backends are far more critical than adding > > additional markup/markdown syntax to enable org mode to replace texinfo. > > IMO such changes are not only misguided, they have the real potential to > > adversely impact org by making things more complicated and harder to > > maintain, destroying the advantage of a very small and easy to use > > markup and moving the focus towards becoming an authoring tool rather > > than a personal data organisation aid. > > Tim, I afraid that we a slipping to overgeneralisation again. > We may or may not need to make changes to Org syntax for the purpose of > writing documentation using Org. > > Can someone please prepare a small example texi file containing examples > of various texi features that are commonly needed and possibly also > demonstrating some concrete instances of problems that exist in texinfo? Does org accept latex structures? Because texinfo is just plain tex, and as consequence remained limited (e.g. no colouring, no bold mathematical expressions' no boxes for formal end of proof). > If we have such an example and know the expected outputs, someone can > then create an equivalent Org file. Then, the discussion will hopefuly > become more constructive. > > Best, > Ihor > > ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-13 1:47 ` Christopher Dimech @ 2022-06-13 2:47 ` Ihor Radchenko 2022-06-13 5:04 ` Christopher Dimech 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-13 2:47 UTC (permalink / raw) To: Christopher Dimech; +Cc: Tim Cross, rms, eliz, monnier, acm, emacs-devel Christopher Dimech <dimech@gmx.com> writes: > Does org accept latex structures? Because texinfo is just plain tex, and as > consequence remained limited (e.g. no colouring, no bold mathematical expressions' > no boxes for formal end of proof). Yes, latex is supported natively. You can mix it into Org text. For non-latex export, it will be either converted to image of left as-is. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-13 2:47 ` Ihor Radchenko @ 2022-06-13 5:04 ` Christopher Dimech 2022-06-13 5:22 ` Org mode and Emacs Werner LEMBERG 0 siblings, 1 reply; 505+ messages in thread From: Christopher Dimech @ 2022-06-13 5:04 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Tim Cross, rms, eliz, monnier, acm, emacs-devel > Sent: Monday, June 13, 2022 at 2:47 PM > From: "Ihor Radchenko" <yantar92@gmail.com> > To: "Christopher Dimech" <dimech@gmx.com> > Cc: "Tim Cross" <theophilusx@gmail.com>, rms@gnu.org, eliz@gnu.org, monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Subject: Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) > > Christopher Dimech <dimech@gmx.com> writes: > > > Does org accept latex structures? Because texinfo is just plain tex, and as > > consequence remained limited (e.g. no colouring, no bold mathematical expressions' > > no boxes for formal end of proof). > > Yes, latex is supported natively. You can mix it into Org text. For > non-latex export, it will be either converted to image of left as-is. > > Best, > Ihor That would be very good development, using org for Official Gnu Documentation. Had discussed with Gavin Smith about latex a couple of years ago, who explained that texinfo would have to be rewritten completely if we are to have a latex engine for it. Lot of work and unlikely to happen. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-13 5:04 ` Christopher Dimech @ 2022-06-13 5:22 ` Werner LEMBERG 2022-06-13 5:59 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Werner LEMBERG @ 2022-06-13 5:22 UTC (permalink / raw) To: dimech; +Cc: yantar92, theophilusx, rms, eliz, monnier, acm, emacs-devel > Had discussed with Gavin Smith about latex a couple of years ago, > who explained that texinfo would have to be rewritten completely if > we are to have a latex engine for it. Lot of work and unlikely to > happen. By the way, has someone informed the Texinfo maintainers that people are whetting knives here on the list, discussing quite fundamental changes about the future of the project? Werner ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-13 5:22 ` Org mode and Emacs Werner LEMBERG @ 2022-06-13 5:59 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-13 5:59 UTC (permalink / raw) To: emacs-devel, Werner LEMBERG, dimech Cc: yantar92, theophilusx, rms, monnier, acm On June 13, 2022 8:22:57 AM GMT+03:00, Werner LEMBERG <wl@gnu.org> wrote: > > > Had discussed with Gavin Smith about latex a couple of years ago, > > who explained that texinfo would have to be rewritten completely if > > we are to have a latex engine for it. Lot of work and unlikely to > > happen. > > By the way, has someone informed the Texinfo maintainers that people > are whetting knives here on the list, discussing quite fundamental > changes about the future of the project? Please don't over-dramatize. No one is "wetting any knives" here, we are just talking. If you followed the discussion, supplanting Texinfo is nowhere in sight yet. This is Emacs, not GCC. And yes, this discussion should move to the Texinfo list, as has been suggested already. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-13 0:39 ` Ihor Radchenko 2022-06-13 1:42 ` Tim Cross 2022-06-13 1:47 ` Christopher Dimech @ 2022-06-13 11:54 ` Eli Zaretskii 2022-06-14 13:32 ` Ihor Radchenko 2 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-13 11:54 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: rms@gnu.org, eliz@gnu.org, monnier@iro.umontreal.ca, acm@muc.de, > emacs-devel@gnu.org > Date: Mon, 13 Jun 2022 08:39:42 +0800 > > Can someone please prepare a small example texi file containing examples > of various texi features that are commonly needed and possibly also > demonstrating some concrete instances of problems that exist in texinfo? Given that you already produced a list of markups, is such an example still wanted? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-13 11:54 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii @ 2022-06-14 13:32 ` Ihor Radchenko 2022-06-14 13:45 ` Eli Zaretskii 2022-06-15 23:15 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-14 13:32 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, rms, monnier, acm, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> Can someone please prepare a small example texi file containing examples >> of various texi features that are commonly needed and possibly also >> demonstrating some concrete instances of problems that exist in texinfo? > > Given that you already produced a list of markups, is such an example > still wanted? It would certainly help with development. For example, I did not look close into allowed markup nesting. File with tricky examples will be helpful as a benchmark for any attempted implementation. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-14 13:32 ` Ihor Radchenko @ 2022-06-14 13:45 ` Eli Zaretskii 2022-06-15 23:15 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-14 13:45 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, rms, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: theophilusx@gmail.com, rms@gnu.org, monnier@iro.umontreal.ca, > acm@muc.de, emacs-devel@gnu.org > Date: Tue, 14 Jun 2022 21:32:33 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> Can someone please prepare a small example texi file containing examples > >> of various texi features that are commonly needed and possibly also > >> demonstrating some concrete instances of problems that exist in texinfo? > > > > Given that you already produced a list of markups, is such an example > > still wanted? > > It would certainly help with development. For example, I did not look > close into allowed markup nesting. File with tricky examples will be > helpful as a benchmark for any attempted implementation. I'm afraid that would take someone who knows more than I do about Org, because I don't have a clear idea what examples to provide to be useful for this purpose. In general the Texinfo manual has examples of every Texinfo feature, of course. But it is a large manual. Specifically wrt nested markups, the most frequent ones I can think off the top of my head are: . @var inside @code . @key inside @kbd . any in-line markup inside @w . @r, which is "escape to regular typeface" and can be used inside any markup that produces other typefaces HTH ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-14 13:32 ` Ihor Radchenko 2022-06-14 13:45 ` Eli Zaretskii @ 2022-06-15 23:15 ` Richard Stallman 2022-06-17 6:43 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Richard Stallman @ 2022-06-15 23:15 UTC (permalink / raw) To: Ihor Radchenko; +Cc: eliz, theophilusx, monnier, acm, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > It would certainly help with development. For example, I did not look > close into allowed markup nesting. File with tricky examples will be > helpful as a benchmark for any attempted implementation. I don't know for certain that every possible nesting "does the right thing". I do know that @var{} is used inside many other constructs. By contrast, @dfn{} would not be nested inside or around other contructs very much. @key can be nested inside @kbd, and it behaves a little differently when nested. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-15 23:15 ` Richard Stallman @ 2022-06-17 6:43 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-17 6:43 UTC (permalink / raw) To: rms; +Cc: eliz, theophilusx, monnier, acm, emacs-devel Richard Stallman <rms@gnu.org> writes: > > It would certainly help with development. For example, I did not look > > close into allowed markup nesting. File with tricky examples will be > > helpful as a benchmark for any attempted implementation. > > I don't know for certain that every possible nesting "does the right > thing". I do know that @var{} is used inside many other constructs. > By contrast, @dfn{} would not be nested inside or around other > contructs very much. @key can be nested inside @kbd, and it behaves > a little differently when nested. Thanks! I replied in a different thread, that includes Org ML in CC. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 22:38 ` Richard Stallman 2022-06-12 22:56 ` Tim Cross @ 2022-06-14 13:18 ` Ihor Radchenko 2022-06-14 13:38 ` Org mode and Emacs Robert Pluim 2022-06-15 23:15 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Richard Stallman 1 sibling, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-14 13:18 UTC (permalink / raw) To: rms; +Cc: theophilusx, eliz, monnier, acm, emacs-devel Richard Stallman <rms@gnu.org> writes: > They are in the Texinfo manual. Look at node "Indicating", which is > section 7.1 in the version I have in Info. Thanks! > > One key reason I worry about going down that road is that I suspect it > > would complicate org's syntax. Two key benefits of org mode is that the > > basic syntax is simple and it maps reasonably consistently acorss > > different output formats. However, this flexibility does come at a cost. > > To provide consistency across export formats, the basic formatting > > 'concepts' need to be somewhat 'generalised', which means at times you > > will loose some of the more advanced or sophisticated formatting power > > of some export back-ends. > > I suspect we are slightly miscommunicating, because Texinfo already > generates several different formats of output, and each markup construct > is carefully defined about how it should appear in each output format. > > So I'm sure it is possible to define additional markup constucts and > make each one do, in each output format, what Texinfo would (or does) > do with it. > > The only hard part is finding syntax for them. As I stated in another message, we may not need to define additional markup constructs. texinfo-specific markup appears to be a little bit too specific for general-purpose formatting aimed by Org. Instead of modifying Org syntax (and forcing all third-party packages adapt), we can leverage the fact that parts of Org syntax are programmable in Elisp. Org provides links as a flexible markup element - its export to different formats can be arbitrarily customized (via function). See A.3 Adding Hyperlink Types of Org manual. For paragraph-level constructs, there is org-export-filter-parse-tree-functions and external https://github.com/alhassy/org-special-block-extras We may implement something more reminiscent to link customization in future. org-export-filter-parse-tree-functions is a bit too generic (one function for everything). In summary, Org does not really need to change syntax in order to support full texinfo markup. AFAIU, the missing parts are some of the texinfo-specific markup elements, abbreviations, and generating index. They all can be implemented as new libraries. Abbreviations and index are WIP at https://github.com/tecosaur/org-glossary The texinfo-specific markup can be another (optionally loaded) Org library. > > As pointed out elswhere in this thread, org could support missing > > texinfo syntax using texinfo specific blocks. However, that isn't a > > great experience from an authoring perspective. It is fine if you only > > need to do it occasionally, but if you have to constantly add such > > blocks in order to get really well formatted texinfo output, > > This is rather vague. Why do you thik this would be difficult to use > Can we find a solution to make it easy to use? This was a reference to Org export blocks: #+begin_export latex This text will only appear when exported to \LaTeX, not to any other format. #+end_export They may be not great experience what one needs to do something like #+begin_export texinfo @var{test} will only appear in .texi output. #+end_export #+begin_export latex \emph{test} will only appear in .tex output. #+end_export many times in the same .org document. However, we have special blocks for this purpose. They are extendable, as I descibed above. Though some improvements in Org core might be needed if we have to use this extensibility more frequently. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-14 13:18 ` Ihor Radchenko @ 2022-06-14 13:38 ` Robert Pluim 2022-06-15 6:13 ` Cusom special block export, similar org org-link :export parameter (was: Org mode and Emacs) Ihor Radchenko 2022-06-15 23:15 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Robert Pluim @ 2022-06-14 13:38 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, theophilusx, eliz, monnier, acm, emacs-devel >>>>> On Tue, 14 Jun 2022 21:18:58 +0800, Ihor Radchenko <yantar92@gmail.com> said: Ihor> #+begin_export latex Ihor> This text will only appear when exported to \LaTeX, not to any other format. Ihor> #+end_export Ihor> They may be not great experience what one needs to do something like Ihor> #+begin_export texinfo Ihor> @var{test} will only appear in .texi output. Ihor> #+end_export Ihor> #+begin_export latex Ihor> \emph{test} will only appear in .tex output. Ihor> #+end_export Ihor> many times in the same .org document. Ihor> However, we have special blocks for this purpose. They are extendable, Ihor> as I descibed above. Though some improvements in Org core might be Ihor> needed if we have to use this extensibility more frequently. My first thought here was Org macros, but as far as I can tell they canʼt be multi-line, which would make them a bit cumbersome to use. Tell us more about special blocks, the documentation on them is a bit sparse. Robert -- ^ permalink raw reply [flat|nested] 505+ messages in thread
* Cusom special block export, similar org org-link :export parameter (was: Org mode and Emacs) 2022-06-14 13:38 ` Org mode and Emacs Robert Pluim @ 2022-06-15 6:13 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-15 6:13 UTC (permalink / raw) To: Robert Pluim Cc: rms, theophilusx, eliz, monnier, acm, emacs-devel, emacs-orgmode Robert Pluim <rpluim@gmail.com> writes: > Ihor> However, we have special blocks for this purpose. They are extendable, > Ihor> as I descibed above. Though some improvements in Org core might be > Ihor> needed if we have to use this extensibility more frequently. > > My first thought here was Org macros, but as far as I can tell they canʼt > be multi-line, which would make them a bit cumbersome to use. Tell us > more about special blocks, the documentation on them is a bit sparse. Macros are meant to configure per-document level of export. Not global, usually. Special blocks are Org blocks with the following syntax: #+begin_environment [optional parameters] #+end_environment With "environment" being any word except "center", "quote", "comment", "example", "export", "src", and "verse". Currently, latex export directly translates such blocks into \begin{environment} \end{environment} plain text export supports special handling of environment=justifyleft, justifyright. html export supports '("article" "aside" "audio" "canvas" "details" "figcaption" "figure" "footer" "header" "menu" "meter" "nav" "output" "progress" "section" "summary" "video") special blocks texinfo export translates special blocks into @environment [options] @end odt exprot supports "annotation" and "textbox" blocks. etc. Because "environment" can be pretty much anything, we can have a pre-processor in org-export-filter-parse-tree-functions that transforms different special blocks into appropriate form for a given export backend (texinfo, latex, ascii, html, etc). This way, we will not have to rely on backend-specific handling. Such pre-processor currently does not exist, but it would be fairly trivial to implement. We may even introduce an interface similar to org-link-set-parameters for easier configuration of various types of special blocks. WDYT? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-14 13:18 ` Ihor Radchenko 2022-06-14 13:38 ` Org mode and Emacs Robert Pluim @ 2022-06-15 23:15 ` Richard Stallman 2022-06-17 6:52 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Richard Stallman @ 2022-06-15 23:15 UTC (permalink / raw) To: Ihor Radchenko; +Cc: theophilusx, eliz, monnier, acm, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > They may be not great experience what one needs to do something like > #+begin_export texinfo > @var{test} will only appear in .texi output. > #+end_export > #+begin_export latex > \emph{test} will only appear in .tex output. > #+end_export > many times in the same .org document. > However, we have special blocks for this purpose. They are extendable, > as I descibed above. Though some improvements in Org core might be > needed if we have to use this extensibility more frequently. Maybe that is sufficient. It would be ok to use, for manuals, a submode which extend the usual Org facilities. But it would be good for that submode to define shorter command names for these. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-15 23:15 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Richard Stallman @ 2022-06-17 6:52 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-17 6:52 UTC (permalink / raw) To: rms; +Cc: theophilusx, eliz, monnier, acm, emacs-devel Richard Stallman <rms@gnu.org> writes: > > However, we have special blocks for this purpose. They are extendable, > > as I descibed above. Though some improvements in Org core might be > > needed if we have to use this extensibility more frequently. > > Maybe that is sufficient. It would be ok to use, for manuals, a > submode which extend the usual Org facilities. Submode is what I am inclined to. All these @var and @dfn constructs only make sense for software documentation. Org is aiming much more general writeup. > But it would be good for that submode to define shorter command names > for these. Could you please elaborate? Which parts of the Org syntax you find too long? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-09 23:10 ` Tim Cross 2022-06-10 5:59 ` Eli Zaretskii @ 2022-06-12 0:42 ` Richard Stallman 2022-06-12 1:39 ` Tim Cross ` (2 more replies) 1 sibling, 3 replies; 505+ messages in thread From: Richard Stallman @ 2022-06-12 0:42 UTC (permalink / raw) To: Tim Cross; +Cc: monnier, acm, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > Making org mode syntax equivalent to texinfo syntax seems like a > mistake to me. If this succeeds, it would be an important advance for the GNU system. We would replace Texinfo with a much cleaner system, easier to use and more maintainable. Not solely for Emacs, but for ALL our documentation! The original idea was to have a light weight syntax which > is easyh to learn, not create a clone of texinfo. Besides, I suspect it > would be very difficult to maintain backwards compatibility. These are real concerns, but they are not real certainties. If we look for solutions, we may find good ones. I hope someone will give it a try. One of Texinfo's crucial features is a wide range of semantic markup constructs, each of which can generate different output depending on the output format. For instance, Texinfo has @var, @emph and @dfn, all of which generate italics in printed output, but they differ in what they generate for other output formats. There are probably 15 other such constructs. These constructs make it possible to carry out our documentation style constructs. Does Org format have the ability to make all these distinctions? If not, I suspect that the Org mode documentation isn't following all our style conventions for documentation. There is nothing fundamentally hard about supporting these distinctions. Can someone please examine which ones Org supports and which ones not, and propose exensions for the ones not yet supported? -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 0:42 ` Richard Stallman @ 2022-06-12 1:39 ` Tim Cross 2022-06-12 2:40 ` Org mode and Emacs T.V Raman 2022-06-12 6:02 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 2022-06-12 1:45 ` Org mode and Emacs Po Lu 2022-06-12 4:53 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Christopher Dimech 2 siblings, 2 replies; 505+ messages in thread From: Tim Cross @ 2022-06-12 1:39 UTC (permalink / raw) To: rms; +Cc: monnier, acm, emacs-devel Richard Stallman <rms@gnu.org> writes: > [[[ To any NSA and FBI agents reading my email: please consider ]]] > [[[ whether defending the US Constitution against all enemies, ]]] > [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > > Making org mode syntax equivalent to texinfo syntax seems like a > > mistake to me. > > If this succeeds, it would be an important advance for the GNU system. > We would replace Texinfo with a much cleaner system, easier to use and > more maintainable. Not solely for Emacs, but for ALL our > documentation! > One key reason I worry about going down that road is that I suspect it would complicate org's syntax. Two key benefits of org mode is that the basic syntax is simple and it maps reasonably consistently acorss different output formats. However, this flexibility does come at a cost. To provide consistency across export formats, the basic formatting 'concepts' need to be somewhat 'generalised', which means at times you will loose some of the more advanced or sophisticated formatting power of some export back-ends. There are times, primarily where you need specialised or specific formatting in a particular format, where you need to drop down to that low level formatting 'language'. This is the org 'escape hatch', which provides a way to better leverage off the specific capabilities of a partricular back-end. For exmaple, you can embed latex in your document which will be added to your formated output when you generate a PDF/PS or *.tex output file. However, that formatting often won't work when you use the same source file to generate an HTML or Markdown or ODT version because there isn't a clear way to translate the latex to that format. As pointed out elswhere in this thread, org could support missing texinfo syntax using texinfo specific blocks. However, that isn't a great experience from an authoring perspective. It is fine if you only need to do it occasionally, but if you have to constantly add such blocks in order to get really well formatted texinfo output, it will become frustrating. Org also supports a powerful custom link format which could be used to add missing syntax elements, but I'm unsure on the usability of such an approach once you have a few such cusotm links. If we want to avoid this situation, the syntax would need to be added to the basic org syntax. However, that will also require all back-ends being able to interpret that syntax in some 'sane' manner, which would likley be considerable work and in some situations, could be extremely difficult to do consistently with good results. It would also add to the overall amount of syntax, potentially making it more complex and harder to learn. Org mode is a great mode, especially for general documentation and information management, todo and time management and to some extent, for literate programming and/or 'executable' documents, such as lab books or even some devops type applicaitons. Being able to have a single source document which can be used to generate 'reasonable' versions in different formats. You can even spend a bit of effort to customise things so that you can get some pretty consistent advanced formatting in some specific export formats. However, I often find when you need the advanced formatting power of a specific back-end, your often better just writing the document in that back-end as it tends to take less effort and results in cleaner source. The other side of th coin is the on-going development of texinfo. I have not written enough texinfo to really understand what the issues are which drive the desire to replace it with something else, such as org. I know there are some criticisms regarding output formats, but I also know there is on-going work to improve that situation. Is the right strategy to work on org mode so that it can replace texinfo or work on texinfo to address limitations (or both?)? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 1:39 ` Tim Cross @ 2022-06-12 2:40 ` T.V Raman 2022-06-12 6:02 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 1 sibling, 0 replies; 505+ messages in thread From: T.V Raman @ 2022-06-12 2:40 UTC (permalink / raw) To: Tim Cross; +Cc: rms, monnier, acm, emacs-devel [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain; charset=gb18030, Size: 383 bytes --] Note that org's export to texinfo is quite good; as an example, I wrote the "Emacspeak Turning Twenty" article in org-mode and later exported it to texinfo among other formats. Org's export to TeX,LaTeX and PDF is also very good; likely superior to its export to Texinfo. -- Thanks, --Raman(I Search, I Find, I Misplace, I Research) 7©4 Id: kg:/m/0285kf1 0Ü8 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 1:39 ` Tim Cross 2022-06-12 2:40 ` Org mode and Emacs T.V Raman @ 2022-06-12 6:02 ` Eli Zaretskii 2022-06-12 22:38 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-12 6:02 UTC (permalink / raw) To: Tim Cross; +Cc: rms, monnier, acm, emacs-devel > From: Tim Cross <theophilusx@gmail.com> > Cc: monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Date: Sun, 12 Jun 2022 11:39:07 +1000 > > The other side of th coin is the on-going development of texinfo. I have > not written enough texinfo to really understand what the issues are > which drive the desire to replace it with something else, such as org. I > know there are some criticisms regarding output formats, but I also know > there is on-going work to improve that situation. Is the right strategy > to work on org mode so that it can replace texinfo or work on texinfo to > address limitations (or both?)? The main problem with Texinfo that I'm aware of is its reliance on TeX for the printed output formats (PDF, PS, etc.). Many restrictions in Texinfo come from the fact that something is impossible to implement in TeX. Another issue that frequently comes up is the HTML format it produces: it currently doesn't support many features expected from good on-line documentation, such as multiple frames etc. A better place to discuss the shortcomings of Texinfo is on the Texinfo mailing list, where we can talk to the few people who are actively developing Texinfo and aware of the main issues. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 6:02 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii @ 2022-06-12 22:38 ` Richard Stallman 2022-06-13 2:29 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Richard Stallman @ 2022-06-12 22:38 UTC (permalink / raw) To: Eli Zaretskii; +Cc: theophilusx, monnier, acm, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > The main problem with Texinfo that I'm aware of is its reliance on TeX > for the printed output formats (PDF, PS, etc.). Many restrictions in > Texinfo come from the fact that something is impossible to implement > in TeX. I am not sure which kind of "something" this refers to. Formatting capabilities? Input syntax? Since we want to take advantage of TeX to make beautiful printed manuals, we will be limited to TeX's formatting capabilities. But I think they are more powerful than anything else. The limitations of TeX's parser are things we could avoid, if we didn't have TeX directly read the texinfo source code. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 22:38 ` Richard Stallman @ 2022-06-13 2:29 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-13 2:29 UTC (permalink / raw) To: rms; +Cc: theophilusx, monnier, acm, emacs-devel > From: Richard Stallman <rms@gnu.org> > Cc: theophilusx@gmail.com, monnier@iro.umontreal.ca, acm@muc.de, > emacs-devel@gnu.org > Date: Sun, 12 Jun 2022 18:38:46 -0400 > > > The main problem with Texinfo that I'm aware of is its reliance on TeX > > for the printed output formats (PDF, PS, etc.). Many restrictions in > > Texinfo come from the fact that something is impossible to implement > > in TeX. > > I am not sure which kind of "something" this refers to. Formatting > capabilities? Input syntax? The fact that Texinfo's TeX implementation is via TeX macros, AFAIU. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 0:42 ` Richard Stallman 2022-06-12 1:39 ` Tim Cross @ 2022-06-12 1:45 ` Po Lu 2022-06-12 2:15 ` Ihor Radchenko 2022-06-12 4:53 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Christopher Dimech 2 siblings, 1 reply; 505+ messages in thread From: Po Lu @ 2022-06-12 1:45 UTC (permalink / raw) To: Richard Stallman; +Cc: Tim Cross, monnier, acm, emacs-devel Richard Stallman <rms@gnu.org> writes: > If this succeeds, it would be an important advance for the GNU system. > We would replace Texinfo with a much cleaner system, easier to use and > more maintainable. Not solely for Emacs, but for ALL our > documentation! Org mode is not easier to use than Texinfo... at least, I didn't figure out how to use it. IIRC it's also much slower to create similarly sized .info files from Org source than it is with makeinfo, and it also requires an entire manual to be contained in a single file. Texinfo also comes with the added benefit of not requiring Emacs to edit or to translate into other formats. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 1:45 ` Org mode and Emacs Po Lu @ 2022-06-12 2:15 ` Ihor Radchenko 2022-06-12 2:36 ` David Masterson ` (2 more replies) 0 siblings, 3 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-12 2:15 UTC (permalink / raw) To: Po Lu; +Cc: Richard Stallman, Tim Cross, monnier, acm, emacs-devel Po Lu <luangruo@yahoo.com> writes: > IIRC it's also much slower to create similarly sized .info files from > Org source than it is with makeinfo Could you please quantify much slower? Do you have concrete example of a .org file which can demonstrate the problem? Also, I think that I have to clarify about texinfo export from Org. Org does not currently generate .info files directly. .org file is first transformed into .texi and then uses makeinfo to translate the generated .texi into .info. If there is an interest in Org being used instead of texinfo, it should be based on the extensibility of Org rather than just making Org replicate what texinfo does. However, I am not sure if all parties in this discussion have a clear idea about concrete advantages of Org compared with texinfo. I am not faimiliar with texinfo. Others may not be as faimilar with Org. I believe that if we want to be serious about using Org for documentation more widely, we should first identify which particular aspects of Org are beneficial for documentation and which particular aspects are available in other documentation generators and must be then provided by Org (if not yet available). > ... , and it also requires an entire > manual to be contained in a single file. This is not true. Org mode supports #+include: directives to incorporate multiple smaller files into one larger file. > Texinfo also comes with the added benefit of not requiring Emacs to edit > or to translate into other formats. You can edit Org files outside Emacs. Say, in vim. The point about exporting to other formats is valid. This last point also raises a question. Can Elisp interpreter and libraries be factored out of Emacs to create a way to execute Elisp programs without installing all the interactive parts of Emacs? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 2:15 ` Ihor Radchenko @ 2022-06-12 2:36 ` David Masterson 2022-06-12 3:06 ` Ihor Radchenko 2022-06-12 3:28 ` Tim Cross 2022-06-12 2:50 ` Po Lu 2022-06-12 6:57 ` Eli Zaretskii 2 siblings, 2 replies; 505+ messages in thread From: David Masterson @ 2022-06-12 2:36 UTC (permalink / raw) To: Ihor Radchenko Cc: Po Lu, Richard Stallman, Tim Cross, monnier, acm, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > Po Lu <luangruo@yahoo.com> writes: > >> Texinfo also comes with the added benefit of not requiring Emacs to edit >> or to translate into other formats. > > You can edit Org files outside Emacs. Say, in vim. > The point about exporting to other formats is valid. > > This last point also raises a question. Can Elisp interpreter and > libraries be factored out of Emacs to create a way to execute Elisp > programs without installing all the interactive parts of Emacs? Isn't Tim Cross(?) working on something like that -- ie. a parser for the Org language. Once we have a solid parser, we can build a standard (set of?) backend(s) for much of Orgmode. -- David Masterson ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 2:36 ` David Masterson @ 2022-06-12 3:06 ` Ihor Radchenko 2022-06-12 3:39 ` David Masterson 2022-06-12 3:28 ` Tim Cross 1 sibling, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-12 3:06 UTC (permalink / raw) To: David Masterson Cc: Po Lu, Richard Stallman, Tim Cross, monnier, acm, emacs-devel David Masterson <dsmasterson@gmail.com> writes: >> This last point also raises a question. Can Elisp interpreter and >> libraries be factored out of Emacs to create a way to execute Elisp >> programs without installing all the interactive parts of Emacs? > > Isn't Tim Cross(?) working on something like that -- ie. a parser for > the Org language. Once we have a solid parser, we can build a standard > (set of?) backend(s) for much of Orgmode. Org already have a parser. Written in Elisp. Export is built on top of the parser. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 3:06 ` Ihor Radchenko @ 2022-06-12 3:39 ` David Masterson 2022-06-12 4:43 ` Tim Cross 0 siblings, 1 reply; 505+ messages in thread From: David Masterson @ 2022-06-12 3:39 UTC (permalink / raw) To: Ihor Radchenko Cc: Po Lu, Richard Stallman, Tim Cross, monnier, acm, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > David Masterson <dsmasterson@gmail.com> writes: > >>> This last point also raises a question. Can Elisp interpreter and >>> libraries be factored out of Emacs to create a way to execute Elisp >>> programs without installing all the interactive parts of Emacs? >> >> Isn't Tim Cross(?) working on something like that -- ie. a parser for >> the Org language. Once we have a solid parser, we can build a standard >> (set of?) backend(s) for much of Orgmode. > > Org already have a parser. Written in Elisp. Export is built on top of > the parser. But Elisp is not portable to a non-Emacs system (say, iPhone). In the long run, it would be better to define a "parseable" language as the standard basis for Org. From that, people can develop (parts of) Org on other platforms (Vim, Beorg, Orgzly) and test/prove that they are compatible with version X of the language. I think Organice was doing this, but I haven't looked at it deeply. Oh, but I see your point about "export". By backend, I was assuming a true parser would generate a standard "internal" language which could be fed into simpler backends to actually do the work. The front-end parser and back-ends could be translated/rebuilt as needed on new platorms (iOS, Android, MS-Windows, etc.). More is needed than this, but that's the idea. -- David Masterson ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 3:39 ` David Masterson @ 2022-06-12 4:43 ` Tim Cross 2022-06-12 5:08 ` Po Lu 2022-06-12 5:53 ` David Masterson 0 siblings, 2 replies; 505+ messages in thread From: Tim Cross @ 2022-06-12 4:43 UTC (permalink / raw) To: David Masterson Cc: Ihor Radchenko, Po Lu, Richard Stallman, monnier, acm, emacs-devel David Masterson <dsmasterson@gmail.com> writes: > Ihor Radchenko <yantar92@gmail.com> writes: > >> David Masterson <dsmasterson@gmail.com> writes: >> >>>> This last point also raises a question. Can Elisp interpreter and >>>> libraries be factored out of Emacs to create a way to execute Elisp >>>> programs without installing all the interactive parts of Emacs? >>> >>> Isn't Tim Cross(?) working on something like that -- ie. a parser for >>> the Org language. Once we have a solid parser, we can build a standard >>> (set of?) backend(s) for much of Orgmode. >> >> Org already have a parser. Written in Elisp. Export is built on top of >> the parser. > > But Elisp is not portable to a non-Emacs system (say, iPhone). In the > long run, it would be better to define a "parseable" language as the > standard basis for Org. From that, people can develop (parts of) Org on > other platforms (Vim, Beorg, Orgzly) and test/prove that they are > compatible with version X of the language. I think Organice was doing > this, but I haven't looked at it deeply. > This is all part of the aims and process. However, the first step is to develop a robust elisp based parser. This helps to ensure the org syntax is consistent and helps identifies ambiguities which need to be fixed as well as provides a reference implementation which developers can use to develop parsers in other languages. > Oh, but I see your point about "export". By backend, I was assuming a > true parser would generate a standard "internal" language which could be > fed into simpler backends to actually do the work. The front-end parser > and back-ends could be translated/rebuilt as needed on new platorms > (iOS, Android, MS-Windows, etc.). More is needed than this, but that's > the idea. Personally, I think it is unlikely we will ever see an org mode for other platforms which is equivalent to Emacs. However, the work which is being done to create a clean parser in elisp and use that as the basis for an API to manipulate org data, generate font-locking and formatting and provide a clean API for exporters to use will make it much easier for people to develop external org support at varying levels. Those doing the main work, like Ihor, are very aware of the desire to facilitate external implementations. It is one reason that a fairly conservative attitude to change is adopted by the project and often, one consideration is what imapct a change would have on external org compatible projects. Sometimes, this is challenging as on one level, we want ot advance and improve org mode, but on the other, we want it to be as stable as possible to reduce adverse impact on external projects. So while the main target is and will always be Emacs, an eye is kept on efforts of other external projects and an effort is made to support them when possible and within Emacs and FSF principals/guidelines. So far, the key contributors have been doing a very good job. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 4:43 ` Tim Cross @ 2022-06-12 5:08 ` Po Lu 2022-06-12 5:20 ` Ihor Radchenko 2022-06-12 5:27 ` Tim Cross 2022-06-12 5:53 ` David Masterson 1 sibling, 2 replies; 505+ messages in thread From: Po Lu @ 2022-06-12 5:08 UTC (permalink / raw) To: Tim Cross Cc: David Masterson, Ihor Radchenko, Richard Stallman, monnier, acm, emacs-devel Tim Cross <theophilusx@gmail.com> writes: >> But Elisp is not portable to a non-Emacs system (say, iPhone). In the >> long run, it would be better to define a "parseable" language as the >> standard basis for Org. From that, people can develop (parts of) Org on >> other platforms (Vim, Beorg, Orgzly) and test/prove that they are >> compatible with version X of the language. I think Organice was doing >> this, but I haven't looked at it deeply. > This is all part of the aims and process. However, the first step is to > develop a robust elisp based parser. This helps to ensure the org syntax > is consistent and helps identifies ambiguities which need to be fixed as > well as provides a reference implementation which developers can use to It is not the goal of Emacs to support tyrant devices such as the iPhone running proprietary operating systems such as iOS. If people are going to refactor Org mode, I hope they will not do it with support for iPhones in mind. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 5:08 ` Po Lu @ 2022-06-12 5:20 ` Ihor Radchenko 2022-06-12 5:27 ` Tim Cross 1 sibling, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-12 5:20 UTC (permalink / raw) To: Po Lu Cc: Tim Cross, David Masterson, Richard Stallman, monnier, acm, emacs-devel Po Lu <luangruo@yahoo.com> writes: >> This is all part of the aims and process. However, the first step is to >> develop a robust elisp based parser. This helps to ensure the org syntax >> is consistent and helps identifies ambiguities which need to be fixed as >> well as provides a reference implementation which developers can use to > > It is not the goal of Emacs to support tyrant devices such as the iPhone > running proprietary operating systems such as iOS. > > If people are going to refactor Org mode, I hope they will not do it > with support for iPhones in mind. I am not sure why you only picked up iPhones from the list suggested by David. The aim is not supporting iPhones. The aim is supporting other GNU and Free Software projects that are making use of Org, which implies conservative approach to breaking changes, especially in syntax. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 5:08 ` Po Lu 2022-06-12 5:20 ` Ihor Radchenko @ 2022-06-12 5:27 ` Tim Cross 1 sibling, 0 replies; 505+ messages in thread From: Tim Cross @ 2022-06-12 5:27 UTC (permalink / raw) To: Po Lu Cc: David Masterson, Ihor Radchenko, Richard Stallman, monnier, acm, emacs-devel Po Lu <luangruo@yahoo.com> writes: > Tim Cross <theophilusx@gmail.com> writes: > >>> But Elisp is not portable to a non-Emacs system (say, iPhone). In the >>> long run, it would be better to define a "parseable" language as the >>> standard basis for Org. From that, people can develop (parts of) Org on >>> other platforms (Vim, Beorg, Orgzly) and test/prove that they are >>> compatible with version X of the language. I think Organice was doing >>> this, but I haven't looked at it deeply. > >> This is all part of the aims and process. However, the first step is to >> develop a robust elisp based parser. This helps to ensure the org syntax >> is consistent and helps identifies ambiguities which need to be fixed as >> well as provides a reference implementation which developers can use to > > It is not the goal of Emacs to support tyrant devices such as the iPhone > running proprietary operating systems such as iOS. > > If people are going to refactor Org mode, I hope they will not do it > with support for iPhones in mind. There has never been suggestion that what we are doing is to facilitate development on non-free platforms. However, if you develop good clean code and clear algorithms which others are able to use as a reference to understand the syntax and semantics of a system, you cannot control how they will use that information. In fact, it can be argued that making such information readily available and accessible is a large part of the freedoms being promoted by the FSF. This is completely different from, for example, providing an interface which can be used by a non-free platform. Besides, if org is as bad and difficult to learn as you have indicated in other posts, nobody will bother implementing it on those platforms you seem so threatened by anyway! ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 4:43 ` Tim Cross 2022-06-12 5:08 ` Po Lu @ 2022-06-12 5:53 ` David Masterson 2022-06-12 6:56 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: David Masterson @ 2022-06-12 5:53 UTC (permalink / raw) To: Tim Cross Cc: Ihor Radchenko, Po Lu, Richard Stallman, monnier, acm, emacs-devel Tim Cross <theophilusx@gmail.com> writes: > David Masterson <dsmasterson@gmail.com> writes: > >> Ihor Radchenko <yantar92@gmail.com> writes: >> >>> David Masterson <dsmasterson@gmail.com> writes: >>> >>>>> This last point also raises a question. Can Elisp interpreter and >>>>> libraries be factored out of Emacs to create a way to execute Elisp >>>>> programs without installing all the interactive parts of Emacs? >>>> >>>> Isn't Tim Cross(?) working on something like that -- ie. a parser for >>>> the Org language. Once we have a solid parser, we can build a standard >>>> (set of?) backend(s) for much of Orgmode. >>> >>> Org already have a parser. Written in Elisp. Export is built on top of >>> the parser. >> >> But Elisp is not portable to a non-Emacs system (say, iPhone). In the >> long run, it would be better to define a "parseable" language as the >> standard basis for Org. From that, people can develop (parts of) Org on >> other platforms (Vim, Beorg, Orgzly) and test/prove that they are >> compatible with version X of the language. I think Organice was doing >> this, but I haven't looked at it deeply. >> > > This is all part of the aims and process. However, the first step is to > develop a robust elisp based parser. This helps to ensure the org syntax > is consistent and helps identifies ambiguities which need to be fixed as > well as provides a reference implementation which developers can use to > develop parsers in other languages. Semantic/Bovine ?? Agree >> Oh, but I see your point about "export". By backend, I was assuming a >> true parser would generate a standard "internal" language which could be >> fed into simpler backends to actually do the work. The front-end parser >> and back-ends could be translated/rebuilt as needed on new platorms >> (iOS, Android, MS-Windows, etc.). More is needed than this, but that's >> the idea. > > Personally, I think it is unlikely we will ever see an org mode for > other platforms which is equivalent to Emacs. However, the work which is > being done to create a clean parser in elisp and use that as the basis > for an API to manipulate org data, generate font-locking and formatting > and provide a clean API for exporters to use will make it much easier > for people to develop external org support at varying levels. Agree > Those doing the main work, like Ihor, are very aware of the desire to > facilitate external implementations. It is one reason that a fairly > conservative attitude to change is adopted by the project and often, one > consideration is what imapct a change would have on external org > compatible projects. Sometimes, this is challenging as on one level, we > want ot advance and improve org mode, but on the other, we want it to be > as stable as possible to reduce adverse impact on external projects. So > while the main target is and will always be Emacs, an eye is kept on > efforts of other external projects and an effort is made to support them > when possible and within Emacs and FSF principals/guidelines. > > So far, the key contributors have been doing a very good job. That's good. -- David Masterson ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 5:53 ` David Masterson @ 2022-06-12 6:56 ` Ihor Radchenko 2022-06-12 18:29 ` David Masterson 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-12 6:56 UTC (permalink / raw) To: David Masterson Cc: Tim Cross, Po Lu, Richard Stallman, monnier, acm, emacs-devel David Masterson <dsmasterson@gmail.com> writes: >> This is all part of the aims and process. However, the first step is to >> develop a robust elisp based parser. This helps to ensure the org syntax >> is consistent and helps identifies ambiguities which need to be fixed as >> well as provides a reference implementation which developers can use to >> develop parsers in other languages. > > Semantic/Bovine ?? Org is not context-free. Also, Org maintaners previously rejected the idea of implementing Org parser not in Elisp. Mainly because it would limit the ability to maintain and contribute to Org - one would need to learn another programming language to alter anything in Org syntax. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 6:56 ` Ihor Radchenko @ 2022-06-12 18:29 ` David Masterson 2022-06-14 5:09 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: David Masterson @ 2022-06-12 18:29 UTC (permalink / raw) To: Ihor Radchenko Cc: Tim Cross, Po Lu, Richard Stallman, monnier, acm, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > David Masterson <dsmasterson@gmail.com> writes: > >>> This is all part of the aims and process. However, the first step is to >>> develop a robust elisp based parser. This helps to ensure the org syntax >>> is consistent and helps identifies ambiguities which need to be fixed as >>> well as provides a reference implementation which developers can use to >>> develop parsers in other languages. >> >> Semantic/Bovine ?? > > Org is not context-free. But could it be moved in that direction? (ie. Organice) > Also, Org maintaners previously rejected the idea of implementing Org > parser not in Elisp. Mainly because it would limit the ability to > maintain and contribute to Org - one would need to learn another > programming language to alter anything in Org syntax. Hmmm. That would make it difficult to keep the language "parseable" by a different parser. Elisp would not provide the checks for (say) keeping the language context-free. -- David Masterson ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 18:29 ` David Masterson @ 2022-06-14 5:09 ` Ihor Radchenko 2022-06-19 23:48 ` David Masterson 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-14 5:09 UTC (permalink / raw) To: David Masterson Cc: Tim Cross, Po Lu, Richard Stallman, monnier, acm, emacs-devel David Masterson <dsmasterson@gmail.com> writes: >>> Semantic/Bovine ?? >> >> Org is not context-free. > > But could it be moved in that direction? (ie. Organice) I don't think so. It is motivated by the fundamental Org syntax design, AFAIU. (mostly by first match wins design). We are not going to change fundamentals of the Org syntax. It will break backward compatibility. >> Also, Org maintaners previously rejected the idea of implementing Org >> parser not in Elisp. Mainly because it would limit the ability to >> maintain and contribute to Org - one would need to learn another >> programming language to alter anything in Org syntax. > > Hmmm. That would make it difficult to keep the language "parseable" by a > different parser. Elisp would not provide the checks for (say) keeping > the language context-free. At this point, we are trying to "freeze" Org syntax as much as possible. So, major changes are not expected. Different parsers should not suffer from future changes (if they do, we should not make those changes to start with). As for keeping checks, we do have a set of parser tests using ERT. So, major breakage will be prevented. On top of this, we plan to make the parser tests more friendly to third-party tools: https://orgmode.org/list/87fsqzi4tw.fsf@localhost Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-14 5:09 ` Ihor Radchenko @ 2022-06-19 23:48 ` David Masterson 2022-06-20 0:03 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: David Masterson @ 2022-06-19 23:48 UTC (permalink / raw) To: Ihor Radchenko Cc: Tim Cross, Po Lu, Richard Stallman, monnier, acm, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > David Masterson <dsmasterson@gmail.com> writes: > >>>> Semantic/Bovine ?? >>> >>> Org is not context-free. >> >> But could it be moved in that direction? (ie. Organice) > > I don't think so. It is motivated by the fundamental Org syntax design, > AFAIU. (mostly by first match wins design). We are not going to change > fundamentals of the Org syntax. It will break backward compatibility. Could Org be moved toward a "well-defined" grammar that could be separated from the Emacs implementation to allow other systems (iOS, Android, Windows) to implement (at least part of) a "standard" Org? Could the backward compatibility be covered by an Emacs library where necessary? >>> Also, Org maintaners previously rejected the idea of implementing Org >>> parser not in Elisp. Mainly because it would limit the ability to >>> maintain and contribute to Org - one would need to learn another >>> programming language to alter anything in Org syntax. >> >> Hmmm. That would make it difficult to keep the language "parseable" by a >> different parser. Elisp would not provide the checks for (say) keeping >> the language context-free. > > At this point, we are trying to "freeze" Org syntax as much as possible. > So, major changes are not expected. Different parsers should not suffer > from future changes (if they do, we should not make those changes to > start with). > > As for keeping checks, we do have a set of parser tests using ERT. So, > major breakage will be prevented. On top of this, we plan to make the > parser tests more friendly to third-party tools: > https://orgmode.org/list/87fsqzi4tw.fsf@localhost This sounds good. -- David Masterson ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-19 23:48 ` David Masterson @ 2022-06-20 0:03 ` Ihor Radchenko 2022-06-20 0:24 ` David Masterson 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-20 0:03 UTC (permalink / raw) To: David Masterson Cc: Tim Cross, Po Lu, Richard Stallman, monnier, acm, emacs-devel David Masterson <dsmasterson@gmail.com> writes: >>> But could it be moved in that direction? (ie. Organice) >> >> I don't think so. It is motivated by the fundamental Org syntax design, >> AFAIU. (mostly by first match wins design). We are not going to change >> fundamentals of the Org syntax. It will break backward compatibility. > > Could Org be moved toward a "well-defined" grammar that could be > separated from the Emacs implementation to allow other systems (iOS, > Android, Windows) to implement (at least part of) a "standard" Org? > Could the backward compatibility be covered by an Emacs library where > necessary? Yes. See https://orgmode.org/worg/dev/org-syntax.html I hope it is well-defined enough for you. I don't think that we need major changes to allow implementation in other systems. There is already a number of existing third-party parsers for Org: https://github.com/200ok-ch/org-parser https://github.com/tgbugs/laundry https://github.com/milisims/tree-sitter-org https://github.com/tecosaur/Org.jl Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-20 0:03 ` Ihor Radchenko @ 2022-06-20 0:24 ` David Masterson 0 siblings, 0 replies; 505+ messages in thread From: David Masterson @ 2022-06-20 0:24 UTC (permalink / raw) To: Ihor Radchenko Cc: Tim Cross, Po Lu, Richard Stallman, monnier, acm, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > David Masterson <dsmasterson@gmail.com> writes: > >>>> But could it be moved in that direction? (ie. Organice) >>> >>> I don't think so. It is motivated by the fundamental Org syntax design, >>> AFAIU. (mostly by first match wins design). We are not going to change >>> fundamentals of the Org syntax. It will break backward compatibility. >> >> Could Org be moved toward a "well-defined" grammar that could be >> separated from the Emacs implementation to allow other systems (iOS, >> Android, Windows) to implement (at least part of) a "standard" Org? >> Could the backward compatibility be covered by an Emacs library where >> necessary? > > Yes. See https://orgmode.org/worg/dev/org-syntax.html > I hope it is well-defined enough for you. > I don't think that we need major changes to allow implementation in > other systems. There is already a number of existing third-party parsers > for Org: > > https://github.com/200ok-ch/org-parser > https://github.com/tgbugs/laundry > https://github.com/milisims/tree-sitter-org > https://github.com/tecosaur/Org.jl Excellent! -- David Masterson ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 2:36 ` David Masterson 2022-06-12 3:06 ` Ihor Radchenko @ 2022-06-12 3:28 ` Tim Cross 1 sibling, 0 replies; 505+ messages in thread From: Tim Cross @ 2022-06-12 3:28 UTC (permalink / raw) To: David Masterson Cc: Ihor Radchenko, Po Lu, Richard Stallman, monnier, acm, emacs-devel David Masterson <dsmasterson@gmail.com> writes: > Ihor Radchenko <yantar92@gmail.com> writes: > >> Po Lu <luangruo@yahoo.com> writes: >> >>> Texinfo also comes with the added benefit of not requiring Emacs to edit >>> or to translate into other formats. >> >> You can edit Org files outside Emacs. Say, in vim. >> The point about exporting to other formats is valid. >> >> This last point also raises a question. Can Elisp interpreter and >> libraries be factored out of Emacs to create a way to execute Elisp >> programs without installing all the interactive parts of Emacs? > > Isn't Tim Cross(?) working on something like that -- ie. a parser for > the Org language. Once we have a solid parser, we can build a standard > (set of?) backend(s) for much of Orgmode. No. Ihor is the one working on the parser. I think Ihor's question about isolating the Elisp interpreter is about options to make org mode work outside of Emacs. There are frequent questions to the org list about making org available to other editors/environments. However, the big problem is that much of the really powerful featurs of org mode are intrinsically tied to elisp functionality - for exmaple all the babel and backend export support. While it is possible to write a parser in any language which would enable basic org markup/formatting, that does not solve the problem of executable blocks, babel/noweb and interaction with back end exporters etc. This would be one example of where something like an elisp LSP module would be useful. However, the idea of an elisp LSP module was discouraged by a couple of people on this list over concerns that such a module would enable non-free platforms to take advantage of elisp - a fear which I think is overstated and which I think one person referred to as "Jumping at shadows" The overall conclusion was that such a module would be considerable amount of work. Some suggested you could use emacsclient to create something which could be used in such a manner. However, I suspect that if you go to the effort of installing and configuring emacs, you would just use emacs. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 2:15 ` Ihor Radchenko 2022-06-12 2:36 ` David Masterson @ 2022-06-12 2:50 ` Po Lu 2022-06-12 3:54 ` chad 2022-06-12 6:57 ` Eli Zaretskii 2 siblings, 1 reply; 505+ messages in thread From: Po Lu @ 2022-06-12 2:50 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Richard Stallman, Tim Cross, monnier, acm, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > Could you please quantify much slower? Making the Org manual takes about 27 seconds on my system: GEN org.texi GEN ../../info/org.info real 0m27.210s user 0m25.567s sys 0m0.783s Which is longer than it takes for the entire (much more massive) Emacs Lisp reference manual: GEN ../../info/elisp.info real 0m24.561s user 0m23.660s sys 0m0.515s And generating the .info file from the Org manual texinfo source takes only: GEN ../../info/org.info real 0m5.381s user 0m5.109s sys 0m0.141s So it's clear that the org to texinfo export takes most of the time. > I believe that if we want to be serious about using Org for > documentation more widely, we should first identify which particular > aspects of Org are beneficial for documentation and which particular > aspects are available in other documentation generators and must be then > provided by Org (if not yet available). I don't see why we should be serious about using Org for our documentation, when most people already know texinfo and are quite happy with it. > This is not true. Org mode supports #+include: directives to incorporate > multiple smaller files into one larger file. That's nice, but why isn't the Org manual itself structured that way? > You can edit Org files outside Emacs. Say, in vim. But does it work very well? > This last point also raises a question. Can Elisp interpreter and > libraries be factored out of Emacs to create a way to execute Elisp > programs without installing all the interactive parts of Emacs? Probably not without a lot of work. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 2:50 ` Po Lu @ 2022-06-12 3:54 ` chad 2022-06-12 5:04 ` Po Lu 2022-06-12 6:21 ` Eli Zaretskii 0 siblings, 2 replies; 505+ messages in thread From: chad @ 2022-06-12 3:54 UTC (permalink / raw) To: Po Lu Cc: Ihor Radchenko, Richard Stallman, Tim Cross, Stefan Monnier, Alan Mackenzie, EMACS development team [-- Attachment #1: Type: text/plain, Size: 1018 bytes --] On Sat, Jun 11, 2022 at 10:52 PM Po Lu <luangruo@yahoo.com> wrote: > I don't see why we should be serious about using Org for our > documentation, when most people already know texinfo and are quite happy > with it. > I think a reasonable examination of the emacs-devel archives as well as the common practices of most of the people publishing emacs lisp packages today would lead to a very different conclusion. There are several threads about maintenance concerns around makeinfo/texinfo, and many discussions about replacing texinfo with, for example, HTML.There are periodic threads where people claim that they won't try to add their project to GNU because the burden of learning and using texinfo is too high, although those have died down in volume since it became more practical to translate other formats to texinfo. None of this is to say that Org is the "right" format for Emacs documentation. I just don't think it's correct to say "most people already know texinfo and are quite happy with it". ~Chad [-- Attachment #2: Type: text/html, Size: 1430 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 3:54 ` chad @ 2022-06-12 5:04 ` Po Lu 2022-06-12 7:02 ` Ihor Radchenko ` (2 more replies) 2022-06-12 6:21 ` Eli Zaretskii 1 sibling, 3 replies; 505+ messages in thread From: Po Lu @ 2022-06-12 5:04 UTC (permalink / raw) To: chad Cc: Ihor Radchenko, Richard Stallman, Tim Cross, Stefan Monnier, Alan Mackenzie, EMACS development team chad <yandros@gmail.com> writes: > I think a reasonable examination of the emacs-devel archives as well > as the common practices of most of the people publishing emacs lisp > packages today would lead to a very different conclusion. There are > several threads about maintenance concerns around makeinfo/texinfo, > and many discussions about replacing texinfo with, for example, > HTML. Published Emacs Lisp packages in the wild typically come with no documentation at all, aside from doc strings and a single README file. Org mode happens to be popular for the latter, but I don't know why. The learning curve is high and the results seem to be unjustified. An easier format such as HTML or Markdown can be used instead. > There are periodic threads where people claim that they won't try to > add their project to GNU because the burden of learning and using > texinfo is too high, although those have died down in volume since it > became more practical to translate other formats to texinfo. Why would Org mode be any different? Unlike HTML, it is completely specific to Emacs, which actually makes it worse than Texinfo, because Texinfo is at least widely used throughout the entire GNU project. And anyway, I make the following offer: if someone doesn't want to contribute documentation (or code) because he doesn't know Texinfo, he can write the documentation in plain text, and I will translate it to Texinfo for him. IIRC the GDB developers made a similar offer, and for the many years that it existed nobody ever took advantage of it, which shows that Texinfo is not a serious barrier to contributors. Further more, using Org mode for documentation will make Emacs lose many people who are actually writing documentation, right now. At least I have no interest in ever learning Org mode, and there seem to be no volunteers who will translate Texinfo (or plain text) to Org mode for me if we begin to use it for documentation in the future. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 5:04 ` Po Lu @ 2022-06-12 7:02 ` Ihor Radchenko 2022-06-12 22:38 ` Richard Stallman 2022-06-12 22:38 ` Richard Stallman 2 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-12 7:02 UTC (permalink / raw) To: Po Lu Cc: chad, Richard Stallman, Tim Cross, Stefan Monnier, Alan Mackenzie, EMACS development team Po Lu <luangruo@yahoo.com> writes: > Published Emacs Lisp packages in the wild typically come with no > documentation at all, aside from doc strings and a single README file. > > Org mode happens to be popular for the latter, but I don't know why. > The learning curve is high and the results seem to be unjustified. An > easier format such as HTML or Markdown can be used instead. Which indicates that others may not agree with your assertion about the learning curve. For simple README https://gitlab.com/publicvoit/orgdown/-/blob/master/doc/Orgdown1-Syntax-Examples.org is more than enough. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 5:04 ` Po Lu 2022-06-12 7:02 ` Ihor Radchenko @ 2022-06-12 22:38 ` Richard Stallman 2022-06-12 22:38 ` Richard Stallman 2 siblings, 0 replies; 505+ messages in thread From: Richard Stallman @ 2022-06-12 22:38 UTC (permalink / raw) To: Po Lu; +Cc: yandros, yantar92, theophilusx, monnier, acm, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > Published Emacs Lisp packages in the wild typically come with no > documentation at all, aside from doc strings and a single README file. Depending on the quality of the README file, this could be imperfect but adequate, or it could be lousy. Please don't accept it as a standard to judge other things by. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 5:04 ` Po Lu 2022-06-12 7:02 ` Ihor Radchenko 2022-06-12 22:38 ` Richard Stallman @ 2022-06-12 22:38 ` Richard Stallman 2 siblings, 0 replies; 505+ messages in thread From: Richard Stallman @ 2022-06-12 22:38 UTC (permalink / raw) To: Po Lu; +Cc: yandros, yantar92, theophilusx, monnier, acm, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > And anyway, I make the following offer: if someone doesn't want to > contribute documentation (or code) because he doesn't know Texinfo, he > can write the documentation in plain text, and I will translate it to > Texinfo for him. Thank you for making this offer. I hope that many contributors will take you up on it. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 3:54 ` chad 2022-06-12 5:04 ` Po Lu @ 2022-06-12 6:21 ` Eli Zaretskii 1 sibling, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-12 6:21 UTC (permalink / raw) To: chad; +Cc: luangruo, yantar92, rms, theophilusx, monnier, acm, emacs-devel > From: chad <yandros@gmail.com> > Date: Sat, 11 Jun 2022 23:54:39 -0400 > Cc: Ihor Radchenko <yantar92@gmail.com>, Richard Stallman <rms@gnu.org>, > Tim Cross <theophilusx@gmail.com>, > Stefan Monnier <monnier@iro.umontreal.ca>, Alan Mackenzie <acm@muc.de>, > EMACS development team <emacs-devel@gnu.org> > > I don't see why we should be serious about using Org for our > documentation, when most people already know texinfo and are quite happy > with it. > > I think a reasonable examination of the emacs-devel archives as well as the common practices of most of > the people publishing emacs lisp packages today would lead to a very different conclusion. There are > several threads about maintenance concerns around makeinfo/texinfo, and many discussions about > replacing texinfo with, for example, HTML.There are periodic threads where people claim that they won't try > to add their project to GNU because the burden of learning and using texinfo is too high, although those have > died down in volume since it became more practical to translate other formats to texinfo. The real issue behind these claims is that developers seldom like writing good documentation. So learning to use a sophisticated markup system such as Texinfo is overkill for them. On top of that, if the package is not an Emacs package, there's a problem that Texinfo editing is not really well supported outside of Emacs, which is another reason for complaining by people who don't use Emacs for developing their package. Good HTML is no easier to write than good Texinfo, but the difference is that there are tools out there other than Emacs which can be used to create HTML, especially if the manual is relatively simple (as many Free Software manuals are). Bottom line: the complaints are real, but I'm not sure they help in this discussion, because there's no known alternative to Texinfo for creating good documentation of the kind that we are used to in Emacs. GCC folks are switching to Sphinx, but that has its own problems, and without good support for it in Emacs (and no, rst.el is not enough, as it doesn't provide enough markup commands, AFAICT) it isn't (yet) a viable alternative. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-12 2:15 ` Ihor Radchenko 2022-06-12 2:36 ` David Masterson 2022-06-12 2:50 ` Po Lu @ 2022-06-12 6:57 ` Eli Zaretskii 2 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-12 6:57 UTC (permalink / raw) To: Ihor Radchenko; +Cc: luangruo, rms, theophilusx, monnier, acm, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Richard Stallman <rms@gnu.org>, Tim Cross <theophilusx@gmail.com>, > monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Date: Sun, 12 Jun 2022 10:15:59 +0800 > > Po Lu <luangruo@yahoo.com> writes: > > > IIRC it's also much slower to create similarly sized .info files from > > Org source than it is with makeinfo > > Could you please quantify much slower? > Do you have concrete example of a .org file which can demonstrate the > problem? With an unoptimized build of Emacs 29, generation of org.texi from org.org takes 2.5 minutes here. An optimized build of Emacs 28 does that in 30 sec on one system and 1 minute on another. Generation of org.info then takes between 3 and 4 seconds. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-12 0:42 ` Richard Stallman 2022-06-12 1:39 ` Tim Cross 2022-06-12 1:45 ` Org mode and Emacs Po Lu @ 2022-06-12 4:53 ` Christopher Dimech 2 siblings, 0 replies; 505+ messages in thread From: Christopher Dimech @ 2022-06-12 4:53 UTC (permalink / raw) To: rms; +Cc: Tim Cross, monnier, acm, emacs-devel > Sent: Sunday, June 12, 2022 at 12:42 PM > From: "Richard Stallman" <rms@gnu.org> > To: "Tim Cross" <theophilusx@gmail.com> > Cc: monnier@iro.umontreal.ca, acm@muc.de, emacs-devel@gnu.org > Subject: Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) > > [[[ To any NSA and FBI agents reading my email: please consider ]]] > [[[ whether defending the US Constitution against all enemies, ]]] > [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > > Making org mode syntax equivalent to texinfo syntax seems like a > > mistake to me. > > If this succeeds, it would be an important advance for the GNU system. > We would replace Texinfo with a much cleaner system, easier to use and > more maintainable. Not solely for Emacs, but for ALL our > documentation! > > The original idea was to have a light weight syntax which > > is easyh to learn, not create a clone of texinfo. Besides, I suspect it > > would be very difficult to maintain backwards compatibility. > > These are real concerns, but they are not real certainties. If we look > for solutions, we may find good ones. > > I hope someone will give it a try. An important aspect moving forward is to permit the use of latex rather than tex. > One of Texinfo's crucial features is a wide range of semantic markup > constructs, each of which can generate different output depending on > the output format. For instance, Texinfo has @var, @emph and @dfn, > all of which generate italics in printed output, but they differ in > what they generate for other output formats. There are probably 15 > other such constructs. > > These constructs make it possible to carry out our documentation style > constructs. > > Does Org format have the ability to make all these distinctions? > If not, I suspect that the Org mode documentation isn't following > all our style conventions for documentation. > > There is nothing fundamentally hard about supporting these > distinctions. Can someone please examine which ones Org supports and > which ones not, and propose exensions for the ones not yet supported? > > -- > Dr Richard Stallman (https://stallman.org) > Chief GNUisance of the GNU Project (https://gnu.org) > Founder, Free Software Foundation (https://fsf.org) > Internet Hall-of-Famer (https://internethalloffame.org) > > > > ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-09 22:31 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Richard Stallman 2022-06-09 23:10 ` Tim Cross @ 2022-06-10 13:56 ` Ihor Radchenko 2022-06-10 19:32 ` Akib Azmain Turja 2 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-10 13:56 UTC (permalink / raw) To: rms; +Cc: Tim Cross, monnier, acm, emacs-devel Richard Stallman <rms@gnu.org> writes: > If we are going to redesign Org mode syntax, > can we please add room for extensibility > so that an extension of Org mode syntax > could have all the features of Texinfo? Adding to Tim's statement, there is currently no plan to redesign Org _syntax_. What is going on now is a slow process of consolidating our parser to be more consistent + clarifying the synatax description. See https://orgmode.org/worg/dev/org-syntax.html There are still some places in the parser and in the syntax description that a vague or contradict each other. As for Texinfo features, Org syntax is extensible enough to cover all of them. At least, from quick skimming through the texinfo manual TOC. Org does not have dedicated syntax for all the features in "6.1 Indicating Definitions, Commands, etc.", but they can be represented as special link types. Also, Org does not provide index generation for all the export backends, which has nothing to do with syntax. It's just not implemented in our generic export library. Only dedicated Texinfo exporter supprots the index generation. > If the new syntax can express all the distinctions that Texinfo > can already express, that would make it possible to switch to it > for our documentation sources. > > This needs to include conversion into some variant of TeX syntax so > that we can generate high-quality printed manuals through TeX. Org is able to export to TeX and many other formats. However, I do not know what features are required in the context of manuals. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-09 22:31 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Richard Stallman 2022-06-09 23:10 ` Tim Cross 2022-06-10 13:56 ` Ihor Radchenko @ 2022-06-10 19:32 ` Akib Azmain Turja 2 siblings, 0 replies; 505+ messages in thread From: Akib Azmain Turja @ 2022-06-10 19:32 UTC (permalink / raw) To: Richard Stallman, Tim Cross; +Cc: monnier, acm, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1627 bytes --] Richard Stallman <rms@gnu.org> writes: > [[[ To any NSA and FBI agents reading my email: please consider ]]] > [[[ whether defending the US Constitution against all enemies, ]]] > [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > If we are going to redesign Org mode syntax, > can we please add room for extensibility > so that an extension of Org mode syntax > could have all the features of Texinfo? AFAIK, no, Org syntax is not being changed. It seems like Org syntax is already much extensible. > > If the new syntax can express all the distinctions that Texinfo > can already express, that would make it possible to switch to it > for our documentation sources. > > This needs to include conversion into some variant of TeX syntax so > that we can generate high-quality printed manuals through TeX. Converting all Texinfo to Org would require significant effort, which I think should be spend on new packages or existing packages like GCC, Guix, Emacs (and perhaps Hurd to complete the GNU system, LOL). Also switching to Org would probably make vi (no, I'm don't use it) users angry. (No flamewar please.) I think we should better make texinfo-mode better. > > -- > Dr Richard Stallman (https://stallman.org) > Chief GNUisance of the GNU Project (https://gnu.org) > Founder, Free Software Foundation (https://fsf.org) > Internet Hall-of-Famer (https://internethalloffame.org) > > > -- Akib Azmain Turja This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-07 18:14 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Stefan Monnier 2022-06-07 18:26 ` Org mode and Emacs Lars Ingebrigtsen 2022-06-07 22:10 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Tim Cross @ 2022-06-08 13:22 ` Ihor Radchenko 2022-06-08 13:33 ` Stefan Kangas 2022-06-08 14:23 ` Org mode and Emacs Stefan Monnier 2 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-08 13:22 UTC (permalink / raw) To: Stefan Monnier; +Cc: Alan Mackenzie, Tim Cross, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > AFAICT the problem seen from Emacs, is that Org is large (even for > a basic uses, which occasionally leads to high load times) and that it > doesn't follow all the usual Emacs conventions, such as > overriding/remapping standard key-bindings (the resulting behaviors may > sometimes be similar to the standard one, but even when that's the case > the redefinition can easily bite the user). I am not sure if I follow the argument. Major modes are allowed to change defaults. For example, special modes often change truncate-lines value. Org mode is also tweaking defaults (yes, many of them). I do not see any problem here in general. If you have some specific cases when Org mode alters Emacs defaults in a way that bites the user, please give concrete examples. Otherwise, your criticism is not very constructive. > The problem seen from Org is that Emacs doesn't use Org enough :-) > [ This paragraph's shorter length probably reflects the fact that I'm > less familiar with Org than with Emacs. ] > > I think the way forward is to define a "basic org-mode". This one could > be used at many more places where there's currently an occasional desire > to use Org that's resisted because of the above problems. > Ideally, org-mode would then be defined as an extension of this "basic > org-mode". Also ideally some of those extensions would be reworked so > they can be used "Emacs-wide" rather than only in Org (obviously, that > can only make sense for some of them). > > We could start this "basic org-mode" as a trivial copycat of > `outline-mode` and then start adding Org features to it. The driving > constraint is to keep it lightweight and rules-abiding enough that there > won't be any resistance to using it, while at the same time making sure > that the features added to it can be removed from the "org-mode > extension", so that org-mode and "basic org-mode" don't diverge. This is reasonable. RMS also asked for this years back https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org Since we cannot start Org from scratch, factoring out individual modules is taking a lot of time and having the hostile attitudes expressed in some of the emails in this thread is not exactly encouraging. If you want Org to be more modular, please help by reporting inconsistencies or misbehavior to Org ML. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs (was: Convert README.org to plain text README while installing package) 2022-06-08 13:22 ` Ihor Radchenko @ 2022-06-08 13:33 ` Stefan Kangas 2022-06-08 14:23 ` Org mode and Emacs Stefan Monnier 1 sibling, 0 replies; 505+ messages in thread From: Stefan Kangas @ 2022-06-08 13:33 UTC (permalink / raw) To: Ihor Radchenko Cc: Stefan Monnier, Alan Mackenzie, Tim Cross, Emacs developers Ihor Radchenko <yantar92@gmail.com> writes: > Since we cannot start Org from scratch, factoring out individual modules > is taking a lot of time and having the hostile attitudes expressed in > some of the emails in this thread is not exactly encouraging. Thanks for working on modularizing Org mode, it sounds promising. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-08 13:22 ` Ihor Radchenko 2022-06-08 13:33 ` Stefan Kangas @ 2022-06-08 14:23 ` Stefan Monnier 2022-06-08 15:08 ` Ihor Radchenko 2022-06-12 22:38 ` Richard Stallman 1 sibling, 2 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-08 14:23 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Alan Mackenzie, Tim Cross, emacs-devel Ihor Radchenko [2022-06-08 21:22:07] wrote: > Stefan Monnier <monnier@iro.umontreal.ca> writes: >> AFAICT the problem seen from Emacs, is that Org is large (even for >> a basic uses, which occasionally leads to high load times) and that it >> doesn't follow all the usual Emacs conventions, such as >> overriding/remapping standard key-bindings (the resulting behaviors may >> sometimes be similar to the standard one, but even when that's the case >> the redefinition can easily bite the user). > I am not sure if I follow the argument. Major modes are allowed to > change defaults. For example, special modes often change truncate-lines > value. Org mode is also tweaking defaults (yes, many of them). I do not > see any problem here in general. > If you have some specific cases when Org mode alters Emacs defaults in a > way that bites the user, please give concrete examples. Otherwise, your > criticism is not very constructive. [ Alan gave one or two concrete examples of things that bit him. ] This is not a criticism, just a description of how Org is perceived from the side of "old-time Emacs users who aren't Org users". The key in what you wrote above is the "yes, many of them", which means that even tho those tweaks are minor they sum up to something that old-time users will almost inevitably bump into. It's not a problem for Org itself, and there are good reasons for each one of those tweaks, I'm sure, but it does create opposition to using Org "in core", such as in etc/NEWS. > This is reasonable. RMS also asked for this years back > https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org > Since we cannot start Org from scratch, factoring out individual modules > is taking a lot of time and having the hostile attitudes expressed in > some of the emails in this thread is not exactly encouraging. I know it's a lot of work and I'm glad you (plural) are taking it on, and I'm not very happy about some of the things that have been said in that thread, which is why I tried to start this thread. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-08 14:23 ` Org mode and Emacs Stefan Monnier @ 2022-06-08 15:08 ` Ihor Radchenko 2022-06-12 22:38 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-08 15:08 UTC (permalink / raw) To: Stefan Monnier; +Cc: Alan Mackenzie, Tim Cross, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >> If you have some specific cases when Org mode alters Emacs defaults in a >> way that bites the user, please give concrete examples. Otherwise, your >> criticism is not very constructive. > > [ Alan gave one or two concrete examples of things that bit him. ] Alan gave two concrete examples: >>> I put org.org into a buffer. I saw lots of lines ending in ..., which is >>> fair enough. I try C-c C-s to show a subtree, and instead get a calendar >>> in a new window. Brilliant! Did I mention I hate context dependent >>> commands? So I have to type M-x outline-show-subtree, which works. I >>> read the screenful of text, then type C-S-<down> to scroll it up. >>> Instead of my standard scrolling command, I get an error message about >>> "Not at a clock log". Org mode has stolen the key sequences >>> C-S-<up>/<down>, amongst many others. So how am I meant to scroll text? where he 1. Complains about major mode shadowing key binding reserved for major modes, according to D.2 Key Binding Conventions section of the Elisp manual: > • Sequences consisting of ‘C-c’ followed by a control character or a > digit are reserved for major modes. 2. C-S-<down>, which is a valid point. However, arrow commands is one of the few core concepts used in Org major mode. I am not sure if re-binding C-S-<down> is so much of a sin on Org mode side. Emacs manual 49.3 Customizing Key Bindings says: > Since most modes define their own key bindings, activating a mode > might override your custom key bindings. A small number of keys are > reserved for user-defined bindings, and should not be used by modes, so > key bindings using those keys are safer in this regard. The reserved > key sequences are those consisting of ‘C-c’ followed by a letter (either > upper or lower case), and function keys <F5> through <F9> without > modifiers (*note Modifier Keys::). As a compromise, Org may provide some kind of read-only mode just to navigate the document. The arrow bindings can be disabled then. They are for editing. >>> I see many of these bindings as context dependent, with the name of the >>> function named after the key sequence, not the functionality - e.g. >>> org-shiftcontroldown. That is not the way the rest of Emacs is >>> constructed. I hate context dependence (except when it is properly >>> implemented, e.g. by major modes). Only a handful of bindings are context dependent. In particular, control/shift/meta-arrow keys, <TAB>, and C-c C-c. They are most frequently used when editing and navigating Org buffers. It would help if "properly implemented" were defined more precisely in the above statement. > This is not a criticism, just a description of how Org is perceived from > the side of "old-time Emacs users who aren't Org users". > The key in what you wrote above is the "yes, many of them", which means > that even tho those tweaks are minor they sum up to something that > old-time users will almost inevitably bump into. > > It's not a problem for Org itself, and there are good reasons for each > one of those tweaks, I'm sure, but it does create opposition to using > Org "in core", such as in etc/NEWS. I understand. However, such criticism is not helpful. Only concrete examples can lead to actual Org improvements in this area. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-06-08 14:23 ` Org mode and Emacs Stefan Monnier 2022-06-08 15:08 ` Ihor Radchenko @ 2022-06-12 22:38 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Richard Stallman @ 2022-06-12 22:38 UTC (permalink / raw) To: Stefan Monnier; +Cc: yantar92, acm, theophilusx, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > Since we cannot start Org from scratch, factoring out individual modules > > is taking a lot of time and having the hostile attitudes expressed in > > some of the emails in this thread is not exactly encouraging. > I know it's a lot of work and I'm glad you (plural) are taking it on, I feel the same. To move some of the features out of Org, so that they are available in all Emacs modes, will make enhance their availabiliy to all Emacs users. Also, it could eventually make Org itself simpler -- and less daunting to try out and learn. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 16:02 ` Alan Mackenzie 2022-06-07 18:14 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Stefan Monnier @ 2022-06-07 21:51 ` Tim Cross 2022-06-08 19:26 ` chad 1 sibling, 1 reply; 505+ messages in thread From: Tim Cross @ 2022-06-07 21:51 UTC (permalink / raw) To: Alan Mackenzie; +Cc: emacs-devel Alan Mackenzie <acm@muc.de> writes: > I resent you continally repeating this unfounded assertion. There are > 784 entries in org.info's key index. That is the current state. That is > the truth. How can that be "incorrect"? Even if 750 of the 784 bindings > somehow "don't come into play", they still exist, and are still > problematic. Are you saying that only every tenth entry actually > signifies an actual binding, or something like that? > I will make a last attempt to clarify. When this thread started, the discussion was around whether readme.org files should be converted to or replaced with just plain 'traditional' readme files - waht would once probably have been called ASCII files, though now these are utf8 I'm not sure calling them ascii is accurate. It was then suggested an alternative would be to have the readme.org files formatted using org-mode formatting - enabling org mode features for folding, formatting of tables, links, source code examples etc. It was also stated this would be a read-only buffer. The point which has been missed an probably should have been made clearer is that nobody was talking about a full blown org mode. The suggestion was to use primarily org mode rendering/presentation capabilities in a read-only buffer. My point was that most of the key bindings and complexity you were referring to were not relevant in this scenario. Of the 784 key bindings you refer to, the majority of them would never come into play because they are bindings which only exist is specialised sub-modes of org mode, like the agenda, which would not appear in the rendering of a readme.org file. When you open an org file, you get 236 org related key bindings, not 7984. All those additional key bindings only get loaded/defined if you open specialised org modes like the agenda, which does not have a key binding by default, so your unlikely to accidentally initiate that mode. So going back to my original post, which was simply saying that implying you would get 784 additional key bindings just because you open an org file was misleading because you simply don't get that many additional key bindings when you open an org file - in fact, you get only about 1/3 of that many. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 21:51 ` Convert README.org to plain text README while installing package Tim Cross @ 2022-06-08 19:26 ` chad 0 siblings, 0 replies; 505+ messages in thread From: chad @ 2022-06-08 19:26 UTC (permalink / raw) To: Tim Cross; +Cc: Alan Mackenzie, EMACS development team [-- Attachment #1: Type: text/plain, Size: 3239 bytes --] On Tue, Jun 7, 2022 at 7:04 PM Tim Cross <theophilusx@gmail.com> wrote: > So going back to my original post, which was simply saying that > implying you would get 784 additional key bindings just because you open > an org file was misleading because you simply don't get that many > additional key bindings when you open an org file - in fact, you get > only about 1/3 of that many. > I'll add to this statement: the vast majority of those keybindings aren't relevant when viewing a read-only org file, which is what we're talking about here. In fact, I used Org (very lightly) for a few years with exactly these two extra keybindings: Control-Return and Meta-Return. I also used Tab to expand/compact headings, so perhaps three, depending on your usage (I was coming from outline-mode, so this wasn't new.) This is a read/write usage, not a read-only usage. Everything else is basically the same. There are a few places where Org changes the meaning of keys to accommodate Org features, but most of these are DWIM-ish adaptations, or they're guarded by an Org warning that Org uses the keys for something else (for example, shift-selection). A few years in, I discovered that Org would automatically reflow the plain-text tables I sometimes used, and I learned another use for the Tab key; as near as I can tell, this feature had been present for months if not years without ever bothering me. These days, most of my work involves creative writing and light number crunching, not coding; as part of that, I use dozens of text files over several active topics. I keep these in a simple hierarchy of plain Org files. These load in emacs instantaneously. I search through them using standard text-searching tools (mostly find and grep, sometimes trying out things like fd, ack, ripgrep, and deadgrep). I once started reading the Org manual (several years ago) and bounced off pretty quickly; I haven't been back in years (although I hear that the manual has improved quite a lot since then, and it's on my overly-long todo list for "someday".) I say all this not to try to convince anyone to try Org; in fact, if you're not already using outline-mode a bunch, then you're probably only interested in Org if there's a specific use that you'd like for emacs (agenda, note-keeping, literate programming, etc). That said, there are people, especially elisp package authors, who use it and like it and will continue to use it to write package documentation. In the (few) packages that I personally load, most have a README.org, a few have a README.md, and none have a plain README. Put another way, this isn't about encouraging authors to use Org or not; the authors have chosen already. For people who don't really want to try to "learn Org": I hear and understand. I've been there myself; time is usually rarer than interesting ideas to explore. I'm writing this in the hope that it encourages people to not be afraid of looking at .org files even if you don't have the time, interest, or inclination to "learn Org", because while the ocean is indeed (seemingly?) quite deep, the shallow end is quite comfortable. In the meantime, we can all hope that the arrival of the "modular Org" is sooner rather than later. Hope that helps, ~Chad [-- Attachment #2: Type: text/html, Size: 3819 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 13:57 ` Tim Cross 2022-06-06 16:02 ` Eli Zaretskii @ 2022-06-06 19:19 ` Alan Mackenzie 2022-06-07 10:50 ` Protesilaos Stavrou 2022-06-07 0:43 ` Po Lu 2 siblings, 1 reply; 505+ messages in thread From: Alan Mackenzie @ 2022-06-06 19:19 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel Hello, Tim. On Mon, Jun 06, 2022 at 23:57:55 +1000, Tim Cross wrote: > Alan Mackenzie <acm@muc.de> writes: > > Hello, Tim. > > On Mon, Jun 06, 2022 at 10:19:38 +1000, Tim Cross wrote: [ .... ] > >> I also find arguments based around org being complex and difficult > >> to learn to be somewhat overstated. > > There are 784 key bindings in org mode. How can you say that this > > isn't complex and difficult to learn? > Very simply because the vast majority of those key bindings only come > into play when yhou use advanced features of org mode, such as the > agenda or table editing or noweb mode. If your just using basic org > mode features, very very few of those key bindings even come into play. > Just go throgh that list and remove any bindings relating to agenda > mode, table editing mode, source block modes, clock table mode, list > editing mode, etc and you end up with very few key bindings (all of > which are udner the C-c prefix, unlike your previous claim). > >> Org is powerful and very configurable, .... > > That contradicts your previous statement to some extent. Any program > > which is very configurable _needs_ to be configured. > Absolute nonsense. Just because you can configure somethig doesn't > automatically mean you have to configure it. Emacs is extremely > configurable, but you can use it just fine without doing any > configuration at all. I find emacs -Q to be all but unusable, except for testing things with. Only with my configurations does it become (for me) usable. I expect the same to hold for other users. > >> .... which means there can be a lot to learn if you want to leverage > >> off all it has to offer. > > I don't want to do this. I want to avoid org mode being loaded into my > > Emacs; it fouls up my key bindings to a significant extent. Particularly > > if I just want to read a 50 line README. > Other posts have already explained this is NOT what is being proposed > and that it would not affect your key bindings in the slightest. All > that is being proposed here is that a read only buffer will use org mode > to format/display the content. Very simple and not the monster you > insist. "Very simple" has a habit of becoming increasingly complicated over time. > >> However, like emacs, the basics are very simple and easy to learn. > > I don't agree that the basics of Emacs are simple and easy to learn at > > all. That's a strong reason why other editors have become popular. Why > > should I have to spend time learning a super-complicated mode just to > > read a 50 line README? > Well, basically, you don't. That has already bene explained and does not > need to be reitterated here. However, to be clear, all that is being > proposed is that org formatting is applied to the contents of this > read-only buffer. There will not be huge numbers of > unwanted/unexpected key bindings ..... For how long? Politically, org mode is not part of the Emacs core - it is an add on application for those that want it. If the maintainers of org mode wanted to push it into the Emacs core, then they would proceed, firstly, by getting a foot in the door, then steadily increasing pressure to add more and more until org mode was in the position, say, that Info currently is. This must not happen (for reasons discussed below). > .... and there won't be a huge number of other changes. You will likely > have some additional key bindings which may make navigating larger > README files easier and that is about it. All that is really being > proposed here is org font locking and outline mode navigation. All very > simple really. Hmm. > >> While I'm not arguing that org should be forced upon everyone .... > > If a README file is README.org, then org mode is forced upon anybody > > wishing to read it in Emacs. This is why README files should be plain > > text. > >> .... and I would agree we need to keep potential load time issues in > >> mind, there seems to be lots in this thread over stating the issues > >> and jumping to extremes. > > I think org mode is an extreme, with its 784 key bindings which seem > > to violate written and unwritten conventions. > That 750 key bindings is an extreme over statement and not what is > being proposed. Yet again, people jumping to extremes based on > ignorance and paranoia. Spend the time to go through the key bindings > listed in the org manual and you will soon realise that the majority of > those bindings only come into effect in specialised modes - none of > which are relevant to a READM.org ile (for example, all the key > bindings relating to agenda mode). I see many of these bindings as context dependent, with the name of the function named after the key sequence, not the functionality - e.g. org-shiftcontroldown. That is not the way the rest of Emacs is constructed. I hate context dependence (except when it is properly implemented, e.g. by major modes). > I find it odd that someone can on one hand argue so hard against using a > mode based on the complexity and vast nubmer of key bindings and at the > same time admit they have never spent the time to learn the mode. There are only so many hours in the day. I do not "admit" not spending the time learning it, since there is nothing shameful about this. I state it as a fact. I do not want to spend/waste time on it, and it is my fear I will be forced into it by a stealthy increase in org mode's prevelance, if I do not protest. > This seems like an arguument based on fear and uncertainty about > something you don't know rather thaa one based on fact. I put org.org into a buffer. I saw lots of lines ending in ..., which is fair enough. I try C-c C-s to show a subtree, and instead get a calendar in a new window. Brilliant! Did I mention I hate context dependent commands? So I have to type M-x outline-show-subtree, which works. I read the screenful of text, then type C-S-<down> to scroll it up. Instead of my standard scrolling command, I get an error message about "Not at a clock log". Org mode has stolen the key sequences C-S-<up>/<down>, amongst many others. So how am I meant to scroll text? And so on. Without some serious configuration, org mode is for me unusable. Moving commands around key sequences would be a nightmare, given the 784 commands (or even the 230 counted by Eli). And moving org-shiftcontroldown onto a different key sequence, assuming I could find one, would make the function name meaningless. > Perhaps look more closely at what was actually being proposed (which > was not a full org mode situation) and spend enough time to at least > understand the basics before condemnation. I see what looks like an attempt to force org mode into the Emacs core by stealth, a bit at a time. Already you are talking about adding functionality for folding. > >> All that seems to really be under consideration is to enable rendering > >> of *org files in help buffers using org font locking and perhaps > >> enabling folding, which could be very beneficial for large readme files > >> and would not matter for small ones. I also suspect this is something > >> which could be disabled with a simple variable setting for those who > >> really don't like it. And then after adding folding (by some mysterious key sequence), some bright spark will want to add something else "for large readme files". The consequence will be large READMEs which cannot be conveniently read without this new feature. And so it goes on. We already have a format in Emacs for big documentation, Info. Info has about 40 commands, not 230 or 784. > > "It" could best be avoided by having plain text README files. I think we should strongly encourage package writers to include plain text READMEs in their packages. -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 19:19 ` Alan Mackenzie @ 2022-06-07 10:50 ` Protesilaos Stavrou 2022-06-07 12:07 ` Philip Kaludercic 2022-06-07 12:27 ` Stefan Monnier 0 siblings, 2 replies; 505+ messages in thread From: Protesilaos Stavrou @ 2022-06-07 10:50 UTC (permalink / raw) To: Alan Mackenzie, Tim Cross; +Cc: emacs-devel Hello Alan, Tim, everyone, > From: Alan Mackenzie <acm@muc.de> > Date: Mon, 06 Jun 2022 19:19:41 +0000 > >> > "It" could best be avoided by having plain text README files. > > I think we should strongly encourage package writers to include plain > text READMEs in their packages. As the author of several packages on GNU ELPA which use a README.org: 1. I agree with Alan's main point: we do not need Org in Help buffers. Neither directly nor indirectly (if we are having that discussion). 2. I write all my documentation using the .org format because it can be exported to HTML and Info, among others. 3. Org is efficient for large files as it can, for example, create persistent links/references between headings. Those are preserved in .info and .html exports. Org also has nice features, such as the ability to declare a buffer-local macro which expands into, say, "version1.2.3" or "exported on DATE" where DATE is dynamic. There are some cases where such functionality is helpful. Concerning point 2, the GNU ELPA machinery converts the Org source into an Info file. (Those who install the package can access it with C-h i.) While I handle the export process into HTML. This arrangement provides two viable options for users who (i) want to read the documentation yet (ii) do not wish to load org.el and its accoutrements. A third option, in the form of a plain README, is a useful addition, provided it is done automatically by GNU ELPA. As far as I am concerned, the README.org is a fully fledged resource, not a generic, "here are the absolute basics" kind of file (not that there is anything wrong with those). As such, writing a separate README, instead of exporting README.org into plain text, would add to the maintenance burden---I would rather avoid that. All the best, Protesilaos (or simply "Prot") -- Protesilaos Stavrou https://protesilaos.com ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 10:50 ` Protesilaos Stavrou @ 2022-06-07 12:07 ` Philip Kaludercic 2022-06-07 12:23 ` Protesilaos Stavrou 2022-06-07 12:27 ` Stefan Monnier 1 sibling, 1 reply; 505+ messages in thread From: Philip Kaludercic @ 2022-06-07 12:07 UTC (permalink / raw) To: Protesilaos Stavrou; +Cc: Alan Mackenzie, Tim Cross, emacs-devel Hi Prot, Protesilaos Stavrou <info@protesilaos.com> writes: > As far as I am concerned, the README.org is a fully fledged resource, > not a generic, "here are the absolute basics" kind of file (not that > there is anything wrong with those). As such, writing a separate > README, instead of exporting README.org into plain text, would add to > the maintenance burden---I would rather avoid that. Most packages already come with a commentary section. While these sometimes just say "read the readme" and others contain unnecessary information like how to install the package, many are well written and give the appropriate, initial pointers for getting started with a new package (basic configuration recommendations, what the main entry points are, what options to look out for). As mentioned before in this thread, I don't think C-h P should give the same information as the manual -- be in in org-mode or rendered as ASCII/Unicode. This is too much information for someone who wants to just wants to know what does a package "foobar" do (this is especially critical when package names are not self explanatory/generic, as was discussed a few weeks back). So it might make sense to instruct the ELPA build system to not use README files, and instead just rely on the commentary section of the main file. If you check out the elpaa--get-README function elpa-admin.el, you'll already see that "README.md" files are commented out, because ;; Most README.md files seem to be ;; currently worse than the Commentary: ;; section :-( which is indicative of a general problem, of how README files are used since GitHub (I have commented previously on the issue in this thread https://news.ycombinator.com/item?id=30336703). Once again I would recommend publishing guidelines and recommendations on good packaging practices on elpa.gnu.org, with tips and examples on what a good commentary section looks like. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 12:07 ` Philip Kaludercic @ 2022-06-07 12:23 ` Protesilaos Stavrou 0 siblings, 0 replies; 505+ messages in thread From: Protesilaos Stavrou @ 2022-06-07 12:23 UTC (permalink / raw) To: Philip Kaludercic; +Cc: Alan Mackenzie, Tim Cross, emacs-devel Hello Philip, > From: Philip Kaludercic <philipk@posteo.net> > Date: Tue, 07 Jun 2022 12:07:21 +0000 > >> As far as I am concerned, the README.org is a fully fledged resource, >> not a generic, "here are the absolute basics" kind of file (not that >> there is anything wrong with those). As such, writing a separate >> README, instead of exporting README.org into plain text, would add to >> the maintenance burden---I would rather avoid that. > > Most packages already come with a commentary section. While these > sometimes just say "read the readme" and others contain unnecessary > information like how to install the package, many are well written and > give the appropriate, initial pointers for getting started with a new > package (basic configuration recommendations, what the main entry points > are, what options to look out for). > > As mentioned before in this thread, I don't think C-h P should give the > same information as the manual -- be in in org-mode or rendered as > ASCII/Unicode. This is too much information for someone who wants to > just wants to know what does a package "foobar" do (this is especially > critical when package names are not self explanatory/generic, as was > discussed a few weeks back). > > [... 14 lines elided] > > Once again I would recommend publishing guidelines and recommendations > on good packaging practices on elpa.gnu.org, with tips and examples on > what a good commentary section looks like. Agreed! I do use the Commentary for this purpose and am happy to conform with such guidelines. -- Protesilaos Stavrou https://protesilaos.com ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 10:50 ` Protesilaos Stavrou 2022-06-07 12:07 ` Philip Kaludercic @ 2022-06-07 12:27 ` Stefan Monnier 1 sibling, 0 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-07 12:27 UTC (permalink / raw) To: Protesilaos Stavrou; +Cc: Alan Mackenzie, Tim Cross, emacs-devel > Concerning point 2, the GNU ELPA machinery converts the Org source into > an Info file. (Those who install the package can access it with C-h i.) > While I handle the export process into HTML. Side note: elpa.gnu.org also converts the docs to HTML (both for Org and for Texinfo docs), and puts a link to it under "Manual" as can be seen on http://elpa.gnu.org/packages/modus-themes.html [ This HTML isn't ideal, admittedly. Help welcome. ] > This arrangement provides two viable options for users who (i) want to > read the documentation yet (ii) do not wish to load org.el and its > accoutrements. A third option, in the form of a plain README, is a > useful addition, provided it is done automatically by GNU ELPA. I have the impression that there's a bit of misunderstanding here. There are two pieces of "docs" that elpa.gnu.org handles: the `:readme` and the `:doc`. One is meant for an actual manual that can be browsed with Info, while the other is expected to be a short blurb, to help prospective users understand what the package is about. Both accept the Org format, but the `:doc` does not accept "plain text" (nor Markdown). Many packages currently use the same file for both (typically called `README.org`), which tends to be suboptimal (it's often neither a great `:readme` nor a great `:doc`), but works fine. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 13:57 ` Tim Cross 2022-06-06 16:02 ` Eli Zaretskii 2022-06-06 19:19 ` Alan Mackenzie @ 2022-06-07 0:43 ` Po Lu 2022-06-07 6:46 ` Kévin Le Gouguec 2 siblings, 1 reply; 505+ messages in thread From: Po Lu @ 2022-06-07 0:43 UTC (permalink / raw) To: Tim Cross; +Cc: Alan Mackenzie, emacs-devel Tim Cross <theophilusx@gmail.com> writes: > That 750 key bindings is an extreme over statement and not what is being > proposed. Yet again, people jumping to extremes based on ignorance and > paranoia. Spend the time to go through the key bindings listed in the > org manual and you will soon realise that the majority of those bindings > only come into effect in specialised mdoes - none of which are relevant > to a READM.org ile (for example, all the key bindings relating to agenda > mode). > > I find it odd that someone can on one hand argue so hard against using a > mode based on the complexity and vast nubmer of key bindings and at the > same time admit they have never spent the time to learn the mode. This > seems like an arguument based on fear and uncertainty about something > you don't know rather thaa one based on fact. Perhaps look more closely > at what was actually being proposed (which was not a full org mode > situation) and spend enough time to at least udnerstand the basics > before condemnation. What's more odd is expecting someone to go through the entire Org manual in order to write and edit README files. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 0:43 ` Po Lu @ 2022-06-07 6:46 ` Kévin Le Gouguec 2022-06-07 7:53 ` Po Lu 0 siblings, 1 reply; 505+ messages in thread From: Kévin Le Gouguec @ 2022-06-07 6:46 UTC (permalink / raw) To: Po Lu; +Cc: Tim Cross, Alan Mackenzie, emacs-devel Po Lu <luangruo@yahoo.com> writes: > Tim Cross <theophilusx@gmail.com> writes: > >> That 750 key bindings is an extreme over statement and not what is being >> proposed. Yet again, people jumping to extremes based on ignorance and >> paranoia. Spend the time to go through the key bindings listed in the >> org manual and you will soon realise that the majority of those bindings >> only come into effect in specialised mdoes - none of which are relevant >> to a READM.org ile (for example, all the key bindings relating to agenda >> mode). >> >> I find it odd that someone can on one hand argue so hard against using a >> mode based on the complexity and vast nubmer of key bindings and at the >> same time admit they have never spent the time to learn the mode. This >> seems like an arguument based on fear and uncertainty about something >> you don't know rather thaa one based on fact. Perhaps look more closely >> at what was actually being proposed (which was not a full org mode >> situation) and spend enough time to at least udnerstand the basics >> before condemnation. > > What's more odd is expecting someone to go through the entire Org manual > in order to write and edit README files. This thread has been a bit disheartening to read from the sidelines; as much as I empathize with wanting to keep Emacs accessible by avoiding code bloat and keeping cognitive load low, it is hard to know what to reply to statements such as this one. No one has ever had to "go through the entire Org manual in order to write and edit README files". Just like no one has ever had to read the Emacs manual cover-to-cover in order to do anything with it (they _can_, of course, but they don't _have to_). My first years with Org were spent using (1) TAB folding (2) outline navigation commands, and that's it. We already have *Help* commands which combine those features on the development branch: see C-h b with the recently added describe-bindings-outline user option. I don't see how enabling these section-folding and navigation features could hurt the accessibility of package READMEs written in a structured format. Alan rightly points out that Org overreaches with unhelpful key bindings, and the benchmarks posted earlier show that loading Org wholesale for displaying READMEs would be irksome; to me that merely suggests that whatever subset of Org features we find relevant for _consulting_ READMEs should be better modularized. So I would expect the discussion to focus on: (1) what this subset should be (e.g. should it include font-locking), (2) whether we want to add "escape hatches" for whoever struggles with those "rich" READMEs. E.g. a package-show-plain-readmes option, which could do any number of things depending on the underlying markup format. For Org specifically, it could pass the README through the (ill-named, and not limited to ASCII) org-ascii-export-as-ascii, which converts Org markup to something more appropriate for font-lock-less viewing (sections underlined with equals and hyphens; inline links moved to their own paragraphs). It might even make sense for (Non)GNU ELPA to automatically run that function on Org READMEs in order to ensure the "bare" variant is always instantly available to package users. Instead it feels like a lot of words are spent on (1) Org is bloated and complex: yes, agreed, (2) Package maintainers should give users the courtesy of maintaining two READMEs: no, I don't see a universe where that will fly. I hope at the end of all this, we can find some middle ground between maintainers who happen to find their productivity increased by using Org, and maintainers who cringe at every cycle their CPU spends on Org code. I believe this middle ground exists. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 6:46 ` Kévin Le Gouguec @ 2022-06-07 7:53 ` Po Lu 2022-06-07 22:15 ` Kévin Le Gouguec 0 siblings, 1 reply; 505+ messages in thread From: Po Lu @ 2022-06-07 7:53 UTC (permalink / raw) To: Kévin Le Gouguec; +Cc: Tim Cross, Alan Mackenzie, emacs-devel Kévin Le Gouguec <kevin.legouguec@gmail.com> writes: > This thread has been a bit disheartening to read from the sidelines; as > much as I empathize with wanting to keep Emacs accessible by avoiding > code bloat and keeping cognitive load low, it is hard to know what to > reply to statements such as this one. > > No one has ever had to "go through the entire Org manual in order to > write and edit README files". Just like no one has ever had to read the > Emacs manual cover-to-cover in order to do anything with it (they _can_, > of course, but they don't _have to_). So what was the purpose of the following statement? >> Spend the time to go through the key bindings listed in the org >> manual > I don't see how enabling these section-folding and navigation features > could hurt the accessibility of package READMEs written in a structured > format. Alan rightly points out that Org overreaches with unhelpful key > bindings, and the benchmarks posted earlier show that loading Org > wholesale for displaying READMEs would be irksome; to me that merely > suggests that whatever subset of Org features we find relevant for > _consulting_ READMEs should be better modularized. IMO, packages should provide a plain text README, alongside a rich text version in a relatively standard format like HTML. By using HTML, it becomes readable not only in Emacs, but also any other program that understands HTML. HTML is also understood by many more people than Org mode. It's okay for package maintainers to write their documentation using Org, and then to export it to HTML. But please don't insist that all documentation be written using Org mode, or that it's okay for packages to only provide documentation in that format. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 7:53 ` Po Lu @ 2022-06-07 22:15 ` Kévin Le Gouguec 2022-06-08 0:36 ` Po Lu 0 siblings, 1 reply; 505+ messages in thread From: Kévin Le Gouguec @ 2022-06-07 22:15 UTC (permalink / raw) To: Po Lu; +Cc: Tim Cross, Alan Mackenzie, emacs-devel Po Lu <luangruo@yahoo.com> writes: > Kévin Le Gouguec <kevin.legouguec@gmail.com> writes: > >> No one has ever had to "go through the entire Org manual in order to >> write and edit README files". Just like no one has ever had to read the >> Emacs manual cover-to-cover in order to do anything with it (they _can_, >> of course, but they don't _have to_). > > So what was the purpose of the following statement? > >>> Spend the time to go through the key bindings listed in the org >>> manual Cannot speak for Alan nor Tim, but my understanding of the exchange that led to the bit you quote is that Tim was inviting Alan to check how many of these 784 bindings had any relevance to the topic under discussion (navigating and, to a lesser extent, authoring Org READMEs) in order to show that "M-x org TAB" was not a good heuristic to assess how much complexity _needs_ to be shoved onto an unsuspecting README reader. In Tim's own words (a bit before the one you quote): > That 750 key bindings is an extreme over statement and not what is being > proposed. So your paraphrasing ("expecting someone to go through the entire Org manual in order to write and edit README files") struck me as disconnected from (my own understanding of) Tim's argument: AFAIU, * he wasn't saying _Org users_ need to go through the manual, * he was confronting the "784 bindings" argument by saying it was not a particularly useful heuristic, inviting _us_ (participants of the present discussion) to check for ourselves what proportion of these bindings are actually relevant to the discussion. Again though, cannot speak for Tim; I can only explain my undertanding of the discussion, and why I was dumbfounded by your reply. > IMO, packages should provide a plain text README, alongside a rich text > version in a relatively standard format like HTML. > > By using HTML, it becomes readable not only in Emacs, but also any other > program that understands HTML. HTML is also understood by many more > people than Org mode. > > It's okay for package maintainers to write their documentation using > Org, and then to export it to HTML. But please don't insist that all > documentation be written using Org mode, or that it's okay for packages > to only provide documentation in that format. If there's been advocating for all documentation to be written in Org, I've (dis)missed that, so sorry for not following closely enough. As for packages only providing documentation in Org: again, I think the way forward is automation at the package archive level. Asking authors to maintain two sync'ed READMEs stands a small chance of success IMO. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 22:15 ` Kévin Le Gouguec @ 2022-06-08 0:36 ` Po Lu 2022-06-08 6:41 ` Kévin Le Gouguec 0 siblings, 1 reply; 505+ messages in thread From: Po Lu @ 2022-06-08 0:36 UTC (permalink / raw) To: Kévin Le Gouguec; +Cc: Tim Cross, Alan Mackenzie, emacs-devel Kévin Le Gouguec <kevin.legouguec@gmail.com> writes: > So your paraphrasing ("expecting someone to go through the entire Org > manual in order to write and edit README files") struck me as > disconnected from (my own understanding of) Tim's argument: AFAIU, > > * he wasn't saying _Org users_ need to go through the manual, > > * he was confronting the "784 bindings" argument by saying it was not a > particularly useful heuristic, inviting _us_ (participants of the > present discussion) to check for ourselves what proportion of these > bindings are actually relevant to the discussion. > > Again though, cannot speak for Tim; I can only explain my undertanding > of the discussion, and why I was dumbfounded by your reply. Going through the key bindings in the Org manual means to essentially read the entire manual. > If there's been advocating for all documentation to be written in Org, > I've (dis)missed that, so sorry for not following closely enough. That's how this discussion started, with Stefan K asking for all documentation to be written in Org mode. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 0:36 ` Po Lu @ 2022-06-08 6:41 ` Kévin Le Gouguec 0 siblings, 0 replies; 505+ messages in thread From: Kévin Le Gouguec @ 2022-06-08 6:41 UTC (permalink / raw) To: Po Lu; +Cc: Tim Cross, Alan Mackenzie, emacs-devel Po Lu <luangruo@yahoo.com> writes: > Kévin Le Gouguec <kevin.legouguec@gmail.com> writes: > >> So your paraphrasing ("expecting someone to go through the entire Org >> manual in order to write and edit README files") struck me as >> disconnected from (my own understanding of) Tim's argument: AFAIU, >> >> * he wasn't saying _Org users_ need to go through the manual, >> >> * he was confronting the "784 bindings" argument by saying it was not a >> particularly useful heuristic, inviting _us_ (participants of the >> present discussion) to check for ourselves what proportion of these >> bindings are actually relevant to the discussion. >> >> Again though, cannot speak for Tim; I can only explain my undertanding >> of the discussion, and why I was dumbfounded by your reply. > > Going through the key bindings in the Org manual means to essentially > read the entire manual. Yes, going through keybindings ≈ reading the entire manual. My point is that asking _participants of a discussion_ to do X in order to make the stakes clearer (my understanding of Tim's position) ≪ asking anyone who wants to write and edit README files to do X (your paraphrasing) >> If there's been advocating for all documentation to be written in Org, >> I've (dis)missed that, so sorry for not following closely enough. > > That's how this discussion started, with Stefan K asking for all > documentation to be written in Org mode. Assuming you are referring to <CADwFkm=6_xP6zRSkDP1kreMKCXGBmp8Ze2WtUMeqq9-gEuGcxw@mail.gmail.com>? The actual OP, <87leuca7v7.fsf@disroot.org>, started with something much more modest: converting Org READMEs to plaintext. Then, AFAIU, came ideas to (in no particular order, and not mutually exclusive) (1) render the Org syntax rather than converting it "down" to plaintext, (2) enable Org wholesale, (3) break down Org into smaller pieces in order to make it less of a pain to fit the "display a rich README" use-case. I admit to focusing on those points of the discussion, forgetting about StefanK's message, and then getting very confused why Org's complexity _as an authorship tool_ mattered so much. Apologies for not paying enough attention. Anyway, Philip posted a patch to the effect of what the OP suggested in <877d5ut6z6.fsf@posteo.net>. Digging into the ELPA admin branch, I also see that the machinery for the conversion is exactly what I expected: it lets Org's export library do the heavy lifting. So AFAICT reason prevailed; package maintainers who enjoy writing READMEs in Org can keep doing so without impacting their users; and despite the forecast for today being pretty rainy over here, I'm starting my day in a resolutely sunny mood. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 0:19 ` Tim Cross 2022-06-06 9:59 ` Philip Kaludercic 2022-06-06 11:33 ` Alan Mackenzie @ 2022-06-06 16:56 ` Michael Albinus 2022-06-07 20:57 ` Jean Louis 3 siblings, 0 replies; 505+ messages in thread From: Michael Albinus @ 2022-06-06 16:56 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel Tim Cross <theophilusx@gmail.com> writes: Hi Tim, >> I have no problem if there are structured README.org or README.md files >> in parallel. But a README file should be plain text. > > I've seen this mentioned multiple times in this thread and it doesn't > make sense to me. > > Org files *are* plain text. This is one of (perhaps the biggest) selling > points for org mode. They don't use any form of 'binary' data and can be > read just fine in any text editor or just using cat/less/more whatever. > They may look a little *ugly*, especially with respect to URLs, but are > still quite readable - a lot more readable than other plain text formats > such as xml or html or json etc. My point in this discussion is, that package's README files are also visited by people not using Emacs. I'm confident that Emacs is able to present an org-mode structure nicely, but this isn't true outside the Emacs world. Yes, we're speaking about Emacs packages first-hand. But even they are distributed outside the Emacs or GitHub/GitLab infrastructure. So we need a single starting point for users, w/o org-mode dependencies. Best regards, Michael. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 0:19 ` Tim Cross ` (2 preceding siblings ...) 2022-06-06 16:56 ` Michael Albinus @ 2022-06-07 20:57 ` Jean Louis 2022-06-08 6:50 ` Tim Cross 3 siblings, 1 reply; 505+ messages in thread From: Jean Louis @ 2022-06-07 20:57 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel * Tim Cross <theophilusx@gmail.com> [2022-06-06 15:57]: > Org files *are* plain text. This is one of (perhaps the biggest) selling > points for org mode. We may call it "plain text" and problem is not that we can open Org files with any editor as plain text, but in formatting. People format Org files in such ways that they are not readable, they may not make spacing where it would be otherwise required, in other words, plain text files do not look nearly as readable as for example RFC text, like this one: https://www.rfc-editor.org/rfc/rfc1.txt -- Jean Take action in Free Software Foundation campaigns: https://www.fsf.org/campaigns In support of Richard M. Stallman https://stallmansupport.org/ ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 20:57 ` Jean Louis @ 2022-06-08 6:50 ` Tim Cross 2022-06-08 7:25 ` Ihor Radchenko ` (2 more replies) 0 siblings, 3 replies; 505+ messages in thread From: Tim Cross @ 2022-06-08 6:50 UTC (permalink / raw) To: Jean Louis; +Cc: emacs-devel Jean Louis <bugs@gnu.support> writes: > * Tim Cross <theophilusx@gmail.com> [2022-06-06 15:57]: >> Org files *are* plain text. This is one of (perhaps the biggest) selling >> points for org mode. > > We may call it "plain text" and problem is not that we can open Org > files with any editor as plain text, but in formatting. People format > Org files in such ways that they are not readable, they may not make > spacing where it would be otherwise required, in other words, plain > text files do not look nearly as readable as for example RFC text, > like this one: https://www.rfc-editor.org/rfc/rfc1.txt Please give me an example of org mode 'not make space where it would be otherwise required'. Can you provide a single example of org mode syntax which is not readable in any text editor. There are quite a few projects on Github/Gitlab which have readme.org files - can you point to one which cannot be read with a plain text editor? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 6:50 ` Tim Cross @ 2022-06-08 7:25 ` Ihor Radchenko 2022-06-08 7:43 ` Tim Cross 2022-06-08 11:39 ` Jean Louis 2022-06-11 6:40 ` Jean Louis 2 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-08 7:25 UTC (permalink / raw) To: Tim Cross; +Cc: Jean Louis, emacs-devel Tim Cross <theophilusx@gmail.com> writes: > Jean Louis <bugs@gnu.support> writes: > >> * Tim Cross <theophilusx@gmail.com> [2022-06-06 15:57]: >>> Org files *are* plain text. This is one of (perhaps the biggest) selling >>> points for org mode. >> >> We may call it "plain text" and problem is not that we can open Org >> files with any editor as plain text, but in formatting. People format >> Org files in such ways that they are not readable, they may not make >> spacing where it would be otherwise required, in other words, plain >> text files do not look nearly as readable as for example RFC text, >> like this one: https://www.rfc-editor.org/rfc/rfc1.txt > > Please give me an example of org mode 'not make space where it would be > otherwise required'. Can you provide a single example of org mode > syntax which is not readable in any text editor. There are quite a few > projects on Github/Gitlab which have readme.org files - can you point to > one which cannot be read with a plain text editor? I'd say that Jean made a fair point. In Org, authors may not care as much about, for example, indentation. Especially when the org files are written with org-indent-mode turned on. If you look at https://www.rfc-editor.org/rfc/rfc1.txt, the paragraphs a nicely indented and can be distinguished from the headers. The top-level headers are numbered and can be easily distinguished from the lower-level headers. The paragraph text is also filled appropriately, unlike some Org documents written with truncate-lines set to nil. So, without Emacs, Org files (some of them!) may be harder to read compared to properly formatted ASCII. I am not 100% sure if we need to do anything about this observation. One practical conclusion that can be made is that we might incorporate filling, numbering, and indentation into ox-org.el. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 7:25 ` Ihor Radchenko @ 2022-06-08 7:43 ` Tim Cross 2022-06-08 11:27 ` Jean Louis 2022-06-08 13:24 ` Ihor Radchenko 0 siblings, 2 replies; 505+ messages in thread From: Tim Cross @ 2022-06-08 7:43 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Jean Louis, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > Tim Cross <theophilusx@gmail.com> writes: > >> Jean Louis <bugs@gnu.support> writes: >> >>> * Tim Cross <theophilusx@gmail.com> [2022-06-06 15:57]: >>>> Org files *are* plain text. This is one of (perhaps the biggest) selling >>>> points for org mode. >>> >>> We may call it "plain text" and problem is not that we can open Org >>> files with any editor as plain text, but in formatting. People format >>> Org files in such ways that they are not readable, they may not make >>> spacing where it would be otherwise required, in other words, plain >>> text files do not look nearly as readable as for example RFC text, >>> like this one: https://www.rfc-editor.org/rfc/rfc1.txt >> >> Please give me an example of org mode 'not make space where it would be >> otherwise required'. Can you provide a single example of org mode >> syntax which is not readable in any text editor. There are quite a few >> projects on Github/Gitlab which have readme.org files - can you point to >> one which cannot be read with a plain text editor? > > I'd say that Jean made a fair point. > In Org, authors may not care as much about, for example, indentation. > Especially when the org files are written with org-indent-mode turned > on. > > If you look at https://www.rfc-editor.org/rfc/rfc1.txt, the paragraphs a > nicely indented and can be distinguished from the headers. The top-level > headers are numbered and can be easily distinguished from the > lower-level headers. The paragraph text is also filled appropriately, > unlike some Org documents written with truncate-lines set to nil. > > So, without Emacs, Org files (some of them!) may be harder to read > compared to properly formatted ASCII. > > I am not 100% sure if we need to do anything about this observation. > One practical conclusion that can be made is that we might incorporate > filling, numbering, and indentation into ox-org.el. > On one level, fair enough. However, this has nothing to do with org mode in the sense it is not enforced or caused by org mode. My point is that there is nothing in the org mode syntax which makes it impossible to read the contents with any plain text editor. You can also write a poorly formatted ascii file which lacks indentation, spaces etc in any editor. There is nothing in org syntax which forces people to set truncate lines to nil or prevents them from indenting or wrapping or adding underlines or extra spaces around headings or .... If you want to write rfc compliant files with org mode, you can. It isn't the org mode syntax which is the issue. Besides, just because a plain text file doesn't look as nice or isn't a subjectively readable doesn't mean it isn't a plain text file anymore. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 7:43 ` Tim Cross @ 2022-06-08 11:27 ` Jean Louis 2022-06-08 13:24 ` Ihor Radchenko 1 sibling, 0 replies; 505+ messages in thread From: Jean Louis @ 2022-06-08 11:27 UTC (permalink / raw) To: emacs-devel, Tim Cross, Ihor Radchenko I think that one of points conveyed with the term "plain text" means readable. That is what README should be about. Even the title of the file (README) indicates it is for reading. Org files are source files, they shall for public be exported to the actual text. The term "plain text" is not beneficial in this subject. Those files should be readable without Emacs. Take as analogy the TeX system, or Texinfo, that is also plain text but not readable. Jean ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 7:43 ` Tim Cross 2022-06-08 11:27 ` Jean Louis @ 2022-06-08 13:24 ` Ihor Radchenko 1 sibling, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-08 13:24 UTC (permalink / raw) To: Tim Cross; +Cc: Jean Louis, emacs-devel Tim Cross <theophilusx@gmail.com> writes: > On one level, fair enough. However, this has nothing to do with org mode > in the sense it is not enforced or caused by org mode. My point is that > there is nothing in the org mode syntax which makes it impossible to > read the contents with any plain text editor. You can also write a > poorly formatted ascii file which lacks indentation, spaces etc in any editor. Yes, but the poor formatting is immediately noticeable in ASCII. Not in Org. One may provide an org-lint checker for stylistic formatting issues maybe? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 6:50 ` Tim Cross 2022-06-08 7:25 ` Ihor Radchenko @ 2022-06-08 11:39 ` Jean Louis 2022-06-11 6:40 ` Jean Louis 2 siblings, 0 replies; 505+ messages in thread From: Jean Louis @ 2022-06-08 11:39 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel It was not the point if file is readable by computer program named text editor. The point is if it is readable for human. There are many Emacs users who don't use Org. They are not supposed to decipher brackets, properties, links, and lack of authors' sense in writing Org files in human readable manner. This whole subject is related to accessibility, Org files could be accessible and thus well readable if they would be just simple text without any markup, without headings, but they are practically not so. Authors write headings without space to its paragraph, it destroys the original meaning of the heading, file does not convey the intended idea unless you are Emscd wizard. Difficulties in reading reject users. I will give you examples later. On June 8, 2022 6:50:24 AM UTC, Tim Cross <theophilusx@gmail.com> wrote: > >Jean Louis <bugs@gnu.support> writes: > >> * Tim Cross <theophilusx@gmail.com> [2022-06-06 15:57]: >>> Org files *are* plain text. This is one of (perhaps the biggest) >selling >>> points for org mode. >> >> We may call it "plain text" and problem is not that we can open Org >> files with any editor as plain text, but in formatting. People format >> Org files in such ways that they are not readable, they may not make >> spacing where it would be otherwise required, in other words, plain >> text files do not look nearly as readable as for example RFC text, >> like this one: https://www.rfc-editor.org/rfc/rfc1.txt > >Please give me an example of org mode 'not make space where it would be >otherwise required'. Can you provide a single example of org mode >syntax which is not readable in any text editor. There are quite a few >projects on Github/Gitlab which have readme.org files - can you point >to >one which cannot be read with a plain text editor? Jean ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 6:50 ` Tim Cross 2022-06-08 7:25 ` Ihor Radchenko 2022-06-08 11:39 ` Jean Louis @ 2022-06-11 6:40 ` Jean Louis 2022-06-11 11:16 ` Protesilaos Stavrou 2022-06-12 2:23 ` Tim Cross 2 siblings, 2 replies; 505+ messages in thread From: Jean Louis @ 2022-06-11 6:40 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel * Tim Cross <theophilusx@gmail.com> [2022-06-08 09:57]: > Please give me an example of org mode 'not make space where it would be > otherwise required'. Can you provide a single example of org mode > syntax which is not readable in any text editor. There are quite a few > projects on Github/Gitlab which have readme.org files - can you point to > one which cannot be read with a plain text editor? Hello Tim, my response time is not so fast. I am now reviewing package descriptions: - adoc-mode -- it uses Markdown to display descriptions, this is similar case to using Org. However, it is pretty human readable. In any case, both Markdown or Org formats should be converted to plain text. - ample-theme -- uses Org for package description, I consider this description not human readable. It is Emacs package theme. It is not even related to Org mode, but user has to look into following: > * ample-theme.el > - A Light and Dark pair of themes for Emacs. > - Full colored gui and 256 color support only for now > ** Installation > If MELPA is set up: > #+BEGIN_SRC emacs-lisp > (package-install 'ample-theme) > (load-theme 'ample t) > #+END_SRC > ** If you get ugly colors in terminal: > #+BEGIN_SRC shell-script > # throw this in your ~/.bash_profile > export TERM="xterm-256color" > #+END_SRC > ** Ample-light-theme vs Ample-theme > [[http://i.imgur.com/86VLSV9.png]] In the above Org type of a description, author does not have a sense even to separate headings with the blank space. And how does "#+BEGIN_SRC emacs-lisp" help user to install the theme? Can we understand that user simply wants to install the theme and that "#+BEGIN_SRC emacs-lisp" is there not in right place, it is confusing. Author expects of the user to know what is Org mode in addition to installation of a theme, that is not proportional. Users installing themes may not need to know nothing about Org. Recommended for everybody to read: Usability 101: Introduction to Usability https://www.nngroup.com/articles/usability-101-introduction-to-usability/ Usability Testing 101 https://www.nngroup.com/articles/usability-testing-101/ More reviews: - package corfu, totally not human readable as it begins with: > #+title: corfu.el - Completion Overlay Region FUnction > #+author: Daniel Mendler > #+language: en > #+export_file_name: corfu.texi > #+texinfo_dir_category: Emacs > #+texinfo_dir_title: Corfu: (corfu). > #+texinfo_dir_desc: Completion Overlay Region FUnction > #+html: <a href="https://www.gnu.org/software/emacs/"><img alt="GNU Emacs" src="https://github.com/minad/corfu/blob/screenshots/emacs.svg?raw=true"/></a> > #+html: <a href="http://elpa.gnu.org/packages/corfu.html"><img alt="GNU ELPA" src="https://elpa.gnu.org/packages/corfu.svg"/></a> > #+html: <a href="http://elpa.gnu.org/devel/corfu.html"><img > #alt="GNU-devel ELPA" > #src="https://elpa.gnu.org/devel/corfu.svg"/></a - package crdt -- pretty confusing when reading that Org, something like this: > =M-x crdt-list-sessions= lists all sessions. it expects user to know that "=" is markup. It cannot help to full understanding. Users of CRDT need not be Org users. Even many Org users do not know full Org markup. - package ftable > #+TITLE: ftable.el > This package provides some convenient commands for filling a table, i.e., adjusting the layout of the table so it can fit in n columns. > [[./ftable.gif]] How is that tag "#+TITLE: ftable.el" helping user? We cannot assume that users will know about the meaning of "#+" and why shall package description again repeat the title of the package, that is already in the header of the package. And how am I supposed to know what is "[[./ftable.gif]]" and where it is? I have no access to it. It communicates zero information to user. It confuses user. Usability would be greatly improved if there would be simple link to the image, as the package description is anyway showing links correctly. - package leaf -- this one completely expects user to know about Org markup. It has terrible readibility score. - package logos -- until user comes to read anything about description, must go through this garbage: > #+title: logos.el: simple focus mode for Emacs with page breaks or outlines > #+author: Protesilaos Stavrou > #+email: info@protesilaos.com > #+language: en > #+options: ':t toc:nil author:t email:t num:t > #+startup: content > #+macro: stable-version 0.3.0 > #+macro: release-date 2022-03-30 > #+macro: development-version 0.4.0-dev > #+macro: file @@texinfo:@file{@@$1@@texinfo:}@@ > #+macro: space @@texinfo:@: @@ > #+macro: kbd @@texinfo:@kbd{@@$1@@texinfo:}@@ > #+export_file_name: logos.texi > #+texinfo_filename: logos.info > #+texinfo_dir_category: Emacs misc features > #+texinfo_dir_title: Logos: (logos) > #+texinfo_dir_desc: Simple focus mode with page breaks or outlines > #+texinfo_header: @set MAINTAINERSITE @uref{https://protesilaos.com,maintainer webpage} > #+texinfo_header: @set MAINTAINER Protesilaos Stavrou > #+texinfo_header: @set MAINTAINEREMAIL @email{info@protesilaos.com} > #+texinfo_header: @set MAINTAINERCONTACT @uref{mailto:info@protesilaos.com,contact the maintainer} > #+texinfo: @insertcopying - package org-modern -- author did not check if those links work. Example communication by author: > The screenshots shows [[file:example.org][example.org]] with > =org-modern-mode= turned on and off Links appear there, however, the file example.org is not clickable, it could easily be made properly clickable. - package osm -- not readable due to Org settings in package description. Those settings do not belong to package description. - package pdf-tools -- same thing, user is supposed to read description. Things like below are confusing: > :PROPERTIES: > :CREATED: [2021-12-29 Wed 18:34] > :ID: 5a884389-6aec-498a-90d5-f37168809b4f > :EXPORT_FILE_NAME: index > :END: Those tags are by no means relevant to package description. User wants PDF tools, and what is user supposed to mean about the ID hashes and what else... - package posframe -- readability is minimized due to usage of heavier Org markup - package svg-lib -- user cannot click on provided screenshots in Org markup. Why do that? Provide correct links. - package system-packages -- Org markup degrades human readability I have reviewed just some of packages installed on my side. There may be many others. Readability and usability is greatest issue in computing and helping user with understanding. One shall compare Org package descriptions to so many other packages without Org and find out how Org greatly degrades readability and thus understanding on what that package is about and how to use it. -- Jean Take action in Free Software Foundation campaigns: https://www.fsf.org/campaigns In support of Richard M. Stallman https://stallmansupport.org/ ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-11 6:40 ` Jean Louis @ 2022-06-11 11:16 ` Protesilaos Stavrou 2022-06-12 11:32 ` Jean Louis 2022-06-12 2:23 ` Tim Cross 1 sibling, 1 reply; 505+ messages in thread From: Protesilaos Stavrou @ 2022-06-11 11:16 UTC (permalink / raw) To: Jean Louis, Tim Cross; +Cc: emacs-devel > From: Jean Louis <bugs@gnu.support> > Date: Sat, 11 Jun 2022 09:40:44 +0300 > > [... 109 lines elided] > > - package logos -- until user comes to read anything about > description, must go through this garbage: > > [... 68 lines elided] Hello Jean, What you describe as "garbage" package authors consider essential to providing high quality documentation in a variety of formats. The README.org is converted into an Info manual and has a corresponding HTML page both on GNU ELPA and the author's (me) website: - <https://elpa.gnu.org/packages/logos.html> - <https://protesilaos.com/emacs/logos> - Evaluate this to read the Info manual: (info "(logos) Top") Nobody forces you to read the Org source and there are good alternatives on offer. All the best, Protesilaos (or simply "Prot") -- Protesilaos Stavrou https://protesilaos.com ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-11 11:16 ` Protesilaos Stavrou @ 2022-06-12 11:32 ` Jean Louis 0 siblings, 0 replies; 505+ messages in thread From: Jean Louis @ 2022-06-12 11:32 UTC (permalink / raw) To: Protesilaos Stavrou; +Cc: Tim Cross, emacs-devel * Protesilaos Stavrou <info@protesilaos.com> [2022-06-11 14:17]: > What you describe as "garbage" package authors consider essential to > providing high quality documentation in a variety of formats. The > README.org is converted into an Info manual and has a corresponding HTML > page both on GNU ELPA and the author's (me) website: I do not doubt good intentions. I use the word "garbage" from a viewpoint of user who does not know Org mode and who is interested to understand package description. Markup impairs the understanding of package descriptions. Markup does not belong in package description. -- Jean Take action in Free Software Foundation campaigns: https://www.fsf.org/campaigns In support of Richard M. Stallman https://stallmansupport.org/ ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-11 6:40 ` Jean Louis 2022-06-11 11:16 ` Protesilaos Stavrou @ 2022-06-12 2:23 ` Tim Cross 2022-06-12 11:41 ` Jean Louis 1 sibling, 1 reply; 505+ messages in thread From: Tim Cross @ 2022-06-12 2:23 UTC (permalink / raw) To: Jean Louis; +Cc: emacs-devel Jean Louis <bugs@gnu.support> writes: > * Tim Cross <theophilusx@gmail.com> [2022-06-08 09:57]: >> Please give me an example of org mode 'not make space where it would be >> otherwise required'. Can you provide a single example of org mode >> syntax which is not readable in any text editor. There are quite a few >> projects on Github/Gitlab which have readme.org files - can you point to >> one which cannot be read with a plain text editor? > > Hello Tim, > > my response time is not so fast. > > I am now reviewing package descriptions: > > Recommended for everybody to read: > > Usability 101: Introduction to Usability > https://www.nngroup.com/articles/usability-101-introduction-to-usability/ > > Usability Testing 101 > https://www.nngroup.com/articles/usability-testing-101/ > I gues sour differences here are that you have conflated issues of readabiity and usability into the definition of plain text. Few of the criticisms you have raised IMO are specific to org mode or due to org modes syntax. Many of them are due to the individual author's layout and not the result of org mode syntax. You can also write documents which are very difficult to read using just plain ASCII. They are both plain text. I also think you are over-stating the difficulty of reading the raw org file contents. While I would agree they are asier to read when correctly formatted, they are not impossible to understand without it. For example, to say something like > package corfu, totally not human readable as it begins with: > #+title: corfu.el - Completion Overlay Region FUnction > #+author: Daniel Mendler > #+language: en > #+export_file_name: corfu.texi > #+texinfo_dir_category: Emacs > #+texinfo_dir_title: Corfu: (corfu). > #+texinfo_dir_desc: Completion Overlay Region FUnction is I think totally over stating the situation. I showed the above to someone who is not a technical person and only uses computers for email, web and office work and they were able to understand it perfectly well. They didn't know what the texinfo lines meant, but they understood what the title was, who the author was, the language and even the texinfo_dir_desc being a description. I think most people are far more capable at reading such information than you assert. They might not understand all of it, but then again, I often read documents where I don't understand all of it. The point is, there is a big difference betrween being able to read contents and understand them. As a comparison, consider some RFCs which you mentioned previously. I've read RFCs which perfectly comply with the formatting you referenced that were (to me) totally unintelligible. Sometimes this is because the writing is poor or the concepts are too alien or it is simply too complicated for my limited understanding. However, this is still a plain text document. For me, a file is plain text if you can edit and save it using just a plain text editor and the file will still be valid. I also consider HTML files to be plain text, (but far less readable than org files). On the other hand, *.docx are not plain text and cannot be edited with a plain text editor, saved and still be valid. Likewise, other formats which use non-printable control characters are not plain text. I will try to be clearer when I say something is plain text. Readability and useability are separate from file formats - they are a higher, but related, concern. Plain text does not imply anything about readability or usability other than a plain text file can be opened and physically read using a plain text editor or a utility like 'cat'. The ability of the individual to interpret that information does not affect whether it is plain text. If, for example, you write lots of HTML, an HTML file is easily understood when displayed with cat. Of course, appropriate formatting within a plain text document can affect how easily the document can be read and I think it is good to try and format the document to be as easy to read as possible. However, for org mode, this is really under the control of the author. Org data files can be formatted in a way which makes them easier to read and there are various Emacs settings which can be used which affects this (wrap, visual line mode, adding blank lines at the end of sections etc). The point is, many of your criticisms are not due to org syntax or org enforced forrmatting, which is one reason I disagree with assertions that org files are not plain text. I'm sure this is apoint we will never agree on, so happy to leave it there. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-12 2:23 ` Tim Cross @ 2022-06-12 11:41 ` Jean Louis 2022-06-12 21:58 ` [External] : " Drew Adams 2022-06-13 22:37 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Jean Louis @ 2022-06-12 11:41 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel * Tim Cross <theophilusx@gmail.com> [2022-06-12 06:15]: > I gues sour differences here are that you have conflated issues of > readabiity and usability into the definition of plain text. I did not. I speak of understanding of package descriptions. When user clicks on a package within Emacs that user expects package description, not a markup. It is totally irrelevant for user if markup and tags, properties are plain text in the context of editors. What matters is if user is helped to understand the package description. Or maybe user is obstructed to understand it. > Few of the criticisms you have raised IMO are specific to org mode > or due to org modes syntax. They are not specific, it just happens that I had in my package list several Org type package descriptions. If too complex Markdown markup is used it will also obstruct user to understand package description. > Many of them are due to the individual author's layout and not the > result of org mode syntax. You can also write documents which are > very difficult to read using just plain ASCII. They are both plain > text. I do not think that "plain text" is relevant. HTML markup is also plain text, it is irrelevant in the context of a user reading package descriptions. User is not supposed to read HTML tags either, right? > I also think you are over-stating the difficulty of reading the raw org > file contents. While I would agree they are asier to read when correctly > formatted, they are not impossible to understand without it. You may stay in your viewpoint of knowledgable Emacs user, in that sense there is also no need for package descriptions, then we can simply tell users to read sources (irony). One has to mentally become a user who does not know Org, the one who simply wants to find out what package is about. Only so one can understand it. -- Jean Take action in Free Software Foundation campaigns: https://www.fsf.org/campaigns In support of Richard M. Stallman https://stallmansupport.org/ ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-12 11:41 ` Jean Louis @ 2022-06-12 21:58 ` Drew Adams 2022-06-13 22:37 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Drew Adams @ 2022-06-12 21:58 UTC (permalink / raw) To: Jean Louis, Tim Cross; +Cc: emacs-devel@gnu.org > I did not. I speak of understanding of package descriptions. When user > clicks on a package within Emacs that user expects package > description, not a markup. Dunno (I'm not really following this thread), but maybe this example is relevant in discussing the difference in effect between viewing/reading really _plain_ plain-text and viewing/reading plain-text that has some mark-up, labels, instructions, or whatever you might call it mixed in with the text: Consider the difference between viewing/reading a Commentary section in a Lisp file and viewing/reading it in the buffer that `M-x finder-commentary' creates. This difference is about as minimal as you can get. And yet `finder-commentary' is useful, just to give you a friendlier, simpler, more searchable, and more readable presentation. All that command does is show you a buffer with just the Commentary text - no code, comment chars removed, etc., and with the `...'-quoted names of known libraries converted to links to those libraries. It's very rudimentary, but it shows well the benefit of removing even just a few, slight distractions to give users a really _plain_ plain-text description, in the interest of reading it as doc. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-12 11:41 ` Jean Louis 2022-06-12 21:58 ` [External] : " Drew Adams @ 2022-06-13 22:37 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Richard Stallman @ 2022-06-13 22:37 UTC (permalink / raw) To: Jean Louis; +Cc: theophilusx, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > I did not. I speak of understanding of package descriptions. When user > clicks on a package within Emacs that user expects package > description, not a markup. > It is totally irrelevant for user if markup and tags, properties are > plain text in the context of editors. The idea of adding markup to text is that a program will read that marked-up text and output text which is nicely formatted. The marked-up text is harder to read, but the markup effects in the output make it easier to read. If we use markup in some text, we shouldn't normally show users that marked-up source text; we should normally show users the result of processing the markup. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 10:26 ` Tassilo Horn 2022-06-05 11:15 ` Michael Albinus @ 2022-06-05 11:23 ` Ihor Radchenko 2022-06-06 0:33 ` Lars Ingebrigtsen 2022-06-05 11:29 ` Lars Ingebrigtsen ` (3 subsequent siblings) 5 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-05 11:23 UTC (permalink / raw) To: Tassilo Horn Cc: Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Tassilo Horn <tsdh@gnu.org> writes: >> It also takes a relatively long time to load Org mode, so opening a >> README.org-type file for the first time will always cause an annoying >> pause. > > That's true but wouldn't it be possible to extract the font-lock/faces > related parts in a separate file with no dependencies which can be > loaded quickly enough? I mean, the purpose would only be to display org > and markdown READMEs with the highlighting intended by the package > author but there's no point in enabling the gazillon of other features > (from editing to time-tracking to spreadsheet computations) those > packages might provide in addition. We, Org devs, are working on modularizing Org. Currently, extracting fontification is not trivial because it depends on most of Org being loaded. If everything goes as planned (famous last words), fontification will only require Org parser and Org fontification library. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 11:23 ` Ihor Radchenko @ 2022-06-06 0:33 ` Lars Ingebrigtsen 2022-06-06 0:41 ` Lars Ingebrigtsen 2022-06-06 0:59 ` Ihor Radchenko 0 siblings, 2 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-06 0:33 UTC (permalink / raw) To: Ihor Radchenko Cc: Tassilo Horn, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > We, Org devs, are working on modularizing Org. > Currently, extracting fontification is not trivial because it depends on > most of Org being loaded. > > If everything goes as planned (famous last words), fontification will > only require Org parser and Org fontification library. Perhaps it would be a smaller project to just make a super-simple Org parser/displayer to handle what we typically see in a README.org file? I.e., something that does nice fontification of #+ blocks and *** headings and =this= (etc.), and then just removes all the rest of the markup. I mean, it sounds like it could be a very small, lightweight library to get to a 90% solution that would be acceptable to show in the *Help* buffer. And then we wouldn't have to convert anything. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 0:33 ` Lars Ingebrigtsen @ 2022-06-06 0:41 ` Lars Ingebrigtsen 2022-06-06 0:59 ` Ihor Radchenko 1 sibling, 0 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-06 0:41 UTC (permalink / raw) To: Ihor Radchenko Cc: Tassilo Horn, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > I mean, it sounds like it could be a very small, lightweight library to > get to a 90% solution that would be acceptable to show in the *Help* > buffer. And then we wouldn't have to convert anything. (And the same for README.md, of course.) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 0:33 ` Lars Ingebrigtsen 2022-06-06 0:41 ` Lars Ingebrigtsen @ 2022-06-06 0:59 ` Ihor Radchenko 2022-06-06 1:07 ` Lars Ingebrigtsen 1 sibling, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-06 0:59 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Tassilo Horn, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Ihor Radchenko <yantar92@gmail.com> writes: > >> We, Org devs, are working on modularizing Org. >> Currently, extracting fontification is not trivial because it depends on >> most of Org being loaded. >> >> If everything goes as planned (famous last words), fontification will >> only require Org parser and Org fontification library. > > Perhaps it would be a smaller project to just make a super-simple Org > parser/displayer to handle what we typically see in a README.org file? > I.e., something that does nice fontification of #+ blocks and *** > headings and =this= (etc.), and then just removes all the rest of the > markup. > > I mean, it sounds like it could be a very small, lightweight library to > get to a 90% solution that would be acceptable to show in the *Help* > buffer. And then we wouldn't have to convert anything. In short, no. Org is not context free, so we at least need org-element.el parser to handle various edge cases. In particular, there are quite annoying edge cases with fontification of links containing */- markup symbols. One could extract a subset of allowed Org elements from org-element.el, but it would also involve refactoring org-element--current-element, making the whole effort a maintenance burden against future changes in Org. And even if that job is done, we cannot remove links from the parser. Yet, according to the previous benchmarks in this thread, loading links (that is what org-modules contains by default) is giving the largest overheads. IMHO, the proposed lightweight library will require similar efforts to modularizing Org + loading it with some minimal variable settings (for example, org-modules set to nil). Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 0:59 ` Ihor Radchenko @ 2022-06-06 1:07 ` Lars Ingebrigtsen 2022-06-06 1:33 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-06 1:07 UTC (permalink / raw) To: Ihor Radchenko Cc: Tassilo Horn, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > In short, no. Is that a challenge? 😀 > And even if that job is done, we cannot remove links from the parser. > Yet, according to the previous benchmarks in this thread, loading links > (that is what org-modules contains by default) is giving the largest > overheads. What do you mean by "links" in this context? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 1:07 ` Lars Ingebrigtsen @ 2022-06-06 1:33 ` Ihor Radchenko 2022-06-06 7:39 ` Tassilo Horn ` (2 more replies) 0 siblings, 3 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-06 1:33 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Tassilo Horn, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Ihor Radchenko <yantar92@gmail.com> writes: > >> In short, no. > > Is that a challenge? 😀 Sorry, I do not understand what you mean here. >> And even if that job is done, we cannot remove links from the parser. >> Yet, according to the previous benchmarks in this thread, loading links >> (that is what org-modules contains by default) is giving the largest >> overheads. > > What do you mean by "links" in this context? In this specific paragraph, I was referring to file:, id:, help:, and https: links. They (except help:) can be easily found in various README files. Setting up links involves allocation of fairly large strings (see org-element--set-regexps), which, I suspect, is creating a lot of load on Emacs GC and slowing down the loading (especially when Emacs session is already running for some time and GC must sweep across all the available objects). Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 1:33 ` Ihor Radchenko @ 2022-06-06 7:39 ` Tassilo Horn 2022-06-08 12:56 ` Ihor Radchenko 2022-06-06 12:54 ` Lars Ingebrigtsen 2022-06-06 12:56 ` Lars Ingebrigtsen 2 siblings, 1 reply; 505+ messages in thread From: Tassilo Horn @ 2022-06-06 7:39 UTC (permalink / raw) To: Ihor Radchenko Cc: Lars Ingebrigtsen, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: >>> In short, no. >> >> Is that a challenge? 😀 > > Sorry, I do not understand what you mean here. Lars is asking if he should hack up a solution which will probably work well enough for 95% of all existing README.org/md files, so just say "yes"! ;-) Bye, Tassilo ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 7:39 ` Tassilo Horn @ 2022-06-08 12:56 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-08 12:56 UTC (permalink / raw) To: Tassilo Horn Cc: Lars Ingebrigtsen, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Tassilo Horn <tsdh@gnu.org> writes: > Ihor Radchenko <yantar92@gmail.com> writes: > >>>> In short, no. >>> >>> Is that a challenge? 😀 >> >> Sorry, I do not understand what you mean here. > > Lars is asking if he should hack up a solution which will probably work > well enough for 95% of all existing README.org/md files, so just say > "yes"! ;-) :) Done. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 1:33 ` Ihor Radchenko 2022-06-06 7:39 ` Tassilo Horn @ 2022-06-06 12:54 ` Lars Ingebrigtsen 2022-06-06 12:56 ` Lars Ingebrigtsen 2 siblings, 0 replies; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-06 12:54 UTC (permalink / raw) To: Ihor Radchenko Cc: Tassilo Horn, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > In this specific paragraph, I was referring to file:, id:, help:, and > https: links. So you were talking about these? [[http://angg.twu.net/emacsconf2019.html][How to record executable notes with eev - and how to play them back]] Then I posit that it's trivial to get to a speedy 90% solution for these. Anyway, for the people that say we should convert README.org files to "text" and display that -- that's not a very satisfactory solution. To take an arbitrary README.org -- the one in eev. The text version starts off with: --- The best introductions to eev are: - "[How to record executable notes with eev - and how to play them back]": my talk at the EmacsConf 2019. Executable notes are mostly made of [sexp hyperlinks] and [eepitch blocks]. This talk has a quick explanation of sexp hyperlinks at [slides 5 and 6], a mention --- which looks nice, but all those []'s are links, so you get a block down below that's like: --- [How to record executable notes with eev - and how to play them back] <http://angg.twu.net/emacsconf2019.html> [sexp hyperlinks] <http://angg.twu.net/eev-intros/find-eev-quick-intro.html#3> [eepitch blocks] <http://angg.twu.net/eev-intros/find-eev-quick-intro.html#6> --- Which isn't very nice. It should be displayed as: -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 1:33 ` Ihor Radchenko 2022-06-06 7:39 ` Tassilo Horn 2022-06-06 12:54 ` Lars Ingebrigtsen @ 2022-06-06 12:56 ` Lars Ingebrigtsen 2022-06-08 12:55 ` Ihor Radchenko 2 siblings, 1 reply; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-06 12:56 UTC (permalink / raw) To: Ihor Radchenko Cc: Tassilo Horn, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1368 bytes --] (Sorry, I may have sent this twice.) Ihor Radchenko <yantar92@gmail.com> writes: > In this specific paragraph, I was referring to file:, id:, help:, and > https: links. So you were talking about these? [[http://angg.twu.net/emacsconf2019.html][How to record executable notes with eev - and how to play them back]] Then I posit that it's trivial to get to a speedy 90% solution for these. Anyway, for the people that say we should convert README.org files to "text" and display that -- that's not a very satisfactory solution. To take an arbitrary README.org -- the one in eev. The text version starts off with: --- The best introductions to eev are: - "[How to record executable notes with eev - and how to play them back]": my talk at the EmacsConf 2019. Executable notes are mostly made of [sexp hyperlinks] and [eepitch blocks]. This talk has a quick explanation of sexp hyperlinks at [slides 5 and 6], a mention --- which looks OK, but all those []'s are links, so you get a block down below that's like: --- [How to record executable notes with eev - and how to play them back] <http://angg.twu.net/emacsconf2019.html> [sexp hyperlinks] <http://angg.twu.net/eev-intros/find-eev-quick-intro.html#3> [eepitch blocks] <http://angg.twu.net/eev-intros/find-eev-quick-intro.html#6> --- Which isn't very nice. It should be displayed as: [-- Attachment #2: Type: image/png, Size: 58071 bytes --] [-- Attachment #3: Type: text/plain, Size: 247 bytes --] With clickable links and all. I suspect that we can get there with not all that much effort if somebody's interested in just doing the work. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 12:56 ` Lars Ingebrigtsen @ 2022-06-08 12:55 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-08 12:55 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Tassilo Horn, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: >> In this specific paragraph, I was referring to file:, id:, help:, and >> https: links. > > So you were talking about these? > > [[http://angg.twu.net/emacsconf2019.html][How to record executable notes with eev - and how to play them back]] > > Then I posit that it's trivial to get to a speedy 90% solution for these. If you are interested to do this, feel free to go ahead. If you need any help to get started, feel free to ask me or post a question in Org ML. I am a bit biased about fontification because of https://orgmode.org/list/87ee7c9quk.fsf@localhost I keep thinking about various edge cases. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 10:26 ` Tassilo Horn 2022-06-05 11:15 ` Michael Albinus 2022-06-05 11:23 ` Ihor Radchenko @ 2022-06-05 11:29 ` Lars Ingebrigtsen 2022-06-05 13:39 ` Stefan Monnier 2022-06-05 11:45 ` Po Lu ` (2 subsequent siblings) 5 siblings, 1 reply; 505+ messages in thread From: Lars Ingebrigtsen @ 2022-06-05 11:29 UTC (permalink / raw) To: Tassilo Horn Cc: Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Tassilo Horn <tsdh@gnu.org> writes: > So the choice of format is natural for package authors. And they > frequently contain markup like headings, bullet lists, bold, italics, > and code snippets which simply look much better than their text-only > alternative (even in their editing/non-rendered versions). I think most of the hostility non-Org users have towards Org files is the experience of opening one, only seeing a couple of lines with "..." after them and then going "AAARGH", killing the buffer, and using `find-file-literally' on it. But Org files are totally innocuous if you say `M-x org-show-subtree'. package.el could totally use org files instead of text files for the readmes if it ran `org-show-subtree' when displaying the Org readmes -- most people wouldn't even notice that they were reading an Org file (beyond the nice fontification). -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 11:29 ` Lars Ingebrigtsen @ 2022-06-05 13:39 ` Stefan Monnier 2022-06-05 14:24 ` Alan Mackenzie 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-05 13:39 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Tassilo Horn, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Lars Ingebrigtsen [2022-06-05 13:29:28] wrote: > Tassilo Horn <tsdh@gnu.org> writes: >> So the choice of format is natural for package authors. And they >> frequently contain markup like headings, bullet lists, bold, italics, >> and code snippets which simply look much better than their text-only >> alternative (even in their editing/non-rendered versions). > I think most of the hostility non-Org users have towards Org files is > the experience of opening one, only seeing a couple of lines with "..." > after them and then going "AAARGH", killing the buffer, and using > `find-file-literally' on it. There's also the annoying delay, of course (as in "oh, so it takes a long time to load in order to then mess it up!?") We should take these as challenges to improve org-mode rather than as reasons to stay away from it. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 13:39 ` Stefan Monnier @ 2022-06-05 14:24 ` Alan Mackenzie 2022-06-05 14:38 ` Stefan Monnier 0 siblings, 1 reply; 505+ messages in thread From: Alan Mackenzie @ 2022-06-05 14:24 UTC (permalink / raw) To: Stefan Monnier Cc: Lars Ingebrigtsen, Tassilo Horn, Po Lu, Michael Albinus, Stefan Kangas, Akib Azmain Turja, emacs-devel Hello, Stefan. On Sun, Jun 05, 2022 at 09:39:13 -0400, Stefan Monnier wrote: > Lars Ingebrigtsen [2022-06-05 13:29:28] wrote: > > Tassilo Horn <tsdh@gnu.org> writes: > >> So the choice of format is natural for package authors. And they > >> frequently contain markup like headings, bullet lists, bold, italics, > >> and code snippets which simply look much better than their text-only > >> alternative (even in their editing/non-rendered versions). > > I think most of the hostility non-Org users have towards Org files is > > the experience of opening one, only seeing a couple of lines with "..." > > after them and then going "AAARGH", killing the buffer, and using > > `find-file-literally' on it. > There's also the annoying delay, of course (as in "oh, so it takes > a long time to load in order to then mess it up!?") > We should take these as challenges to improve org-mode rather than as > reasons to stay away from it. We should not inflict org mode on random Emacs users, e.g. by making it necessary in order to use *Help* fully. Org mode is a complicated, difficult to learn package which cannot be to everybody's taste. To see just how complicated and difficult to learn it is, have a look at the Key index in org.info. There are 784 entries in that Key index. That is surely more key bindings than I know and use in total. A massive number of these are bindings which don't start with C-c C-<letter>. Many of these will most assuredly clash with other bindings users have set up. Don't get me wrong, I've no objection to people liking and using a package like org mode. Where I draw the line is when I am myself forced to use it as part of accessing some unrelated feature. > Stefan -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 14:24 ` Alan Mackenzie @ 2022-06-05 14:38 ` Stefan Monnier 0 siblings, 0 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-05 14:38 UTC (permalink / raw) To: Alan Mackenzie Cc: Lars Ingebrigtsen, Tassilo Horn, Po Lu, Michael Albinus, Stefan Kangas, Akib Azmain Turja, emacs-devel > Don't get me wrong, I've no objection to people liking and using a > package like org mode. Where I draw the line is when I am myself forced > to use it as part of accessing some unrelated feature. Noone's proposing to force you to use it. The proposition is to use org-mode internally when displaying a README.org file so as to make the *Help* text more pretty. The user wouldn't need to know that Org is being used and the same key bindings of *Help* buffers would still do the same as before (maybe additional key-bindings would be offered, e.g. to fold&unfold parts, of course). Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 10:26 ` Tassilo Horn ` (2 preceding siblings ...) 2022-06-05 11:29 ` Lars Ingebrigtsen @ 2022-06-05 11:45 ` Po Lu 2022-06-05 11:59 ` Ihor Radchenko 2022-06-05 11:59 ` Akib Azmain Turja 2022-06-08 17:21 ` Yoni Rabkin 5 siblings, 1 reply; 505+ messages in thread From: Po Lu @ 2022-06-05 11:45 UTC (permalink / raw) To: Tassilo Horn Cc: Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Tassilo Horn <tsdh@gnu.org> writes: > People nowadays write README.org and README.md files instead of > plain-text READMEs because the packages' repositories are hosted on > some forge (sr.ht, github, gitlab, gitea, whatever) which supports > rendering those formats in a nice way. So the choice of format is > natural for package authors. And they frequently contain markup like > headings, bullet lists, bold, italics, and code snippets which simply > look much better than their text-only alternative (even in their > editing/non-rendered versions). But that doesn't mean all documentation should take on such a format. People read their READMEs in Emacs, not on a forge. > That's true but wouldn't it be possible to extract the font-lock/faces > related parts in a separate file with no dependencies which can be > loaded quickly enough? I mean, the purpose would only be to display org > and markdown READMEs with the highlighting intended by the package > author but there's no point in enabling the gazillon of other features > (from editing to time-tracking to spreadsheet computations) those > packages might provide in addition. That wouldn't be Org mode anymore then, just an outline with some additional fontification rules. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 11:45 ` Po Lu @ 2022-06-05 11:59 ` Ihor Radchenko 2022-06-05 13:11 ` Akib Azmain Turja 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-05 11:59 UTC (permalink / raw) To: Po Lu Cc: Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Po Lu <luangruo@yahoo.com> writes: >> That's true but wouldn't it be possible to extract the font-lock/faces >> related parts in a separate file with no dependencies which can be >> loaded quickly enough? I mean, the purpose would only be to display org >> and markdown READMEs with the highlighting intended by the package >> author but there's no point in enabling the gazillon of other features >> (from editing to time-tracking to spreadsheet computations) those >> packages might provide in addition. > > That wouldn't be Org mode anymore then, just an outline with some > additional fontification rules. Isn't *Help* buffer supposed to be read-only? There is no point loading Org editing features if editing is not going to happen. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 11:59 ` Ihor Radchenko @ 2022-06-05 13:11 ` Akib Azmain Turja 2022-06-05 15:32 ` Tassilo Horn 2022-06-05 15:52 ` Philip Kaludercic 0 siblings, 2 replies; 505+ messages in thread From: Akib Azmain Turja @ 2022-06-05 13:11 UTC (permalink / raw) To: Ihor Radchenko, Po Lu Cc: Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1318 bytes --] Ihor Radchenko <yantar92@gmail.com> writes: > Po Lu <luangruo@yahoo.com> writes: > >>> That's true but wouldn't it be possible to extract the font-lock/faces >>> related parts in a separate file with no dependencies which can be >>> loaded quickly enough? I mean, the purpose would only be to display org >>> and markdown READMEs with the highlighting intended by the package >>> author but there's no point in enabling the gazillon of other features >>> (from editing to time-tracking to spreadsheet computations) those >>> packages might provide in addition. >> >> That wouldn't be Org mode anymore then, just an outline with some >> additional fontification rules. > > Isn't *Help* buffer supposed to be read-only? There is no point loading > Org editing features if editing is not going to happen. > > Best, > Ihor Yes, *Help* is read only. I can't understand why people are suggesting to Org mode in package description. I'm again saying that the README.org should be converted (exported) to plain text on package installation (I won't recommend to do this server, since its probably SaaSS and would increase server load). -- Akib Azmain Turja This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 13:11 ` Akib Azmain Turja @ 2022-06-05 15:32 ` Tassilo Horn 2022-06-05 17:46 ` [External] : " Drew Adams 2022-06-05 15:52 ` Philip Kaludercic 1 sibling, 1 reply; 505+ messages in thread From: Tassilo Horn @ 2022-06-05 15:32 UTC (permalink / raw) To: Akib Azmain Turja Cc: Ihor Radchenko, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel Akib Azmain Turja <akib@disroot.org> writes: Hi Akib, > I can't understand why people are suggesting to Org mode in package > description. I'm not sure if we are miscommunicating. Of course, nobody should be mandated to write his package's README as org file. But as a matter of fact, many package authors do it that way or in markdown simply because both are good formats for structured text and are supported by most forges (markdown even more so), i.e., the rendered version of the README becomes the project's homepage without any additional work. IMHO, that's good: it encourages package authors to write good READMEs because it's the very first thing a prospective user will see. So again, I was only suggesting that if (and only if) a package declares a :readme "README.org" / "README.md", it would be nice if it was *displayed* as in the "native" modes. I certainly acknowledge technical arguments, e.g., loading the org package is too slow (right now). Let me ask the other way round: why can't you understand that someone thinks it's a good idea if the package description is displayed as the author intended? > I'm again saying that the README.org should be converted (exported) to > plain text on package installation (I won't recommend to do this > server, since its probably SaaSS and would increase server load). I've just tried org-exporting the vertico README.org to a plain-text UTF-8 version and have to confess, the result is really, really intriguing. So, yes, such a conversion is at least much better as the status quo where the org syntax is displayed literally without highlighting. (I almost don't perceive the org/markdown syntax when there's syntax highlighting but if there isn't, it looks annoying.) One problem with the "conversion on installation" approach you suggest is that this would load org, too, so the arguments against using/loading org for display would also apply here. Bye, Tassilo ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-05 15:32 ` Tassilo Horn @ 2022-06-05 17:46 ` Drew Adams 2022-06-05 17:57 ` Tassilo Horn 0 siblings, 1 reply; 505+ messages in thread From: Drew Adams @ 2022-06-05 17:46 UTC (permalink / raw) To: Tassilo Horn, Akib Azmain Turja Cc: Ihor Radchenko, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel@gnu.org > why can't you understand that someone thinks > it's a good idea if the package description > is displayed as the author intended? I'd say as the _user_ intends, not the author. Displaying a package description is for the benefit of the user, not the author. It's about what's most useful for users. Best is this, I think: 1. Require a plain-text README, as a minimum. 2. Allow other formats, with appropriate file extensions (e.g. .md, .org). I see no reason for any limit on the kinds of format, for #2. There should be, and likely are, simple ways to generate a plain-text README from this or that more structured format. Condition #1 shouldn't be an obstacle to anyone, I'd think. > I've just tried org-exporting the vertico README.org to a plain-text > UTF-8 version and have to confess, the result is really, really > intriguing. So, yes, such a conversion is at least much better as the > status quo where the org syntax is displayed literally without > highlighting. (I almost don't perceive the org/markdown syntax when > there's syntax highlighting but if there isn't, it looks annoying.) For Org, at least, that seems to confirm my guess that such conversion's already available. > One problem with the "conversion on installation" > approach you suggest is that this would load org, > too, so the arguments against using/loading > org for display would also apply here. Why shouldn't a plain-text README be required at the outset, as part of every package? (Another possibility, if a plain-text README that's equivalent to a structured readme isn't provided, is to allow a plain-text README that tells you how to convert the structured readme to a plain-text version. But I don't see why plain text shouldn't just be a requirement.) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: [External] : Re: Convert README.org to plain text README while installing package 2022-06-05 17:46 ` [External] : " Drew Adams @ 2022-06-05 17:57 ` Tassilo Horn 2022-06-05 20:12 ` Stefan Monnier 2022-06-06 0:23 ` Drew Adams 0 siblings, 2 replies; 505+ messages in thread From: Tassilo Horn @ 2022-06-05 17:57 UTC (permalink / raw) To: Drew Adams Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel Drew Adams <drew.adams@oracle.com> writes: >> why can't you understand that someone thinks >> it's a good idea if the package description >> is displayed as the author intended? > > I'd say as the _user_ intends, not the author. You and me are both users and it seems we have different intentions/preferences. > Displaying a package description is for the > benefit of the user, not the author. It's > about what's most useful for users. Why do you think that a free software package author doesn't write his documentation for this very purpose? Exactly for this reason she has structured the README in sections, **emphasized the important parts**, included usage/config examples as highlighted #+BEGIN_SRC emacs-lisp (code snippets) #+END_SRC and added clickable links to [[helpful resources][https://...]]. Why is it "most useful" for users to strip away parts of that? > Best is this, I think: > > 1. Require a plain-text README, as a minimum. > 2. Allow other formats, with appropriate file > extensions (e.g. .md, .org). > > I see no reason for any limit on the kinds of > format, for #2. > > There should be, and likely are, simple ways > to generate a plain-text README from this or > that more structured format. Condition #1 > shouldn't be an obstacle to anyone, I'd think. If there is some automatic conversion like for org, the obstacle is just that I as a package author would need to commit a generated file with the danger that contributors send patches against the generated document instead of the source. Not a big deal but also not great. >> I've just tried org-exporting the vertico README.org to a plain-text >> UTF-8 version and have to confess, the result is really, really >> intriguing. So, yes, such a conversion is at least much better as the >> status quo where the org syntax is displayed literally without >> highlighting. (I almost don't perceive the org/markdown syntax when >> there's syntax highlighting but if there isn't, it looks annoying.) > > For Org, at least, that seems to confirm my > guess that such conversion's already available. Yes, indeed. I haven't seen anything like that for markdown (which is probably even more popular for READMEs than org), though. Bye, Tassilo ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: [External] : Re: Convert README.org to plain text README while installing package 2022-06-05 17:57 ` Tassilo Horn @ 2022-06-05 20:12 ` Stefan Monnier 2022-06-06 0:24 ` Drew Adams 2022-06-06 0:23 ` Drew Adams 1 sibling, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-05 20:12 UTC (permalink / raw) To: Tassilo Horn Cc: Drew Adams, Akib Azmain Turja, Ihor Radchenko, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel > Yes, indeed. I haven't seen anything like that for markdown (which is > probably even more popular for READMEs than org), though. I think that's simply because Markdown (as the name suggests) is designed such that the Markdown source can be very close to the best possible plain text rendering, significantly reducing the motivation to design plain text renderer. [ That's also why I tend to prefer Markdown for small documents, where I tend to find Org's markup a bit too loud. ] Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-05 20:12 ` Stefan Monnier @ 2022-06-06 0:24 ` Drew Adams 2022-06-06 0:48 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Drew Adams @ 2022-06-06 0:24 UTC (permalink / raw) To: Stefan Monnier, Tassilo Horn Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel@gnu.org > > Yes, indeed. I haven't seen anything like that for markdown (which > > is probably even more popular for READMEs than org), though. > > I think that's simply because Markdown (as the name suggests) is > designed such that the Markdown source can be very close to the best > possible plain text rendering, significantly reducing the motivation to > design plain text renderer. Yes. And that closeness also means that it's generally not so difficult to go from simple plain-text without markup to Markdown, and vice versa. > [ That's also why I tend to prefer Markdown for small documents, where > I tend to find Org's markup a bit too loud. ] Yes again. There are lots of nice, rendered or renderable fancy formats, including Org and Markdown, HTML, PDF, Info/TexInfo. Packages should, I think, be able to provide whatever they like, as long as they also provide some simple README. (And again, that simple, plain-text README need not try to say or do all that another format might do. IMO.) ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-06 0:24 ` Drew Adams @ 2022-06-06 0:48 ` Ihor Radchenko 2022-06-06 1:18 ` Drew Adams 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-06 0:48 UTC (permalink / raw) To: Drew Adams Cc: Stefan Monnier, Tassilo Horn, Akib Azmain Turja, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel@gnu.org Drew Adams <drew.adams@oracle.com> writes: > There are lots of nice, rendered or renderable > fancy formats, including Org and Markdown, HTML, > PDF, Info/TexInfo. > > Packages should, I think, be able to provide > whatever they like, as long as they also provide > some simple README. > > (And again, that simple, plain-text README need > not try to say or do all that another format > might do. IMO.) While what you say makes sense, can we reasonably expect the package authors to provide an extra README file for every single package? If it was the case, this thread would simply not show up. ASCII README would be always available. ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-06 0:48 ` Ihor Radchenko @ 2022-06-06 1:18 ` Drew Adams 2022-06-06 1:38 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Drew Adams @ 2022-06-06 1:18 UTC (permalink / raw) To: Ihor Radchenko Cc: Stefan Monnier, Tassilo Horn, Akib Azmain Turja, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel@gnu.org > > There are lots of nice, rendered or renderable > > fancy formats, including Org and Markdown, HTML, > > PDF, Info/TexInfo. > > > > Packages should, I think, be able to provide > > whatever they like, as long as they also provide > > some simple README. > > > > (And again, that simple, plain-text README need > > not try to say or do all that another format > > might do. IMO.) > > While what you say makes sense, can we reasonably expect the package > authors to provide an extra README file for every single package? I think we can. But only if we ask for it. > If it was the case, this thread would simply not > show up. ASCII README would be always available. No, not if we've never requested that it be included. If we request it then it'll be provided more often, I expect. ___ [ But we can't even get vanilla Emacs itself to add README's for Emacs releases on MS Windows anymore, as it says we can (in the overall Windows README). https://lists.gnu.org/archive/html/emacs-devel/2022-05/msg01378.html ] ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-06 1:18 ` Drew Adams @ 2022-06-06 1:38 ` Ihor Radchenko 2022-06-06 1:41 ` Drew Adams ` (2 more replies) 0 siblings, 3 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-06 1:38 UTC (permalink / raw) To: Drew Adams Cc: Stefan Monnier, Tassilo Horn, Akib Azmain Turja, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel@gnu.org Drew Adams <drew.adams@oracle.com> writes: >> If it was the case, this thread would simply not >> show up. ASCII README would be always available. > > No, not if we've never requested that it be included. > > If we request it then it'll be provided more often, > I expect. Maybe. As a single data point, if I were asked to provide such a README, I would not bother writing it separately. Instead, I would simply export my README.org to ASCII. And it would feel like an unnecessary burden, honestly. If I were asked to provide a README as an option, I would not bother at all and just leave README.org. Do you know how many ELPA packages provide README.org vs README.md vs README in ASCII text? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-06 1:38 ` Ihor Radchenko @ 2022-06-06 1:41 ` Drew Adams 2022-06-06 1:47 ` Stefan Monnier 2022-06-06 7:44 ` Tassilo Horn 2 siblings, 0 replies; 505+ messages in thread From: Drew Adams @ 2022-06-06 1:41 UTC (permalink / raw) To: Ihor Radchenko Cc: Stefan Monnier, Tassilo Horn, Akib Azmain Turja, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel@gnu.org > Do you know how many ELPA packages provide README.org > vs README.md vs README in ASCII text? Not I. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: [External] : Re: Convert README.org to plain text README while installing package 2022-06-06 1:38 ` Ihor Radchenko 2022-06-06 1:41 ` Drew Adams @ 2022-06-06 1:47 ` Stefan Monnier 2022-06-06 7:44 ` Tassilo Horn 2 siblings, 0 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-06 1:47 UTC (permalink / raw) To: Ihor Radchenko; +Cc: emacs-devel@gnu.org > As a single data point, if I were asked to provide such a README, I > would not bother writing it separately. Instead, I would simply export > my README.org to ASCII. And it would feel like an unnecessary burden, > honestly. Don't worry, as de-facto maintainer of GNU ELPA, I'd rather discourage that kind of duplication anyway (and elpa.gnu.org already builds the text rendering of .org and.md READMEs anyway, so it would be perverse to ask the authors to provide another one). Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: [External] : Re: Convert README.org to plain text README while installing package 2022-06-06 1:38 ` Ihor Radchenko 2022-06-06 1:41 ` Drew Adams 2022-06-06 1:47 ` Stefan Monnier @ 2022-06-06 7:44 ` Tassilo Horn 2022-06-06 9:34 ` Ihor Radchenko 2 siblings, 1 reply; 505+ messages in thread From: Tassilo Horn @ 2022-06-06 7:44 UTC (permalink / raw) To: Ihor Radchenko Cc: Drew Adams, Stefan Monnier, Akib Azmain Turja, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > Do you know how many ELPA packages provide README.org vs README.md vs > README in ASCII text? GNU ELPA: 348 packages, 7 README.org / 3 README.md (one commented out) NonGNU ELPA: 160 packages, 9 README.org / 22 README.md (one commented out) But those are only the packages which have explicitly specified a :readme property. I'm not sure but think ELPA will pick up a README.md/org if there's no README automatically so it might not needed to be declared in which case my stats aren't very meaningful. I have no stats for MELPA but I guess there the ratio is even more shifted towards org/md. Bye, Tassilo ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: [External] : Re: Convert README.org to plain text README while installing package 2022-06-06 7:44 ` Tassilo Horn @ 2022-06-06 9:34 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-06 9:34 UTC (permalink / raw) To: Tassilo Horn Cc: Drew Adams, Stefan Monnier, Akib Azmain Turja, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel Tassilo Horn <tsdh@gnu.org> writes: > Ihor Radchenko <yantar92@gmail.com> writes: > >> Do you know how many ELPA packages provide README.org vs README.md vs >> README in ASCII text? > > GNU ELPA: 348 packages, 7 README.org / 3 README.md (one commented out) There are certainly more. I have found at least 19 README.org on ELPA in the first 70-ish packages. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-05 17:57 ` Tassilo Horn 2022-06-05 20:12 ` Stefan Monnier @ 2022-06-06 0:23 ` Drew Adams 2022-06-06 7:20 ` Tassilo Horn 1 sibling, 1 reply; 505+ messages in thread From: Drew Adams @ 2022-06-06 0:23 UTC (permalink / raw) To: Tassilo Horn Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel@gnu.org All I meant there was that it's good if a package description is written from a user point of view - what it does for users, not just what the author intended in writing the package. The motivation and design underlying the package are important, and _can_ be useful to document (for users and others). But most useful is what you can do with it as a user. > > Displaying a package description is for the > > benefit of the user, not the author. It's > > about what's most useful for users. > > Why do you think that a free software package author > doesn't write his documentation for this very purpose? I didn't presume anything about how or why every pkg author writes their pkg doc. I assume that most do try to provide useful doc for users. My point was only that the description should be useful to users. Whether the description "is displayed as the author intended" is, on the other hand, essentially a nice-to-have. (And I, as one user, really like to have it.) And I was clear that authors should be able to provide whatever display format/method they like. But I think _also_ providing a plain-text description of some sort (a plain README) should be a minimal requirement - at least something that's requested with "Please at least provide a plain-text README." > Exactly for this reason she has > structured the README in sections, **emphasized > the important parts**, included usage/config > examples as highlighted Absolutely nothing wrong with that - quite the contrary. That's to be hoped for, even expected. No one said anything to the contrary. > and added clickable links to [[helpful > resources][https://...]]. No one said that authors shouldn't be able to provide helpful links, images, and other artifacts. Even flashes and fireworks, if they like. > Why is it "most useful" for users to strip > away parts of that? No one said that that's the case. Not I, at least. Are you looking for a straw man? All I suggested was that authors should, at a minimum, provide a simple, plain-text file README. I explicitly even said that (IMO) that file need not say the same thing as whatever their other, formatted README.* files say. Why is a plain-text README desirable - useful for everyone? Because it requires minimal paraphernalia to easily read it or otherwise process it. Lowest common denominator. This should be a no-brainer, I think. > > Best is this, I think: > > > > 1. Require a plain-text README, as a minimum. > > 2. Allow other formats, with appropriate file > > extensions (e.g. .md, .org). > > > > I see no reason for any limit on the kinds of > > format, for #2. > > > > There should be, and likely are, simple ways > > to generate a plain-text README from this or > > that more structured format. Condition #1 > > shouldn't be an obstacle to anyone, I'd think. > > If there is some automatic conversion like for org, > the obstacle is just that I as a package author > would need to commit a generated file with the danger > that contributors send patches against the generated > document instead of the source. Not a big deal but > also not great. As you say, neither a big deal, nor a great obstacle. > > For Org, at least, that seems to confirm my > > guess that such conversion's already available. > > Yes, indeed. I haven't seen anything like that for > markdown (which is probably even more popular for > READMEs than org), though. Even producing a plain-text version by manual editing is likely not a big deal in most cases. That's my guess, at least. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: [External] : Re: Convert README.org to plain text README while installing package 2022-06-06 0:23 ` Drew Adams @ 2022-06-06 7:20 ` Tassilo Horn 2022-06-06 15:10 ` Drew Adams 0 siblings, 1 reply; 505+ messages in thread From: Tassilo Horn @ 2022-06-06 7:20 UTC (permalink / raw) To: Drew Adams Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel Drew Adams <drew.adams@oracle.com> writes: >> Why is it "most useful" for users to strip >> away parts of that? > > No one said that that's the case. Not I, at > least. Are you looking for a straw man? No, but maybe I didn't understand your suggestion which sounded to me as if you say that there should be a mandatory plain-text README to be displayed in Emacs plus as many "rich" versions as the package author wants. However, those wouldn't be displayed in Emacs. Or rather, they would only be displayed when you find-file them inside ~/.emacs.d/elpa/ which is quite unlikely to happen (unless the README says there are also alternative versions of the package destription available). >> Yes, indeed. I haven't seen anything like that for >> markdown (which is probably even more popular for >> READMEs than org), though. > > Even producing a plain-text version by manual editing > is likely not a big deal in most cases. That's my > guess, at least. Having to synchronize two identical (wrt. contents) versions of a document manually, one with markup and one without, is annoying IMO. Bye, Tassilo ^ permalink raw reply [flat|nested] 505+ messages in thread
* RE: [External] : Re: Convert README.org to plain text README while installing package 2022-06-06 7:20 ` Tassilo Horn @ 2022-06-06 15:10 ` Drew Adams 0 siblings, 0 replies; 505+ messages in thread From: Drew Adams @ 2022-06-06 15:10 UTC (permalink / raw) To: Tassilo Horn Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, emacs-devel@gnu.org, Michael Albinus, Alan Mackenzie, Stefan Kangas > maybe I didn't understand your suggestion which > sounded to me as if you say that there should be > a mandatory plain-text README to be displayed in > Emacs plus as many "rich" versions as the package > author wants. My suggestion is apparently moot now, as Stefan has already decided what will/won't be done, wrt GNU ELPA. But yes I suggested that a plain README be required. I don't think I said anything about it being displayed. And I'm pretty sure I didn't say anything about any other formats _not_ being displayed. I don't have a set opinion about what should be displayed by default. (I'd hope that, whatever is the default for displaying, users could somehow control what is actually the case.) I did say "required". I later changed the wording to clarify that I meant something more like "requested politely", e.g., Please consider supplying a plain README. > However, those wouldn't be displayed in Emacs. No; see above. I didn't mean to say, and I don't think I said, anything about what gets displayed. Now that you bring that up, I guess my preference would be that package authors can control what gets displayed by default (see above, about users being perhaps able to override the default). But others have brought up the question of needing this or that tool/infrastructure, to be able to display this or that "rich" format. I haven't been focused on the question of display, except to say that it takes very little to be able to read a simple, plain-text readme. My request for including plain README was just to include it, not to display it. Make it available. > Or rather, they would only be displayed when you > find-file them inside ~/.emacs.d/elpa/ which is > quite unlikely to happen (unless the README says > there are also alternative versions of the package > destription available). I hadn't thought about any particular mechanisms for making this or that format available, or displaying one or the other format. My suggestion (concern) was just to try to get authors to (also) include a plain README. > >> Yes, indeed. I haven't seen anything like that for > >> markdown (which is probably even more popular for > >> READMEs than org), though. > > > > Even producing a plain-text version by manual editing > > is likely not a big deal in most cases. That's my > > guess, at least. > > Having to synchronize two identical (wrt. contents) versions of a > document manually, one with markup and one without, is annoying IMO. As I said, the simple, plain README need not have the same content as any more elaborate README.*. Its purpose could be just to let you know something basic about the package - perhaps even including instructions for how to display a more elaborate README.*. To me, a plain README could serve as a minimal place to start, requiring next to no paraphernalia or infrastructure to read it. It need not be long or in any way complete/definitive. It should at least serve as minimal help to get going. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 13:11 ` Akib Azmain Turja 2022-06-05 15:32 ` Tassilo Horn @ 2022-06-05 15:52 ` Philip Kaludercic 2022-06-05 17:44 ` Stefan Monnier 1 sibling, 1 reply; 505+ messages in thread From: Philip Kaludercic @ 2022-06-05 15:52 UTC (permalink / raw) To: Akib Azmain Turja Cc: Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel Akib Azmain Turja <akib@disroot.org> writes: > Ihor Radchenko <yantar92@gmail.com> writes: > >> Po Lu <luangruo@yahoo.com> writes: >> >>>> That's true but wouldn't it be possible to extract the font-lock/faces >>>> related parts in a separate file with no dependencies which can be >>>> loaded quickly enough? I mean, the purpose would only be to display org >>>> and markdown READMEs with the highlighting intended by the package >>>> author but there's no point in enabling the gazillon of other features >>>> (from editing to time-tracking to spreadsheet computations) those >>>> packages might provide in addition. >>> >>> That wouldn't be Org mode anymore then, just an outline with some >>> additional fontification rules. >> >> Isn't *Help* buffer supposed to be read-only? There is no point loading >> Org editing features if editing is not going to happen. >> >> Best, >> Ihor > > Yes, *Help* is read only. > > I can't understand why people are suggesting to Org mode in package > description. I'm again saying that the README.org should be converted > (exported) to plain text on package installation (I won't recommend to > do this server, since its probably SaaSS and would increase server > load). I don't understand how this thread diverged, this proposal seems absolutely sensible. Why should a package that is not installed get converted to plain text, while when installed you see the org source code? From what I see this could be fixed if elpa-admin added a "README-elpa" with the rendered text into the tarball. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:52 ` Philip Kaludercic @ 2022-06-05 17:44 ` Stefan Monnier 2022-06-06 10:35 ` Philip Kaludercic 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-05 17:44 UTC (permalink / raw) To: Philip Kaludercic Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel > From what I see this could be fixed if elpa-admin added a "README-elpa" > with the rendered text into the tarball. I think you're right. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 17:44 ` Stefan Monnier @ 2022-06-06 10:35 ` Philip Kaludercic 2022-06-07 17:55 ` Stefan Monnier 0 siblings, 1 reply; 505+ messages in thread From: Philip Kaludercic @ 2022-06-06 10:35 UTC (permalink / raw) To: Stefan Monnier Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel [-- Attachment #1: Type: text/plain, Size: 232 bytes --] Stefan Monnier <monnier@iro.umontreal.ca> writes: >> From what I see this could be fixed if elpa-admin added a "README-elpa" >> with the rendered text into the tarball. > > I think you're right. I believe this should do the job: [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Render-Org-documentation-in-a-plain-text-README-elpa.patch --] [-- Type: text/x-diff, Size: 1752 bytes --] From 9d973044a346860d3fc6164fe75ad8cd9721a595 Mon Sep 17 00:00:00 2001 From: Philip Kaludercic <philipk@posteo.net> Date: Mon, 6 Jun 2022 12:34:40 +0200 Subject: [PATCH] Render Org documentation in a plain-text README-elpa file * elpa-admin.el (elpaa--make-one-tarball-1): Call elpaa--write-plain-readme. (elpaa--write-plain-readme): Add new function. --- elpa-admin.el | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/elpa-admin.el b/elpa-admin.el index ecf4bc90d1..60da684f1f 100644 --- a/elpa-admin.el +++ b/elpa-admin.el @@ -621,6 +621,7 @@ auxillary files unless TARBALL-ONLY is non-nil ." (elpaa--make pkg-spec dir) (elpaa--build-Info pkg-spec dir destdir)) (elpaa--write-pkg-file dir pkgname metadata revision) + (elpaa--write-plain-readme dir pkg-spec) ;; FIXME: Allow renaming files or selecting a subset of the files! (cl-assert (not (string-match "[][*\\|?]" pkgname))) (cl-assert (not (string-match "[][*\\|?]" vers))) @@ -1162,6 +1163,17 @@ Rename DIR/ to PKG-VERS/, and return the descriptor." nil pkg-file))) +(defun elpaa--write-plain-readme (pkg-dir pkg-spec) + "Render a plain text readme from PKG-SPEC in PKG-DIR. +This is only done if necessary, that is if the readme contents +are not already taken to be formatted in plain text." + (let* ((readme-content (elpaa--get-README pkg-spec pkg-dir))) + (unless (eq (car readme-content ) 'text/plain) + (write-region + (elpaa--section-to-plain-text readme-content) + nil + (expand-file-name "README-elpa" pkg-dir))))) + (defun elpaa-batch-generate-description-file (&rest _) "(Re)build the <PKG>-pkg.el file for particular packages." (while command-line-args-left -- 2.36.1 ^ permalink raw reply related [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-06 10:35 ` Philip Kaludercic @ 2022-06-07 17:55 ` Stefan Monnier 2022-06-24 7:18 ` Akib Azmain Turja 2022-06-24 15:42 ` Philip Kaludercic 0 siblings, 2 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-07 17:55 UTC (permalink / raw) To: Philip Kaludercic Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel > From 9d973044a346860d3fc6164fe75ad8cd9721a595 Mon Sep 17 00:00:00 2001 > From: Philip Kaludercic <philipk@posteo.net> > Date: Mon, 6 Jun 2022 12:34:40 +0200 > Subject: [PATCH] Render Org documentation in a plain-text README-elpa file > > * elpa-admin.el (elpaa--make-one-tarball-1): Call elpaa--write-plain-readme. > (elpaa--write-plain-readme): Add new function. That looks OK, feel free to push that to `elpa-admin`. Comments: - you'd get more bonus points by arranging for the code to (re)use the existing plain text (which is currently produced later when (re)generating the HTML page, so it would take a fair bit of code reorg to get that. In the mean time, a FIXME comment seems in order). - Your code may end up "rendering" a Markdown file for no obvious benefit since the rendering is a no-op (tho I think this is very hypothetical and likely can't happen because of other constraints). - We might also add a `README-elpa` file when the readme file has a name not recognized by `package--get-description`. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 17:55 ` Stefan Monnier @ 2022-06-24 7:18 ` Akib Azmain Turja 2022-06-24 7:45 ` Philip Kaludercic 2022-06-24 15:42 ` Philip Kaludercic 1 sibling, 1 reply; 505+ messages in thread From: Akib Azmain Turja @ 2022-06-24 7:18 UTC (permalink / raw) To: Stefan Monnier, Philip Kaludercic Cc: Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1336 bytes --] Stefan Monnier <monnier@iro.umontreal.ca> writes: >> From 9d973044a346860d3fc6164fe75ad8cd9721a595 Mon Sep 17 00:00:00 2001 >> From: Philip Kaludercic <philipk@posteo.net> >> Date: Mon, 6 Jun 2022 12:34:40 +0200 >> Subject: [PATCH] Render Org documentation in a plain-text README-elpa file >> >> * elpa-admin.el (elpaa--make-one-tarball-1): Call elpaa--write-plain-readme. >> (elpaa--write-plain-readme): Add new function. > > That looks OK, feel free to push that to `elpa-admin`. > Comments: > - you'd get more bonus points by arranging for the code > to (re)use the existing plain text (which is currently produced later > when (re)generating the HTML page, so it would take a fair bit of > code reorg to get that. In the mean time, a FIXME comment seems > in order). > - Your code may end up "rendering" a Markdown file for no obvious benefit > since the rendering is a no-op (tho I think this is very hypothetical > and likely can't happen because of other constraints). > - We might also add a `README-elpa` file when the readme file has a name > not recognized by `package--get-description`. > > > Stefan > Any update? -- Akib Azmain Turja This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-24 7:18 ` Akib Azmain Turja @ 2022-06-24 7:45 ` Philip Kaludercic 2022-07-15 12:24 ` Akib Azmain Turja 0 siblings, 1 reply; 505+ messages in thread From: Philip Kaludercic @ 2022-06-24 7:45 UTC (permalink / raw) To: Akib Azmain Turja Cc: Stefan Monnier, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel Akib Azmain Turja <akib@disroot.org> writes: > Stefan Monnier <monnier@iro.umontreal.ca> writes: > >>> From 9d973044a346860d3fc6164fe75ad8cd9721a595 Mon Sep 17 00:00:00 2001 >>> From: Philip Kaludercic <philipk@posteo.net> >>> Date: Mon, 6 Jun 2022 12:34:40 +0200 >>> Subject: [PATCH] Render Org documentation in a plain-text README-elpa file >>> >>> * elpa-admin.el (elpaa--make-one-tarball-1): Call elpaa--write-plain-readme. >>> (elpaa--write-plain-readme): Add new function. >> >> That looks OK, feel free to push that to `elpa-admin`. >> Comments: >> - you'd get more bonus points by arranging for the code >> to (re)use the existing plain text (which is currently produced later >> when (re)generating the HTML page, so it would take a fair bit of >> code reorg to get that. In the mean time, a FIXME comment seems >> in order). >> - Your code may end up "rendering" a Markdown file for no obvious benefit >> since the rendering is a no-op (tho I think this is very hypothetical >> and likely can't happen because of other constraints). >> - We might also add a `README-elpa` file when the readme file has a name >> not recognized by `package--get-description`. >> >> >> Stefan >> > > Any update? Sorry, this got buried in my inbox, I will try to update the patch today. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-24 7:45 ` Philip Kaludercic @ 2022-07-15 12:24 ` Akib Azmain Turja 2022-07-15 13:07 ` Philip Kaludercic 0 siblings, 1 reply; 505+ messages in thread From: Akib Azmain Turja @ 2022-07-15 12:24 UTC (permalink / raw) To: Philip Kaludercic Cc: Stefan Monnier, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel [-- Attachment #1: Type: text/plain, Size: 275 bytes --] I just found a pretty description with `describe-package`. Thanks. -- Akib Azmain Turja Find me on Mastodon at @akib@hostux.social. This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-07-15 12:24 ` Akib Azmain Turja @ 2022-07-15 13:07 ` Philip Kaludercic 2022-07-15 14:54 ` Akib Azmain Turja 0 siblings, 1 reply; 505+ messages in thread From: Philip Kaludercic @ 2022-07-15 13:07 UTC (permalink / raw) To: Akib Azmain Turja Cc: Stefan Monnier, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel Akib Azmain Turja <akib@disroot.org> writes: > I just found a pretty description with `describe-package`. Thanks. What package was it? Had it been installed locally? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-07-15 13:07 ` Philip Kaludercic @ 2022-07-15 14:54 ` Akib Azmain Turja 0 siblings, 0 replies; 505+ messages in thread From: Akib Azmain Turja @ 2022-07-15 14:54 UTC (permalink / raw) To: Philip Kaludercic Cc: Stefan Monnier, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel [-- Attachment #1: Type: text/plain, Size: 572 bytes --] Philip Kaludercic <philipk@posteo.net> writes: > Akib Azmain Turja <akib@disroot.org> writes: > >> I just found a pretty description with `describe-package`. Thanks. > > What package was it? Had it been installed locally? A few which I installed/upgraded recently, including: * cape * consult * corfu * embark * lin * marginalia * osm * svg-lib -- Akib Azmain Turja Find me on Mastodon at @akib@hostux.social. This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-07 17:55 ` Stefan Monnier 2022-06-24 7:18 ` Akib Azmain Turja @ 2022-06-24 15:42 ` Philip Kaludercic 2022-06-24 17:50 ` Stefan Monnier 1 sibling, 1 reply; 505+ messages in thread From: Philip Kaludercic @ 2022-06-24 15:42 UTC (permalink / raw) To: Stefan Monnier Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel [-- Attachment #1: Type: text/plain, Size: 566 bytes --] Stefan Monnier <monnier@iro.umontreal.ca> writes: >> From 9d973044a346860d3fc6164fe75ad8cd9721a595 Mon Sep 17 00:00:00 2001 >> From: Philip Kaludercic <philipk@posteo.net> >> Date: Mon, 6 Jun 2022 12:34:40 +0200 >> Subject: [PATCH] Render Org documentation in a plain-text README-elpa file >> >> * elpa-admin.el (elpaa--make-one-tarball-1): Call elpaa--write-plain-readme. >> (elpaa--write-plain-readme): Add new function. > > That looks OK, feel free to push that to `elpa-admin`. > Comments: The following should address most of your points, just one question, [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Render-Org-documentation-in-a-plain-text-README-elpa.patch --] [-- Type: text/x-diff, Size: 4063 bytes --] From c60903f5e916b22678c43c0fba0aa73a9e0bf471 Mon Sep 17 00:00:00 2001 From: Philip Kaludercic <philipk@posteo.net> Date: Mon, 6 Jun 2022 12:34:40 +0200 Subject: [PATCH] Render Org documentation in a plain-text README-elpa file * elpa-admin.el (elpaa--make-one-tarball-1): Call elpaa--write-plain-readme. (elpaa--write-plain-readme): Add new function. (elpaa--html-make-pkg): Reuse the rendered plaintext from `elpaa--write-plain-readme'. --- elpa-admin.el | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/elpa-admin.el b/elpa-admin.el index 58c0ec2e0a..173d247e87 100644 --- a/elpa-admin.el +++ b/elpa-admin.el @@ -610,7 +610,8 @@ auxillary files unless TARBALL-ONLY is non-nil ." (lambda (file) (string-match re file) (cons (match-string 1 file) file)) - (directory-files destdir nil re))))) + (directory-files destdir nil re)))) + rendered) (when ldir (cl-pushnew (list (file-name-as-directory ldir) "") renames :test #'equal)) @@ -621,6 +622,7 @@ auxillary files unless TARBALL-ONLY is non-nil ." (elpaa--make pkg-spec dir) (elpaa--build-Info pkg-spec dir destdir)) (elpaa--write-pkg-file dir pkgname metadata revision) + (setq rendered (elpaa--write-plain-readme dir pkg-spec)) ;; FIXME: Allow renaming files or selecting a subset of the files! (cl-assert (not (string-match "[][*\\|?]" pkgname))) (cl-assert (not (string-match "[][*\\|?]" vers))) @@ -677,7 +679,7 @@ auxillary files unless TARBALL-ONLY is non-nil ." (elpaa--html-make-pkg pkgdesc pkg-spec `((,vers . ,(file-name-nondirectory tarball)) . ,oldtarballs) - dir)))) + dir rendered)))) 'new))) (defun elpaa--makeenv (version revision) @@ -1162,6 +1164,22 @@ Rename DIR/ to PKG-VERS/, and return the descriptor." nil pkg-file))) +(defun elpaa--write-plain-readme (pkg-dir pkg-spec) + "Render a plain text readme from PKG-SPEC in PKG-DIR. +This is only done if necessary, that is if the readme contents +are not already taken to be formatted in plain text or when the +readme file has an unconventional name" + (let ((readme-file (elpaa--spec-get pkg-spec :readme)) + (known-readme-names ;see `package--get-description' + '("README-elpa" "README-elpa.md" "README" "README.rst" "README.org")) + (readme-content (elpaa--get-README pkg-spec pkg-dir))) + (if (or (not (eq (car readme-content) 'text/plain)) + (not (member readme-file known-readme-names))) + (let ((rendered (elpaa--section-to-plain-text readme-content))) + (write-region rendered nil (expand-file-name "README-elpa" pkg-dir)) + rendered) + readme-content))) + (defun elpaa-batch-generate-description-file (&rest _) "(Re)build the <PKG>-pkg.el file for particular packages." (while command-line-args-left @@ -1489,7 +1507,7 @@ arbitrary code." )) (insert "</dd>\n")))) -(defun elpaa--html-make-pkg (pkg pkg-spec files srcdir) +(defun elpaa--html-make-pkg (pkg pkg-spec files srcdir plain-readme) (let* ((name (symbol-name (car pkg))) (latest (package-version-join (aref (cdr pkg) 0))) (mainsrcfile (expand-file-name (elpaa--main-file pkg-spec) srcdir)) @@ -1542,7 +1560,7 @@ arbitrary code." <pre>M-x <span class=\"kw\">package-install</span> RET <span class=\"kw\">%s</span> RET</pre>" name)) (let* ((readme-content (elpaa--get-README pkg-spec srcdir)) - (readme-text (elpaa--section-to-plain-text readme-content)) + (readme-text plain-readme) (readme-html (elpaa--section-to-html readme-content)) (readme-output-filename (concat name "-readme.txt"))) (write-region readme-text nil readme-output-filename) -- 2.36.1 [-- Attachment #3: Type: text/plain, Size: 818 bytes --] > - you'd get more bonus points by arranging for the code > to (re)use the existing plain text (which is currently produced later > when (re)generating the HTML page, so it would take a fair bit of > code reorg to get that. In the mean time, a FIXME comment seems > in order). > - Your code may end up "rendering" a Markdown file for no obvious benefit > since the rendering is a no-op (tho I think this is very hypothetical > and likely can't happen because of other constraints). Why should this be? Both this and the previous iteration of the patch check (eq (car readme-content) 'text/plain), and currently there is only text/plain (which includes markdown) and text/org? > - We might also add a `README-elpa` file when the readme file has a name > not recognized by `package--get-description`. ^ permalink raw reply related [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-24 15:42 ` Philip Kaludercic @ 2022-06-24 17:50 ` Stefan Monnier 2022-06-26 10:25 ` Philip Kaludercic 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-24 17:50 UTC (permalink / raw) To: Philip Kaludercic Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel > The following should address most of your points, just one question, Nicely hidden inside the signature :-) > > - Your code may end up "rendering" a Markdown file for no obvious benefit > > since the rendering is a no-op (tho I think this is very hypothetical > > and likely can't happen because of other constraints). > Why should this be? Both this and the previous iteration of the patch > check (eq (car readme-content) 'text/plain), and currently there is only > text/plain (which includes markdown) and text/org? text/plain does not include markdown. Markdown files will have type text/markdown. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-24 17:50 ` Stefan Monnier @ 2022-06-26 10:25 ` Philip Kaludercic 2022-06-26 12:22 ` Stefan Monnier 0 siblings, 1 reply; 505+ messages in thread From: Philip Kaludercic @ 2022-06-26 10:25 UTC (permalink / raw) To: Stefan Monnier Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >> The following should address most of your points, just one question, > > Nicely hidden inside the signature :-) > >> > - Your code may end up "rendering" a Markdown file for no obvious benefit >> > since the rendering is a no-op (tho I think this is very hypothetical >> > and likely can't happen because of other constraints). >> Why should this be? Both this and the previous iteration of the patch >> check (eq (car readme-content) 'text/plain), and currently there is only >> text/plain (which includes markdown) and text/org? > > text/plain does not include markdown. Markdown files will have type > text/markdown. But in that case, markdown files could still be "rendered", by replacing regular double quotes with fancy quotes, removing unnecessary code blocks, etc. E.g. using "pandoc -t plain", this file --8<---------------cut here---------------start------------->8--- # This is a title So and so *this* and __that__, "they say", yes! some more stuff =============== How about some [code](https://code.com)? ```lisp (defun call-me () (call-me) (call-me) (anytime)) ``` or like this $ ls -l ### The Remainder So what is left? * A list * A number list 1. one 2. two 3. three * a horizontal rule --- that's it --8<---------------cut here---------------end--------------->8--- is converted into --8<---------------cut here---------------start------------->8--- This is a title So and so this and that, “they say”, yes! some more stuff How about some code? (defun call-me () (call-me) (call-me) (anytime)) or like this $ ls -l The Remainder So what is left? - A list - A number list 1. one 2. two 3. three - a horizontal rule ------------------------------------------------------------------------ that’s it --8<---------------cut here---------------end--------------->8--- While not perfect (and I am not arguing to necessarily use pandoc), I do think that there is enough that can be done with markup while still dealing with plain text. > > Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-26 10:25 ` Philip Kaludercic @ 2022-06-26 12:22 ` Stefan Monnier 2022-06-26 13:08 ` Philip Kaludercic 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-26 12:22 UTC (permalink / raw) To: Philip Kaludercic Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel > But in that case, markdown files could still be "rendered", by replacing In theory, yes. In practice, currently they aren't. > is converted into FWIW I find this particular conversion made things overall worse :-( Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-26 12:22 ` Stefan Monnier @ 2022-06-26 13:08 ` Philip Kaludercic 2022-06-26 15:23 ` Stefan Monnier 0 siblings, 1 reply; 505+ messages in thread From: Philip Kaludercic @ 2022-06-26 13:08 UTC (permalink / raw) To: Stefan Monnier Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >> But in that case, markdown files could still be "rendered", by replacing > > In theory, yes. In practice, currently they aren't. My bad, I had forgotten that `elpaa--extension-to-mime' could also return text/markdown, which confused me here. I'll update the patch to only work on text/x-org and push it. >> is converted into > > FWIW I find this particular conversion made things overall worse :-( Hmm, what in particular? As I said it doesn't have to be this specific formatting. It could also just be cleaned up markdown, moving links below paragraphs, wrapping lines and stuff like that. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-26 13:08 ` Philip Kaludercic @ 2022-06-26 15:23 ` Stefan Monnier 2022-06-27 10:04 ` Philip Kaludercic 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-26 15:23 UTC (permalink / raw) To: Philip Kaludercic Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel >> FWIW I find this particular conversion made things overall worse :-( > Hmm, what in particular? I didn't like the two spaces after the "bullets" and more importantly the information about what is a title was completely lost. > As I said it doesn't have to be this specific formatting. Sure, I just mentioned it because I found it rather odd to return a worse result than the identity function. > It could also just be cleaned up markdown, moving links below > paragraphs, wrapping lines and stuff like that. Then again, this can (should?) be applied to the source directly. Don't get me wrong: I'm definitely not opposed to rendering Markdown to plain text. I've seen a fair bit of "GFM" files that are not great to read as plain text, so there is clearly room for improvement. Maybe throwing out the badges would be a good first step ;-) Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-26 15:23 ` Stefan Monnier @ 2022-06-27 10:04 ` Philip Kaludercic 2022-06-27 12:12 ` Stefan Monnier 2022-06-27 14:29 ` Yuri Khan 0 siblings, 2 replies; 505+ messages in thread From: Philip Kaludercic @ 2022-06-27 10:04 UTC (permalink / raw) To: Stefan Monnier Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >>> FWIW I find this particular conversion made things overall worse :-( >> Hmm, what in particular? > > I didn't like the two spaces after the "bullets" and more importantly > the information about what is a title was completely lost. > >> As I said it doesn't have to be this specific formatting. > > Sure, I just mentioned it because I found it rather odd to return > a worse result than the identity function. My example was a pretty clean markdown file, which is usually readable enough in its own right, so most post-processing doesn't help. But I agree, it isn't as easy as just using pandoc's "plain" output. >> It could also just be cleaned up markdown, moving links below >> paragraphs, wrapping lines and stuff like that. > > Then again, this can (should?) be applied to the source directly. As in sending upstream patches? From my experience a lot of maintainers (regrettably) prefer something that renders well on sites like GitHub or GitLab than that is readable as plain text. Or do you mean writing custom code to process markdown files? > Don't get me wrong: I'm definitely not opposed to rendering Markdown to > plain text. I've seen a fair bit of "GFM" files that are not great to > read as plain text, so there is clearly room for improvement. > Maybe throwing out the badges would be a good first step ;-) Then again, this ties into the README files that look better when rendered on GitHub/Lab, and where it is worth considering if *not* using them would be of more use. If this is about providing a good overview when using C-h P, using a good commentary section (that does not include comments on how to install the package, contribution guidelines, screenshots, etc.) might be better to begin with. This is also related to the point I had raised earlier in the thread that when a package generates a manual from a Org README, using the same file as a README file results in too much (redundant) information. > Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-27 10:04 ` Philip Kaludercic @ 2022-06-27 12:12 ` Stefan Monnier 2022-06-27 14:29 ` Yuri Khan 1 sibling, 0 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-27 12:12 UTC (permalink / raw) To: Philip Kaludercic Cc: Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel > As in sending upstream patches? That's what I meant, yes. > From my experience a lot of maintainers (regrettably) prefer something > that renders well on sites like GitHub or GitLab than that is readable > as plain text. The kinds of changes mentioned shouldn't make any difference in terms of rendering. > This is also related to the point I had raised earlier in the thread > that when a package generates a manual from a Org README, using the same > file as a README file results in too much (redundant) information. I don't think it's our role to fix this problem: the (Non)GNU infrastructure offers a place to put a "README" and another to put a "manual" and it's up to the package's authors to use this as they think is best. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-27 10:04 ` Philip Kaludercic 2022-06-27 12:12 ` Stefan Monnier @ 2022-06-27 14:29 ` Yuri Khan 2022-06-27 16:44 ` Philip Kaludercic 1 sibling, 1 reply; 505+ messages in thread From: Yuri Khan @ 2022-06-27 14:29 UTC (permalink / raw) To: Philip Kaludercic Cc: Stefan Monnier, Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, Emacs developers On Mon, 27 Jun 2022 at 17:05, Philip Kaludercic <philipk@posteo.net> wrote: > > Maybe throwing out the badges would be a good first step ;-) > > Then again, this ties into the README files that look better when > rendered on GitHub/Lab, and where it is worth considering if *not* > using them would be of more use. Badges do not have to be ugly in markdown source. Or, at least, they can be much less ugly. Let’s dissect [Eglot’s README.md][1] just as an example (only because it has badges; otherwise, I think Eglot’s README.md is as non-ugly as possible): [1]: https://raw.githubusercontent.com/joaotavora/eglot/master/README.md [![Build status](https://github.com/joaotavora/eglot/actions/workflows/test.yml/badge.svg)](https://github.com/joaotavora/eglot/actions/workflows/test.yml) [![GNU ELPA](https://elpa.gnu.org/packages/eglot.svg)](https://elpa.gnu.org/packages/eglot.html) [![MELPA](https://melpa.org/packages/eglot-badge.svg)](https://melpa.org/#/eglot) Here we have three source lines, each longer than 80 characters. They are not separated with blank lines, so they comprise a single paragraph; the badges are rendered in a single line. Each badge is a hyperlink [](); brackets contain the link text and parentheses specify the link target URL. There is alternative syntax where you put a short reference identifier at the place of use, and define that reference on a separate line. [![Build status](https://github.com/joaotavora/eglot/actions/workflows/test.yml/badge.svg)][build] [![GNU ELPA](https://elpa.gnu.org/packages/eglot.svg)][elpa] [![MELPA](https://melpa.org/packages/eglot-badge.svg)][melpa] [build]: https://github.com/joaotavora/eglot/actions/workflows/test.yml [elpa]: https://elpa.gnu.org/packages/eglot.html [melpa]: https://melpa.org/#/eglot Next, each badge’s text is actually an image ![](). Brackets contain the alt text fallback for user agents that do not display images; and parens specify the image URL. The same external reference syntax works: [![Build status][build-badge]][build] [![GNU ELPA][elpa-badge]][elpa] [![MELPA][melpa-badge]][melpa] [build]: https://github.com/joaotavora/eglot/actions/workflows/test.yml [elpa]: https://elpa.gnu.org/packages/eglot.html [melpa]: https://melpa.org/#/eglot [build-badge]: https://github.com/joaotavora/eglot/actions/workflows/test.yml/badge.svg [elpa-badge]: https://elpa.gnu.org/packages/eglot.svg [melpa-badge]: https://melpa.org/packages/eglot-badge.svg Now, only one line is longer than 80 characters. URLs specified this way can appear anywhere in the markdown source. Eglot’s README actually uses this syntax for all other (non-badge) hyperlinks and puts URLs at the bottom. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-27 14:29 ` Yuri Khan @ 2022-06-27 16:44 ` Philip Kaludercic 0 siblings, 0 replies; 505+ messages in thread From: Philip Kaludercic @ 2022-06-27 16:44 UTC (permalink / raw) To: Yuri Khan Cc: Stefan Monnier, Akib Azmain Turja, Ihor Radchenko, Po Lu, Tassilo Horn, Michael Albinus, Alan Mackenzie, Stefan Kangas, Emacs developers Yuri Khan <yuri.v.khan@gmail.com> writes: > On Mon, 27 Jun 2022 at 17:05, Philip Kaludercic <philipk@posteo.net> wrote: > >> > Maybe throwing out the badges would be a good first step ;-) >> >> Then again, this ties into the README files that look better when >> rendered on GitHub/Lab, and where it is worth considering if *not* >> using them would be of more use. > > Badges do not have to be ugly in markdown source. Or, at least, they > can be much less ugly. > > Let’s dissect [Eglot’s README.md][1] just as an example (only because > it has badges; otherwise, I think Eglot’s README.md is as non-ugly as > possible): > [...] > > [![Build status][build-badge]][build] > [![GNU ELPA][elpa-badge]][elpa] > [![MELPA][melpa-badge]][melpa] > > [build]: https://github.com/joaotavora/eglot/actions/workflows/test.yml > [elpa]: https://elpa.gnu.org/packages/eglot.html > [melpa]: https://melpa.org/#/eglot > > [build-badge]: > https://github.com/joaotavora/eglot/actions/workflows/test.yml/badge.svg > [elpa-badge]: https://elpa.gnu.org/packages/eglot.svg > [melpa-badge]: https://melpa.org/packages/eglot-badge.svg > > Now, only one line is longer than 80 characters. > > URLs specified this way can appear anywhere in the markdown source. > Eglot’s README actually uses this syntax for all other (non-badge) > hyperlinks and puts URLs at the bottom. I get the impression that this trades horizontally wasted space for vertically wasted space (setting aside that by reordering the URLs they don't have to appear in the middle of the document). The issue here is that in the plain-text buffer that C-h P provides, the badges are simply always redundant, so either they should somehow be handled by the *Help* buffer, removed from the buffer or (as I argue) we should not display the README that is usually targeted at a different audience, and instead recommend to write clean, brief, informative commentary sections in the main files. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 10:26 ` Tassilo Horn ` (3 preceding siblings ...) 2022-06-05 11:45 ` Po Lu @ 2022-06-05 11:59 ` Akib Azmain Turja 2022-06-08 17:21 ` Yoni Rabkin 5 siblings, 0 replies; 505+ messages in thread From: Akib Azmain Turja @ 2022-06-05 11:59 UTC (permalink / raw) To: Tassilo Horn, Po Lu Cc: Michael Albinus, Alan Mackenzie, Stefan Kangas, emacs-devel [-- Attachment #1: Type: text/plain, Size: 2448 bytes --] Tassilo Horn <tsdh@gnu.org> writes: > Po Lu <luangruo@yahoo.com> writes: > >>>>> I think we should actively encourage using it for most types of >>>>> documentation, certainly for README type files. >>>> >>>> I disagree entirely with you. Org mode is highly complicated, >>>> obscure (I've never managed to get a feel for what it does), and >>>> difficult to learn (I've never managed it). A text file is far >>>> easier to read for those not familiar with org. We're talking about >>>> READMEs here. They're typically 20 to 100 lines long. A text file >>>> is ideal for these. >>> >>> 1+. README is a text file which is consulted in order to get >>> information about the related software. There's no rule to visit it >>> by Emacs only. > > People nowadays write README.org and README.md files instead of > plain-text READMEs because the packages' repositories are hosted on some > forge (sr.ht, github, gitlab, gitea, whatever) which supports rendering > those formats in a nice way. So the choice of format is natural for > package authors. And they frequently contain markup like headings, > bullet lists, bold, italics, and code snippets which simply look much > better than their text-only alternative (even in their > editing/non-rendered versions). > >> Hear, hear! >> >> It also takes a relatively long time to load Org mode, so opening a >> README.org-type file for the first time will always cause an annoying >> pause. > > That's true but wouldn't it be possible to extract the font-lock/faces > related parts in a separate file with no dependencies which can be > loaded quickly enough? I mean, the purpose would only be to display org > and markdown READMEs with the highlighting intended by the package > author but there's no point in enabling the gazillon of other features > (from editing to time-tracking to spreadsheet computations) those > packages might provide in addition. > > Bye, > Tassilo I can't understand why you are suggesting to emulate (or enable) Org syntax highlighting in package description. I'm again saying that the README.org should be converted (exported) to plain text on package installation (I won't recommend to do this server, since its probably SaaSS and would increase server load). -- Akib Azmain Turja This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 10:26 ` Tassilo Horn ` (4 preceding siblings ...) 2022-06-05 11:59 ` Akib Azmain Turja @ 2022-06-08 17:21 ` Yoni Rabkin 5 siblings, 0 replies; 505+ messages in thread From: Yoni Rabkin @ 2022-06-08 17:21 UTC (permalink / raw) To: Tassilo Horn Cc: Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Akib Azmain Turja, emacs-devel Tassilo Horn <tsdh@gnu.org> writes: > Po Lu <luangruo@yahoo.com> writes: > >>>>> I think we should actively encourage using it for most types of >>>>> documentation, certainly for README type files. >>>> >>>> I disagree entirely with you. Org mode is highly complicated, >>>> obscure (I've never managed to get a feel for what it does), and >>>> difficult to learn (I've never managed it). A text file is far >>>> easier to read for those not familiar with org. We're talking about >>>> READMEs here. They're typically 20 to 100 lines long. A text file >>>> is ideal for these. >>> >>> 1+. README is a text file which is consulted in order to get >>> information about the related software. There's no rule to visit it >>> by Emacs only. > > People nowadays write README.org and README.md files instead of > plain-text READMEs because the packages' repositories are hosted on some > forge (sr.ht, github, gitlab, gitea, whatever) which supports rendering > those formats in a nice way. So the choice of format is natural for > package authors. And they frequently contain markup like headings, > bullet lists, bold, italics, and code snippets which simply look much > better than their text-only alternative (even in their > editing/non-rendered versions). I personally don't think that org is nice, or natural. Those are subjective statements. Relying on the fact that lots of people use org and formatted READMEs on github and such is not a justification either; lots and lots of people use windows... Making Emacs look like whatever happens to be popular shouldn't be done at the expense of existing users. Org is popular, yes, but Emacs shouldn't be a popularity contest. The fact that there are people, like me, asking to be able to read a text file as a text file should be respected. I would be fine with being able to: (setq packages-org-to-text t) Or similar, and never have to to be confronted with ellipses and foreign (to me) key-bindings when I'm trying to read a few lines of text. Even better, if I had a simple: (org-disable-org-mode t) From then on I won't have to worry about being forced to read plain text through a reincarnated implementation of "hypertext" from the 80s. First do no harm. -- "Cut your own wood and it will warm you twice" ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 8:51 ` Po Lu 2022-06-05 10:26 ` Tassilo Horn @ 2022-06-05 15:15 ` Ihor Radchenko 2022-06-05 15:22 ` Eli Zaretskii 2022-06-05 17:40 ` Stefan Monnier 1 sibling, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-05 15:15 UTC (permalink / raw) To: Po Lu Cc: Michael Albinus, Alan Mackenzie, Stefan Kangas, Tassilo Horn, Akib Azmain Turja, Emacs developers Po Lu <luangruo@yahoo.com> writes: > Hear, hear! > > It also takes a relatively long time to load Org mode, so opening a > README.org-type file for the first time will always cause an annoying > pause. I've heard about the problem with loading multiple times and I do agree that loading can be slow, especially when various kinds of extra functionality is requested. However, I am wondering about the actual numbers. How long is long? Can you execute (setq org-modules nil) (benchmark-progn (find-file "/path/to/test.org")) And share the results? Using Emacs 28 and built-in Org, the above took Elapsed time: 0.161496s (0.092033s in 12 GCs) On my system. I am not sure if it should be considered slow. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:15 ` Ihor Radchenko @ 2022-06-05 15:22 ` Eli Zaretskii 2022-06-05 15:27 ` Eli Zaretskii ` (2 more replies) 2022-06-05 17:40 ` Stefan Monnier 1 sibling, 3 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-05 15:22 UTC (permalink / raw) To: Ihor Radchenko Cc: luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Michael Albinus <michael.albinus@gmx.de>, Alan Mackenzie <acm@muc.de>, > Stefan Kangas <stefan@marxist.se>, Tassilo Horn <tsdh@gnu.org>, Akib > Azmain Turja <akib@disroot.org>, Emacs developers <emacs-devel@gnu.org> > Date: Sun, 05 Jun 2022 23:15:27 +0800 > > I've heard about the problem with loading multiple times and I do agree > that loading can be slow, especially when various kinds of extra > functionality is requested. However, I am wondering about the actual > numbers. How long is long? This: (benchmark-run 1 (load "org")) takes around 1.7 sec here, including 1.1 sec used by GC. This is in an unoptimized build of Emacs 29 on MS-Windows. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:22 ` Eli Zaretskii @ 2022-06-05 15:27 ` Eli Zaretskii 2022-06-05 15:41 ` Ihor Radchenko 2022-06-05 16:16 ` Tassilo Horn 2 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-05 15:27 UTC (permalink / raw) To: yantar92; +Cc: luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel > Date: Sun, 05 Jun 2022 18:22:24 +0300 > From: Eli Zaretskii <eliz@gnu.org> > Cc: luangruo@yahoo.com, michael.albinus@gmx.de, acm@muc.de, > stefan@marxist.se, tsdh@gnu.org, akib@disroot.org, emacs-devel@gnu.org > > This: > > (benchmark-run 1 (load "org")) > > takes around 1.7 sec here, including 1.1 sec used by GC. This is in > an unoptimized build of Emacs 29 on MS-Windows. In an optimized build of Emacs 28 with native-compilation, the above takes 0.5 sec with about 0.1 sec used up by GC. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:22 ` Eli Zaretskii 2022-06-05 15:27 ` Eli Zaretskii @ 2022-06-05 15:41 ` Ihor Radchenko 2022-06-05 15:46 ` Eli Zaretskii 2022-06-05 16:16 ` Tassilo Horn 2 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-05 15:41 UTC (permalink / raw) To: Eli Zaretskii Cc: luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > This: > > (benchmark-run 1 (load "org")) > > takes around 1.7 sec here, including 1.1 sec used by GC. This is in > an unoptimized build of Emacs 29 on MS-Windows. I tried your example on Emacs 29 and Emacs 28 (both optimized with native-comp). On Emacs 29 I got (0.315444431 17 0.154982815) On Emacs 28 I got (0.17239349 11 0.088822737) To make sure that different Org versions do not interfere, I also tried emacs -L /path/to/org/latest/main/branch/lisp -Q with compiled Org repo (I ran make to compile Org into .elc) On Emacs 29 I got (0.311368927 15 0.14880696) On Emacs 28 I got (0.133921351 10 0.076385214) with uncompiled Org report (make clean; make autoloads): On Emacs 29 I got (1.040857068 41 0.5445397559999999) On Emacs 28 I got (0.5897047940000001 28 0.337317824) Clearly, compiled Org version can be loaded fairly fast on Emacs 28 and not so fast on Emacs 29. For comparison, I also tried (benchmark-run 1 (describe-variable 'emacs-version)) On Emacs 29 I got (0.076433304 4 0.030115377999999998) On Emacs 28 I got (0.081279133 3 0.035413045000000004) I consider the last test to represent the compiled built-in help-mode. org-mode loading is just 2x slower than help-mode on Emacs 28. Not so great on Emacs 29, but it is something about Emacs 29 rather than about org-mode. Do you have any ideas? Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:41 ` Ihor Radchenko @ 2022-06-05 15:46 ` Eli Zaretskii 2022-06-05 15:53 ` Ihor Radchenko 2022-06-05 15:53 ` Eli Zaretskii 0 siblings, 2 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-05 15:46 UTC (permalink / raw) To: Ihor Radchenko Cc: luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: luangruo@yahoo.com, michael.albinus@gmx.de, acm@muc.de, > stefan@marxist.se, tsdh@gnu.org, akib@disroot.org, emacs-devel@gnu.org > Date: Sun, 05 Jun 2022 23:41:06 +0800 > > org-mode loading is just 2x slower than help-mode on Emacs 28. > Not so great on Emacs 29, but it is something about Emacs 29 rather than > about org-mode. Do you have any ideas? No, sorry. Suggest to profile it to see what takes time. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:46 ` Eli Zaretskii @ 2022-06-05 15:53 ` Ihor Radchenko 2022-06-05 16:25 ` Eli Zaretskii 2022-06-05 15:53 ` Eli Zaretskii 1 sibling, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-05 15:53 UTC (permalink / raw) To: Eli Zaretskii Cc: luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Ihor Radchenko <yantar92@gmail.com> >> Cc: luangruo@yahoo.com, michael.albinus@gmx.de, acm@muc.de, >> stefan@marxist.se, tsdh@gnu.org, akib@disroot.org, emacs-devel@gnu.org >> Date: Sun, 05 Jun 2022 23:41:06 +0800 >> >> org-mode loading is just 2x slower than help-mode on Emacs 28. >> Not so great on Emacs 29, but it is something about Emacs 29 rather than >> about org-mode. Do you have any ideas? > > No, sorry. Suggest to profile it to see what takes time. Well. I tried, but it is not terribly helpful: 251 85% - command-execute 251 85% - funcall-interactively 251 85% - eval-last-sexp 251 85% - elisp--eval-last-sexp 251 85% - progn 251 85% - benchmark-call 251 85% - #<lambda 0x14e0668> 251 85% - load 251 85% - load-with-code-conversion 130 44% - require 120 40% - load-with-code-conversion 70 23% + internal-macroexpand-for-load 30 10% - require 20 6% - load-with-code-conversion 20 6% - require 10 3% load-with-code-conversion 10 3% + byte-code 10 3% byte-code 101 34% + internal-macroexpand-for-load 10 3% + custom-declare-variable 43 14% - ... 33 11% Automatic GC 10 3% + byte-compile-preprocess Looks like macroexpand takes time, but I have no idea how to interpret this. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:53 ` Ihor Radchenko @ 2022-06-05 16:25 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-05 16:25 UTC (permalink / raw) To: Ihor Radchenko Cc: luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: luangruo@yahoo.com, michael.albinus@gmx.de, acm@muc.de, > stefan@marxist.se, tsdh@gnu.org, akib@disroot.org, emacs-devel@gnu.org > Date: Sun, 05 Jun 2022 23:53:14 +0800 > > > No, sorry. Suggest to profile it to see what takes time. > > Well. I tried, but it is not terribly helpful: > > 251 85% - command-execute > 251 85% - funcall-interactively > 251 85% - eval-last-sexp > 251 85% - elisp--eval-last-sexp > 251 85% - progn > 251 85% - benchmark-call > 251 85% - #<lambda 0x14e0668> > 251 85% - load > 251 85% - load-with-code-conversion > 130 44% - require > 120 40% - load-with-code-conversion > 70 23% + internal-macroexpand-for-load > 30 10% - require > 20 6% - load-with-code-conversion > 20 6% - require > 10 3% load-with-code-conversion > 10 3% + byte-code > 10 3% byte-code > 101 34% + internal-macroexpand-for-load > 10 3% + custom-declare-variable > 43 14% - ... > 33 11% Automatic GC > 10 3% + byte-compile-preprocess > > Looks like macroexpand takes time, but I have no idea how to interpret > this. I think also load-with-code-conversion, which probably means we are loading gobs of Lisp. That could also explain why macroexpansion takes a significant percentage. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:46 ` Eli Zaretskii 2022-06-05 15:53 ` Ihor Radchenko @ 2022-06-05 15:53 ` Eli Zaretskii 2022-06-05 15:58 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-05 15:53 UTC (permalink / raw) To: yantar92; +Cc: luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel > Date: Sun, 05 Jun 2022 18:46:46 +0300 > From: Eli Zaretskii <eliz@gnu.org> > Cc: luangruo@yahoo.com, michael.albinus@gmx.de, acm@muc.de, > stefan@marxist.se, tsdh@gnu.org, akib@disroot.org, emacs-devel@gnu.org > > > From: Ihor Radchenko <yantar92@gmail.com> > > Cc: luangruo@yahoo.com, michael.albinus@gmx.de, acm@muc.de, > > stefan@marxist.se, tsdh@gnu.org, akib@disroot.org, emacs-devel@gnu.org > > Date: Sun, 05 Jun 2022 23:41:06 +0800 > > > > org-mode loading is just 2x slower than help-mode on Emacs 28. > > Not so great on Emacs 29, but it is something about Emacs 29 rather than > > about org-mode. Do you have any ideas? > > No, sorry. Suggest to profile it to see what takes time. For some more data point, I ran (benchmark-run 1 (find-file "~/org/Plan.org")) Plan.org is a 1500-line file, so not very large, and it uses only very basic Org features: links to URLs. The above tool 5.1 sec (3.1 of which in GC) in non-optimized Emacs 29 and 1.5 sec (0.4 in GC) in optimized Emacs 28. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:53 ` Eli Zaretskii @ 2022-06-05 15:58 ` Ihor Radchenko 2022-06-05 16:27 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-05 15:58 UTC (permalink / raw) To: Eli Zaretskii Cc: luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > For some more data point, I ran > > (benchmark-run 1 (find-file "~/org/Plan.org")) > > Plan.org is a 1500-line file, so not very large, and it uses only very > basic Org features: links to URLs. > > The above tool 5.1 sec (3.1 of which in GC) in non-optimized Emacs 29 > and 1.5 sec (0.4 in GC) in optimized Emacs 28. Does setting org-modules to nil prior opening the file change much for you? (setq org-modules nil) (benchmark-run 1 (find-file "~/org/Plan.org")) Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:58 ` Ihor Radchenko @ 2022-06-05 16:27 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-05 16:27 UTC (permalink / raw) To: Ihor Radchenko Cc: luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: luangruo@yahoo.com, michael.albinus@gmx.de, acm@muc.de, > stefan@marxist.se, tsdh@gnu.org, akib@disroot.org, emacs-devel@gnu.org > Date: Sun, 05 Jun 2022 23:58:40 +0800 > > > The above tool 5.1 sec (3.1 of which in GC) in non-optimized Emacs 29 > > and 1.5 sec (0.4 in GC) in optimized Emacs 28. > > Does setting org-modules to nil prior opening the file change much for > you? > > (setq org-modules nil) > (benchmark-run 1 (find-file "~/org/Plan.org")) Yes, it cuts the time in more than half: only 2.0 sec out of 5.1, in the unoptimized build of Emacs 29. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:22 ` Eli Zaretskii 2022-06-05 15:27 ` Eli Zaretskii 2022-06-05 15:41 ` Ihor Radchenko @ 2022-06-05 16:16 ` Tassilo Horn 2 siblings, 0 replies; 505+ messages in thread From: Tassilo Horn @ 2022-06-05 16:16 UTC (permalink / raw) To: Eli Zaretskii Cc: Ihor Radchenko, luangruo, michael.albinus, acm, stefan, akib, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> I've heard about the problem with loading multiple times and I do >> agree that loading can be slow, especially when various kinds of >> extra functionality is requested. However, I am wondering about the >> actual numbers. How long is long? > > This: > > (benchmark-run 1 (load "org")) > > takes around 1.7 sec here, including 1.1 sec used by GC. This is in > an unoptimized build of Emacs 29 on MS-Windows. Here it is "Elapsed time: 0.233381s (0.142206s in 12 GCs)" after emacs -Q with an unoptimized but default native-compiled emacs built from the current master on GNU/Linux (on a 8 years old notebook). Bye, Tassilo ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 15:15 ` Ihor Radchenko 2022-06-05 15:22 ` Eli Zaretskii @ 2022-06-05 17:40 ` Stefan Monnier 2022-06-08 12:51 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-05 17:40 UTC (permalink / raw) To: Ihor Radchenko Cc: Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Tassilo Horn, Akib Azmain Turja, Emacs developers > I've heard about the problem with loading multiple times and I do agree > that loading can be slow, especially when various kinds of extra > functionality is requested. However, I am wondering about the actual > numbers. How long is long? For me it's just the act of loading Org even for very plain org-mode files (plain enough that they fall within the common subset covered by `outline-mode`; the only reason I use org-mode for those files is because it highlight and "activates" URLs). I think it's fairly hard to quantify accurately: last time I tried to benchmark it, the numbers (around 1s) were much lower than what I was experiencing subjectively (I'd estimate 5s or more). So, AFAICT, the time to load Org mode is strongly affected by the size of the running session or maybe by the presence of various third party packages, which means benchmarking emacs --batch --eval '(require `org)' doesn't give a representative picture of the problem. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 17:40 ` Stefan Monnier @ 2022-06-08 12:51 ` Ihor Radchenko 2022-06-08 14:17 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-08 12:51 UTC (permalink / raw) To: Stefan Monnier Cc: Po Lu, Michael Albinus, Alan Mackenzie, Stefan Kangas, Tassilo Horn, Akib Azmain Turja, Emacs developers Stefan Monnier <monnier@iro.umontreal.ca> writes: >> I've heard about the problem with loading multiple times and I do agree >> that loading can be slow, especially when various kinds of extra >> functionality is requested. However, I am wondering about the actual >> numbers. How long is long? > > I think it's fairly hard to quantify accurately: last time I tried to > benchmark it, the numbers (around 1s) were much lower than what I was > experiencing subjectively (I'd estimate 5s or more). So, AFAICT, the > time to load Org mode is strongly affected by the size of the running > session or maybe by the presence of various third party packages, which > means benchmarking > > emacs --batch --eval '(require `org)' > > doesn't give a representative picture of the problem. I suspect that garbage collection is strongly affecting your loading times. It is taking half of the time even with clean Emacs. I imagine that GC sweeps are even more costly when you have a sizable allocated memory. Loading ol-* staff appears to generate a lot of GC runs because of mapconcats over all the links types with O(N_links^2) strings being allocated. I took a note of this. I am sure that memory usage can be improved on loading. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 12:51 ` Ihor Radchenko @ 2022-06-08 14:17 ` Eli Zaretskii 2022-06-08 14:38 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-06-08 14:17 UTC (permalink / raw) To: Ihor Radchenko Cc: monnier, luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Po Lu <luangruo@yahoo.com>, Michael Albinus <michael.albinus@gmx.de>, > Alan Mackenzie <acm@muc.de>, Stefan Kangas <stefan@marxist.se>, Tassilo > Horn <tsdh@gnu.org>, Akib Azmain Turja <akib@disroot.org>, Emacs > developers <emacs-devel@gnu.org> > Date: Wed, 08 Jun 2022 20:51:35 +0800 > > > emacs --batch --eval '(require `org)' > > > > doesn't give a representative picture of the problem. > > I suspect that garbage collection is strongly affecting your loading > times. It is taking half of the time even with clean Emacs. I imagine > that GC sweeps are even more costly when you have a sizable allocated > memory. Which, btw, yet another problem: why does loading Org produce so much garbage? Can this be kept in check somehow? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 14:17 ` Eli Zaretskii @ 2022-06-08 14:38 ` Ihor Radchenko 2022-06-08 14:43 ` Stefan Monnier 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-08 14:38 UTC (permalink / raw) To: Eli Zaretskii Cc: monnier, luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> I suspect that garbage collection is strongly affecting your loading >> times. It is taking half of the time even with clean Emacs. I imagine >> that GC sweeps are even more costly when you have a sizable allocated >> memory. > > Which, btw, yet another problem: why does loading Org produce so much > garbage? Can this be kept in check somehow? AFAIU, so much garbage is produced simply because nobody looked into memory usage. It is not the easiest thing to track in Emacs. Can it be kept in check? Maybe. We need to identify first which parts are taking excess memory. org-link-set-parameters appears to be one of the bottlenecks. I do not know any others. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 14:38 ` Ihor Radchenko @ 2022-06-08 14:43 ` Stefan Monnier 2022-06-08 15:44 ` Ihor Radchenko 2022-06-08 16:02 ` Eli Zaretskii 0 siblings, 2 replies; 505+ messages in thread From: Stefan Monnier @ 2022-06-08 14:43 UTC (permalink / raw) To: Ihor Radchenko Cc: Eli Zaretskii, luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel >> Which, btw, yet another problem: why does loading Org produce so much >> garbage? Can this be kept in check somehow? > > AFAIU, so much garbage is produced simply because nobody looked into > memory usage. It is not the easiest thing to track in Emacs. > > Can it be kept in check? Maybe. We need to identify first which parts > are taking excess memory. org-link-set-parameters appears to be one of > the bottlenecks. I do not know any others. w.r.t spending time in the GC, since the GC is run based on the amount of memory that has been allocated (as opposed to the amount of garbage generated, for example. which we sadly can't know in advance), the "mem" profiler (aka `M-x profilter-start RET mem RET`) can be very useful since it does a sampling-based profiling where the "time" is measured in number of byte allocated (or some approximation thereof). Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 14:43 ` Stefan Monnier @ 2022-06-08 15:44 ` Ihor Radchenko 2022-06-08 17:37 ` Stefan Monnier 2022-06-08 16:02 ` Eli Zaretskii 1 sibling, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-06-08 15:44 UTC (permalink / raw) To: Stefan Monnier Cc: Eli Zaretskii, luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >>> Which, btw, yet another problem: why does loading Org produce so much >>> garbage? Can this be kept in check somehow? >> >> AFAIU, so much garbage is produced simply because nobody looked into >> memory usage. It is not the easiest thing to track in Emacs. >> >> Can it be kept in check? Maybe. We need to identify first which parts >> are taking excess memory. org-link-set-parameters appears to be one of >> the bottlenecks. I do not know any others. > > w.r.t spending time in the GC, since the GC is run based on the amount > of memory that has been allocated (as opposed to the amount of garbage > generated, for example. which we sadly can't know in advance), the "mem" > profiler (aka `M-x profilter-start RET mem RET`) can be very useful > since it does a sampling-based profiling where the "time" is measured in > number of byte allocated (or some approximation thereof). I know the 'mem profiler. It can help in some cases. That's how I noticed org-link-set-parameter. However, here is one recent memory profiler report: 1,307,745,624 23% - org-persist-load 1,307,726,070 23% - org-persist-read 1,281,385,671 22% - seq-find 1,281,385,671 22% - seq-do 1,281,385,671 22% - mapc 1,281,385,671 22% - #<compiled 0x43febaeb8a58606> 1,281,385,671 22% - #<compiled -0x148951e07c95fd1a> 1,281,385,671 22% - run-hook-with-args-until-success 333,712 0% + org-element--cache-persist-before-read Unless I misinterpret things, run-hook-with-args-until-success is allocating memory. However, that hook contains a single function which is not allocating much according to the profiler. Moreover, I know myself that the actual memory allocation is done down in org-perist-read when running (read (current-buffer)). I got memory profiler wrong significant amount of time to learn not to trust it too much. cpu profiler is a bit more robust in this regard. At least not until I C-g in the middle of the profiling. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 15:44 ` Ihor Radchenko @ 2022-06-08 17:37 ` Stefan Monnier 2022-06-09 3:11 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-08 17:37 UTC (permalink / raw) To: Ihor Radchenko Cc: Eli Zaretskii, luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel > Moreover, I know myself that the actual memory allocation is done down > in org-perist-read when running (read (current-buffer)). I got memory > profiler wrong significant amount of time to learn not to trust it too > much. Indeed, the memory profiler is not very reliable. Some of that is a conscious implementation choice (e.g. we take samples when a whole new 16KB block is allocated to be divided into several smaller objects rather than for every object allocation). Other problems are things for which I don't have an explanation (and haven't spent the time to try and track them down). I've still found it useful to help figure out in which part of the code most of the allocation takes place, but you need to supplement it without your own knowledge of what the code does :-( Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 17:37 ` Stefan Monnier @ 2022-06-09 3:11 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-06-09 3:11 UTC (permalink / raw) To: Stefan Monnier Cc: Eli Zaretskii, luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > I've still found it useful to help figure out in which part of the code > most of the allocation takes place, but you need to supplement it > without your own knowledge of what the code does :-( Unfortunately, the profiler (both cpu and mem) is particularly useless when I try to measure loading time. For example, my last profiler run on Emacs-29 + native-comp yielded the following: Elapsed time: 0.505677s (0.295615s in 23 GCs) Memory: 32,661,616 98% - command-execute 32,378,659 97% - funcall-interactively 28,537,695 86% - eval-last-sexp 28,537,695 86% - elisp--eval-last-sexp 28,470,377 86% - progn 28,470,377 86% - let 28,469,646 86% - progn 26,927,800 81% - org-mode 18,457,233 55% - org-load-modules-maybe 18,113,241 54% + require 7,294,107 22% - byte-code 4,198,028 12% + require CPU: 251 79% - command-execute 230 72% - funcall-interactively 225 70% - eval-last-sexp 225 70% - elisp--eval-last-sexp 225 70% - progn 225 70% - let 225 70% - progn 218 68% - org-mode 158 49% - org-load-modules-maybe 148 46% - require 148 46% - byte-code 129 40% - require 129 40% - byte-code 119 37% - require 119 37% - byte-code Sure, loading appears to take a lot of time, but loading of which file? The profiler gives no answer whatsoever. I went as far as stripping all the requires from org.el and removing all but defun/defvar/defcustom statements. Still, the loading time was almost 0.3 sec with no easy way to tell which part is creating the bottleneck. In summary, I see no other way of speeding up Org loading than splitting org.el and other big files into modules and hoping for the best. A practical interim solution could be doing (require 'org) in init.el Given that Org mode is compiled and loaded on Emacs startup, the loading generally takes a 0.3-0.5 seconds, which is much better compared to 5.1 seconds reported by Eli. Best, Ihor ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-08 14:43 ` Stefan Monnier 2022-06-08 15:44 ` Ihor Radchenko @ 2022-06-08 16:02 ` Eli Zaretskii 1 sibling, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-06-08 16:02 UTC (permalink / raw) To: Stefan Monnier Cc: yantar92, luangruo, michael.albinus, acm, stefan, tsdh, akib, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Eli Zaretskii <eliz@gnu.org>, luangruo@yahoo.com, > michael.albinus@gmx.de, acm@muc.de, stefan@marxist.se, tsdh@gnu.org, > akib@disroot.org, emacs-devel@gnu.org > Date: Wed, 08 Jun 2022 10:43:44 -0400 > > w.r.t spending time in the GC, since the GC is run based on the amount > of memory that has been allocated (as opposed to the amount of garbage > generated, for example. which we sadly can't know in advance), the "mem" > profiler (aka `M-x profilter-start RET mem RET`) can be very useful > since it does a sampling-based profiling where the "time" is measured in > number of byte allocated (or some approximation thereof). Did you actually try this, and could indeed deduce where most of the garbage is produced? The problem with your idea, IME, is that Emacs calls memory-allocation functions from code that has nothing to do with consing Lisp objects, and it does that _a_lot_. So given a "memory" profile it is not very easy to see which of the parts are related to GC, and many times the hot spots aren't. But it doesn't hurt to try what you suggest, maybe we'd get a lucky break in this case. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 14:17 ` Alan Mackenzie 2022-06-04 14:31 ` Tassilo Horn 2022-06-04 16:09 ` Stefan Kangas @ 2022-06-04 17:32 ` Akib Azmain Turja 2 siblings, 0 replies; 505+ messages in thread From: Akib Azmain Turja @ 2022-06-04 17:32 UTC (permalink / raw) To: Alan Mackenzie, Tassilo Horn; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 2140 bytes --] Alan Mackenzie <acm@muc.de> writes: > Hello, Tassilo. > > On Sat, Jun 04, 2022 at 15:32:10 +0200, Tassilo Horn wrote: >> Akib Azmain Turja <akib@disroot.org> writes: > >> Hi Akib, > >> > Packages on ELPA or NonGNU ELPA with README.org show pretty >> > description both with describe-package and web browser. But after >> > installing a package, describe-package shows the Org source code, >> > which contains many useless things (I mean useless for showing in the >> > *Help* buffer). > >> > Isn't it possible to convert README.org to plain text README while >> > compiling the package, possibly optionally? Changing package--compile >> > function should do the job. > >> An alternative would be to enable (parts of) org-mode in the *Help* >> buffer. I've just tried `M-x describe-package RET corfu RET` which >> comes with a README.org and then enabling org-mode in the *Help* buffer. >> That looked really nice and didn't seem to cause bad effects except that >> the [back] / [forward] buttons stopped working. > > No, no, no, no! Org mode is a highly complicated, obscure mode which is > NOT part of core Emacs, and mustn't become so. What you're proposing is > a slippery slope, where ever greater portions of org mode would get > pushed into the core, causing ever greater problems for those who do not > use org mode. > > You pointed out one such problem yourself, org mode key bindings will > take up key binding space currently used by other modes and by users. Yeah, I don't also use Org mode very often, so enabling Org mode will be an even bigger problem to me. > > Surely the solution has got to be to encourage package authors to write > plain text (or .texi) documentation, by pointing out the difficulties > the non-standard .org format creates. Writing docs in Texinfo is obviously good, but I think its too much work for small packages. > >> Bye, >> Tassilo > > -- > Alan Mackenzie (Nuremberg, Germany). -- Akib Azmain Turja This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 13:32 ` Tassilo Horn 2022-06-04 14:17 ` Alan Mackenzie @ 2022-06-04 17:22 ` Akib Azmain Turja 1 sibling, 0 replies; 505+ messages in thread From: Akib Azmain Turja @ 2022-06-04 17:22 UTC (permalink / raw) To: Tassilo Horn; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1268 bytes --] Tassilo Horn <tsdh@gnu.org> writes: > Akib Azmain Turja <akib@disroot.org> writes: > > Hi Akib, > >> Packages on ELPA or NonGNU ELPA with README.org show pretty >> description both with describe-package and web browser. But after >> installing a package, describe-package shows the Org source code, >> which contains many useless things (I mean useless for showing in the >> *Help* buffer). >> >> Isn't it possible to convert README.org to plain text README while >> compiling the package, possibly optionally? Changing package--compile >> function should do the job. > > An alternative would be to enable (parts of) org-mode in the *Help* > buffer. I've just tried `M-x describe-package RET corfu RET` which > comes with a README.org and then enabling org-mode in the *Help* buffer. > That looked really nice and didn't seem to cause bad effects except that > the [back] / [forward] buttons stopped working. > > Bye, > Tassilo I don't see why enabling some features of Org mode in *Help* buffer is useful. IMHO writing a command to "find-file" the README is better than this. -- Akib Azmain Turja This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 13:14 Convert README.org to plain text README while installing package Akib Azmain Turja 2022-06-04 13:32 ` Tassilo Horn @ 2022-06-04 16:36 ` Stefan Monnier 2022-06-05 12:29 ` Akib Azmain Turja 1 sibling, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-04 16:36 UTC (permalink / raw) To: Akib Azmain Turja; +Cc: emacs-devel Akib Azmain Turja [2022-06-04 19:14:20] wrote: > Packages on ELPA or NonGNU ELPA with README.org show pretty description > both with describe-package and web browser. But after installing a > package, describe-package shows the Org source code, which contains > many useless things (I mean useless for showing in the *Help* buffer). > > Isn't it possible to convert README.org to plain text README while > compiling the package, possibly optionally? Actually, if the package is not yet installed, you should get this "README.org converted to plain text" (which we get by fetching it from `elpa.(non)gnu.org`, because it's distributed separately from the package itself). This is related to the problem that package tarballs do not come with a description of where we can find their README (even tho the site that packages it does know (and uses it in the webpage it generates)) :-( Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-04 16:36 ` Stefan Monnier @ 2022-06-05 12:29 ` Akib Azmain Turja 2022-06-05 13:34 ` Stefan Monnier 0 siblings, 1 reply; 505+ messages in thread From: Akib Azmain Turja @ 2022-06-05 12:29 UTC (permalink / raw) To: Stefan Monnier; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1377 bytes --] Stefan Monnier <monnier@iro.umontreal.ca> writes: > Akib Azmain Turja [2022-06-04 19:14:20] wrote: >> Packages on ELPA or NonGNU ELPA with README.org show pretty description >> both with describe-package and web browser. But after installing a >> package, describe-package shows the Org source code, which contains >> many useless things (I mean useless for showing in the *Help* buffer). >> >> Isn't it possible to convert README.org to plain text README while >> compiling the package, possibly optionally? > > Actually, if the package is not yet installed, you should get this > "README.org converted to plain text" (which we get by fetching it from > `elpa.(non)gnu.org`, because it's distributed separately from the > package itself). > > This is related to the problem that package tarballs do not come with > a description of where we can find their README (even tho the site that > packages it does know (and uses it in the webpage it generates)) :-( > > > Stefan > Yeah, you correctly understood my problem. But I think the convertion (or export) should be done while installation, not while packaging the package (which would probably be SaaSS), for the sake of freedom. -- Akib Azmain Turja This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 12:29 ` Akib Azmain Turja @ 2022-06-05 13:34 ` Stefan Monnier 2022-06-06 3:34 ` Akib Azmain Turja 0 siblings, 1 reply; 505+ messages in thread From: Stefan Monnier @ 2022-06-05 13:34 UTC (permalink / raw) To: Akib Azmain Turja; +Cc: emacs-devel > Yeah, you correctly understood my problem. But I think the convertion > (or export) should be done while installation, not while packaging the > package (which would probably be SaaSS), for the sake of freedom. The server (elpa.gnu.org) already has the READMEorg in its "converted to plain text form". It could simply add it into the tarball. It's not SaaSS. Stefan ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Convert README.org to plain text README while installing package 2022-06-05 13:34 ` Stefan Monnier @ 2022-06-06 3:34 ` Akib Azmain Turja 0 siblings, 0 replies; 505+ messages in thread From: Akib Azmain Turja @ 2022-06-06 3:34 UTC (permalink / raw) To: Stefan Monnier; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 728 bytes --] Stefan Monnier <monnier@iro.umontreal.ca> writes: >> Yeah, you correctly understood my problem. But I think the convertion >> (or export) should be done while installation, not while packaging the >> package (which would probably be SaaSS), for the sake of freedom. > > The server (elpa.gnu.org) already has the READMEorg in its "converted to > plain text form". It could simply add it into the tarball. > It's not SaaSS. > > > Stefan > If its not SaaSS then its the best solution, as it won't require to change client side (package.el) code. -- Akib Azmain Turja This message is signed by me with my GnuPG key. It's fingerprint is: 7001 8CE5 819F 17A3 BBA6 66AF E74F 0EFA 922A E7F5 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs @ 2022-09-25 2:52 Payas Relekar 2022-09-25 6:35 ` Bastien 0 siblings, 1 reply; 505+ messages in thread From: Payas Relekar @ 2022-09-25 2:52 UTC (permalink / raw) To: emacs-devel Hi Bastien, Bastien <bzg@gnu.org> writes: > Here is the email I sent when I accepted to switch to using .org as > the native format for Org documentation: > > https://list.orgmode.org/orgmode/87371gfas7.fsf@bzg.fr/ > > As you can read, I wanted to make the switch as an experiment to see > if we were really solving a problem here. > > I believe we didn't get more contributions to the manual by switching > to .org, so I'd be in favor of switching back to using .texi as the > native format for Org's documentation. (Not for 9.6, obviously, more > probably for 10.0 -- I'll discuss this with other Org maintainers.) From your mail, below were the motivators for change: - Let's stabilize editing standards around the org.org file. - Let's test org capabilities against a giant .org file. - Let's make `C-x 4 a' do something useful in an .org section. - Let's write more non-emacs parsers and exporters. - Let's see if we have more contributions to the manual and if we really solved a problem here. While you're best to judge the number of contributions, #1 and #2, or the dogfooding opportunities provided by the switch are immense. One doesn't occasionally run into org documents the size of org.org. It has already resulted in gc enhancements as it was slowing down Emacs build and was optimized. I'll say that alone is a benefit worth keeping. There is also more progress being made on non-emacs parsers[0][1], and perhaps we can reach out if they find org.org useful. Generally speaking, these projects are even smaller (number of contributors-wise) than org-mode, but a shared burden is always nicer. [0]: https://github.com/nvim-orgmode/orgmode [1]: https://github.com/200ok-ch/org-parser/blob/master/resources/org.ebnf > The other topics in this thread (make Org's Texinfo exporter provide > good .texi manuals, make Org more modular, etc.) are interesting, but > they are really separate questions IMHO. Thanks, Payas -- ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-25 2:52 Org mode and Emacs Payas Relekar @ 2022-09-25 6:35 ` Bastien 2022-09-25 7:05 ` Ihor Radchenko ` (2 more replies) 0 siblings, 3 replies; 505+ messages in thread From: Bastien @ 2022-09-25 6:35 UTC (permalink / raw) To: Payas Relekar; +Cc: emacs-devel Hi Payas, Payas Relekar <relekarpayas@gmail.com> writes: > From your mail, below were the motivators for change: Re-reading it, the list was a mix of (1) things to be done for the switch to be sensible (and that will be boosted if the switch happens) and (2) possible nice outcomes. > - Let's stabilize editing standards around the org.org file. (=> 1) Something that we did before the switch. > - Let's test org capabilities against a giant .org file. (=> 2) Yes, this can lead to enhancements in Org like it did recently, but I don't think this is a good reason enough to justify the switch. > - Let's make `C-x 4 a' do something useful in an .org section. (=> 1) Also something now available. > - Let's write more non-emacs parsers and exporters. (=> 2) This was an illusion: I don't think projects like Pandoc use the org-manual.org file to test whether they are good parsers and exporters. For a good reason: nobody really needs to use Pandoc for parsing/exporting the Org manual. > - Let's see if we have more contributions to the manual and if > we really solved a problem here. (=> 2) This didn't happen. > While you're best to judge the number of contributions, #1 and #2, or > the dogfooding opportunities provided by the switch are immense. > > One doesn't occasionally run into org documents the size of org.org. It > has already resulted in gc enhancements as it was slowing down Emacs > build and was optimized. I'll say that alone is a benefit worth keeping. I'm not convinced: slowing down Emacs build to create opportunities for enhancing the Org parser and exporter does not strike me as a good reason. Respecting the GNU standards about manuals ("The preferred document format for the GNU system is the Texinfo formatting language.") and the recent discussions provide good reasons for switching back to .texi, if all maintainers agree. > There is also more progress being made on non-emacs parsers[0][1], and > perhaps we can reach out if they find org.org useful. Generally > speaking, these projects are even smaller (number of contributors-wise) > than org-mode, but a shared burden is always nicer. > > [0]: https://github.com/nvim-orgmode/orgmode > [1]: https://github.com/200ok-ch/org-parser/blob/master/resources/org.ebnf Of course, feel free to contact them, but why would they want to try solving the challenge for exporting org-manual.org? They can create any .org file with the complexity they desire if they need it. Also, independently from this discussion, _we_ should certainly provide an example document containing alls things a parser/exporter should be able to handle. All best, -- Bastien ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-25 6:35 ` Bastien @ 2022-09-25 7:05 ` Ihor Radchenko 2022-09-25 7:47 ` Bastien 2022-09-25 8:01 ` Eli Zaretskii 2022-09-26 3:02 ` Ihor Radchenko 2 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2022-09-25 7:05 UTC (permalink / raw) To: Bastien; +Cc: Payas Relekar, emacs-devel Bastien <bzg@gnu.org> writes: >> While you're best to judge the number of contributions, #1 and #2, or >> the dogfooding opportunities provided by the switch are immense. >> >> One doesn't occasionally run into org documents the size of org.org. It >> has already resulted in gc enhancements as it was slowing down Emacs >> build and was optimized. I'll say that alone is a benefit worth keeping. > > I'm not convinced: slowing down Emacs build to create opportunities > for enhancing the Org parser and exporter does not strike me as a good > reason. As the outcome of that thread, I managed to reduce org->texinfo conversion down to 4sec. https://yhetil.org/emacs-devel/87czf9np98.fsf@localhost I do not think that the overall impact on the build process is something we need to worry about in this context. > Respecting the GNU standards about manuals ("The preferred document > format for the GNU system is the Texinfo formatting language.") and > the recent discussions provide good reasons for switching back to > .texi, if all maintainers agree. Do note that RMS expressed interest in changing this standard to Org, given that Org provides all the necessary tooling to produce high-quality manual. https://yhetil.org/emacs-devel/E1nzQh5-0001OB-22@fencepost.gnu.org I also spawned a thread about this in Org ML. https://list.orgmode.org/87k09frdsv.fsf@localhost/ -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-25 7:05 ` Ihor Radchenko @ 2022-09-25 7:47 ` Bastien 2022-09-25 8:01 ` Ihor Radchenko 2022-09-25 8:22 ` Bastien Guerry 0 siblings, 2 replies; 505+ messages in thread From: Bastien @ 2022-09-25 7:47 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Payas Relekar, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > As the outcome of that thread, I managed to reduce org->texinfo > conversion down to 4sec. https://yhetil.org/emacs-devel/87czf9np98.fsf@localhost > I do not think that the overall impact on the build process is something > we need to worry about in this context. Thanks a lot for working on this, this is a great achievement. To be fair, the impact on the build process was not really my main concern. My concern is that Org does not respect the GNU standards. >> Respecting the GNU standards about manuals ("The preferred document >> format for the GNU system is the Texinfo formatting language.") and >> the recent discussions provide good reasons for switching back to >> .texi, if all maintainers agree. > > Do note that RMS expressed interest in changing this standard to Org, > given that Org provides all the necessary tooling to produce > high-quality manual. > https://yhetil.org/emacs-devel/E1nzQh5-0001OB-22@fencepost.gnu.org I'm aware of this possibility, but there are many "if" :) Also, we can work toward this possibility without using .org for the Org manual. What do you think of my proposal to produce an example document, demonstrating the Org syntax and its possibilities? It would surely help us test our own code (parser and exporters) and help others write parsers/exporters (outside of Emacs). -- Bastien ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-25 7:47 ` Bastien @ 2022-09-25 8:01 ` Ihor Radchenko 2022-09-25 8:22 ` Bastien Guerry 1 sibling, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-09-25 8:01 UTC (permalink / raw) To: Bastien; +Cc: Payas Relekar, emacs-devel Bastien <bzg@gnu.org> writes: > To be fair, the impact on the build process was not really my main > concern. My concern is that Org does not respect the GNU standards. We do. Org does provide manual in Texinfo format. The fact that our texinfo version is auto-generated is irrelevant wrt GNU Standards. > What do you think of my proposal to produce an example document, > demonstrating the Org syntax and its possibilities? > > It would surely help us test our own code (parser and exporters) and > help others write parsers/exporters (outside of Emacs). See https://orgmode.org/list/87fsqzi4tw.fsf@localhost P.S. Note that I am trying to keep discussion relevant to this emacs-devel thread. More serious discussion about switching back from .org to .texinfo should probably happen at Org ML. -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-25 7:47 ` Bastien 2022-09-25 8:01 ` Ihor Radchenko @ 2022-09-25 8:22 ` Bastien Guerry 1 sibling, 0 replies; 505+ messages in thread From: Bastien Guerry @ 2022-09-25 8:22 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Payas Relekar, emacs-devel Bastien <bzg@gnu.org> writes: > What do you think of my proposal to produce an example document, > demonstrating the Org syntax and its possibilities? PS: I see that you raised a similar idea on the Org-mode mailing list, I will follow-up there. -- Bastien Guerry ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-25 6:35 ` Bastien 2022-09-25 7:05 ` Ihor Radchenko @ 2022-09-25 8:01 ` Eli Zaretskii 2022-09-25 19:47 ` Tim Cross 2022-09-26 3:02 ` Ihor Radchenko 2 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2022-09-25 8:01 UTC (permalink / raw) To: Bastien; +Cc: relekarpayas, emacs-devel > From: Bastien <bzg@gnu.org> > Cc: emacs-devel@gnu.org > Date: Sun, 25 Sep 2022 08:35:54 +0200 > > Respecting the GNU standards about manuals ("The preferred document > format for the GNU system is the Texinfo formatting language.") and > the recent discussions provide good reasons for switching back to > .texi, if all maintainers agree. I definitely agree (it makes it easier for me to contribute to the Org manual). But eventually it is the decision of the Org project. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-25 8:01 ` Eli Zaretskii @ 2022-09-25 19:47 ` Tim Cross 2022-09-26 6:12 ` Bastien 2022-09-26 12:10 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Tim Cross @ 2022-09-25 19:47 UTC (permalink / raw) To: emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Bastien <bzg@gnu.org> >> Cc: emacs-devel@gnu.org >> Date: Sun, 25 Sep 2022 08:35:54 +0200 >> >> Respecting the GNU standards about manuals ("The preferred document >> format for the GNU system is the Texinfo formatting language.") and >> the recent discussions provide good reasons for switching back to >> .texi, if all maintainers agree. > > I definitely agree (it makes it easier for me to contribute to the Org > manual). But eventually it is the decision of the Org project. and I would be exactly the opposite. I know and use org-mode. I don't know and have never used texinfo despite over 25 years of Emacs use. Having to learn another formatting solution just to contribute to the formatting solution I use isn't going to be encouraging. The question I wonder about is where are we most likely to get the majority of our contributions from, those who use org mode and know it or those who don't and for those who use org-mode, how many will know texinfo? I know one of the original justifications was to see if it improved contributions towards documentation and it appears this has not been the case. However, I do wonder about that - I have certainly seen numerous manual patches on the list and I wonder how many of those would not occur if the patcher also needed to know texinfo? There may also be other impediments which slows down contributions that are unrelated to the documentation format (I still find determining if something is a known issue or not and the state of progress to resolving it difficult to track - not a criticism of the core maintainers, who I believe do an incredible job. Real problem is the challenge of realising a better process given the very very few core contributors available - basically a resourcing challenge). At the end of the day, I think the dog food argument is important. Having the manual in org format has seen a number of improvements and does provide a good and most importantly large and used example. Having a sample document which developers could use to verify parsers etc would be a good addition, but the problem with such documents is they tend not to be maintained and are not actively used. There is huge value in having a large and reasonably complex document which is being actively updated/enhanced and which is used in the real world to produce documents in various formats which are also actively read and used. It tends to be in active use of generated documents we find more subtle issues, things which tend to be missed in cursory scans of test documents. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-25 19:47 ` Tim Cross @ 2022-09-26 6:12 ` Bastien 2022-09-26 6:35 ` Ihor Radchenko 2022-09-26 12:10 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Bastien @ 2022-09-26 6:12 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel Hi Tim, Tim Cross <theophilusx@gmail.com> writes: > The question I wonder about is where are we most likely to get the > majority of our contributions from, those who use org mode and know it > or those who don't and for those who use org-mode, how many will > know texinfo? Recruiting contributors for Org is also a way to recruit contributors for the GNU project in general, which uses Texinfo as its standard format for manuals. For occasional fixes, I don't think the difference between the .texi and .org format makes that much of a difference. For substantial contributions, it probably does: but contributors of these important changes are probably those for which this difference can easily be overcome -- and *should* be overcome, because they are also potential contributors for the GNU project. > (I still find determining if something is a > known issue or not and the state of progress to resolving it difficult > to track (FWIW I agree, that's the motivation behind my work on Woof!.) > Real problem is the challenge of realising a better > process given the very very few core contributors available - basically > a resourcing challenge). What we don't see so far is the contributors we lose because we use .org as the format for the manual: Eli is one and there are probably others. > At the end of the day, I think the dog food argument is > important. Having the manual in org format has seen a number of > improvements and does provide a good and most importantly large and used > example. Having a sample document which developers could use to verify > parsers etc would be a good addition, but the problem with such > documents is they tend not to be maintained and are not actively > used. There is huge value in having a large and reasonably complex > document which is being actively updated/enhanced and which is used in > the real world to produce documents in various formats which are also > actively read and used. It tends to be in active use of generated > documents we find more subtle issues, things which tend to be > missed in cursory scans of test documents. Full disclosure: the dog food argument never convinced me. Dog fooding /per se/ never makes any sense, unless you motivate it with another good reason. I suspect our (lispian?) brains is fascinated by recursive stuff (a rose is a rose is a rose) but this is something we should resist. Anyway, I won't insist on this anymore, the decision will be that of all Org core maintainers, of course. -- Bastien ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 6:12 ` Bastien @ 2022-09-26 6:35 ` Ihor Radchenko 2022-09-26 6:57 ` Bastien ` (2 more replies) 0 siblings, 3 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-09-26 6:35 UTC (permalink / raw) To: Bastien; +Cc: Tim Cross, emacs-devel Bastien <bzg@gnu.org> writes: > Hi Tim, > > Tim Cross <theophilusx@gmail.com> writes: > >> The question I wonder about is where are we most likely to get the >> majority of our contributions from, those who use org mode and know it >> or those who don't and for those who use org-mode, how many will >> know texinfo? > > Recruiting contributors for Org is also a way to recruit contributors > for the GNU project in general, which uses Texinfo as its standard > format for manuals. I am now more concerned about Org contributors. I am not sure if we have a luxury to push our contributors into GNU project by forcing them to learn texinfo in addition to Org (note that I do not say that we should not encourage people to contribute to GNU; just that we should not discourage people from becoming Org contributors in the first place). >> Real problem is the challenge of realising a better >> process given the very very few core contributors available - basically >> a resourcing challenge). > > What we don't see so far is the contributors we lose because we use > .org as the format for the manual: Eli is one and there are probably > others. I am wondering how many Emacs developers contributed to Org in the past. In particular, before we switched away from .texi manual. -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 6:35 ` Ihor Radchenko @ 2022-09-26 6:57 ` Bastien 2023-08-18 17:09 ` Ihor Radchenko 2022-09-26 8:24 ` Eli Zaretskii 2022-09-26 8:32 ` Jean Louis 2 siblings, 1 reply; 505+ messages in thread From: Bastien @ 2022-09-26 6:57 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Tim Cross, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > I am now more concerned about Org contributors. I am not sure if we have > a luxury to push our contributors into GNU project by forcing them to > learn texinfo in addition to Org (note that I do not say that we should > not encourage people to contribute to GNU; just that we should not > discourage people from becoming Org contributors in the first place). I will write to you off-list this week to try to find a way to define our strategy here (beyond the sole topic of the manual). I suggest we have an online event after the EmacsConf (Dec. 11th would be good) to include the community in our decisions about setting up a strategy for Org future maintenance and contributions. > I am wondering how many Emacs developers contributed to Org in the past. > In particular, before we switched away from .texi manual. My impression is that these contributions have been small in size of text modified but very useful and regular, at least as much useful as the occasional ones received from Org casual contributors. -- Bastien ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 6:57 ` Bastien @ 2023-08-18 17:09 ` Ihor Radchenko 2023-08-18 18:31 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-18 17:09 UTC (permalink / raw) To: Bastien; +Cc: Ihor Radchenko, Tim Cross, emacs-devel Bastien <bzg@gnu.org> writes: >> I am wondering how many Emacs developers contributed to Org in the past. >> In particular, before we switched away from .texi manual. > > My impression is that these contributions have been small in size of > text modified but very useful and regular, at least as much useful as > the occasional ones received from Org casual contributors. Hmm. I tried (in Emacs git repo), skipping contributions from Org maintainers: git shortlog -- doc/misc/org.texi (see below for shortened output) The overwhelming majority of the commits are (1) typos; (2) texinfo markup changes (not needed as long as we follow the conventions in ox-texinfo); (3) copyright year updates; (4) some common wording conventions, like using "GNU" or https in urls. This was since 2007. git shortlog -- doc/misc/org.org is since Feb 2021 (3 years) Eli Zaretskii (1): <copyright> Glenn Morris (3): <moving from texi to org source> <copyright> Hanno Perrey (1): ; * doc/misc/org.org: fix capture context example Stefan Kangas (14): <typos> Avoid treating number as an enum in the org manual Drop support for the dead third-party w3 package <merges> Recommend NonGNU ELPA over MELPA Delete broken link to Network Theory Ltd. ; Prefer HTTPS to HTTP in many URLs Total 19 commits from Emacs contributors. For the same 3 year period org.texi saw git shortlog --since 2018 -- doc/misc/org.texi 21 commits. As expected, we are no longer seeing texinfo markup fixes, except a single work around a bug in Emacs Info command. All other types of the contributions from Emacs devs are not stopping. I conclude that .org version of the manual did not cause reduced contributions from Emacs devs compared to .texi version. And I believe that automated export is useful to avoid dealing with texinfo issues in addition to our own .org issues we would have to address anyway. ------------------- git shortlog -- doc/misc/org.texi Alan Mackenzie (1): Expunge "allow" + infinitive without direct object from source and doc. Andreas Schwab (2): <2 commits fixing texinfo markup> Chong Yidong (3): * org.texi (Org Plot): Fix tags (Bug#3507). * org.texi (Workflow states, Agenda commands): Fix tags (Bug#3508). <1 merge> Eli Zaretskii (4): <3 commits fixing texinfo markup> <1 commit fixing typo> Glenn Morris (57): <several commits moving files around Emacs repo> Update Back-Cover Text as per maintain.info. Change to GFDL 1.2. Refer to license in Emacs manual. <typos> <copyright year updates> Peter Tury <tury.peter at gmail.com> (tiny change) Peter Tury <tury.peter at gmail.com> (tiny change) Nuke arch-tags. Standardize possessive apostrophe usage in manuals, docs, and comments Ref: http://lists.gnu.org/archive/html/emacs-devel/2012-02/msg00649.html <texinfo markup fixes> Standardize case of "Front-Cover Texts" in texi file permissions notices. <merges> Replace doc references to load-hooks Distribute the real source for some doc/misc manuals (bug#45143) Juanma Barranquero (7): <typos> Karl Berry (1): <texinfo markup fixes> Lars Ingebrigtsen (1): <texinfo markup fixes> Mario Lang (2): <typos> Martin Rudalics (1): <texinfo markup> Mauro Aranda (1): <texinfo markup> Michael Albinus (6): Document remote file name syntax change <texinfo markup> * doc/misc/org.texi (Installation): Fix clone commands. Miles Bader (2): <merges> Paul Eggert (54): <spelling> <copyright years> <texinfo markup> Simplify use of current-time and friends. Fix some 24-hour time stamps in documentation. Modernize usage of 'macOS' in doc and comments Prefer HTTPS to FTP and HTTP in documentation <merges> Update some URLs Radon Rosborough (1): Add early init file, stop package-initialize insertion Stefan Kangas (2): ; Prefer https to http in more URLs Avoid recommending Google Stefan Monnier (3): <merges> * src/print.c (syms_of_print) <print_quoted>: Set default to true Stephen Eglen (1): <typos> Ville Skyttä (1): <typos> -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-18 17:09 ` Ihor Radchenko @ 2023-08-18 18:31 ` Eli Zaretskii 2023-08-18 18:49 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2023-08-18 18:31 UTC (permalink / raw) To: Ihor Radchenko; +Cc: bzg, yantar92, theophilusx, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: Ihor Radchenko <yantar92@gmail.com>, Tim Cross <theophilusx@gmail.com>, > emacs-devel@gnu.org > Date: Fri, 18 Aug 2023 17:09:42 +0000 > > I conclude that .org version of the manual did not cause reduced > contributions from Emacs devs compared to .texi version. And I believe > that automated export is useful to avoid dealing with texinfo issues in > addition to our own .org issues we would have to address anyway. You are comparing negligibly small numbers of contributions, so any conclusions from such comparison are questionable at best. My personal conclusion is that Emacs developers don't contribute too much to Org's documentation because Emacs developers who are used to work on documentation seldom if ever use Org for any significant work. So they don't have significant changes to contribute, except fixing typos and markup. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-18 18:31 ` Eli Zaretskii @ 2023-08-18 18:49 ` Ihor Radchenko 2023-08-18 19:11 ` Eli Zaretskii 2023-08-23 2:12 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-08-18 18:49 UTC (permalink / raw) To: Eli Zaretskii; +Cc: bzg, yantar92, theophilusx, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > You are comparing negligibly small numbers of contributions, so any > conclusions from such comparison are questionable at best. > My personal conclusion is that Emacs developers don't contribute too > much to Org's documentation because Emacs developers who are used to > work on documentation seldom if ever use Org for any significant work. > So they don't have significant changes to contribute, except fixing > typos and markup. I think that I need to clarify about the purpose of my "analysis" (I know that data is not very conclusive, but that all we got). I was looking into the idea that we can get more help from Emacs contributors if we switch the format back from .org to .texi. (Mostly, out of curiosity; and because it was hanging in my TODO list) According to the above, I do not see that we can achieve such goal: 1. Absolute number of contributions to Org manual was never large (not by Org-only contrubutors) 2. Relative numbers, although hard to compare, also do not indicate that the switch would be helpful in any way. Having said that, RMS officially asked Org team to work towards allowing Org to become the new GNU documentation standard. https://list.orgmode.org/orgmode/87bkqx4jyg.fsf@localhost/ So, the above idea was rather theoretical. Unless we disregard RMS request, Org manual will be a good reference template to explore real-life caveats of using .org as the true documentation source. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-18 18:49 ` Ihor Radchenko @ 2023-08-18 19:11 ` Eli Zaretskii 2023-08-18 19:31 ` Ihor Radchenko 2023-08-23 2:12 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2023-08-18 19:11 UTC (permalink / raw) To: Ihor Radchenko; +Cc: bzg, yantar92, theophilusx, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: bzg@gnu.org, yantar92@gmail.com, theophilusx@gmail.com, emacs-devel@gnu.org > Date: Fri, 18 Aug 2023 18:49:53 +0000 > > I was looking into the idea that we can get more help from Emacs > contributors if we switch the format back from .org to .texi. I'd welcome such a change, FWIW. > Having said that, RMS officially asked Org team to work towards allowing > Org to become the new GNU documentation standard. > https://list.orgmode.org/orgmode/87bkqx4jyg.fsf@localhost/ > So, the above idea was rather theoretical. Unless we disregard RMS > request, Org manual will be a good reference template to explore > real-life caveats of using .org as the true documentation source. I don't know why Richard made that request. From my POV, switching from Texinfo to _any_ significantly different markup has the significant disadvantage that people who routinely improve our manuals will have to learn a new language, and that will likely lower their motivation. For better and for worse, Texinfo is our documentation system. It is a good system, and is actively developed (version 7.1 will be released soon), so abandoning it for something that is less familiar to those who take care of the documentation is a net loss, IMO. We have enough real problems on our hands to afford inventing new ones. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-18 19:11 ` Eli Zaretskii @ 2023-08-18 19:31 ` Ihor Radchenko 2023-08-19 5:51 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-18 19:31 UTC (permalink / raw) To: Eli Zaretskii; +Cc: bzg, yantar92, theophilusx, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> I was looking into the idea that we can get more help from Emacs >> contributors if we switch the format back from .org to .texi. > > I'd welcome such a change, FWIW. But how would Org benefit from it? For good reasons or not, we have already switched to .org format. And switching back now would require extra effort that needs to be justified. (Even without taking into account RMS' request). >> Having said that, RMS officially asked Org team to work towards allowing >> Org to become the new GNU documentation standard. >> https://list.orgmode.org/orgmode/87bkqx4jyg.fsf@localhost/ > ... > I don't know why Richard made that request. From my POV, switching > from Texinfo to _any_ significantly different markup has the > significant disadvantage that people who routinely improve our manuals > will have to learn a new language, and that will likely lower their > motivation. > > For better and for worse, Texinfo is our documentation system. It is > a good system, and is actively developed (version 7.1 will be released > soon), so abandoning it for something that is less familiar to those > who take care of the documentation is a net loss, IMO. We have enough > real problems on our hands to afford inventing new ones. Feel free to reply to RMS' email. I am sure that he had good reasons to make the official request. I am not in position to argue on his behalf. If you prefer, the same email is also on emacs-devel: https://yhetil.org/emacs-devel/E1ocmvz-0002iB-2M@fencepost.gnu.org/ -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-18 19:31 ` Ihor Radchenko @ 2023-08-19 5:51 ` Eli Zaretskii 2023-08-19 9:04 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2023-08-19 5:51 UTC (permalink / raw) To: Ihor Radchenko; +Cc: bzg, yantar92, theophilusx, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: bzg@gnu.org, yantar92@gmail.com, theophilusx@gmail.com, emacs-devel@gnu.org > Date: Fri, 18 Aug 2023 19:31:31 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> I was looking into the idea that we can get more help from Emacs > >> contributors if we switch the format back from .org to .texi. > > > > I'd welcome such a change, FWIW. > > But how would Org benefit from it? I explained that below this part in my email. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-19 5:51 ` Eli Zaretskii @ 2023-08-19 9:04 ` Ihor Radchenko 2023-08-19 9:26 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-19 9:04 UTC (permalink / raw) To: Eli Zaretskii; +Cc: bzg, yantar92, theophilusx, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> > I'd welcome such a change, FWIW. >> >> But how would Org benefit from it? > > I explained that below this part in my email. I thought that you were referring to RMS' request. > I don't know why Richard made that request. From my POV, switching > from Texinfo to _any_ significantly different markup has the > significant disadvantage that people who routinely improve our manuals > will have to learn a new language, and that will likely lower their > motivation. > For better and for worse, Texinfo is our documentation system. It is > a good system, and is actively developed (version 7.1 will be released > soon), so abandoning it for something that is less familiar to those > who take care of the documentation is a net loss, IMO. We have enough > real problems on our hands to afford inventing new ones. This may be true in general, but not for Org development. People contributing to the Org manual are expected to understand Org markup. So, it is Texinfo that becomes the new language to learn and that creates an extra contribution barrier. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-19 9:04 ` Ihor Radchenko @ 2023-08-19 9:26 ` Eli Zaretskii 2023-08-19 9:44 ` Ihor Radchenko 2023-08-21 1:12 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Eli Zaretskii @ 2023-08-19 9:26 UTC (permalink / raw) To: Ihor Radchenko; +Cc: bzg, yantar92, theophilusx, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: bzg@gnu.org, yantar92@gmail.com, theophilusx@gmail.com, emacs-devel@gnu.org > Date: Sat, 19 Aug 2023 09:04:38 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> > I'd welcome such a change, FWIW. > >> > >> But how would Org benefit from it? > > > > I explained that below this part in my email. > > I thought that you were referring to RMS' request. I was. But Richard reads this list, so he will see what I wrote and respond if he wants to. > > I don't know why Richard made that request. From my POV, switching > > from Texinfo to _any_ significantly different markup has the > > significant disadvantage that people who routinely improve our manuals > > will have to learn a new language, and that will likely lower their > > motivation. > > > For better and for worse, Texinfo is our documentation system. It is > > a good system, and is actively developed (version 7.1 will be released > > soon), so abandoning it for something that is less familiar to those > > who take care of the documentation is a net loss, IMO. We have enough > > real problems on our hands to afford inventing new ones. > > This may be true in general, but not for Org development. > People contributing to the Org manual are expected to understand Org > markup. So, it is Texinfo that becomes the new language to learn and > that creates an extra contribution barrier. If the idea is that the main contributors to the Org documentation are Org users, and the rest of Emacs developers don't really count for this purpose, then why did you count the contributions of the latter to begin with? they should be immaterial for you, AFAIU. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-19 9:26 ` Eli Zaretskii @ 2023-08-19 9:44 ` Ihor Radchenko 2023-08-19 10:19 ` Po Lu 2023-08-19 10:42 ` Eli Zaretskii 2023-08-21 1:12 ` Richard Stallman 1 sibling, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-08-19 9:44 UTC (permalink / raw) To: Eli Zaretskii; +Cc: bzg, yantar92, theophilusx, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> This may be true in general, but not for Org development. >> People contributing to the Org manual are expected to understand Org >> markup. So, it is Texinfo that becomes the new language to learn and >> that creates an extra contribution barrier. > > If the idea is that the main contributors to the Org documentation are > Org users, and the rest of Emacs developers don't really count for > this purpose, then why did you count the contributions of the latter > to begin with? they should be immaterial for you, AFAIU. They are not immaterial. The work contributed to Org manual by Emacs devs, even if small, is certainly appreciated. However, decision to switch back to .texi is a major one and will affect all the contributors, not just Emacs devs. Thus, I am trying to weight the overall impact on the _Org_ development and maintenance. I looked into git logs specifically to see the contribution pattern of Emacs developers and whether it was influenced by switching from .texi to .org. I only found that Texinfo markup-related contributions disappeared (naturally). Other contributions remained at their levels: typo fixes and enforcing Emacs-wide terminology and wording standards. Neither typo fixes nor enforcing certain terminology uses are limited by the fact that Org manual is not in .texi format. (Correct me if I am wrong). On the other hand, people contributing non-trivial patches for Org manual should naturally be familiar with Org mode. So, using Org markup should not be a handicap for them, while forcing texinfo may sometimes be. That's why, not disregarding Emacs dev contributions, I do not see what positive change switching back to .texi would bring to Org development. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-19 9:44 ` Ihor Radchenko @ 2023-08-19 10:19 ` Po Lu 2023-08-19 10:47 ` Eli Zaretskii 2023-08-19 10:48 ` Ihor Radchenko 2023-08-19 10:42 ` Eli Zaretskii 1 sibling, 2 replies; 505+ messages in thread From: Po Lu @ 2023-08-19 10:19 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Eli Zaretskii, bzg, yantar92, theophilusx, emacs-devel Ihor Radchenko <yantar92@posteo.net> writes: > Eli Zaretskii <eliz@gnu.org> writes: > >>> This may be true in general, but not for Org development. >>> People contributing to the Org manual are expected to understand Org >>> markup. So, it is Texinfo that becomes the new language to learn and >>> that creates an extra contribution barrier. >> >> If the idea is that the main contributors to the Org documentation are >> Org users, and the rest of Emacs developers don't really count for >> this purpose, then why did you count the contributions of the latter >> to begin with? they should be immaterial for you, AFAIU. > > They are not immaterial. The work contributed to Org manual by Emacs > devs, even if small, is certainly appreciated. However, decision to > switch back to .texi is a major one and will affect all the > contributors, not just Emacs devs. Thus, I am trying to weight the ^^^^^^^^^^^^^^^ Is Org not part of Emacs? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-19 10:19 ` Po Lu @ 2023-08-19 10:47 ` Eli Zaretskii 2023-08-19 10:48 ` Ihor Radchenko 1 sibling, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2023-08-19 10:47 UTC (permalink / raw) To: Po Lu; +Cc: yantar92, bzg, yantar92, theophilusx, emacs-devel > From: Po Lu <luangruo@yahoo.com> > Cc: Eli Zaretskii <eliz@gnu.org>, bzg@gnu.org, yantar92@gmail.com, > theophilusx@gmail.com, emacs-devel@gnu.org > Date: Sat, 19 Aug 2023 18:19:45 +0800 > > Ihor Radchenko <yantar92@posteo.net> writes: > > > They are not immaterial. The work contributed to Org manual by Emacs > > devs, even if small, is certainly appreciated. However, decision to > > switch back to .texi is a major one and will affect all the > > contributors, not just Emacs devs. Thus, I am trying to weight the > ^^^^^^^^^^^^^^^ > > Is Org not part of Emacs? Its development is separate. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-19 10:19 ` Po Lu 2023-08-19 10:47 ` Eli Zaretskii @ 2023-08-19 10:48 ` Ihor Radchenko 1 sibling, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-08-19 10:48 UTC (permalink / raw) To: Po Lu; +Cc: Eli Zaretskii, bzg, yantar92, theophilusx, emacs-devel Po Lu <luangruo@yahoo.com> writes: >> They are not immaterial. The work contributed to Org manual by Emacs >> devs, even if small, is certainly appreciated. However, decision to >> switch back to .texi is a major one and will affect all the >> contributors, not just Emacs devs. Thus, I am trying to weight the > ^^^^^^^^^^^^^^^ > > Is Org not part of Emacs? Org is maintained separately from Emacs, but distributed together with Emacs. That said, by "Emacs devs" I mostly meant experienced Elisp contributors who are not familiar with Org markup. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-19 9:44 ` Ihor Radchenko 2023-08-19 10:19 ` Po Lu @ 2023-08-19 10:42 ` Eli Zaretskii 2023-08-19 10:54 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2023-08-19 10:42 UTC (permalink / raw) To: Ihor Radchenko; +Cc: bzg, yantar92, theophilusx, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: bzg@gnu.org, yantar92@gmail.com, theophilusx@gmail.com, emacs-devel@gnu.org > Date: Sat, 19 Aug 2023 09:44:08 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > > If the idea is that the main contributors to the Org documentation are > > Org users, and the rest of Emacs developers don't really count for > > this purpose, then why did you count the contributions of the latter > > to begin with? they should be immaterial for you, AFAIU. > > They are not immaterial. The work contributed to Org manual by Emacs > devs, even if small, is certainly appreciated. However, decision to > switch back to .texi is a major one and will affect all the > contributors, not just Emacs devs. Thus, I am trying to weight the > overall impact on the _Org_ development and maintenance. > > I looked into git logs specifically to see the contribution pattern of > Emacs developers and whether it was influenced by switching from .texi > to .org. I only found that Texinfo markup-related contributions > disappeared (naturally). Other contributions remained at their levels: > typo fixes and enforcing Emacs-wide terminology and wording standards. > > Neither typo fixes nor enforcing certain terminology uses are limited by > the fact that Org manual is not in .texi format. (Correct me if I am > wrong). > > On the other hand, people contributing non-trivial patches for Org > manual should naturally be familiar with Org mode. So, using Org markup > should not be a handicap for them, while forcing texinfo may sometimes > be. > > That's why, not disregarding Emacs dev contributions, I do not see what > positive change switching back to .texi would bring to Org development. We are going in circles. I already explained, or tried to explain, what would be the positive change. It's your call, obviously. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-19 10:42 ` Eli Zaretskii @ 2023-08-19 10:54 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-08-19 10:54 UTC (permalink / raw) To: Eli Zaretskii; +Cc: bzg, yantar92, theophilusx, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> That's why, not disregarding Emacs dev contributions, I do not see what >> positive change switching back to .texi would bring to Org development. > > We are going in circles. I already explained, or tried to explain, > what would be the positive change. It's your call, obviously. I understand that. However, I do not fully agree with the arguments you used to support this statement. And I replied explaining why I do not fully agree. Let me know if I missed something you wanted to point out. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-19 9:26 ` Eli Zaretskii 2023-08-19 9:44 ` Ihor Radchenko @ 2023-08-21 1:12 ` Richard Stallman 2023-08-21 7:56 ` Philip Kaludercic 1 sibling, 1 reply; 505+ messages in thread From: Richard Stallman @ 2023-08-21 1:12 UTC (permalink / raw) To: Eli Zaretskii; +Cc: yantar92, bzg, yantar92, theophilusx, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > > I don't know why Richard made that request. I don't remember making that request. I must have done so, but I don't know why. Can someone please show me that message? Once i see its date, I can find the pertinent messages in my saved old mail, and think about this. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-21 1:12 ` Richard Stallman @ 2023-08-21 7:56 ` Philip Kaludercic 0 siblings, 0 replies; 505+ messages in thread From: Philip Kaludercic @ 2023-08-21 7:56 UTC (permalink / raw) To: Richard Stallman Cc: Eli Zaretskii, yantar92, bzg, yantar92, theophilusx, emacs-devel Richard Stallman <rms@gnu.org> writes: > [[[ To any NSA and FBI agents reading my email: please consider ]]] > [[[ whether defending the US Constitution against all enemies, ]]] > [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > > > > I don't know why Richard made that request. > > I don't remember making that request. I must have done so, but I > don't know why. > > Can someone please show me that message? Once i see its date, > I can find the pertinent messages in my saved old mail, and > think about this. These were the two messages referenced in the thread: https://yhetil.org/emacs-devel/E1ocmvz-0002iB-2M@fencepost.gnu.org/ https://list.orgmode.org/orgmode/87bkqx4jyg.fsf@localhost/ ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-18 18:49 ` Ihor Radchenko 2023-08-18 19:11 ` Eli Zaretskii @ 2023-08-23 2:12 ` Richard Stallman 2023-08-23 9:44 ` Ihor Radchenko 2023-08-24 11:52 ` Bastien Guerry 1 sibling, 2 replies; 505+ messages in thread From: Richard Stallman @ 2023-08-23 2:12 UTC (permalink / raw) To: Ihor Radchenko; +Cc: eliz, bzg, yantar92, theophilusx, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > Having said that, RMS officially asked Org team to work towards allowing > Org to become the new GNU documentation standard. > https://list.orgmode.org/orgmode/87bkqx4jyg.fsf@localhost/ The words quoted above can be interpreted in multiple ways, so when I read it the first time, it didn't seem like anything I would have sad. Now I see it does fit, when interpreted differently. My suggestion was not that we adopt the current Org format for GNU documentation. The current Org format doesn't have all the features that are needed -- it can't do the job that Texinfo does for us. But I think the Org format could be _extended_ into a format that would have all the features we need -- and then, with work, it could become superior to Texinfo. That would be an advance overall for documentation, and Org developers will appreciate that Org plays a role in it. The idea is based on a factual assumption that I think is true but it couldn't hurt to check. As I understand it, Org format is fits into a family of formats, also including Markup and Sphinx, which are similar in the basics. Because of this, I think a lot of people would find it easier to learn to use Org mode (including the extensions that will be needed) than Texinfo. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 2:12 ` Richard Stallman @ 2023-08-23 9:44 ` Ihor Radchenko 2023-08-23 11:01 ` Colin Baxter 2023-08-24 11:52 ` Bastien Guerry 1 sibling, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-23 9:44 UTC (permalink / raw) To: rms; +Cc: eliz, bzg, yantar92, theophilusx, emacs-devel Richard Stallman <rms@gnu.org> writes: > The idea is based on a factual assumption that I think is true > but it couldn't hurt to check. As I understand it, Org format is > fits into a family of formats, also including Markup and Sphinx, > which are similar in the basics. At least, Wikipedia classifies Org into "Lightweight Markup" family: https://en.wikipedia.org/wiki/Lightweight_markup_language -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 9:44 ` Ihor Radchenko @ 2023-08-23 11:01 ` Colin Baxter 2023-08-23 11:12 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Colin Baxter @ 2023-08-23 11:01 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, eliz, bzg, yantar92, theophilusx, emacs-devel >>>>> Ihor Radchenko <yantar92@posteo.net> writes: > Richard Stallman <rms@gnu.org> writes: >> The idea is based on a factual assumption that I think is true >> but it couldn't hurt to check. As I understand it, Org format is >> fits into a family of formats, also including Markup and Sphinx, >> which are similar in the basics. > At least, Wikipedia classifies Org into "Lightweight Markup" > family: https://en.wikipedia.org/wiki/Lightweight_markup_language "Lightweight Markup" appears to be the author's invention, with no reference or citation given. Colin Baxter. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 11:01 ` Colin Baxter @ 2023-08-23 11:12 ` Ihor Radchenko 2023-08-23 12:49 ` Po Lu 2023-08-23 16:53 ` Colin Baxter 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-08-23 11:12 UTC (permalink / raw) To: m43cap; +Cc: rms, eliz, bzg, yantar92, theophilusx, emacs-devel Colin Baxter <m43cap@yandex.com> writes: >>>>>> Ihor Radchenko <yantar92@posteo.net> writes: > > > Richard Stallman <rms@gnu.org> writes: > >> The idea is based on a factual assumption that I think is true > >> but it couldn't hurt to check. As I understand it, Org format is > >> fits into a family of formats, also including Markup and Sphinx, > >> which are similar in the basics. > > > At least, Wikipedia classifies Org into "Lightweight Markup" > > family: https://en.wikipedia.org/wiki/Lightweight_markup_language > > "Lightweight Markup" appears to be the author's invention, with no > reference or citation given. Why is it a problem? I have seen this term elsewhere as well. In any case, the key point is not the terminology, but the fact that other people independently grouped Org markup with multiple other markups that are widely used. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 11:12 ` Ihor Radchenko @ 2023-08-23 12:49 ` Po Lu 2023-08-23 17:18 ` Colin Baxter 2023-08-25 1:11 ` Richard Stallman 2023-08-23 16:53 ` Colin Baxter 1 sibling, 2 replies; 505+ messages in thread From: Po Lu @ 2023-08-23 12:49 UTC (permalink / raw) To: Ihor Radchenko; +Cc: m43cap, rms, eliz, bzg, yantar92, theophilusx, emacs-devel Ihor Radchenko <yantar92@posteo.net> writes: > Why is it a problem? I have seen this term elsewhere as well. > > In any case, the key point is not the terminology, but the fact that > other people independently grouped Org markup with multiple other > markups that are widely used. I may not be the individual best qualified to speak apropos this subject, but I can't imagine any reason learning Texinfo should be any more challenging than learning a version of Org extended to sufficiently address every one of Texinfo's applications. Annulling any hypothetical advantage -- should it exist -- is the significant body of GNU developers who are already proficient in Texinfo, many of whom bear no interest in Org. Richard, what makes you believe that? Can you provide any concrete examples of Texinfo constructs that would be simpler in Org? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 12:49 ` Po Lu @ 2023-08-23 17:18 ` Colin Baxter 2023-08-23 17:47 ` Ihor Radchenko 2023-08-25 1:11 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Colin Baxter @ 2023-08-23 17:18 UTC (permalink / raw) To: Po Lu; +Cc: Ihor Radchenko, rms, eliz, bzg, yantar92, theophilusx, emacs-devel >>>>> Po Lu <luangruo@yahoo.com> writes: > Ihor Radchenko <yantar92@posteo.net> writes: >> Why is it a problem? I have seen this term elsewhere as well. >> >> In any case, the key point is not the terminology, but the fact >> that other people independently grouped Org markup with multiple >> other markups that are widely used. > I may not be the individual best qualified to speak apropos this > subject, but I can't imagine any reason learning Texinfo should be > any more challenging than learning a version of Org extended to > sufficiently address every one of Texinfo's applications. > Annulling any hypothetical advantage -- should it exist -- is the > significant body of GNU developers who are already proficient in > Texinfo, many of whom bear no interest in Org. > Richard, what makes you believe that? Can you provide any > concrete examples of Texinfo constructs that would be simpler in > Org? Perhaps this ship sailed long ago but would removing org-mode from emacs core go some way towards making the choice between using texinfo or org-mode irrelevant? As an external package, org-mode's documentation could then evolve independently, as its users think best. Colin Baxter. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 17:18 ` Colin Baxter @ 2023-08-23 17:47 ` Ihor Radchenko 2023-08-23 18:02 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-23 17:47 UTC (permalink / raw) To: m43cap; +Cc: Po Lu, rms, eliz, bzg, yantar92, theophilusx, emacs-devel Colin Baxter <m43cap@yandex.com> writes: > > Richard, what makes you believe that? Can you provide any > > concrete examples of Texinfo constructs that would be simpler in > > Org? > > Perhaps this ship sailed long ago but would removing org-mode from emacs > core go some way towards making the choice between using texinfo or > org-mode irrelevant? As an external package, org-mode's documentation > could then evolve independently, as its users think best. IMHO, the problem with sources of Org manual is not that much of a big deal. Eli favours all the manuals to be in .texi format because it is standard. However, the current situation is certainly not _that_ unbearable to the level where we need to consider drastic measures like removing the whole Org from Emacs. Not even close. The far more important discussion here is about Org as potential replacement to Texinfo (after Org is modified to support software manual-specific markup). I am not an expert in Texinfo, so I cannot judge if it is a good call or not. On Texinfo side, there might be problems existing in Texinfo that will not exist in Org, as I recall from previous discussions. But I cannot tell any specifics. On Org side, Org allows fine-tuned configuration of export for each individual export target. And generating parts of the manuals programmatically. AFAIK, this is not easily possible in Texinfo. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 17:47 ` Ihor Radchenko @ 2023-08-23 18:02 ` Eli Zaretskii 2023-08-23 18:08 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2023-08-23 18:02 UTC (permalink / raw) To: Ihor Radchenko Cc: m43cap, luangruo, rms, bzg, yantar92, theophilusx, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: Po Lu <luangruo@yahoo.com>, rms@gnu.org, eliz@gnu.org, bzg@gnu.org, > yantar92@gmail.com, theophilusx@gmail.com, emacs-devel@gnu.org > Date: Wed, 23 Aug 2023 17:47:43 +0000 > > On Org side, Org allows fine-tuned configuration of export for each > individual export target. And generating parts of the manuals > programmatically. AFAIK, this is not easily possible in Texinfo. Yes, it is. At least the separate configuration part. See the section "Customization Variables" and its sub-sections in the Texinfo manual. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 18:02 ` Eli Zaretskii @ 2023-08-23 18:08 ` Ihor Radchenko 2023-08-23 18:18 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-23 18:08 UTC (permalink / raw) To: Eli Zaretskii Cc: m43cap, luangruo, rms, bzg, yantar92, theophilusx, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> On Org side, Org allows fine-tuned configuration of export for each >> individual export target. And generating parts of the manuals >> programmatically. AFAIK, this is not easily possible in Texinfo. > > Yes, it is. At least the separate configuration part. See the > section "Customization Variables" and its sub-sections in the Texinfo > manual. I did not mean variables affecting all the output. I meant more fine-grained staff, like direct html/latex/odt/... snippets only embedded when exporting to html/latex/odt/... For example, see https://orgmode.org/manual/Quoting-LaTeX-code.html or https://orgmode.org/manual/Quoting-HTML-tags.html -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 18:08 ` Ihor Radchenko @ 2023-08-23 18:18 ` Eli Zaretskii 2023-08-23 18:36 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2023-08-23 18:18 UTC (permalink / raw) To: Ihor Radchenko Cc: m43cap, luangruo, rms, bzg, yantar92, theophilusx, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: m43cap@yandex.com, luangruo@yahoo.com, rms@gnu.org, bzg@gnu.org, > yantar92@gmail.com, theophilusx@gmail.com, emacs-devel@gnu.org > Date: Wed, 23 Aug 2023 18:08:42 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> On Org side, Org allows fine-tuned configuration of export for each > >> individual export target. And generating parts of the manuals > >> programmatically. AFAIK, this is not easily possible in Texinfo. > > > > Yes, it is. At least the separate configuration part. See the > > section "Customization Variables" and its sub-sections in the Texinfo > > manual. > > I did not mean variables affecting all the output. I meant more > fine-grained staff, like direct html/latex/odt/... snippets only > embedded when exporting to html/latex/odt/... > > For example, see https://orgmode.org/manual/Quoting-LaTeX-code.html or > https://orgmode.org/manual/Quoting-HTML-tags.html Sorry, I don't understand what that means, even after looking at the URLs. "Direct snippets only embedded" I cannot parse, sorry. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 18:18 ` Eli Zaretskii @ 2023-08-23 18:36 ` Ihor Radchenko 2023-08-23 18:46 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-23 18:36 UTC (permalink / raw) To: Eli Zaretskii Cc: m43cap, luangruo, rms, bzg, yantar92, theophilusx, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> I did not mean variables affecting all the output. I meant more >> fine-grained staff, like direct html/latex/odt/... snippets only >> embedded when exporting to html/latex/odt/... >> >> For example, see https://orgmode.org/manual/Quoting-LaTeX-code.html or >> https://orgmode.org/manual/Quoting-HTML-tags.html > > Sorry, I don't understand what that means, even after looking at the > URLs. "Direct snippets only embedded" I cannot parse, sorry. For example, you can set a different color, for a single link, just when exporting to html. Or put a page break at certain place just in odt export. Or set image size differently in html and latex, for a single image. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 18:36 ` Ihor Radchenko @ 2023-08-23 18:46 ` Eli Zaretskii 2023-08-23 18:50 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2023-08-23 18:46 UTC (permalink / raw) To: Ihor Radchenko Cc: m43cap, luangruo, rms, bzg, yantar92, theophilusx, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: m43cap@yandex.com, luangruo@yahoo.com, rms@gnu.org, bzg@gnu.org, > yantar92@gmail.com, theophilusx@gmail.com, emacs-devel@gnu.org > Date: Wed, 23 Aug 2023 18:36:36 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> I did not mean variables affecting all the output. I meant more > >> fine-grained staff, like direct html/latex/odt/... snippets only > >> embedded when exporting to html/latex/odt/... > >> > >> For example, see https://orgmode.org/manual/Quoting-LaTeX-code.html or > >> https://orgmode.org/manual/Quoting-HTML-tags.html > > > > Sorry, I don't understand what that means, even after looking at the > > URLs. "Direct snippets only embedded" I cannot parse, sorry. > > For example, you can set a different color, for a single link, just when > exporting to html. Or put a page break at certain place just in odt > export. Or set image size differently in html and latex, for a single > image. Texinfo has @ifhtml..@end ifhtml blocks, where you can do stuff for HTML output only. And similar for all other formats; see "Conditional Commands" in the Texinfo manual. There's also @html..@end html, where you can write raw HTML (and similar for other formats; see "Raw Formatter Commands" in the Texinfo manual. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 18:46 ` Eli Zaretskii @ 2023-08-23 18:50 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-08-23 18:50 UTC (permalink / raw) To: Eli Zaretskii Cc: m43cap, luangruo, rms, bzg, yantar92, theophilusx, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> For example, you can set a different color, for a single link, just when >> exporting to html. Or put a page break at certain place just in odt >> export. Or set image size differently in html and latex, for a single >> image. > > Texinfo has @ifhtml..@end ifhtml blocks, where you can do stuff for > HTML output only. And similar for all other formats; see "Conditional > Commands" in the Texinfo manual. There's also @html..@end html, where > you can write raw HTML (and similar for other formats; see "Raw > Formatter Commands" in the Texinfo manual. This is indeed the same functionality. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 12:49 ` Po Lu 2023-08-23 17:18 ` Colin Baxter @ 2023-08-25 1:11 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Richard Stallman @ 2023-08-25 1:11 UTC (permalink / raw) To: Po Lu; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > Richard, what makes you believe that? Can you provide any concrete > examples of Texinfo constructs that would be simpler in Org? That's not what the issue is. Org format is similar to Markdown and I'd expect that millions of users use Markdown. Those millions of people would be willing to learn all the other details if those details were in a framework like Markdown. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 11:12 ` Ihor Radchenko 2023-08-23 12:49 ` Po Lu @ 2023-08-23 16:53 ` Colin Baxter 2023-08-23 17:56 ` Tassilo Horn 1 sibling, 1 reply; 505+ messages in thread From: Colin Baxter @ 2023-08-23 16:53 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, eliz, bzg, yantar92, theophilusx, emacs-devel >>>>> Ihor Radchenko <yantar92@posteo.net> writes: > Colin Baxter <m43cap@yandex.com> writes: >>>>>>> Ihor Radchenko <yantar92@posteo.net> writes: >> >> > Richard Stallman <rms@gnu.org> writes: >> The idea is based on >> a factual assumption that I think is true >> but it couldn't hurt >> to check. As I understand it, Org format is >> fits into a >> family of formats, also including Markup and Sphinx, >> which are >> similar in the basics. >> >> > At least, Wikipedia classifies Org into "Lightweight Markup" > >> family: https://en.wikipedia.org/wiki/Lightweight_markup_language >> >> "Lightweight Markup" appears to be the author's invention, with >> no reference or citation given. > Why is it a problem? I have seen this term elsewhere as well. Without a reference to consult, I do not understand what the author means by the term "lightweight". Colin Baxter. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 16:53 ` Colin Baxter @ 2023-08-23 17:56 ` Tassilo Horn 0 siblings, 0 replies; 505+ messages in thread From: Tassilo Horn @ 2023-08-23 17:56 UTC (permalink / raw) To: m43cap; +Cc: Ihor Radchenko, rms, eliz, bzg, yantar92, theophilusx, emacs-devel Colin Baxter <m43cap@yandex.com> writes: > >> "Lightweight Markup" appears to be the author's invention, with > >> no reference or citation given. > > > Why is it a problem? I have seen this term elsewhere as well. > > Without a reference to consult, I do not understand what the author > means by the term "lightweight". I think the common definition of lightweight markup is that it's still easily readable by humans. For example, HTML is pretty much unreadable without a browser rendering it. In contrast, Org and Markdown can be easily read as plain text without a special renderer or editor, e.g., with just "less". Bye, Tassilo ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-23 2:12 ` Richard Stallman 2023-08-23 9:44 ` Ihor Radchenko @ 2023-08-24 11:52 ` Bastien Guerry 2023-08-24 17:51 ` T.V Raman 2023-08-25 1:14 ` Richard Stallman 1 sibling, 2 replies; 505+ messages in thread From: Bastien Guerry @ 2023-08-24 11:52 UTC (permalink / raw) To: Richard Stallman; +Cc: Ihor Radchenko, eliz, yantar92, theophilusx, emacs-devel FWIW, I think it is good to improve the .org format and Org tools so that they are as good as .texi and Texinfo tools when it comes to writing and publishing technical documentation. Because most of Org syntax is easy to learn, this may help more developers enjoy writing good documentation. I don't think we need to consider replacing Texinfo by Org as the GNU documentat format, though. I understand such a possibility can be a motivation boost for Org developers, but it could also be a pitfall, inducing focus on the wrong priorities. Let's simply try to improve Org in general, and see if more GNU maintainers want to use it as their native documentat format (the example of Org's documentation shows it's already possible.) 2 cts, -- Bastien Guerry ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-24 11:52 ` Bastien Guerry @ 2023-08-24 17:51 ` T.V Raman 2023-08-24 17:55 ` Ihor Radchenko 2023-08-25 1:16 ` Richard Stallman 2023-08-25 1:14 ` Richard Stallman 1 sibling, 2 replies; 505+ messages in thread From: T.V Raman @ 2023-08-24 17:51 UTC (permalink / raw) To: Bastien Guerry Cc: Richard Stallman, Ihor Radchenko, eliz, yantar92, theophilusx, emacs-devel [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain; charset=gb18030, Size: 663 bytes --] I agree. I think if we find gaps, we should first try to fix this in the "export org to texi" code, and that will then reveal places where we indeed need to evolve org as a format. In fact, if we discover that org-to-texi cant express everything that texi can, we should first consider adding styling options to org-to-texi flow, and only if that fails, would we need to add new constructs to org. Off the top of my head, the only things I can think of that org may not presently cover is the ability to indicate index entries in the org source -- Thanks, --Raman(I Search, I Find, I Misplace, I Research) 7©4 Id: kg:/m/0285kf1 0Ü8 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-24 17:51 ` T.V Raman @ 2023-08-24 17:55 ` Ihor Radchenko 2023-08-24 18:35 ` T.V Raman 2023-08-25 1:16 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-24 17:55 UTC (permalink / raw) To: T.V Raman Cc: Bastien Guerry, Richard Stallman, eliz, yantar92, theophilusx, emacs-devel, Timothy "T.V Raman" <raman@google.com> writes: > Off the top of my head, the only things I can think of that org may not > presently cover is the ability to indicate index entries in the org > source For texi export, we allow #+cindex/#+findex/#+vindex. Also, Timothy has been working on https://github.com/tecosaur/org-glossary to bring it natively into Org. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-24 17:55 ` Ihor Radchenko @ 2023-08-24 18:35 ` T.V Raman 0 siblings, 0 replies; 505+ messages in thread From: T.V Raman @ 2023-08-24 18:35 UTC (permalink / raw) To: yantar92; +Cc: raman, bzg, rms, eliz, yantar92, theophilusx, emacs-devel, orgmode this is awesome to know -- glad we're already on it. Ihor Radchenko writes: > "T.V Raman" <raman@google.com> writes: > > > Off the top of my head, the only things I can think of that org may not > > presently cover is the ability to indicate index entries in the org > > source > > For texi export, we allow #+cindex/#+findex/#+vindex. > Also, Timothy has been working on > https://github.com/tecosaur/org-glossary to bring it natively into Org. > > -- > Ihor Radchenko // yantar92, > Org mode contributor, > Learn more about Org mode at <https://orgmode.org/>. > Support Org development at <https://liberapay.com/org-mode>, > or support my work at <https://liberapay.com/yantar92> -- Thanks, --Raman(I Search, I Find, I Misplace, I Research) ♉ Id: kg:/m/0285kf1 🦮 -- Thanks, --Raman(I Search, I Find, I Misplace, I Research) ♉ Id: kg:/m/0285kf1 🦮 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-24 17:51 ` T.V Raman 2023-08-24 17:55 ` Ihor Radchenko @ 2023-08-25 1:16 ` Richard Stallman 2023-08-25 11:45 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Richard Stallman @ 2023-08-25 1:16 UTC (permalink / raw) To: T.V Raman; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > Off the top of my head, the only things I can think of that org may not > presently cover is the ability to indicate index entries in the org > source How does one express these constructs in Org format? @code @samo @emph @def @var @i @url @key @kbd > In fact, if we discover that > org-to-texi cant express everything that texi can, we should first > consider adding styling options to org-to-texi flow, and only if that > fails, would we need to add new constructs to org. I am skeptical that that would address these issues, but I can't be sure since I don't know what that does. Can you please show a hypotherical example of solvin a problem this way? For instance, how would you propose to use them to distinguish between @var, @def and @emph? They look alike in TeX output. It is in OTHER formats that they produce different output. 1. What would that approach look like in the document source? I can't be sure -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-25 1:16 ` Richard Stallman @ 2023-08-25 11:45 ` Richard Stallman 0 siblings, 0 replies; 505+ messages in thread From: Richard Stallman @ 2023-08-25 11:45 UTC (permalink / raw) To: rms; +Cc: raman, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] Oops! I wrote > @code > @samo > @emph > @def > @var > @i > @url > @key > @kbd but it is not @def, it is @dfn. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-24 11:52 ` Bastien Guerry 2023-08-24 17:51 ` T.V Raman @ 2023-08-25 1:14 ` Richard Stallman 2023-08-25 9:04 ` Bastien Guerry 1 sibling, 1 reply; 505+ messages in thread From: Richard Stallman @ 2023-08-25 1:14 UTC (permalink / raw) To: Bastien Guerry; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > FWIW, I think it is good to improve the .org format and Org tools > so that they are as good as .texi and Texinfo tools when it comes > to writing and publishing technical documentation. The term "as good as" may suggest, incorrectly, that this is a matter of comparing the two formats over some sense of _quality_. But that's not what this is about. The improvements I've proposed for Org format are a matter of _supporting the range of necessary constructs_. Texinfo supports them all because I implemented them all in Texinfo. As a result, you can express in Texinfo everything that has proved necessary (so far, at least) in GNU documentation. At present, that is not the case with Org format. There are situations in which Org format provides no way to express what needs to be said. > Let's simply try to improve Org in general, and see if more GNU > maintainers want to use it as their native documentat format (the > example of Org's documentation shows it's already possible.) We need to be careful here. What does the existence of Org mode documentation written in Org format actually show -- given that the format doesn't support all the constructs that are needed in general? It might show that the Org mode documentation doesn't make all the textual distinctions properly -- that it fails to follow our style guide. If so, then it is "possible" but only with flawed output. But not necesarily. Perhaps it shows that the Org mode documentation needs only a limited subset of those constructs, and those are all implemened in Ogr format. If so, that could mean that Org format is fine for the Org mode manual in prticular, but is not adequate for the whole range of our documentation. Either way, to make Org format adequate for that whole range of constructs, in all the output formats, will require working specifically towards that goal. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-25 1:14 ` Richard Stallman @ 2023-08-25 9:04 ` Bastien Guerry 2023-08-25 18:56 ` Philip Kaludercic 2023-08-26 2:04 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Bastien Guerry @ 2023-08-25 9:04 UTC (permalink / raw) To: Richard Stallman; +Cc: emacs-devel Richard Stallman <rms@gnu.org> writes: > The term "as good as" may suggest, incorrectly, that this is a matter > of comparing the two formats over some sense of _quality_. But that's > not what this is about. The improvements I've proposed for Org format > are a matter of _supporting the range of necessary constructs_. I'm confident we can support the necessary constructs in Org. > > Let's simply try to improve Org in general, and see if more GNU > > maintainers want to use it as their native documentat format (the > > example of Org's documentation shows it's already possible.) > > We need to be careful here. What does the existence of Org mode > documentation written in Org format actually show -- given that the > format doesn't support all the constructs that are needed in general? > > It might show that the Org mode documentation doesn't make all the > textual distinctions properly -- that it fails to follow our style > guide. If so, then it is "possible" but only with flawed output. If a .texi expert can report such flaws in the Org manual, we can then fix them and, if needed, implement the necessary constructs. > But not necesarily. Perhaps it shows that the Org mode documentation > needs only a limited subset of those constructs, and those are all > implemened in Ogr format. If so, that could mean that Org format is > fine for the Org mode manual in prticular, but is not adequate for the > whole range of our documentation. I believe this is more plausible. > Either way, to make Org format adequate for that whole range of > constructs, in all the output formats, will require working > specifically towards that goal. Agreed, and this is what Org maintainers are working on. -- Bastien Guerry ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-25 9:04 ` Bastien Guerry @ 2023-08-25 18:56 ` Philip Kaludercic 2023-08-26 10:46 ` Ihor Radchenko 2023-08-26 2:04 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Philip Kaludercic @ 2023-08-25 18:56 UTC (permalink / raw) To: Bastien Guerry; +Cc: Richard Stallman, emacs-devel Bastien Guerry <bzg@gnu.org> writes: > Richard Stallman <rms@gnu.org> writes: > >> The term "as good as" may suggest, incorrectly, that this is a matter >> of comparing the two formats over some sense of _quality_. But that's >> not what this is about. The improvements I've proposed for Org format >> are a matter of _supporting the range of necessary constructs_. > > I'm confident we can support the necessary constructs in Org. > >> > Let's simply try to improve Org in general, and see if more GNU >> > maintainers want to use it as their native documentat format (the >> > example of Org's documentation shows it's already possible.) >> >> We need to be careful here. What does the existence of Org mode >> documentation written in Org format actually show -- given that the >> format doesn't support all the constructs that are needed in general? >> >> It might show that the Org mode documentation doesn't make all the >> textual distinctions properly -- that it fails to follow our style >> guide. If so, then it is "possible" but only with flawed output. > > If a .texi expert can report such flaws in the Org manual, we can then > fix them and, if needed, implement the necessary constructs. I don't know if this is of any use, but the initial manual for Compat (https://elpa.gnu.org/packages/compat.html) was written using Org and ox-texinfo and I later switched to writing .texi directly. This commit here documents the switch that includes a partial rewrite. https://git.sr.ht/~pkal/compat/commit/dd48603a136881a5321de4419be95ea873496172 Some things here might be difficult to map, such as the proper usage of reference macros or the different kinds of markup from (texinfo) Indicating. > >> But not necesarily. Perhaps it shows that the Org mode documentation >> needs only a limited subset of those constructs, and those are all >> implemened in Ogr format. If so, that could mean that Org format is >> fine for the Org mode manual in prticular, but is not adequate for the >> whole range of our documentation. > > I believe this is more plausible. > >> Either way, to make Org format adequate for that whole range of >> constructs, in all the output formats, will require working >> specifically towards that goal. > > Agreed, and this is what Org maintainers are working on. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-25 18:56 ` Philip Kaludercic @ 2023-08-26 10:46 ` Ihor Radchenko 2023-08-31 2:09 ` Richard Stallman 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-26 10:46 UTC (permalink / raw) To: Philip Kaludercic; +Cc: Bastien Guerry, Richard Stallman, emacs-devel Philip Kaludercic <philipk@posteo.net> writes: >> If a .texi expert can report such flaws in the Org manual, we can then >> fix them and, if needed, implement the necessary constructs. > > I don't know if this is of any use, but the initial manual for Compat > (https://elpa.gnu.org/packages/compat.html) was written using Org and > ox-texinfo and I later switched to writing .texi directly. This commit > here documents the switch that includes a partial rewrite. > > https://git.sr.ht/~pkal/compat/commit/dd48603a136881a5321de4419be95ea873496172 > > Some things here might be difficult to map, such as the proper usage of > reference macros or the different kinds of markup from (texinfo) Indicating. ox-texinfo certainly misses some specialalized Texinfo syntax, like @defn, @var, etc: doc/Documentation_Standards.org: + Texinfo commands such as @var and @defoption are not used. The preference for this type of thing is that the user browses the customize groups. If you want or need to refer to, say, a variable then document it as "the variable @code{org-startup-folded}" It is not export backend fault per se - Org markup simply does not define specialized markups more granular than ~code~. So, users have to use macros like {{{kbd(C-c SPC)}}} that expands to direct texinfo export snippet @@texinfo:@kbd{@@C-c SPC@@texinfo:}@@ Though we do not provide similar macros for @var/@env/etc. We might. Or we might allow custom inline special markup as I suggested in https://orgmode.org/list/87bkqx4jyg.fsf@localhost Another limitation, I can see in the commit is using @xref, which Org does not do. ox-texinfo now transforms <info:#node> links into @ref only. On the other hand, Org provides a powerful citation syntax [cite:see @citation-key], which may be also utilized if we support info files as bibliography source. This will be a superset of what @xref/@ref/@pxref does as cite/variant is an equivalent of \autocite in LaTeX. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-26 10:46 ` Ihor Radchenko @ 2023-08-31 2:09 ` Richard Stallman 2023-08-31 8:50 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Richard Stallman @ 2023-08-31 2:09 UTC (permalink / raw) To: Ihor Radchenko; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > It is not export backend fault per se - Org markup simply does not > define specialized markups more granular than ~code~. > So, users have to use macros like {{{kbd(C-c SPC)}}} that expands to > direct texinfo export snippet ... That source construct might be acceptable, though it has 6 extra characters and that would be rather inconvenient. Could you manage to reduce the number of them? Could you explain what is implied/presumed by "direct texinfo export"? Are you saying that Org would be a front-end for Texinfo, for all output formats? I'd hope it could generate Info files and HTML directly. > Though we do not provide similar macros for @var/@env/etc. We might. They are very often used. To make org adequate as a replacement for Texinfo, they would have to be predefined. (It is ok if the file has to specify loading some extension to define them.) > Or we might allow custom inline special markup as I suggested in > https://orgmode.org/list/87bkqx4jyg.fsf@localhost I skimmed that long message but I couldn't see any suggestion about @var, @env, etc. Could you please send me the actual proposal, by itself, rather than an indirection to it? > On the other hand, Org provides a powerful citation syntax > [cite:see @citation-key], which may be also utilized if we support > info files as bibliography source. This will be a superset of what > @xref/@ref/@pxref does as cite/variant is an equivalent of \autocite in > LaTeX. If that is smarter than the current Texinfo commands and can replace them with no loss of flexibility and capability, that would be fine. Provided it handles all output formats correctly. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-31 2:09 ` Richard Stallman @ 2023-08-31 8:50 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-08-31 8:50 UTC (permalink / raw) To: rms; +Cc: emacs-devel Richard Stallman <rms@gnu.org> writes: > > It is not export backend fault per se - Org markup simply does not > > define specialized markups more granular than ~code~. > > So, users have to use macros like {{{kbd(C-c SPC)}}} that expands to > > direct texinfo export snippet ... > > That source construct might be acceptable, though it has 6 extra characters > and that would be rather inconvenient. Could you manage to reduce the number > of them? > Could you explain what is implied/presumed by "direct texinfo export"? > Are you saying that Org would be a front-end for Texinfo, for all > output formats? I'd hope it could generate Info files and HTML directly. To clarify, I was describing what already exists in Org. We leverage Texinfo itself as intermediate format to export to Info. The above "kbd" macro is specifically designed for Texinfo export. We may eventually have an option to export to Info directly, but not yet. Though nothing stops people from implementing such functionality. Adding new export Backends is not very hard: https://orgmode.org/worg/dev/org-export-reference.html#back-end > > Though we do not provide similar macros for @var/@env/etc. We might. > > They are very often used. To make org adequate as a replacement for > Texinfo, they would have to be predefined. (It is ok if the file > has to specify loading some extension to define them.) > > > Or we might allow custom inline special markup as I suggested in > > https://orgmode.org/list/87bkqx4jyg.fsf@localhost > > I skimmed that long message but I couldn't see any suggestion about > @var, @env, etc. Could you please send me the actual proposal, by > itself, rather than an indirection to it? For more complete support of software manual-specific markup, we plan to introduce more versatile, hackable markup that can be extended by users to fit their needs. See https://list.orgmode.org/orgmode/87a6b8pbhg.fsf@posteo.net/ and https://list.orgmode.org/orgmode/87mtaez8do.fsf@localhost/ One of the ideas was to follow TeXinfo markup closely, extending with some Org-specific needs: Simple case: @name{<contents>} If we need to escape { or } inside: @name{{<contents}} Extra configuration: @name[:key value ...]{<contents>} @color[:name red]{[[file:image.png][description]]} Then, users or packages can define the details of how to fontify or export a given @name{...} markup. That way, we can add anything, including @var/@env/... We already do something similar for links: https://orgmode.org/manual/Adding-Hyperlink-Types.html (org-man-export), except that links have limitations that prevent them from being used as markup. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-25 9:04 ` Bastien Guerry 2023-08-25 18:56 ` Philip Kaludercic @ 2023-08-26 2:04 ` Richard Stallman 2023-08-26 5:50 ` Eli Zaretskii 1 sibling, 1 reply; 505+ messages in thread From: Richard Stallman @ 2023-08-26 2:04 UTC (permalink / raw) To: Bastien Guerry; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > It might show that the Org mode documentation doesn't make all the > > textual distinctions properly -- that it fails to follow our style > > guide. If so, then it is "possible" but only with flawed output. > If a .texi expert can report such flaws in the Org manual, we can then > fix them and, if needed, implement the necessary constructs. Please look at the list I sent yesterday. (See below.) Can Org mode handle each of them, and generate from it the correct output for each output format? > @code > @samo > @emph > @dfn > @var > @i > @url > @key > @kbd If it can, the next thing to do is to look through the node in the Texinfo manual which documents these, and see what other constructs are in that node, and see if Org mode has equivalent constructs for all of those Texinfo constructs. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-26 2:04 ` Richard Stallman @ 2023-08-26 5:50 ` Eli Zaretskii 2023-08-26 16:49 ` Jose E. Marchesi 2023-08-27 1:33 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Eli Zaretskii @ 2023-08-26 5:50 UTC (permalink / raw) To: rms; +Cc: bzg, emacs-devel > From: Richard Stallman <rms@gnu.org> > Cc: emacs-devel@gnu.org > Date: Fri, 25 Aug 2023 22:04:13 -0400 > > Please look at the list I sent yesterday. (See below.) Can Org mode > handle each of them, and generate from it the correct output for each > output format? > > > @code > > @samo > > @emph > > @dfn > > @var > > @i > > @url > > @key > > @kbd I think it can, but a typical Texinfo manual these days uses many more commands, not just those. So this list is insufficient to judge these issues. The actual list is much longer. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-26 5:50 ` Eli Zaretskii @ 2023-08-26 16:49 ` Jose E. Marchesi 2023-08-26 16:56 ` Ihor Radchenko 2023-08-27 1:32 ` Richard Stallman 2023-08-27 1:33 ` Richard Stallman 1 sibling, 2 replies; 505+ messages in thread From: Jose E. Marchesi @ 2023-08-26 16:49 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rms, bzg, emacs-devel >> From: Richard Stallman <rms@gnu.org> >> Cc: emacs-devel@gnu.org >> Date: Fri, 25 Aug 2023 22:04:13 -0400 >> >> Please look at the list I sent yesterday. (See below.) Can Org mode >> handle each of them, and generate from it the correct output for each >> output format? >> >> > @code >> > @samo >> > @emph >> > @dfn >> > @var >> > @i >> > @url >> > @key >> > @kbd > > I think it can, but a typical Texinfo manual these days uses many more > commands, not just those. So this list is insufficient to judge these > issues. The actual list is much longer. I think the point here is not just these particular marks, or some other particular set. It is that "markdown-style" formats, like org-mode, are not very good supporting "inline" marking. For inline marks, Texinfo uses a conventient generic and extensible form: @NAME{CONTENT}. "markdown-style" formats, on the contrary, generally support a closed set of marks that use delimited based ad-hoc syntax, such as *foo*, =foo=, "foo", `foo' and so on. Block marks are usually supported well enough by these formats. In the case of org it would be #+BEGIN_WHATEVER end #+END_WHATEVER I suppose. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-26 16:49 ` Jose E. Marchesi @ 2023-08-26 16:56 ` Ihor Radchenko 2023-08-26 20:28 ` Philip Kaludercic ` (2 more replies) 2023-08-27 1:32 ` Richard Stallman 1 sibling, 3 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-08-26 16:56 UTC (permalink / raw) To: Jose E. Marchesi; +Cc: Eli Zaretskii, rms, bzg, emacs-devel "Jose E. Marchesi" <jemarch@gnu.org> writes: > I think the point here is not just these particular marks, or some other > particular set. It is that "markdown-style" formats, like org-mode, are > not very good supporting "inline" marking. > > For inline marks, Texinfo uses a conventient generic and extensible > form: @NAME{CONTENT}. That's what we plan to add to Org. Not just because of this discussion; there are other reasons why we want such custom inline markup. > Block marks are usually supported well enough by these formats. In the > case of org it would be #+BEGIN_WHATEVER end #+END_WHATEVER I suppose. Yup. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-26 16:56 ` Ihor Radchenko @ 2023-08-26 20:28 ` Philip Kaludercic 2023-08-26 20:58 ` Ihor Radchenko 2023-08-27 1:32 ` Richard Stallman 2023-08-30 8:11 ` Bastien Guerry 2 siblings, 1 reply; 505+ messages in thread From: Philip Kaludercic @ 2023-08-26 20:28 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Jose E. Marchesi, Eli Zaretskii, rms, bzg, emacs-devel Ihor Radchenko <yantar92@posteo.net> writes: > "Jose E. Marchesi" <jemarch@gnu.org> writes: > >> I think the point here is not just these particular marks, or some other >> particular set. It is that "markdown-style" formats, like org-mode, are >> not very good supporting "inline" marking. >> >> For inline marks, Texinfo uses a conventient generic and extensible >> form: @NAME{CONTENT}. > > That's what we plan to add to Org. Not just because of this discussion; > there are other reasons why we want such custom inline markup. Do you have a reference where I could read up on how this is going? >> Block marks are usually supported well enough by these formats. In the >> case of org it would be #+BEGIN_WHATEVER end #+END_WHATEVER I suppose. > > Yup. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-26 20:28 ` Philip Kaludercic @ 2023-08-26 20:58 ` Ihor Radchenko 2023-08-30 8:11 ` Bastien Guerry 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-26 20:58 UTC (permalink / raw) To: Philip Kaludercic; +Cc: Jose E. Marchesi, Eli Zaretskii, rms, bzg, emacs-devel Philip Kaludercic <philipk@posteo.net> writes: >>> For inline marks, Texinfo uses a conventient generic and extensible >>> form: @NAME{CONTENT}. >> >> That's what we plan to add to Org. Not just because of this discussion; >> there are other reasons why we want such custom inline markup. > > Do you have a reference where I could read up on how this is going? https://list.orgmode.org/orgmode/87a6b8pbhg.fsf@posteo.net/ -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-26 20:58 ` Ihor Radchenko @ 2023-08-30 8:11 ` Bastien Guerry 0 siblings, 0 replies; 505+ messages in thread From: Bastien Guerry @ 2023-08-30 8:11 UTC (permalink / raw) To: Ihor Radchenko Cc: Philip Kaludercic, Jose E. Marchesi, Eli Zaretskii, rms, emacs-devel Ihor Radchenko <yantar92@posteo.net> writes: > https://list.orgmode.org/orgmode/87a6b8pbhg.fsf@posteo.net/ OK, forget my previous request, I'll read from here. -- Bastien Guerry ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-26 16:56 ` Ihor Radchenko 2023-08-26 20:28 ` Philip Kaludercic @ 2023-08-27 1:32 ` Richard Stallman 2023-08-27 8:32 ` Ihor Radchenko 2023-08-30 8:11 ` Bastien Guerry 2 siblings, 1 reply; 505+ messages in thread From: Richard Stallman @ 2023-08-27 1:32 UTC (permalink / raw) To: Ihor Radchenko; +Cc: jemarch, eliz, bzg, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > Block marks are usually supported well enough by these formats. In the > > case of org it would be #+BEGIN_WHATEVER end #+END_WHATEVER I suppose. That syntax would do the job, but they are more clumsy than the syntax used by Texinfo: @WHATEVER and @end WHATEVER. In order for a modified Org mode syntax to be a step up, it should look as clean as Texinfo. Might it be posisble to simplify the syntax that Org would use for this? -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-27 1:32 ` Richard Stallman @ 2023-08-27 8:32 ` Ihor Radchenko 2023-08-28 1:32 ` Richard Stallman 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-27 8:32 UTC (permalink / raw) To: rms; +Cc: jemarch, eliz, bzg, emacs-devel Richard Stallman <rms@gnu.org> writes: > > > Block marks are usually supported well enough by these formats. In the > > > case of org it would be #+BEGIN_WHATEVER end #+END_WHATEVER I suppose. > > That syntax would do the job, but they are more clumsy than the syntax > used by Texinfo: @WHATEVER and @end WHATEVER. In order for a modified > Org mode syntax to be a step up, it should look as clean as Texinfo. > > Might it be posisble to simplify the syntax that Org would use for > this? I am not convinced that it is necessary. Doing so will either be backward-incompatible or introduce more syntax constructs to Org, making the syntax more complicated. And I do not find the current syntax "clumsy". -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-27 8:32 ` Ihor Radchenko @ 2023-08-28 1:32 ` Richard Stallman 2023-08-29 8:29 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Richard Stallman @ 2023-08-28 1:32 UTC (permalink / raw) To: Ihor Radchenko; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > > > Block marks are usually supported well enough by these formats. In the > > > > case of org it would be #+BEGIN_WHATEVER end #+END_WHATEVER I suppose. > > > > That syntax would do the job, but they are more clumsy than the syntax > > used by Texinfo: @WHATEVER and @end WHATEVER. In order for a modified > > Org mode syntax to be a step up, it should look as clean as Texinfo. > > > > Might it be posisble to simplify the syntax that Org would use for > > this? > I am not convinced that it is necessary. I won't claim it is _necessary_, but it would sure make these constructs more convenient. > Doing so will either be backward-incompatible or introduce more syntax > constructs to Org, making the syntax more complicated. Maybe it is ok for it to be backward-incompatible. I don't know for certain, but it looks like these constructs were recently added. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-28 1:32 ` Richard Stallman @ 2023-08-29 8:29 ` Ihor Radchenko 2023-09-01 1:18 ` Richard Stallman 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-29 8:29 UTC (permalink / raw) To: rms; +Cc: emacs-devel Richard Stallman <rms@gnu.org> writes: > > > That syntax would do the job, but they are more clumsy than the syntax > > > used by Texinfo: @WHATEVER and @end WHATEVER. In order for a modified > > > Org mode syntax to be a step up, it should look as clean as Texinfo. > > > > > > Might it be posisble to simplify the syntax that Org would use for > > > this? > > > I am not convinced that it is necessary. > > I won't claim it is _necessary_, but it would sure make these constructs > more convenient. Maybe. But also more confusing. In org, we can have #+begin_src latex ... #+end_src latex or #+begin_export latex ... #+end_export or #+begin_example latex ... #+end_example or #+begin: name ... #+end: If we add simplified syntax like #+latex ... #+end latex or, less consistently with the rest of Org @latex ... @end latex it is actually not very intuitive that such construct implies export, but not something else. In addition, we already have #+latex: latex one-liner as a simple form, limited to a single line that avoids #+begin_export latex latex one-liner #+end_export -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-29 8:29 ` Ihor Radchenko @ 2023-09-01 1:18 ` Richard Stallman 2023-09-01 6:46 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Richard Stallman @ 2023-09-01 1:18 UTC (permalink / raw) To: Ihor Radchenko; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > or, less consistently with the rest of Org > @latex > ... > @end latex > it is actually not very intuitive that such construct implies export, > but not something else. I'd like to understand this issue. Could you please explain what "export" means here? And what is the "something else", that it might have been? It appears to be a distinction that doesn't exist in Texinfo. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-01 1:18 ` Richard Stallman @ 2023-09-01 6:46 ` Ihor Radchenko 2023-09-04 1:32 ` Richard Stallman 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-09-01 6:46 UTC (permalink / raw) To: rms; +Cc: emacs-devel Richard Stallman <rms@gnu.org> writes: > > @latex > > ... > > @end latex > > > it is actually not very intuitive that such construct implies export, > > but not something else. > > I'd like to understand this issue. Could you please explain what > "export" means here? And what is the "something else", that it might > have been? It appears to be a distinction that doesn't exist in > Texinfo. You are right, Texinfo has a very narrow focus - authoring manuals in multiple output formats. Org does a lot more. ----- One feature of Org we are discussing here is "exporting": https://orgmode.org/manual/Exporting.html - an ability to convert Org documents into other textual formats. Similar to Texinfo's raw formatter commands (https://www.gnu.org/software/texinfo/manual/texinfo/html_node/Raw-Formatter-Commands.html), Org provides a special markup that specifies text to be placed into specific tex/html/odt/etc export output: #+begin_export html ... #+end_export or #+HTML: <single line> or @@html:<inline html>@@ See https://orgmode.org/manual/Quoting-HTML-tags.html, for example. ----- However, Org has more features, served by other Org's markup. ----- One of the popular features is implementing literal programming (babel). https://orgmode.org/manual/Features-Overview.html: Org can manage the source code in the block delimited by ‘#+BEGIN_SRC’ … ‘#+END_SRC’ in several ways that can simplify housekeeping tasks essential to modern source code maintenance. Org can edit, format, extract, export, and publish source code blocks. #+begin_src emacs-lisp :results value (+ 1 2) #+end_src #+RESULTS[b6b278abd782307b5eca9b1b761906bae5292bcc]: : 3 Users can execute arbitrary pieces of source code in arbitrary programming languages, get the results inline and even mix input/output of code written in different languages: #+name: bash-input #+begin_src bash :results output table :var file="file.csv" cat $file #+end_src #+begin_src elisp :var table=bash-input(file="table-input.csv") (mapcar (lambda (line) (format "%S" line)) table) #+end_src The code evaluation results can even be used to generate exported text. Here is a slightly edited snippet from Org manual that generates a table of all export customizations: #+begin_src emacs-lisp :exports results :results table :eval yes (require 'ox) (let ((alist org-export-options-alist)) (let (result) (dolist (spec alist) (when (and (nth 3 spec) (symbolp (nth 3 spec))) ; has customization (push (list (format "~%s~" (car spec)) (format "~%s~" (nth 3 spec))) result))) (nreverse result))) #+end_src You can see the output in https://orgmode.org/manual/Publishing-options.html Furthermore, users can "tangle" code from the Org document, extracting only the appropriate source code pieces into actual code-only files. https://orgmode.org/manual/Extracting-Source-Code.html So, one can keep documentation and notes (with markup!) together with the actual code, tangle the org file to produce the program, or export the org file to produce readable notes - full literal programming implementation. ----- Apart from snippets of source code, Org has a markup for non-code literal text that is not a subject of Org markup: #+begin_example Some example from a text file. #+end_example Some people even prefer to keep snippets of source code as example blocks as well (when a sole purpose of such source block is exporting): #+begin_example bash ls $HOME #+end_example ----- Finally, there are dynamic blocks that can be used to generate parts of Org file programatically from Elisp: https://orgmode.org/manual/Dynamic-Blocks.html #+begin: blockname <parameters> <automatically generated text> #+end: One example of built-in dynamic block is clocktable that is used by people that record their work time using Org mode (https://orgmode.org/manual/Clocking-Work-Time.html, https://orgmode.org/manual/The-clock-table.html, https://olddeuteronomy.github.io/post/org-export-clocktables/) ----- To conclude, the common pattern for Org's block markup is #+begin<something> ... #+end<something> What you proposed with @latex: ... @end is similar to dropping <something> part. However, as you see in the above, there is a variety of different block markups serving for entirely different purposes and just saying @latex (or rather #+latex, if we follow the common Org block convention) does make it intuitive whether the block is "export", or "example" or "src" or dynamic. Furthermore, #+latex is already used for export one-liners. In theory, we might drop "begin" parts and get #+src elisp ... #+end #+example ... #+end #+export html #+end but this is akin converting Pascal's begin/end into C's {/} - possible, but an alternative convention is already established. And, generally, we, in Org, try to refrain from changing syntax too much - a number of external projects are relying upon the existing syntax. So, making significant syntax changes should not be taken lightly. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-01 6:46 ` Ihor Radchenko @ 2023-09-04 1:32 ` Richard Stallman 2023-09-04 22:05 ` Juergen Fenn 2023-09-06 11:29 ` Ihor Radchenko 0 siblings, 2 replies; 505+ messages in thread From: Richard Stallman @ 2023-09-04 1:32 UTC (permalink / raw) To: Ihor Radchenko; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > One feature of Org we are discussing here is "exporting": > https://orgmode.org/manual/Exporting.html - an ability to convert Org > documents into other textual formats. I think I understand what you mean by exporting. Maybe I understand what you have in mind for "exporting" Org documents as Texinfo so as to make GNU manuals. But I think I see a disappointing difference between the two. Texinfo for manuals is semantic markup, but the proposed use of Org mode for manuals is not. With Texinfo, you specify what the manual contents should be, and in each format you get that contents as output. See what I mean? I was hoping to see that same smoothness with Org mode -- that you specify the contents and Org mode owuld give youa manual with that contents. But instead, it seems that Org mode for a manual would specify Texinfo text to "export", and Org would generate Texinfo source to export which Texinfo could then convert convert into manuals as output. This exporting might _work_, but it not be as good, overall, as Texinfo. It would be disappointing. Could it be it possible to make Org mode generate the manual output formats using contructs that are semantic in meaning, that specify semantic markup for the contents of the manual -- and have Org mode generate the output formats to represent the semantics of the manual contents? That would be a good replacement for Texinfo. I think this from Bastien Guerry Yes, that's the idea behind using one-char constructs for basic inline markup (eg ~code~, /emphasis/, etc.) and two-chars constructs for more specific inline markup (eg ~~defn~~, =~defvar~=, etc.). Is a sort of thing that might be smooth in this sense. It would specify semantics of the contents, rather than "what to export". It might bot be the best way (as Po Lu says), but at least it is semantic markup. > For more complete support of software manual-specific markup, we plan to > introduce more versatile, hackable markup that can be extended by users > to fit their needs. See > https://list.orgmode.org/orgmode/87a6b8pbhg.fsf@posteo.net/ and > https://list.orgmode.org/orgmode/87mtaez8do.fsf@localhost/ That coukd be the sort of thing that would be as smooth so Texinfo format. > One of the ideas was to follow TeXinfo markup closely, extending with > some Org-specific needs: > Simple case: > @name{<contents>} > If we need to escape { or } inside: > @name{{<contents}} > Extra configuration: > @name[:key value ...]{<contents>} This describes a syntax, but not the semantics. For > Simple case: > @name{<contents>} what's an example of what _name_ might be, and what would it mean? -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-04 1:32 ` Richard Stallman @ 2023-09-04 22:05 ` Juergen Fenn 2023-09-05 11:04 ` Ihor Radchenko 2023-09-06 0:58 ` Richard Stallman 2023-09-06 11:29 ` Ihor Radchenko 1 sibling, 2 replies; 505+ messages in thread From: Juergen Fenn @ 2023-09-04 22:05 UTC (permalink / raw) To: emacs-devel Am 04.09.23 um 03:32 Uhr schrieb Richard Stallman: > But I think I see a disappointing difference between the two. Texinfo > for manuals is semantic markup, but the proposed use of Org mode for > manuals is not. With Texinfo, you specify what the manual contents > should be, and in each format you get that contents as output. See what > I mean? This is an interesting point indeed. I wonder, too, whether we could add semantic markup to Org? It's not exactly what lightweight markup languages such as Markdown or Org are meant to be. But we would not necessarily need this in core, a semantic add-on would probably be enough. We would need an interface that allows for more or less freely defining semantic markup to Org text. A semantic org mode. Hm. Best regards, Jürgen. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-04 22:05 ` Juergen Fenn @ 2023-09-05 11:04 ` Ihor Radchenko 2023-09-06 0:58 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-09-05 11:04 UTC (permalink / raw) To: Juergen Fenn; +Cc: emacs-devel Juergen Fenn <jfenn@gmx.net> writes: > Am 04.09.23 um 03:32 Uhr schrieb Richard Stallman: >> But I think I see a disappointing difference between the two. Texinfo >> for manuals is semantic markup, but the proposed use of Org mode for >> manuals is not. With Texinfo, you specify what the manual contents >> should be, and in each format you get that contents as output. See what >> I mean? > > > This is an interesting point indeed. I wonder, too, whether we could add > semantic markup to Org? It's not exactly what lightweight markup > languages such as Markdown or Org are meant to be. But we would not > necessarily need this in core, a semantic add-on would probably be > enough. We would need an interface that allows for more or less freely > defining semantic markup to Org text. A semantic org mode. Hm. This is exactly what I have in mind. Similar idea is implemented in https://github.com/alhassy/org-special-block-extras -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-04 22:05 ` Juergen Fenn 2023-09-05 11:04 ` Ihor Radchenko @ 2023-09-06 0:58 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Richard Stallman @ 2023-09-06 0:58 UTC (permalink / raw) To: Juergen Fenn; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > This is an interesting point indeed. I wonder, too, whether we could add > semantic markup to Org? It's not exactly what lightweight markup > languages such as Markdown or Org are meant to be. But we would not > necessarily need this in core, a semantic add-on would probably be > enough. Implementing it in an extension would be ok -- it could still do the job that way. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-04 1:32 ` Richard Stallman 2023-09-04 22:05 ` Juergen Fenn @ 2023-09-06 11:29 ` Ihor Radchenko 2023-09-06 12:33 ` Eli Zaretskii 2023-09-09 0:38 ` Richard Stallman 1 sibling, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-09-06 11:29 UTC (permalink / raw) To: rms; +Cc: emacs-devel Richard Stallman <rms@gnu.org> writes: > I was hoping to see that same smoothness with Org mode -- that you > specify the contents and Org mode owuld give youa manual with that > contents. But instead, it seems that Org mode for a manual would > specify Texinfo text to "export", and Org would generate Texinfo source > to export which Texinfo could then convert convert into manuals as output. I think I need to clarify. This is how export to info works _now_. However, nothing stops us from implementing direct export org->info without intermediate org->texi step. Someone just needs to do this job. > Could it be it possible to make Org mode generate the manual output > formats using contructs that are semantic in meaning, that specify > semantic markup for the contents of the manual -- and have Org mode > generate the output formats to represent the semantics of the manual > contents? That would be a good replacement for Texinfo. > I think this from Bastien Guerry > > Yes, that's the idea behind using one-char constructs for basic inline > markup (eg ~code~, /emphasis/, etc.) and two-chars constructs for more > specific inline markup (eg ~~defn~~, =~defvar~=, etc.). > > Is a sort of thing that might be smooth in this sense. It would specify > semantics of the contents, rather than "what to export". > > It might bot be the best way (as Po Lu says), but at least it is > semantic markup. Sorry, but I am lost here. Currently, Org mode does not support all the markup supported by texinfo. But we will change that. Meanwhile, Org implements info/manual export by ad-hoc extension allowing to add texinfo-specific markup. This is not ideal, and it does not need to be done after we do add the missing markup (Org equivalents of @kbd, @var, and other) > > One of the ideas was to follow TeXinfo markup closely, extending with > > some Org-specific needs: > > > Simple case: > > @name{<contents>} > > > If we need to escape { or } inside: > > @name{{<contents}} > > > Extra configuration: > > @name[:key value ...]{<contents>} > > This describes a syntax, but not the semantics. > > For > > > Simple case: > > @name{<contents>} > > what's an example of what _name_ might be, and what would it mean? For example, @kbd{C-c C-x C-c} or @var{org-export-backends}. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 11:29 ` Ihor Radchenko @ 2023-09-06 12:33 ` Eli Zaretskii 2023-09-06 12:43 ` Ihor Radchenko 2023-09-09 0:38 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2023-09-06 12:33 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: emacs-devel@gnu.org > Date: Wed, 06 Sep 2023 11:29:02 +0000 > > Richard Stallman <rms@gnu.org> writes: > > > I was hoping to see that same smoothness with Org mode -- that you > > specify the contents and Org mode owuld give youa manual with that > > contents. But instead, it seems that Org mode for a manual would > > specify Texinfo text to "export", and Org would generate Texinfo source > > to export which Texinfo could then convert convert into manuals as output. > > I think I need to clarify. This is how export to info works _now_. > However, nothing stops us from implementing direct export org->info > without intermediate org->texi step. Someone just needs to do this job. Unless the implementation will somehow invoke texi2any under the hood, this would mean a separate implementation of texi2any in Emacs Lisp, something that I don't recommend: it will need to play a constant catchup game with the Texinfo development, and it will be probably quite slow. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 12:33 ` Eli Zaretskii @ 2023-09-06 12:43 ` Ihor Radchenko 2023-09-06 12:49 ` Po Lu 2023-09-06 13:08 ` Eli Zaretskii 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-09-06 12:43 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rms, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> I think I need to clarify. This is how export to info works _now_. >> However, nothing stops us from implementing direct export org->info >> without intermediate org->texi step. Someone just needs to do this job. > > Unless the implementation will somehow invoke texi2any under the hood, > this would mean a separate implementation of texi2any in Emacs Lisp, Yes. We already have org2pdf, org2ascii, org2html, but not org2info. > something that I don't recommend: it will need to play a constant > catchup game with the Texinfo development, and it will be probably > quite slow. I am not sure about catchup game. Unless Texinfo drastically changes the output format, we can simply provide default settings to replicate the Texinfo look of the produced html/pdf/plaintext/info files. As for being slow, it is already not the case. Last time we discussed Org export being slow, I have managed to speed things up on par with Texinfo timings on Org manual. In any case, the first step is extending Org markup to be more suitable for software manuals. Whether we use texinfo under the hood or not might be discussed afterwards. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 12:43 ` Ihor Radchenko @ 2023-09-06 12:49 ` Po Lu 2023-09-06 12:54 ` Ihor Radchenko 2023-09-06 13:08 ` Eli Zaretskii 1 sibling, 1 reply; 505+ messages in thread From: Po Lu @ 2023-09-06 12:49 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Eli Zaretskii, rms, emacs-devel Ihor Radchenko <yantar92@posteo.net> writes: > As for being slow, it is already not the case. Last time we discussed > Org export being slow, I have managed to speed things up on par with > Texinfo timings on Org manual. Did you use Texinfo 4.13 as a baseline for the comparison, or a more recent release written in Perl? ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 12:49 ` Po Lu @ 2023-09-06 12:54 ` Ihor Radchenko 2023-09-06 13:04 ` Po Lu 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-09-06 12:54 UTC (permalink / raw) To: Po Lu; +Cc: Eli Zaretskii, rms, emacs-devel Po Lu <luangruo@yahoo.com> writes: > Ihor Radchenko <yantar92@posteo.net> writes: > >> As for being slow, it is already not the case. Last time we discussed >> Org export being slow, I have managed to speed things up on par with >> Texinfo timings on Org manual. > > Did you use Texinfo 4.13 as a baseline for the comparison, or a more > recent release written in Perl? More recent release. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 12:54 ` Ihor Radchenko @ 2023-09-06 13:04 ` Po Lu 2023-09-06 13:10 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Po Lu @ 2023-09-06 13:04 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Eli Zaretskii, rms, emacs-devel Ihor Radchenko <yantar92@posteo.net> writes: > Po Lu <luangruo@yahoo.com> writes: > >> Ihor Radchenko <yantar92@posteo.net> writes: >> >>> As for being slow, it is already not the case. Last time we discussed >>> Org export being slow, I have managed to speed things up on par with >>> Texinfo timings on Org manual. >> >> Did you use Texinfo 4.13 as a baseline for the comparison, or a more >> recent release written in Perl? > > More recent release. makeinfo 7.0.2 requires approximately 18x more time to generate emacs.info than 4.13, so an accurate performance comparison cannot be made with the Perl versions as a basis. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 13:04 ` Po Lu @ 2023-09-06 13:10 ` Ihor Radchenko 2023-09-06 13:33 ` Po Lu 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-09-06 13:10 UTC (permalink / raw) To: Po Lu; +Cc: Eli Zaretskii, rms, emacs-devel Po Lu <luangruo@yahoo.com> writes: >>>> As for being slow, it is already not the case. Last time we discussed >>>> Org export being slow, I have managed to speed things up on par with >>>> Texinfo timings on Org manual. >>> >>> Did you use Texinfo 4.13 as a baseline for the comparison, or a more >>> recent release written in Perl? >> >> More recent release. > > makeinfo 7.0.2 requires approximately 18x more time to generate > emacs.info than 4.13, so an accurate performance comparison cannot be > made with the Perl versions as a basis. If makeinfo 7.0.2 deems its performance acceptable, I do not see why similar Org's performance should not be. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 13:10 ` Ihor Radchenko @ 2023-09-06 13:33 ` Po Lu 2023-09-06 15:28 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Po Lu @ 2023-09-06 13:33 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Eli Zaretskii, rms, emacs-devel Ihor Radchenko <yantar92@posteo.net> writes: > If makeinfo 7.0.2 deems its performance acceptable, I do not see why > similar Org's performance should not be. Many of us don't find makeinfo 7.0.2's performance tolerable, which is why Emacs continues to support Texinfo 4.13. It's not the only Texinfo user to maintain such compatibility, BTW. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 13:33 ` Po Lu @ 2023-09-06 15:28 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2023-09-06 15:28 UTC (permalink / raw) To: Po Lu; +Cc: yantar92, rms, emacs-devel > From: Po Lu <luangruo@yahoo.com> > Cc: Eli Zaretskii <eliz@gnu.org>, rms@gnu.org, emacs-devel@gnu.org > Date: Wed, 06 Sep 2023 21:33:50 +0800 > > Ihor Radchenko <yantar92@posteo.net> writes: > > > If makeinfo 7.0.2 deems its performance acceptable, I do not see why > > similar Org's performance should not be. > > Many of us don't find makeinfo 7.0.2's performance tolerable, which is > why Emacs continues to support Texinfo 4.13. That's your personal view. I find the time it takes to produce even such large manuals as emacs.info and elisp.info with Texinfo 7 entirely reasonable, at least when Perl extensions are used. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 12:43 ` Ihor Radchenko 2023-09-06 12:49 ` Po Lu @ 2023-09-06 13:08 ` Eli Zaretskii 2023-09-06 13:20 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2023-09-06 13:08 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: rms@gnu.org, emacs-devel@gnu.org > Date: Wed, 06 Sep 2023 12:43:55 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> I think I need to clarify. This is how export to info works _now_. > >> However, nothing stops us from implementing direct export org->info > >> without intermediate org->texi step. Someone just needs to do this job. > > > > Unless the implementation will somehow invoke texi2any under the hood, > > this would mean a separate implementation of texi2any in Emacs Lisp, > > Yes. We already have org2pdf, org2ascii, org2html, but not org2info. > > > something that I don't recommend: it will need to play a constant > > catchup game with the Texinfo development, and it will be probably > > quite slow. > > I am not sure about catchup game. Unless Texinfo drastically changes the > output format, we can simply provide default settings to replicate the > Texinfo look of the produced html/pdf/plaintext/info files. Not the output format, but the language: new directives and sometimes markup are added with every release of Texinfo. Just read their NEWS file. > As for being slow, it is already not the case. Last time we discussed > Org export being slow, I have managed to speed things up on par with > Texinfo timings on Org manual. That was for Org-to-Texinfo conversion. Here we are talking about Org-to-Info, which is much more work. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 13:08 ` Eli Zaretskii @ 2023-09-06 13:20 ` Ihor Radchenko 2023-09-06 15:25 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-09-06 13:20 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rms, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> I am not sure about catchup game. Unless Texinfo drastically changes the >> output format, we can simply provide default settings to replicate the >> Texinfo look of the produced html/pdf/plaintext/info files. > > Not the output format, but the language: new directives and sometimes > markup are added with every release of Texinfo. Just read their NEWS > file. The way I see Org markup extension would make it easy to users add new custom markup, as needed. Then, no frequent changes to the base markup will be necessary to accommodate for less common use cases. In other words, I do not see why Org should support every single Texinfo markup. We just need to provide enough support to be on par in terms of the needs of manual authors. >> As for being slow, it is already not the case. Last time we discussed >> Org export being slow, I have managed to speed things up on par with >> Texinfo timings on Org manual. > > That was for Org-to-Texinfo conversion. Here we are talking about > Org-to-Info, which is much more work. Why so? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 13:20 ` Ihor Radchenko @ 2023-09-06 15:25 ` Eli Zaretskii 2023-09-06 16:12 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2023-09-06 15:25 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: rms@gnu.org, emacs-devel@gnu.org > Date: Wed, 06 Sep 2023 13:20:21 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> I am not sure about catchup game. Unless Texinfo drastically changes the > >> output format, we can simply provide default settings to replicate the > >> Texinfo look of the produced html/pdf/plaintext/info files. > > > > Not the output format, but the language: new directives and sometimes > > markup are added with every release of Texinfo. Just read their NEWS > > file. > > The way I see Org markup extension would make it easy to users add new > custom markup, as needed. Then, no frequent changes to the base markup > will be necessary to accommodate for less common use cases. You missed the "new directives" part. I do recommend reading their NEWS to get the idea of the rate at which new features are added to Texinfo. > In other words, I do not see why Org should support every single Texinfo > markup. We just need to provide enough support to be on par in terms of > the needs of manual authors. GNU Manuals use a large portion of what Texinfo provides, and limiting what they could use when they write in Org would mean some Texinfo features cannot be used that way, which is a disadvantage. People will have to think twice before they switch to Org, because at some point they might want to use a feature you decided not to support. > > That was for Org-to-Texinfo conversion. Here we are talking about > > Org-to-Info, which is much more work. > > Why so? Because the production rules are much more complex. I suggest to take a look at the Perl source code of texi2any to see what it entails. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 15:25 ` Eli Zaretskii @ 2023-09-06 16:12 ` Ihor Radchenko 2023-09-06 16:34 ` Eli Zaretskii 2023-09-09 0:37 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-09-06 16:12 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rms, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> > Not the output format, but the language: new directives and sometimes >> > markup are added with every release of Texinfo. Just read their NEWS >> > file. >> >> The way I see Org markup extension would make it easy to users add new >> custom markup, as needed. Then, no frequent changes to the base markup >> will be necessary to accommodate for less common use cases. > > You missed the "new directives" part. I do recommend reading their > NEWS to get the idea of the rate at which new features are added to > Texinfo. There is a single word "directives" in https://git.savannah.gnu.org/cgit/texinfo.git/plain/NEWS: . #line directives are recognized. >> In other words, I do not see why Org should support every single Texinfo >> markup. We just need to provide enough support to be on par in terms of >> the needs of manual authors. > > GNU Manuals use a large portion of what Texinfo provides, and limiting > what they could use when they write in Org would mean some Texinfo > features cannot be used that way, which is a disadvantage. People > will have to think twice before they switch to Org, because at some > point they might want to use a feature you decided not to support. I understand. However, new features are added to Texinfo for a reason. If the same reason is valid for Org, equivalent features may be added. Similarly, we may add features to Org that have no equivalent in Texinfo. So, this is a normal development process with new features being proposed, discussed, and maybe added. From my point of view, equivalence with Texinfo is not something we care as much about yet. It is just a general direction we can use for Org development. I deem it useful even if we cannot reach all the Texinfo features at the end. But if we can, we can _later_ see further about how we want to incorporate (or not) new Texinfo features. >> > That was for Org-to-Texinfo conversion. Here we are talking about >> > Org-to-Info, which is much more work. >> >> Why so? > > Because the production rules are much more complex. I suggest to take > a look at the Perl source code of texi2any to see what it entails. AFAIU, the conversion is done in /usr/share/texinfo/Texinfo/Convert/Info.pm I do not see anything extraordinary. Org uses similar approach - working with parse tree. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 16:12 ` Ihor Radchenko @ 2023-09-06 16:34 ` Eli Zaretskii 2023-09-07 11:11 ` Ihor Radchenko 2023-09-09 0:37 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Eli Zaretskii @ 2023-09-06 16:34 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, emacs-devel > From: Ihor Radchenko <yantar92@posteo.net> > Cc: rms@gnu.org, emacs-devel@gnu.org > Date: Wed, 06 Sep 2023 16:12:57 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> > Not the output format, but the language: new directives and sometimes > >> > markup are added with every release of Texinfo. Just read their NEWS > >> > file. > >> > >> The way I see Org markup extension would make it easy to users add new > >> custom markup, as needed. Then, no frequent changes to the base markup > >> will be necessary to accommodate for less common use cases. > > > > You missed the "new directives" part. I do recommend reading their > > NEWS to get the idea of the rate at which new features are added to > > Texinfo. > > There is a single word "directives" in > https://git.savannah.gnu.org/cgit/texinfo.git/plain/NEWS: > > . #line directives are recognized. Look for changes whose heading is "Language:". > >> In other words, I do not see why Org should support every single Texinfo > >> markup. We just need to provide enough support to be on par in terms of > >> the needs of manual authors. > > > > GNU Manuals use a large portion of what Texinfo provides, and limiting > > what they could use when they write in Org would mean some Texinfo > > features cannot be used that way, which is a disadvantage. People > > will have to think twice before they switch to Org, because at some > > point they might want to use a feature you decided not to support. > > I understand. However, new features are added to Texinfo for a reason. > If the same reason is valid for Org, equivalent features may be added. The reason why features are added to Texinfo is that those features are useful for writing software documentation. After all, this is the main purpose of Texinfo. So I cannot see how the same reasons could not be valid for Org -- assuming that Org wants to support writing software documentation, not just the Emacs manuals in their current form. > Similarly, we may add features to Org that have no equivalent in > Texinfo. When you add features to Org, no one needs to catch up with them. But when Texinfo adds new features, users of Org who use Org for writing software documentation will expect those features to be supported, since Org will (AFAIU) claim that it can be used for that. > >> Why so? > > > > Because the production rules are much more complex. I suggest to take > > a look at the Perl source code of texi2any to see what it entails. > > AFAIU, the conversion is done in > /usr/share/texinfo/Texinfo/Convert/Info.pm > I do not see anything extraordinary. Org uses similar approach - working > with parse tree. Fine, then I guess you are all set and can simply forget what I said. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 16:34 ` Eli Zaretskii @ 2023-09-07 11:11 ` Ihor Radchenko 2023-09-10 0:22 ` Richard Stallman 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-09-07 11:11 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rms, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> There is a single word "directives" in >> https://git.savannah.gnu.org/cgit/texinfo.git/plain/NEWS: >> >> . #line directives are recognized. > > Look for changes whose heading is "Language:". I did, and in many cases Texinfo added things that are already available in Org. Otherwise, these changes are not really that frequent. I do not see much problem catching up. >> I understand. However, new features are added to Texinfo for a reason. >> If the same reason is valid for Org, equivalent features may be added. > > The reason why features are added to Texinfo is that those features > are useful for writing software documentation. After all, this is the > main purpose of Texinfo. So I cannot see how the same reasons could > not be valid for Org -- assuming that Org wants to support writing > software documentation, not just the Emacs manuals in their current > form. Org is different from Texinfo and certain things that are impossible with Texinfo and require changing grammar are trivial in Org without upstream changes. I really do not see the raised problem as real problem. To clarify, my stance on this idea with better support for authoring software manuals is to be done in several stages: [ we are at this point now ] 1. Provide basic programmable constructs on Org syntax level. This is generally useful and should be done anyway for various reasons. 2. Use the programmable constructs as a basis for an optional library that will allow software manual-specific markup in Org, aiming for an equivalence of export output. This can be useful, at least, for people using Org to write README files. [ here, we may re-consider the whole idea and see if we should move further ] 3. Look into fine-tuning commands provided by Texinfo (like formatting and layout control) and see if they can (or should) be integrated into Org. [ the problem you describe is to be considered seriously here, and I do not see why it should be the show stopper ] 4. Look if people are interested to switch from Texinfo to Org. 5. Look about maintaining parity between Texinfo and Org. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-07 11:11 ` Ihor Radchenko @ 2023-09-10 0:22 ` Richard Stallman 0 siblings, 0 replies; 505+ messages in thread From: Richard Stallman @ 2023-09-10 0:22 UTC (permalink / raw) To: Ihor Radchenko; +Cc: eliz, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] Those proposed steps seem reasonable to me in principle. It is just a matter of making them work. I don't see any reason why any of them should turn out impossible, except _maybe_ the task of finding good syntaxes to use for the various constructs new in Org format. The rest is just writing software, and we know how to do that. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 16:12 ` Ihor Radchenko 2023-09-06 16:34 ` Eli Zaretskii @ 2023-09-09 0:37 ` Richard Stallman 2023-09-09 9:36 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Richard Stallman @ 2023-09-09 0:37 UTC (permalink / raw) To: Ihor Radchenko; +Cc: eliz, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > I understand. However, new features are added to Texinfo for a reason. That is correct. We added each of these features to Texinfo because some GNU manual needed it. Indeed, the simple way to find out all the features GNU manuals need is to look at the Texinfo manual and see what they required us to support. We're comsidering the idea of making Org format adequate for GNU mamuals. Currently it can't express the distinctions that are needed in order to produce the proper output for each of the output formats. If we want to achieve that, we come to this question: * What specific extension syntax would we use for Org equivalents of the Texinfo constructs that Org currently cannot express> > The way I see Org markup extension would make it easy to users add new > custom markup, as needed. Then, no frequent changes to the base markup > will be necessary to accommodate for less common use cases. I see a possible ambiguity and point of confusion. When you say, "extension", do you mean "a package that gets loaded on top of ordinary Org mode"? That's what I thought it meant. Implementing some of the Texinfo constructs in such a package, perhaps called org-texinfo, is an implementation detail as far as I'm concerned. But now I think maybe you mean something else -- that you propose to add some sort of limited macro definition facility and have the missing Texinfo constructs be defined using that. Is that it? To be adequate for this job, the macro definition facility needs to be more powerful than they usually are. The expansion of one construct needs to depend on the output format being generated, and sometimes the expansion of construct A depends on whether it is inside construct B. If the facility can do that, I think it will suffice for nearly all of the missing Texinfo constructs. If you think of this as a method to simplify part of the implementation of Texinfo in Org, it may work. But be prepared for exeptions, constructs that need special handling! If you think of this as a way to keep Org itself free of Texinfo impurities, it won't work. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-09 0:37 ` Richard Stallman @ 2023-09-09 9:36 ` Ihor Radchenko 2023-09-10 0:22 ` Richard Stallman 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-09-09 9:36 UTC (permalink / raw) To: rms; +Cc: eliz, emacs-devel Richard Stallman <rms@gnu.org> writes: > > The way I see Org markup extension would make it easy to users add new > > custom markup, as needed. Then, no frequent changes to the base markup > > will be necessary to accommodate for less common use cases. > > I see a possible ambiguity and point of confusion. When you say, > "extension", do you mean "a package that gets loaded on top of > ordinary Org mode"? That's what I thought it meant. A package or user Elisp snippet. For example, Org currently allows extending hyperlinks like (defun org-man-export (link description format _) "Export a man page link from Org files." (let ((path (format "http://man.he.net/?topic=%s§ion=all" link)) (desc (or description link))) (pcase format (`html (format "<a target=\"_blank\" href=\"%s\">%s</a>" path desc)) (`latex (format "\\href{%s}{%s}" path desc)) (`texinfo (format "@uref{%s,%s}" path desc)) (`ascii (format "%s (%s)" desc path)) (t path)))) (org-link-set-parameters "man" :export #'org-man-export) Then, <man:emacs> links will be formatted arbitrarily during export. The same idea will be for markup syntax: @var{variable-name} will, in future, be defined as (org-markup-set-parameters "var" :export #'my-export-function-for-var) @var is probably something we will have within Org, but if one needs some weird markup for a specific manual, it will equally be possible to define @my-special-markup{contents} via (org-markup-set-parameters "my-special-markup" :export #'custom-export-formatter) > Implementing some of the Texinfo constructs in such a package, perhaps > called org-texinfo, is an implementation detail as far as I'm > concerned. > > But now I think maybe you mean something else -- that you propose > to add some sort of limited macro definition facility and have the > missing Texinfo constructs be defined using that. Is that it? We might allow in-document definitions like: #+markup[html]: my-special-markup <foo>%s</foo> #+markup[latex]: my-special-markup \foo{%s} ... but the `org-markup-set-parameters' is what we preliminarily agreed upon for now. Further features are to be discussed later. > To be adequate for this job, the macro definition facility needs to be > more powerful than they usually are. The expansion of one construct > needs to depend on the output format being generated, and sometimes > the expansion of construct A depends on whether it is inside construct > B. > If the facility can do that, I think it will suffice for nearly all of > the missing Texinfo constructs. If you think of this as a method to > simplify part of the implementation of Texinfo in Org, it may work. Org is already capable to provide access to the full parse tree when expanding links with custom :export function. The same can be done for markup constructs. Much more difficult if one wants in-document definitions though. > But be prepared for exeptions, constructs that need special handling! > If you think of this as a way to keep Org itself free of Texinfo > impurities, it won't work. May you elaborate about special handling? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-09 9:36 ` Ihor Radchenko @ 2023-09-10 0:22 ` Richard Stallman 0 siblings, 0 replies; 505+ messages in thread From: Richard Stallman @ 2023-09-10 0:22 UTC (permalink / raw) To: Ihor Radchenko; +Cc: eliz, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > For example, Org currently allows extending hyperlinks like > (defun org-man-export (link description format _) > "Export a man page link from Org files." This method like it should be able to handle the various output formats for Texinfo constructs -- as long as once in a while we can define a function that's a little ideosyncratic, for a construct that is a little more irregular in its behavior. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-06 11:29 ` Ihor Radchenko 2023-09-06 12:33 ` Eli Zaretskii @ 2023-09-09 0:38 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Richard Stallman @ 2023-09-09 0:38 UTC (permalink / raw) To: Ihor Radchenko; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > Currently, Org mode does not support all the markup supported by > texinfo. But we will change that. > Meanwhile, Org implements info/manual export by ad-hoc extension > allowing to add texinfo-specific markup. This is not ideal, and it does > not need to be done after we do add the missing markup (Org equivalents > of @kbd, @var, and other) That is good news -- it means we have the same goal in mind. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-26 16:56 ` Ihor Radchenko 2023-08-26 20:28 ` Philip Kaludercic 2023-08-27 1:32 ` Richard Stallman @ 2023-08-30 8:11 ` Bastien Guerry 2023-09-02 1:50 ` Richard Stallman 2 siblings, 1 reply; 505+ messages in thread From: Bastien Guerry @ 2023-08-30 8:11 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Jose E. Marchesi, Eli Zaretskii, rms, emacs-devel Ihor Radchenko <yantar92@posteo.net> writes: > That's what we plan to add to Org. Not just because of this > discussion; Why? Where can I find this discussion? If the main reasons for supporting inline markup like @NAME{CONTENT} (or similar) are the current fontification shortcomings, I'm skeptic. The .org falls in the category of lightweight markup formats and I believe it should stick close to this definition. That is, natively support *bold*, /emphasis/, etc. without adding heavy more constructs like @NAME{CONTENT} -- even optionnally. If we need to support @defn{FUNCTION} for .texi export, what about considering ~~FUNCTION~~ ? Using two chars at the beg and end is a natural extension of the current markup syntax and should spare us some ambiguity that we have with one-char constructs. > there are other reasons why we want such custom inline markup. Thanks for pointing at these discussions. -- Bastien Guerry ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-30 8:11 ` Bastien Guerry @ 2023-09-02 1:50 ` Richard Stallman 2023-09-02 8:19 ` Ihor Radchenko 2023-09-02 8:30 ` Alfred M. Szmidt 0 siblings, 2 replies; 505+ messages in thread From: Richard Stallman @ 2023-09-02 1:50 UTC (permalink / raw) To: Bastien Guerry; +Cc: yantar92, jemarch, eliz, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > The .org falls in the category of lightweight markup formats and I > believe it should stick close to this definition. If people keep this unchanged, Org mode will remain useful for the tasks it can do today. But it won't be able to do the job for which we now use Texinfo. There are many different Texinfo markup constructs, so to do the job of Texinfo, Org mode would need to be able to distinguish them all. So the basic question is: to extend Org mode to do that job, or not? -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-02 1:50 ` Richard Stallman @ 2023-09-02 8:19 ` Ihor Radchenko 2023-09-02 8:30 ` Alfred M. Szmidt 1 sibling, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-09-02 8:19 UTC (permalink / raw) To: rms; +Cc: Bastien Guerry, jemarch, eliz, emacs-devel Richard Stallman <rms@gnu.org> writes: > > The .org falls in the category of lightweight markup formats and I > > believe it should stick close to this definition. > > If people keep this unchanged, Org mode will remain useful for the > tasks it can do today. But it won't be able to do the job for which > we now use Texinfo. > There are many different Texinfo markup constructs, so to do the job > of Texinfo, Org mode would need to be able to distinguish them all. > > So the basic question is: to extend Org mode to do that job, or not? I don't think that we need to compromise here. We can keep Org lightweight yet allowing arbitrary markup. We already have arbitrary environment markup (special blocks): #+begin_<block type> ... #+end_<block type> like #+begin_center #+begin_abstract #+begin_myweirdlatexenvironment etc These are all valid Org markups, but only a small subset of them has special handling by Org. Other free-form special blocks are simply ignored (or handled generically), for example, on export, unless the specific export backend implements special handling for a given block type. Supporting TeXinfo-specific inline markup can be done using the same approach, without complicating core Org markup specifications. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-09-02 1:50 ` Richard Stallman 2023-09-02 8:19 ` Ihor Radchenko @ 2023-09-02 8:30 ` Alfred M. Szmidt 1 sibling, 0 replies; 505+ messages in thread From: Alfred M. Szmidt @ 2023-09-02 8:30 UTC (permalink / raw) To: rms; +Cc: bzg, yantar92, jemarch, eliz, emacs-devel > The .org falls in the category of lightweight markup formats and I > believe it should stick close to this definition. If people keep this unchanged, Org mode will remain useful for the tasks it can do today. But it won't be able to do the job for which we now use Texinfo. There are many different Texinfo markup constructs, so to do the job of Texinfo, Org mode would need to be able to distinguish them all. So the basic question is: to extend Org mode to do that job, or not? It would make more sense to extend Texinfo to allow for org-mode like markup (or lighter markup), than try to stuff Texinfo into org-mode. Texinfo is despite its issues, very nice and easy to write. Handling the version chasm between existing Texinfo and "new" Texinfo can easily be solved by having a small @texinfo-version thing at the top. But replacing Texinfo outright with some org-mode variant seems like yak shaving. The big issue with Texinfo is that it is hard to make your own complicated constructs (like @defn ..), and the lack of macro support of any kind. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-26 16:49 ` Jose E. Marchesi 2023-08-26 16:56 ` Ihor Radchenko @ 2023-08-27 1:32 ` Richard Stallman 2023-08-27 8:35 ` Ihor Radchenko 2023-08-30 8:14 ` Bastien Guerry 1 sibling, 2 replies; 505+ messages in thread From: Richard Stallman @ 2023-08-27 1:32 UTC (permalink / raw) To: Jose E. Marchesi; +Cc: eliz, bzg, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > For inline marks, Texinfo uses a conventient generic and extensible > form: @NAME{CONTENT}. > "markdown-style" formats, on the contrary, generally support a closed > set of marks that use delimited based ad-hoc syntax, such as *foo*, > =foo=, "foo", `foo' and so on. The challenge is to extend Org format with an open-ended syntax that could handle many different inline markup opertions. ISTR that Sphinx has an extension which is good for such markup. Could that same syntax fit into Org mode too? -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-27 1:32 ` Richard Stallman @ 2023-08-27 8:35 ` Ihor Radchenko 2023-08-28 1:32 ` Richard Stallman 2023-08-30 8:14 ` Bastien Guerry 1 sibling, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-27 8:35 UTC (permalink / raw) To: rms; +Cc: Jose E. Marchesi, eliz, bzg, emacs-devel Richard Stallman <rms@gnu.org> writes: > ISTR that Sphinx has an extension which is good for such markup. > Could that same syntax fit into Org mode too? May you provide some link that describes the extension you are referring to? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-27 8:35 ` Ihor Radchenko @ 2023-08-28 1:32 ` Richard Stallman 2023-08-28 10:04 ` Ihor Radchenko 0 siblings, 1 reply; 505+ messages in thread From: Richard Stallman @ 2023-08-28 1:32 UTC (permalink / raw) To: Ihor Radchenko; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > ISTR that Sphinx has an extension which is good for such markup. > > Could that same syntax fit into Org mode too? > May you provide some link that describes the extension you are referring > to? Sorry, I may have said that in an unclear way. It is not an extension in the sense of an add-on to Sphinx that you need to install separately. It is a standard part of Sphinx and described in the manual, It is an extension in the sense that they added it to the simple markup language that they started with. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-28 1:32 ` Richard Stallman @ 2023-08-28 10:04 ` Ihor Radchenko 2023-08-28 11:15 ` Yuri Khan 2023-08-31 2:09 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-08-28 10:04 UTC (permalink / raw) To: rms; +Cc: emacs-devel Richard Stallman <rms@gnu.org> writes: > > May you provide some link that describes the extension you are referring > > to? > > Sorry, I may have said that in an unclear way. It is not an extension > in the sense of an add-on to Sphinx that you need to install > separately. It is a standard part of Sphinx and described in the > manual, > > It is an extension in the sense that they added it to the simple markup > language that they started with. Sorry, but I can only see a closed list of inline emphasis constructs in https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#inline-markup I may be misunderstanding you. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-28 10:04 ` Ihor Radchenko @ 2023-08-28 11:15 ` Yuri Khan 2023-08-28 11:21 ` Ihor Radchenko 2023-08-31 2:09 ` Richard Stallman 1 sibling, 1 reply; 505+ messages in thread From: Yuri Khan @ 2023-08-28 11:15 UTC (permalink / raw) To: Ihor Radchenko; +Cc: rms, emacs-devel On Mon, 28 Aug 2023 at 17:05, Ihor Radchenko <yantar92@posteo.net> wrote: > Sorry, but I can only see a closed list of inline emphasis constructs in > https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#inline-markup The `interpreted text`:role: construct is extensible, by defining new role codes. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-28 11:15 ` Yuri Khan @ 2023-08-28 11:21 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2023-08-28 11:21 UTC (permalink / raw) To: Yuri Khan; +Cc: rms, emacs-devel Yuri Khan <yuri.v.khan@gmail.com> writes: > On Mon, 28 Aug 2023 at 17:05, Ihor Radchenko <yantar92@posteo.net> wrote: > >> Sorry, but I can only see a closed list of inline emphasis constructs in >> https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#inline-markup > > The `interpreted text`:role: construct is extensible, by defining new > role codes. Thanks! It is similar to what we already plan for: https://list.orgmode.org/orgmode/87a6b8pbhg.fsf@posteo.net/ Except that we also plan to allow assigning optional attributes to the custom markup. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-28 10:04 ` Ihor Radchenko 2023-08-28 11:15 ` Yuri Khan @ 2023-08-31 2:09 ` Richard Stallman 2023-08-31 8:53 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Richard Stallman @ 2023-08-31 2:09 UTC (permalink / raw) To: Ihor Radchenko; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > Sorry, but I can only see a closed list of inline emphasis constructs in > https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#inline-markup > I may be misunderstanding you. I don't remember it any more precisely, so I can't tell you more. I notice that you're talking about restructuredtext and I am talking about Sphinx. Are the two related? I don't remember. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-31 2:09 ` Richard Stallman @ 2023-08-31 8:53 ` Ihor Radchenko 2023-09-04 1:33 ` Richard Stallman 0 siblings, 1 reply; 505+ messages in thread From: Ihor Radchenko @ 2023-08-31 8:53 UTC (permalink / raw) To: rms; +Cc: emacs-devel Richard Stallman <rms@gnu.org> writes: > > Sorry, but I can only see a closed list of inline emphasis constructs in > > https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#inline-markup > > > I may be misunderstanding you. > > I don't remember it any more precisely, so I can't tell you more. > > I notice that you're talking about restructuredtext and I am talking > about Sphinx. Are the two related? I don't remember. When I looked up Sphinx, the documentation directed me to reStructuredtext for markup description: https://www.sphinx-doc.org/en/master/ Sphinx uses the reStructuredText markup language by default, and can read MyST markdown via third-party extensions. Also, Yuri pointed me to >> The `interpreted text`:role: construct is extensible, by defining new >> role codes. which is similar to what we planned for Org: @role{interpreted text} -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-31 8:53 ` Ihor Radchenko @ 2023-09-04 1:33 ` Richard Stallman 0 siblings, 0 replies; 505+ messages in thread From: Richard Stallman @ 2023-09-04 1:33 UTC (permalink / raw) To: Ihor Radchenko; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > I notice that you're talking about restructuredtext and I am talking > > about Sphinx. Are the two related? I don't remember. > When I looked up Sphinx, the documentation directed me to > reStructuredtext for markup description: I guess you must be right. Thanks. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-27 1:32 ` Richard Stallman 2023-08-27 8:35 ` Ihor Radchenko @ 2023-08-30 8:14 ` Bastien Guerry 2023-08-30 8:32 ` Po Lu 1 sibling, 1 reply; 505+ messages in thread From: Bastien Guerry @ 2023-08-30 8:14 UTC (permalink / raw) To: Richard Stallman; +Cc: Jose E. Marchesi, eliz, emacs-devel Richard Stallman <rms@gnu.org> writes: > > "markdown-style" formats, on the contrary, generally support a closed > > set of marks that use delimited based ad-hoc syntax, such as *foo*, > > =foo=, "foo", `foo' and so on. > > The challenge is to extend Org format with an open-ended syntax that > could handle many different inline markup opertions. Yes, that's the idea behind using one-char constructs for basic inline markup (eg ~code~, /emphasis/, etc.) and two-chars constructs for more specific inline markup (eg ~~defn~~, =~defvar~=, etc.). -- Bastien Guerry ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-30 8:14 ` Bastien Guerry @ 2023-08-30 8:32 ` Po Lu 0 siblings, 0 replies; 505+ messages in thread From: Po Lu @ 2023-08-30 8:32 UTC (permalink / raw) To: Bastien Guerry; +Cc: Richard Stallman, Jose E. Marchesi, eliz, emacs-devel Bastien Guerry <bzg@gnu.org> writes: > Yes, that's the idea behind using one-char constructs for basic inline > markup (eg ~code~, /emphasis/, etc.) and two-chars constructs for more > specific inline markup (eg ~~defn~~, =~defvar~=, etc.). I forsee a problem with this approach: users will be compelled to study and memorize dozens of two character combinations that closely resemble one another. Contrast this to Texinfo, where: The key sequence @kbd{C-x C-f} The string @samp{"sample text"} The function @code{defun} The formula @var{X} An @dfn{example} leaves no latitude for visual ambiguity, whereas it will not take much for one to mistake =~ for -~ or ~~. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2023-08-26 5:50 ` Eli Zaretskii 2023-08-26 16:49 ` Jose E. Marchesi @ 2023-08-27 1:33 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Richard Stallman @ 2023-08-27 1:33 UTC (permalink / raw) To: Eli Zaretskii; +Cc: bzg, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > I think it can, but a typical Texinfo manual these days uses many more > commands, not just those. So this list is insufficient to judge these > issues. The actual list is much longer. You're right. But I think that if Org can conveniently handle the ones I listed, giving each its Texinfo behavior in each output format, that means it has the capacity to handle any additional inline markup features too. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 6:35 ` Ihor Radchenko 2022-09-26 6:57 ` Bastien @ 2022-09-26 8:24 ` Eli Zaretskii 2022-09-26 8:32 ` Jean Louis 2 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-09-26 8:24 UTC (permalink / raw) To: Ihor Radchenko; +Cc: bzg, theophilusx, emacs-devel > From: Ihor Radchenko <yantar92@gmail.com> > Cc: Tim Cross <theophilusx@gmail.com>, emacs-devel@gnu.org > Date: Mon, 26 Sep 2022 14:35:00 +0800 > > I am wondering how many Emacs developers contributed to Org in the past. > In particular, before we switched away from .texi manual. You can easily see that with the following command: git log -- doc/misc/org.texi ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 6:35 ` Ihor Radchenko 2022-09-26 6:57 ` Bastien 2022-09-26 8:24 ` Eli Zaretskii @ 2022-09-26 8:32 ` Jean Louis 2022-09-26 9:54 ` Ihor Radchenko 2 siblings, 1 reply; 505+ messages in thread From: Jean Louis @ 2022-09-26 8:32 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Bastien, Tim Cross, emacs-devel * Ihor Radchenko <yantar92@gmail.com> [2022-09-26 09:35]: > I am now more concerned about Org contributors. I am not sure if we have > a luxury to push our contributors into GNU project by forcing them to > learn texinfo in addition to Org (note that I do not say that we should > not encourage people to contribute to GNU; just that we should not > discourage people from becoming Org contributors in the first > place). You may have contributors write it in Org and then convert Org to texinfo by using pandoc: pandoc -f org -t texinfo your-file.org -- Jean Take action in Free Software Foundation campaigns: https://www.fsf.org/campaigns In support of Richard M. Stallman https://stallmansupport.org/ ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 8:32 ` Jean Louis @ 2022-09-26 9:54 ` Ihor Radchenko 2022-09-26 11:04 ` Robert Pluim 2022-09-27 16:17 ` Richard Stallman 0 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-09-26 9:54 UTC (permalink / raw) To: Jean Louis; +Cc: Bastien, Tim Cross, emacs-devel Jean Louis <bugs@gnu.support> writes: > * Ihor Radchenko <yantar92@gmail.com> [2022-09-26 09:35]: >> I am now more concerned about Org contributors. I am not sure if we have >> a luxury to push our contributors into GNU project by forcing them to >> learn texinfo in addition to Org (note that I do not say that we should >> not encourage people to contribute to GNU; just that we should not >> discourage people from becoming Org contributors in the first >> place). > > You may have contributors write it in Org and then convert Org to > texinfo by using pandoc: > > pandoc -f org -t texinfo your-file.org This is almost what we do now. Org contributors write in Org, and then we use Org itself to convert to texinfo. Org files serve as the source files. What we are discussing here is whether Org should switch from Org sources back to texinfo sources used in Emacs. 1. Switching to texinfo will improve Emacs coherence and potentially increase the pool of Emacs contributors familiar with texinfo (by pulling some of the Org contributors) 2. On the other hand, switching from Org to texinfo sources on Org side will require Org contributors to learn texinfo, which they do not really need now. 3. Yet, switching from Org to texinfo will allow Emacs contributors to send patches in familiar texinfo format to Org. From the perspective of Emacs, using texinfo sources is a good thing. From the perspective of Org, it is not that obvious. Hence this discussion. Note that we are now discussing this on emacs-devel and do not yet have inputs from many Org users/contributors. However, we (Org devs) can get inputs from Emacs devs, which is part of the information we need to see what we want to do about Org vs. Texinfo sources. -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 9:54 ` Ihor Radchenko @ 2022-09-26 11:04 ` Robert Pluim 2022-09-27 16:17 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Robert Pluim @ 2022-09-26 11:04 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Jean Louis, Bastien, Tim Cross, emacs-devel >>>>> On Mon, 26 Sep 2022 17:54:33 +0800, Ihor Radchenko <yantar92@gmail.com> said: Ihor> 1. Switching to texinfo will improve Emacs coherence and potentially Ihor> increase the pool of Emacs contributors familiar with texinfo (by Ihor> pulling some of the Org contributors) Ihor> 2. On the other hand, switching from Org to texinfo sources on Org side Ihor> will require Org contributors to learn texinfo, which they do not Ihor> really need now. Ihor> 3. Yet, switching from Org to texinfo will allow Emacs contributors to Ihor> send patches in familiar texinfo format to Org. Ihor> From the perspective of Emacs, using texinfo sources is a good thing. Ihor> From the perspective of Org, it is not that obvious. Ihor> Hence this discussion. As someone who has written the odd bit of Emacs documentation and (less) org-mode documentation, I find the cognitive load of texinfo or org to be about the same [1][2]; whichever one I wrote most recently is relatively easy to use, and the other I need to relearn. I suspect the same is true for most casual submitters of documentation patches. Robert Footnotes: [1] In a lot of cases itʼs "scroll around to see what the rest of the document is doing and just copy that", which is perhaps inefficent of my time, but works. [2] The load would be less if I didnʼt *also* have to write Markdown for work, but Emacs help a lot with that. 😀 -- ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 9:54 ` Ihor Radchenko 2022-09-26 11:04 ` Robert Pluim @ 2022-09-27 16:17 ` Richard Stallman 2022-09-30 3:41 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Richard Stallman @ 2022-09-27 16:17 UTC (permalink / raw) To: Ihor Radchenko; +Cc: bugs, bzg, theophilusx, emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > This is almost what we do now. Org contributors write in Org, and then we > use Org itself to convert to texinfo. Org files serve as the source > files. In order for that to produce proper Texinfo source as output, there would need to be, in Org format, ways to express the many distinctions that Texinfo source needs to make. Yesterday I posted a list of Texinfo constructs and suggested adding to Org mode ways to distinguish them. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-27 16:17 ` Richard Stallman @ 2022-09-30 3:41 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-09-30 3:41 UTC (permalink / raw) To: rms; +Cc: bugs, bzg, theophilusx, emacs-devel Richard Stallman <rms@gnu.org> writes: > In order for that to produce proper Texinfo source as output, there > would need to be, in Org format, ways to express the many distinctions > that Texinfo source needs to make. Yesterday I posted a list of > Texinfo constructs and suggested adding to Org mode ways to > distinguish them. I have just forwarded that email to Org ML. https://list.orgmode.org/87bkqx4jyg.fsf@localhost/T/#u -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-25 19:47 ` Tim Cross 2022-09-26 6:12 ` Bastien @ 2022-09-26 12:10 ` Richard Stallman 1 sibling, 0 replies; 505+ messages in thread From: Richard Stallman @ 2022-09-26 12:10 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-devel [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] Speaking as the Chief GNUisance, rssponsible for GNU Project standards, I would be happy to adopt an upgraded Org format as a new standard source format for GNU manuals, _provided_ Org format has been extended with the capability to express all the constructions and distinctions that Texinfo can express, generate all the output formats Texinfo can generate, and use TeX to make beautiful printed output. Texinfo can generate these output formats: Info files, HTML, ASCII text, and DVI and PDF files via TeX. Texinfo provides numerous subtle distinctions that show up clearly in each of these output formats. Compare, for example, @var, @dfn and @emph; compare @code, @samp, @file, @command, @option, @kbd, and @key. I am sure people can extend Org software to handle these semantic distinctions and generate these output formats. Since it has been done once, it can be done again. But the work is not trivial. The work has to start by designing what the extended Org format will look like. That part is the crucial part; once it has been specified, people can work independently to implement various parts of handling that format. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-25 6:35 ` Bastien 2022-09-25 7:05 ` Ihor Radchenko 2022-09-25 8:01 ` Eli Zaretskii @ 2022-09-26 3:02 ` Ihor Radchenko 2022-09-26 5:37 ` Po Lu 2022-09-26 6:18 ` Bastien 2 siblings, 2 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-09-26 3:02 UTC (permalink / raw) To: Bastien; +Cc: Payas Relekar, emacs-devel [-- Attachment #1: Type: text/plain, Size: 440 bytes --] Bastien <bzg@gnu.org> writes: >> - Let's see if we have more contributions to the manual and if >> we really solved a problem here. > > (=> 2) This didn't happen. I played around with the git log data and your claim appears to be incorrect. The number of commits affecting manual increases substantially according to my analysis. And it is not just initial spike of fixes after introducing the new format. See the attached histogram. [-- Attachment #2: manual-commits-stats.png --] [-- Type: image/png, Size: 27488 bytes --] [-- Attachment #3: Type: text/plain, Size: 916 bytes --] Methodology: The org manual has been introduced around d330eed7c (2017-12) #+begin_src bash git log main --grep manual --format=%ci > manual-commits.dat cat manual-commits.dat | awk '{print $1}' | cut -d- -f1,2 > manual-commits-month.dat #+end_src #+begin_src gnuplot set timefmt '%Y-%m' set xdata time set title "Number of commits according to \"git log --grep manual\"" font ",20" set tics out set label 1 ".texi → .org" at "2017-12",155 center font ",18" textcolor rgb'red' set arrow from "2017-12",0 to "2017-12",150 nohead front lw 10 lc rgb'red' set style fill solid 0.5 border set yrange [0:160] plot 'manual-commits-month.dat' u 1:(1.0) bins=40 w boxes t'' #+end_src -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 3:02 ` Ihor Radchenko @ 2022-09-26 5:37 ` Po Lu 2022-09-26 5:44 ` Emanuel Berg ` (2 more replies) 2022-09-26 6:18 ` Bastien 1 sibling, 3 replies; 505+ messages in thread From: Po Lu @ 2022-09-26 5:37 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Bastien, Payas Relekar, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > I played around with the git log data and your claim appears to be > incorrect. The number of commits affecting manual increases > substantially according to my analysis. And it is not just initial spike > of fixes after introducing the new format. > > See the attached histogram. > > > > > Methodology: > > The org manual has been introduced around d330eed7c (2017-12) > > #+begin_src bash > git log main --grep manual --format=%ci > manual-commits.dat > cat manual-commits.dat | awk '{print $1}' | cut -d- -f1,2 > manual-commits-month.dat > #+end_src > > #+begin_src gnuplot > set timefmt '%Y-%m' > set xdata time > set title "Number of commits according to \"git log --grep manual\"" font ",20" > set tics out > set label 1 ".texi → .org" at "2017-12",155 center font ",18" textcolor rgb'red' > set arrow from "2017-12",0 to "2017-12",150 nohead front lw 10 lc rgb'red' > set style fill solid 0.5 border > set yrange [0:160] > plot 'manual-commits-month.dat' u 1:(1.0) bins=40 w boxes t'' > #+end_src Maybe that points to Org being more maintainence-heavy than Texinfo? Anyway, I've made the following offer and will make it again: if anyone who does not know Texinfo wants to contribute documentation to Emacs, he can write it in plain text, and I will manually convert it to Texinfo. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 5:37 ` Po Lu @ 2022-09-26 5:44 ` Emanuel Berg 2022-09-26 6:20 ` Bastien Guerry 2022-09-26 6:36 ` Ihor Radchenko 2 siblings, 0 replies; 505+ messages in thread From: Emanuel Berg @ 2022-09-26 5:44 UTC (permalink / raw) To: emacs-devel Po Lu wrote: > Anyway, I've made the following offer and will make it > again: if anyone who does not know Texinfo wants to > contribute documentation to Emacs, he can write it in plain > text, and I will manually convert it to Texinfo. There are tools for that ... -- underground experts united https://dataswamp.org/~incal ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 5:37 ` Po Lu 2022-09-26 5:44 ` Emanuel Berg @ 2022-09-26 6:20 ` Bastien Guerry 2022-09-26 13:58 ` T.V Raman 2022-09-26 6:36 ` Ihor Radchenko 2 siblings, 1 reply; 505+ messages in thread From: Bastien Guerry @ 2022-09-26 6:20 UTC (permalink / raw) To: Po Lu; +Cc: Ihor Radchenko, Payas Relekar, emacs-devel Po Lu <luangruo@yahoo.com> writes: > Anyway, I've made the following offer and will make it again: if anyone > who does not know Texinfo wants to contribute documentation to Emacs, he > can write it in plain text, and I will manually convert it to Texinfo. That's a very good move, one that we should make for Org too. I'll propose something on our guide for contributors. -- Bastien ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 6:20 ` Bastien Guerry @ 2022-09-26 13:58 ` T.V Raman 2022-09-26 16:16 ` Eli Zaretskii 0 siblings, 1 reply; 505+ messages in thread From: T.V Raman @ 2022-09-26 13:58 UTC (permalink / raw) To: Bastien Guerry; +Cc: Po Lu, Ihor Radchenko, Payas Relekar, emacs-devel [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain; charset=gb18030, Size: 1206 bytes --] I think there are two (perhaps more) aspects to writing good documentation and accepting / making contributions: 1. The basics -- write a paragraph or two describing some feature. Orthogonal to that and here is where each format bites in different ways: 2. Integrating that contribution into the main corpus of the work --- at the write place, with the right surrounding context, generating the right table-of-contents and menus (a running nightmare in Texinfo if you dont set things up right at the beginning) and so on. I think (2) might likely contribute negatively to (1) happening more often. So rather than arguing about formats (and it's been pointed out already that there are tools for automatic conversion) we should perhaps: A. Create a simple content repository where contributors can check-in their small to medium contributions in the format of their choice (limited to a few open formats) along with the meta-data B. A small group of more experienced volunteers take on the task of incorporating those contributions into the main work. -- Thanks, --Raman(I Search, I Find, I Misplace, I Research) 7©4 Id: kg:/m/0285kf1 0Ü8 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 13:58 ` T.V Raman @ 2022-09-26 16:16 ` Eli Zaretskii 0 siblings, 0 replies; 505+ messages in thread From: Eli Zaretskii @ 2022-09-26 16:16 UTC (permalink / raw) To: T.V Raman; +Cc: bzg, luangruo, yantar92, relekarpayas, emacs-devel > From: "T.V Raman" <raman@google.com> > Cc: Po Lu <luangruo@yahoo.com>, Ihor Radchenko <yantar92@gmail.com>, Payas > Relekar <relekarpayas@gmail.com>, emacs-devel@gnu.org > Date: Mon, 26 Sep 2022 06:58:31 -0700 > > I think there are two (perhaps more) aspects to writing good > documentation and accepting / making contributions: > > 1. The basics -- write a paragraph or two describing some feature. > Orthogonal to that and here is where each format bites in different > ways: > > 2. Integrating that contribution into the main corpus of the work --- at > the write place, with the right surrounding context, generating the > right table-of-contents and menus (a running nightmare in Texinfo if > you dont set things up right at the beginning) and so on. > > I think (2) might likely contribute negatively to (1) happening more > often. I think the division you suggest is somewhat artificial. Unlike man pages, Info manuals are not a collection of loosely-couples articles; instead, the sections in a good Info manual are always connected and context-aware. Thus, writing the documentation for the manual should _always_ start by studying the "neighborhood" in which that new stuff will live. IOW, putting the new text in context is not an after-thought, it should be considered prior to writing. I don't know what you allude to as "running nightmare in Texinfo". Adding a new node to a manual is since long ago a very simple process: just add it and make sure the @menu of the parent node (if there is one) is updated to have the new node in its correct place. That's it. > So rather than arguing about formats (and it's been pointed out already > that there are tools for automatic conversion) we should perhaps: > > A. Create a simple content repository where contributors can check-in > their small to medium contributions in the format of their choice > (limited to a few open formats) along with the meta-data > B. A small group of more experienced volunteers take on the task of > incorporating those contributions into the main work. Most contributors to Emacs already write pretty good Texinfo; adding finishing touches to that and fixing a few style or markup mistakes is already being done in the background by those who spot those blunders. So I think we already have the above in place. ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 5:37 ` Po Lu 2022-09-26 5:44 ` Emanuel Berg 2022-09-26 6:20 ` Bastien Guerry @ 2022-09-26 6:36 ` Ihor Radchenko 2 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-09-26 6:36 UTC (permalink / raw) To: Po Lu; +Cc: Bastien, Payas Relekar, emacs-devel Po Lu <luangruo@yahoo.com> writes: > Maybe that points to Org being more maintainence-heavy than Texinfo? While I am not familiar with Texinfo, I can only recall a single issue with Org manual that is related to it being written in Org. I would not call it maintainence-heavy. -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 3:02 ` Ihor Radchenko 2022-09-26 5:37 ` Po Lu @ 2022-09-26 6:18 ` Bastien 2022-09-26 6:29 ` Ihor Radchenko 1 sibling, 1 reply; 505+ messages in thread From: Bastien @ 2022-09-26 6:18 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Payas Relekar, emacs-devel Ihor Radchenko <yantar92@gmail.com> writes: > I played around with the git log data and your claim appears to be > incorrect. Thanks for providing these numbers, I stand corrected then. But I'm not *that* impressed as I still doubt we can infer a causal connection between the texi->org switch and the increasing number of commits (I've read the detailed logs.) If you, Nicolas, Tim and other core contributors say that switching back to .texi will definitely slow down your contributions to Org (these has to be considered and said in all honesty, which is hard) that I would consider a strong argument against the switch back to .texi---but that's our collective decision anyway, so I'm fine with whatever everyone decides (not now, around Org 10). -- Bastien ^ permalink raw reply [flat|nested] 505+ messages in thread
* Re: Org mode and Emacs 2022-09-26 6:18 ` Bastien @ 2022-09-26 6:29 ` Ihor Radchenko 0 siblings, 0 replies; 505+ messages in thread From: Ihor Radchenko @ 2022-09-26 6:29 UTC (permalink / raw) To: Bastien; +Cc: Payas Relekar, emacs-devel [-- Attachment #1: Type: text/plain, Size: 429 bytes --] Bastien <bzg@gnu.org> writes: > Ihor Radchenko <yantar92@gmail.com> writes: > >> I played around with the git log data and your claim appears to be >> incorrect. > > Thanks for providing these numbers, I stand corrected then. Actually, the number are not as accurate as I thought. The old manual was in doc/org.texi and had no word "manual" in it. Grepping the file names yields (different colours represent .texi and .org) [-- Attachment #2: manual-commits-v2.png --] [-- Type: image/png, Size: 16592 bytes --] [-- Attachment #3: Type: text/plain, Size: 117 bytes --] which is still an improvement because the total number of commits is just a fraction of the early Org development. [-- Attachment #4: total-commits-stats.png --] [-- Type: image/png, Size: 27303 bytes --] [-- Attachment #5: Type: text/plain, Size: 1036 bytes --] > But I'm not *that* impressed as I still doubt we can infer a causal > connection between the texi->org switch and the increasing number of > commits (I've read the detailed logs.) I am sure that you are a lot more familiar with the historic data. > If you, Nicolas, Tim and other core contributors say that switching > back to .texi will definitely slow down your contributions to Org > (these has to be considered and said in all honesty, which is hard) > that I would consider a strong argument against the switch back to > .texi---but that's our collective decision anyway, so I'm fine with > whatever everyone decides (not now, around Org 10). Learning .texi will certainly be an extra burden for me. Not impossible, but feels awkward that I'd need it for Org. In any case, emacs-devel is not the place to decide this. -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 ^ permalink raw reply [flat|nested] 505+ messages in thread
end of thread, other threads:[~2023-09-10 0:22 UTC | newest] Thread overview: 505+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-06-04 13:14 Convert README.org to plain text README while installing package Akib Azmain Turja 2022-06-04 13:32 ` Tassilo Horn 2022-06-04 14:17 ` Alan Mackenzie 2022-06-04 14:31 ` Tassilo Horn 2022-06-04 17:39 ` Akib Azmain Turja 2022-06-04 16:09 ` Stefan Kangas 2022-06-04 17:27 ` Alan Mackenzie 2022-06-04 21:10 ` Stefan Kangas 2022-06-05 8:38 ` Michael Albinus 2022-06-05 8:51 ` Po Lu 2022-06-05 10:26 ` Tassilo Horn 2022-06-05 11:15 ` Michael Albinus 2022-06-05 16:52 ` [External] : " Drew Adams 2022-06-06 0:19 ` Tim Cross 2022-06-06 9:59 ` Philip Kaludercic 2022-06-06 13:47 ` Tim Cross 2022-06-06 14:15 ` Philip Kaludercic 2022-06-06 11:33 ` Alan Mackenzie 2022-06-06 12:26 ` Akib Azmain Turja 2022-06-06 13:57 ` Tim Cross 2022-06-06 16:02 ` Eli Zaretskii 2022-06-07 6:14 ` Tim Cross 2022-06-07 11:21 ` Eli Zaretskii 2022-06-08 13:12 ` Ihor Radchenko 2022-06-08 14:15 ` Eli Zaretskii 2022-06-08 15:15 ` Ihor Radchenko 2022-06-08 16:16 ` Eli Zaretskii 2022-06-09 9:15 ` Ihor Radchenko 2022-06-09 9:38 ` Eli Zaretskii 2022-06-11 3:52 ` Ihor Radchenko 2022-06-11 6:52 ` Eli Zaretskii 2022-06-12 8:40 ` Ihor Radchenko 2022-06-12 8:56 ` Eli Zaretskii 2022-06-12 9:46 ` Ihor Radchenko 2022-06-12 9:59 ` Eli Zaretskii 2022-06-15 5:13 ` Ihor Radchenko 2022-06-15 12:49 ` Eli Zaretskii 2022-06-17 5:55 ` Ihor Radchenko 2022-06-17 6:40 ` Eli Zaretskii 2022-06-18 4:40 ` Limitations on using Org mode in buffers mixing Org markup with non-Org markup (was: Convert README.org to plain text README while installing package) Ihor Radchenko 2022-06-18 7:10 ` Eli Zaretskii 2022-06-19 5:15 ` Ihor Radchenko 2022-06-18 5:10 ` Convert README.org to plain text README while installing package Po Lu 2022-06-18 11:28 ` Lars Ingebrigtsen 2022-06-18 13:33 ` Stefan Monnier 2022-06-18 15:50 ` tomas 2022-06-18 16:00 ` Visuwesh 2022-06-18 17:13 ` tomas 2022-06-18 17:33 ` Yuri Khan 2022-06-18 23:27 ` [External] : " Drew Adams 2022-06-18 17:45 ` Eli Zaretskii 2022-06-18 17:55 ` Visuwesh 2022-06-18 18:39 ` tomas 2022-06-18 23:25 ` [External] : " Drew Adams 2022-06-19 0:24 ` Tim Cross 2022-06-19 4:35 ` tomas 2022-06-19 5:36 ` Tim Cross 2022-06-19 6:43 ` tomas 2022-06-19 17:03 ` Drew Adams 2022-06-19 1:48 ` Po Lu 2022-06-19 2:08 ` Tim Cross 2022-06-19 3:00 ` Po Lu 2022-06-19 6:07 ` Eli Zaretskii 2022-06-19 6:41 ` Tim Cross 2022-06-18 23:22 ` [External] : " Drew Adams 2022-06-19 11:08 ` Lars Ingebrigtsen 2022-06-19 17:10 ` [External] : " Drew Adams 2022-06-20 6:05 ` Kévin Le Gouguec 2022-06-20 8:49 ` Tim Cross 2022-06-12 13:53 ` Kévin Le Gouguec 2022-06-12 20:07 ` Common keybindings idea for multiple markups Jean Louis 2022-06-08 19:34 ` Convert README.org to plain text README while installing package Tim Cross 2022-06-09 5:18 ` Eli Zaretskii 2022-06-09 19:48 ` Alan Mackenzie 2022-06-11 4:09 ` Ihor Radchenko 2022-06-07 16:02 ` Alan Mackenzie 2022-06-07 18:14 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Stefan Monnier 2022-06-07 18:26 ` Org mode and Emacs Lars Ingebrigtsen 2022-06-07 18:48 ` Stefan Monnier 2022-06-07 18:54 ` Eli Zaretskii 2022-06-07 19:38 ` Stefan Monnier 2022-06-07 20:54 ` Lars Ingebrigtsen 2022-06-07 22:10 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Tim Cross 2022-06-08 6:06 ` Org mode and Emacs Visuwesh 2022-06-08 6:58 ` Tim Cross 2022-06-09 22:31 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Richard Stallman 2022-06-09 23:10 ` Tim Cross 2022-06-10 5:59 ` Eli Zaretskii 2022-06-10 6:20 ` Ihor Radchenko 2022-06-10 6:44 ` Eli Zaretskii 2022-06-11 4:49 ` Tim Cross 2022-06-11 6:58 ` Eli Zaretskii 2022-06-11 7:59 ` Tim Cross 2022-06-11 8:29 ` Eli Zaretskii 2022-06-12 9:05 ` Ihor Radchenko 2022-06-12 9:18 ` Eli Zaretskii 2022-06-12 10:04 ` Ihor Radchenko 2022-06-12 10:15 ` Eli Zaretskii 2022-06-12 10:38 ` Ihor Radchenko 2022-06-12 14:36 ` Eli Zaretskii 2022-06-12 15:31 ` Org mode and Emacs Colin Baxter 2022-06-15 5:19 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Ihor Radchenko 2022-06-15 6:46 ` Org mode and Emacs David Engster 2022-06-15 7:36 ` Ihor Radchenko 2022-06-15 13:01 ` Eli Zaretskii 2022-06-16 5:36 ` Ihor Radchenko 2022-06-16 5:58 ` Eli Zaretskii 2022-06-16 9:55 ` Ihor Radchenko 2022-06-15 13:34 ` David Engster 2022-06-16 6:50 ` Ihor Radchenko 2022-06-16 10:21 ` David Engster 2022-06-15 12:50 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 2022-06-16 7:03 ` Ihor Radchenko 2022-06-16 8:13 ` Eli Zaretskii 2022-06-16 11:12 ` Mattias Engdegård 2022-06-16 12:58 ` Ihor Radchenko 2022-06-16 16:59 ` Org mode and Emacs Stefan Monnier 2022-06-17 7:18 ` Larger GC thresholds for non-interactive Emacs (was: Org mode and Emacs) Ihor Radchenko 2022-06-17 13:49 ` Larger GC thresholds for non-interactive Emacs Stefan Monnier 2022-06-17 15:11 ` Lars Ingebrigtsen 2022-06-17 15:30 ` Lars Ingebrigtsen 2022-06-17 16:05 ` Stefan Monnier 2022-06-17 16:11 ` Lars Ingebrigtsen 2022-06-18 5:35 ` Ihor Radchenko 2022-06-18 13:16 ` Stefan Monnier 2022-06-17 17:48 ` Stefan Monnier 2022-06-17 18:20 ` Lars Ingebrigtsen 2022-06-17 18:39 ` Alan Mackenzie 2022-06-17 22:21 ` Stefan Monnier 2022-06-17 22:31 ` Lars Ingebrigtsen 2022-06-17 22:53 ` Stefan Monnier 2022-06-18 6:11 ` Eli Zaretskii 2022-06-18 12:45 ` Lars Ingebrigtsen 2022-06-18 13:26 ` Stefan Monnier 2022-06-18 2:32 ` Stefan Monnier 2022-06-18 12:49 ` Lars Ingebrigtsen 2022-06-18 13:06 ` Stefan Monnier 2022-06-19 11:03 ` Lars Ingebrigtsen 2022-06-18 13:20 ` Eli Zaretskii 2022-06-19 11:02 ` Lars Ingebrigtsen 2022-06-19 11:11 ` Eli Zaretskii 2022-06-21 2:01 ` Lynn Winebarger 2022-06-22 0:01 ` Lynn Winebarger 2022-06-22 12:59 ` Eli Zaretskii 2022-06-22 23:30 ` Lynn Winebarger 2022-06-23 7:09 ` Eli Zaretskii 2022-06-23 7:18 ` Ihor Radchenko 2022-06-23 7:37 ` Eli Zaretskii 2022-06-23 8:02 ` Ihor Radchenko 2022-06-23 8:24 ` Eli Zaretskii 2022-06-23 9:03 ` Ihor Radchenko 2022-06-23 10:08 ` Eli Zaretskii 2022-06-25 5:10 ` Ihor Radchenko 2022-06-25 6:03 ` Eli Zaretskii 2022-06-27 9:32 ` Ihor Radchenko 2022-06-27 13:20 ` Eli Zaretskii 2022-06-29 9:35 ` Ihor Radchenko 2022-06-25 7:51 ` Yuri Khan 2022-06-25 8:28 ` Eli Zaretskii 2022-06-23 17:07 ` Lynn Winebarger 2022-06-23 18:37 ` Eli Zaretskii 2022-06-23 22:08 ` Lynn Winebarger 2022-06-24 6:53 ` Eli Zaretskii 2022-06-25 16:50 ` Lynn Winebarger 2022-06-22 23:26 ` Stefan Monnier 2022-06-22 23:48 ` Lynn Winebarger 2022-06-20 12:42 ` Lynn Winebarger 2022-06-19 7:25 ` Ihor Radchenko 2022-06-19 8:51 ` Eli Zaretskii 2022-06-19 9:23 ` Ihor Radchenko 2022-06-19 9:23 ` Ihor Radchenko 2022-06-19 22:11 ` Stefan Monnier 2022-06-20 12:21 ` Ihor Radchenko 2022-06-23 22:12 ` Stefan Monnier 2022-06-25 5:13 ` Ihor Radchenko 2022-06-25 6:08 ` Eli Zaretskii 2022-06-25 8:33 ` Stefan Monnier 2022-06-18 5:58 ` Eli Zaretskii 2022-06-18 13:46 ` Stefan Monnier 2022-06-18 14:13 ` Eli Zaretskii 2022-06-18 5:28 ` Ihor Radchenko 2022-07-01 2:34 ` Lisp-level macro to avoid excessive GC in memory-allocating code (was: Larger GC thresholds for non-interactive Emacs) Ihor Radchenko 2022-07-01 6:18 ` Eli Zaretskii 2022-07-01 7:52 ` Ihor Radchenko 2022-07-01 10:45 ` Eli Zaretskii 2022-07-01 11:12 ` Ihor Radchenko 2022-07-01 13:56 ` Lisp-level macro to avoid excessive GC in memory-allocating code Stefan Monnier 2022-07-01 14:10 ` Eli Zaretskii 2022-06-21 13:04 ` Larger GC thresholds for non-interactive Emacs Lars Ingebrigtsen 2022-06-21 13:35 ` Lars Ingebrigtsen 2022-06-16 3:19 ` Org mode and Emacs Pankaj Jangid 2022-06-16 4:03 ` Visuwesh 2022-09-25 2:14 ` Bastien 2022-06-12 14:58 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 2022-06-15 5:48 ` Ihor Radchenko 2022-06-15 16:49 ` Eli Zaretskii 2022-06-17 6:11 ` Ihor Radchenko 2022-06-17 6:41 ` Eli Zaretskii 2022-06-12 19:25 ` Jean Louis 2022-06-13 12:14 ` Eli Zaretskii 2022-06-13 22:37 ` Richard Stallman 2022-06-14 0:43 ` Ihor Radchenko [not found] ` <87h74mv56b.fsf@localhost> 2022-06-17 6:42 ` Org syntax compatibility with texinfo syntax (was: Org mode and Emacs (was: Convert README.org to plain text README while installing package)) Ihor Radchenko 2022-06-14 2:28 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 2022-06-14 2:45 ` Ihor Radchenko 2022-06-14 11:04 ` Eli Zaretskii 2022-06-14 11:18 ` Ihor Radchenko 2022-06-14 11:44 ` Eli Zaretskii 2022-06-12 22:38 ` Richard Stallman 2022-06-13 4:38 ` Org mode and Emacs Werner LEMBERG 2022-06-13 16:28 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Christopher Dimech 2022-06-12 0:42 ` Richard Stallman 2022-06-12 1:27 ` Ihor Radchenko 2022-06-12 5:45 ` Eli Zaretskii 2022-06-12 22:38 ` Richard Stallman 2022-06-12 22:56 ` Tim Cross 2022-06-13 0:39 ` Ihor Radchenko 2022-06-13 1:42 ` Tim Cross 2022-06-13 2:40 ` Ihor Radchenko 2022-06-13 1:47 ` Christopher Dimech 2022-06-13 2:47 ` Ihor Radchenko 2022-06-13 5:04 ` Christopher Dimech 2022-06-13 5:22 ` Org mode and Emacs Werner LEMBERG 2022-06-13 5:59 ` Eli Zaretskii 2022-06-13 11:54 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 2022-06-14 13:32 ` Ihor Radchenko 2022-06-14 13:45 ` Eli Zaretskii 2022-06-15 23:15 ` Richard Stallman 2022-06-17 6:43 ` Ihor Radchenko 2022-06-14 13:18 ` Ihor Radchenko 2022-06-14 13:38 ` Org mode and Emacs Robert Pluim 2022-06-15 6:13 ` Cusom special block export, similar org org-link :export parameter (was: Org mode and Emacs) Ihor Radchenko 2022-06-15 23:15 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Richard Stallman 2022-06-17 6:52 ` Ihor Radchenko 2022-06-12 0:42 ` Richard Stallman 2022-06-12 1:39 ` Tim Cross 2022-06-12 2:40 ` Org mode and Emacs T.V Raman 2022-06-12 6:02 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Eli Zaretskii 2022-06-12 22:38 ` Richard Stallman 2022-06-13 2:29 ` Eli Zaretskii 2022-06-12 1:45 ` Org mode and Emacs Po Lu 2022-06-12 2:15 ` Ihor Radchenko 2022-06-12 2:36 ` David Masterson 2022-06-12 3:06 ` Ihor Radchenko 2022-06-12 3:39 ` David Masterson 2022-06-12 4:43 ` Tim Cross 2022-06-12 5:08 ` Po Lu 2022-06-12 5:20 ` Ihor Radchenko 2022-06-12 5:27 ` Tim Cross 2022-06-12 5:53 ` David Masterson 2022-06-12 6:56 ` Ihor Radchenko 2022-06-12 18:29 ` David Masterson 2022-06-14 5:09 ` Ihor Radchenko 2022-06-19 23:48 ` David Masterson 2022-06-20 0:03 ` Ihor Radchenko 2022-06-20 0:24 ` David Masterson 2022-06-12 3:28 ` Tim Cross 2022-06-12 2:50 ` Po Lu 2022-06-12 3:54 ` chad 2022-06-12 5:04 ` Po Lu 2022-06-12 7:02 ` Ihor Radchenko 2022-06-12 22:38 ` Richard Stallman 2022-06-12 22:38 ` Richard Stallman 2022-06-12 6:21 ` Eli Zaretskii 2022-06-12 6:57 ` Eli Zaretskii 2022-06-12 4:53 ` Org mode and Emacs (was: Convert README.org to plain text README while installing package) Christopher Dimech 2022-06-10 13:56 ` Ihor Radchenko 2022-06-10 19:32 ` Akib Azmain Turja 2022-06-08 13:22 ` Ihor Radchenko 2022-06-08 13:33 ` Stefan Kangas 2022-06-08 14:23 ` Org mode and Emacs Stefan Monnier 2022-06-08 15:08 ` Ihor Radchenko 2022-06-12 22:38 ` Richard Stallman 2022-06-07 21:51 ` Convert README.org to plain text README while installing package Tim Cross 2022-06-08 19:26 ` chad 2022-06-06 19:19 ` Alan Mackenzie 2022-06-07 10:50 ` Protesilaos Stavrou 2022-06-07 12:07 ` Philip Kaludercic 2022-06-07 12:23 ` Protesilaos Stavrou 2022-06-07 12:27 ` Stefan Monnier 2022-06-07 0:43 ` Po Lu 2022-06-07 6:46 ` Kévin Le Gouguec 2022-06-07 7:53 ` Po Lu 2022-06-07 22:15 ` Kévin Le Gouguec 2022-06-08 0:36 ` Po Lu 2022-06-08 6:41 ` Kévin Le Gouguec 2022-06-06 16:56 ` Michael Albinus 2022-06-07 20:57 ` Jean Louis 2022-06-08 6:50 ` Tim Cross 2022-06-08 7:25 ` Ihor Radchenko 2022-06-08 7:43 ` Tim Cross 2022-06-08 11:27 ` Jean Louis 2022-06-08 13:24 ` Ihor Radchenko 2022-06-08 11:39 ` Jean Louis 2022-06-11 6:40 ` Jean Louis 2022-06-11 11:16 ` Protesilaos Stavrou 2022-06-12 11:32 ` Jean Louis 2022-06-12 2:23 ` Tim Cross 2022-06-12 11:41 ` Jean Louis 2022-06-12 21:58 ` [External] : " Drew Adams 2022-06-13 22:37 ` Richard Stallman 2022-06-05 11:23 ` Ihor Radchenko 2022-06-06 0:33 ` Lars Ingebrigtsen 2022-06-06 0:41 ` Lars Ingebrigtsen 2022-06-06 0:59 ` Ihor Radchenko 2022-06-06 1:07 ` Lars Ingebrigtsen 2022-06-06 1:33 ` Ihor Radchenko 2022-06-06 7:39 ` Tassilo Horn 2022-06-08 12:56 ` Ihor Radchenko 2022-06-06 12:54 ` Lars Ingebrigtsen 2022-06-06 12:56 ` Lars Ingebrigtsen 2022-06-08 12:55 ` Ihor Radchenko 2022-06-05 11:29 ` Lars Ingebrigtsen 2022-06-05 13:39 ` Stefan Monnier 2022-06-05 14:24 ` Alan Mackenzie 2022-06-05 14:38 ` Stefan Monnier 2022-06-05 11:45 ` Po Lu 2022-06-05 11:59 ` Ihor Radchenko 2022-06-05 13:11 ` Akib Azmain Turja 2022-06-05 15:32 ` Tassilo Horn 2022-06-05 17:46 ` [External] : " Drew Adams 2022-06-05 17:57 ` Tassilo Horn 2022-06-05 20:12 ` Stefan Monnier 2022-06-06 0:24 ` Drew Adams 2022-06-06 0:48 ` Ihor Radchenko 2022-06-06 1:18 ` Drew Adams 2022-06-06 1:38 ` Ihor Radchenko 2022-06-06 1:41 ` Drew Adams 2022-06-06 1:47 ` Stefan Monnier 2022-06-06 7:44 ` Tassilo Horn 2022-06-06 9:34 ` Ihor Radchenko 2022-06-06 0:23 ` Drew Adams 2022-06-06 7:20 ` Tassilo Horn 2022-06-06 15:10 ` Drew Adams 2022-06-05 15:52 ` Philip Kaludercic 2022-06-05 17:44 ` Stefan Monnier 2022-06-06 10:35 ` Philip Kaludercic 2022-06-07 17:55 ` Stefan Monnier 2022-06-24 7:18 ` Akib Azmain Turja 2022-06-24 7:45 ` Philip Kaludercic 2022-07-15 12:24 ` Akib Azmain Turja 2022-07-15 13:07 ` Philip Kaludercic 2022-07-15 14:54 ` Akib Azmain Turja 2022-06-24 15:42 ` Philip Kaludercic 2022-06-24 17:50 ` Stefan Monnier 2022-06-26 10:25 ` Philip Kaludercic 2022-06-26 12:22 ` Stefan Monnier 2022-06-26 13:08 ` Philip Kaludercic 2022-06-26 15:23 ` Stefan Monnier 2022-06-27 10:04 ` Philip Kaludercic 2022-06-27 12:12 ` Stefan Monnier 2022-06-27 14:29 ` Yuri Khan 2022-06-27 16:44 ` Philip Kaludercic 2022-06-05 11:59 ` Akib Azmain Turja 2022-06-08 17:21 ` Yoni Rabkin 2022-06-05 15:15 ` Ihor Radchenko 2022-06-05 15:22 ` Eli Zaretskii 2022-06-05 15:27 ` Eli Zaretskii 2022-06-05 15:41 ` Ihor Radchenko 2022-06-05 15:46 ` Eli Zaretskii 2022-06-05 15:53 ` Ihor Radchenko 2022-06-05 16:25 ` Eli Zaretskii 2022-06-05 15:53 ` Eli Zaretskii 2022-06-05 15:58 ` Ihor Radchenko 2022-06-05 16:27 ` Eli Zaretskii 2022-06-05 16:16 ` Tassilo Horn 2022-06-05 17:40 ` Stefan Monnier 2022-06-08 12:51 ` Ihor Radchenko 2022-06-08 14:17 ` Eli Zaretskii 2022-06-08 14:38 ` Ihor Radchenko 2022-06-08 14:43 ` Stefan Monnier 2022-06-08 15:44 ` Ihor Radchenko 2022-06-08 17:37 ` Stefan Monnier 2022-06-09 3:11 ` Ihor Radchenko 2022-06-08 16:02 ` Eli Zaretskii 2022-06-04 17:32 ` Akib Azmain Turja 2022-06-04 17:22 ` Akib Azmain Turja 2022-06-04 16:36 ` Stefan Monnier 2022-06-05 12:29 ` Akib Azmain Turja 2022-06-05 13:34 ` Stefan Monnier 2022-06-06 3:34 ` Akib Azmain Turja -- strict thread matches above, loose matches on Subject: below -- 2022-09-25 2:52 Org mode and Emacs Payas Relekar 2022-09-25 6:35 ` Bastien 2022-09-25 7:05 ` Ihor Radchenko 2022-09-25 7:47 ` Bastien 2022-09-25 8:01 ` Ihor Radchenko 2022-09-25 8:22 ` Bastien Guerry 2022-09-25 8:01 ` Eli Zaretskii 2022-09-25 19:47 ` Tim Cross 2022-09-26 6:12 ` Bastien 2022-09-26 6:35 ` Ihor Radchenko 2022-09-26 6:57 ` Bastien 2023-08-18 17:09 ` Ihor Radchenko 2023-08-18 18:31 ` Eli Zaretskii 2023-08-18 18:49 ` Ihor Radchenko 2023-08-18 19:11 ` Eli Zaretskii 2023-08-18 19:31 ` Ihor Radchenko 2023-08-19 5:51 ` Eli Zaretskii 2023-08-19 9:04 ` Ihor Radchenko 2023-08-19 9:26 ` Eli Zaretskii 2023-08-19 9:44 ` Ihor Radchenko 2023-08-19 10:19 ` Po Lu 2023-08-19 10:47 ` Eli Zaretskii 2023-08-19 10:48 ` Ihor Radchenko 2023-08-19 10:42 ` Eli Zaretskii 2023-08-19 10:54 ` Ihor Radchenko 2023-08-21 1:12 ` Richard Stallman 2023-08-21 7:56 ` Philip Kaludercic 2023-08-23 2:12 ` Richard Stallman 2023-08-23 9:44 ` Ihor Radchenko 2023-08-23 11:01 ` Colin Baxter 2023-08-23 11:12 ` Ihor Radchenko 2023-08-23 12:49 ` Po Lu 2023-08-23 17:18 ` Colin Baxter 2023-08-23 17:47 ` Ihor Radchenko 2023-08-23 18:02 ` Eli Zaretskii 2023-08-23 18:08 ` Ihor Radchenko 2023-08-23 18:18 ` Eli Zaretskii 2023-08-23 18:36 ` Ihor Radchenko 2023-08-23 18:46 ` Eli Zaretskii 2023-08-23 18:50 ` Ihor Radchenko 2023-08-25 1:11 ` Richard Stallman 2023-08-23 16:53 ` Colin Baxter 2023-08-23 17:56 ` Tassilo Horn 2023-08-24 11:52 ` Bastien Guerry 2023-08-24 17:51 ` T.V Raman 2023-08-24 17:55 ` Ihor Radchenko 2023-08-24 18:35 ` T.V Raman 2023-08-25 1:16 ` Richard Stallman 2023-08-25 11:45 ` Richard Stallman 2023-08-25 1:14 ` Richard Stallman 2023-08-25 9:04 ` Bastien Guerry 2023-08-25 18:56 ` Philip Kaludercic 2023-08-26 10:46 ` Ihor Radchenko 2023-08-31 2:09 ` Richard Stallman 2023-08-31 8:50 ` Ihor Radchenko 2023-08-26 2:04 ` Richard Stallman 2023-08-26 5:50 ` Eli Zaretskii 2023-08-26 16:49 ` Jose E. Marchesi 2023-08-26 16:56 ` Ihor Radchenko 2023-08-26 20:28 ` Philip Kaludercic 2023-08-26 20:58 ` Ihor Radchenko 2023-08-30 8:11 ` Bastien Guerry 2023-08-27 1:32 ` Richard Stallman 2023-08-27 8:32 ` Ihor Radchenko 2023-08-28 1:32 ` Richard Stallman 2023-08-29 8:29 ` Ihor Radchenko 2023-09-01 1:18 ` Richard Stallman 2023-09-01 6:46 ` Ihor Radchenko 2023-09-04 1:32 ` Richard Stallman 2023-09-04 22:05 ` Juergen Fenn 2023-09-05 11:04 ` Ihor Radchenko 2023-09-06 0:58 ` Richard Stallman 2023-09-06 11:29 ` Ihor Radchenko 2023-09-06 12:33 ` Eli Zaretskii 2023-09-06 12:43 ` Ihor Radchenko 2023-09-06 12:49 ` Po Lu 2023-09-06 12:54 ` Ihor Radchenko 2023-09-06 13:04 ` Po Lu 2023-09-06 13:10 ` Ihor Radchenko 2023-09-06 13:33 ` Po Lu 2023-09-06 15:28 ` Eli Zaretskii 2023-09-06 13:08 ` Eli Zaretskii 2023-09-06 13:20 ` Ihor Radchenko 2023-09-06 15:25 ` Eli Zaretskii 2023-09-06 16:12 ` Ihor Radchenko 2023-09-06 16:34 ` Eli Zaretskii 2023-09-07 11:11 ` Ihor Radchenko 2023-09-10 0:22 ` Richard Stallman 2023-09-09 0:37 ` Richard Stallman 2023-09-09 9:36 ` Ihor Radchenko 2023-09-10 0:22 ` Richard Stallman 2023-09-09 0:38 ` Richard Stallman 2023-08-30 8:11 ` Bastien Guerry 2023-09-02 1:50 ` Richard Stallman 2023-09-02 8:19 ` Ihor Radchenko 2023-09-02 8:30 ` Alfred M. Szmidt 2023-08-27 1:32 ` Richard Stallman 2023-08-27 8:35 ` Ihor Radchenko 2023-08-28 1:32 ` Richard Stallman 2023-08-28 10:04 ` Ihor Radchenko 2023-08-28 11:15 ` Yuri Khan 2023-08-28 11:21 ` Ihor Radchenko 2023-08-31 2:09 ` Richard Stallman 2023-08-31 8:53 ` Ihor Radchenko 2023-09-04 1:33 ` Richard Stallman 2023-08-30 8:14 ` Bastien Guerry 2023-08-30 8:32 ` Po Lu 2023-08-27 1:33 ` Richard Stallman 2022-09-26 8:24 ` Eli Zaretskii 2022-09-26 8:32 ` Jean Louis 2022-09-26 9:54 ` Ihor Radchenko 2022-09-26 11:04 ` Robert Pluim 2022-09-27 16:17 ` Richard Stallman 2022-09-30 3:41 ` Ihor Radchenko 2022-09-26 12:10 ` Richard Stallman 2022-09-26 3:02 ` Ihor Radchenko 2022-09-26 5:37 ` Po Lu 2022-09-26 5:44 ` Emanuel Berg 2022-09-26 6:20 ` Bastien Guerry 2022-09-26 13:58 ` T.V Raman 2022-09-26 16:16 ` Eli Zaretskii 2022-09-26 6:36 ` Ihor Radchenko 2022-09-26 6:18 ` Bastien 2022-09-26 6:29 ` Ihor Radchenko
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).