Some Issues I Encountered While Developing Metanote

Some Issues I Encountered While Developing Metanote

[note meta]

[-- Attachment #1: Type: text/plain, Size: 3355 bytes --]

Dear org mode developers,

I'm the author of Metanote. A few days ago, I posted a thread on Reddit
with the title "Let's Talk About the Story of Metanote and Free Software."
https://www.reddit.com/r/emacs/comments/16db9gz/lets_talk_about_the_story_of_metanote_and_free/
You  can learn more about Metanote and me from there. In the replies to
that post, someone mentioned here, so here I am.

During the development of Metanote, I encountered some issues, as follows:
(Note that my testing environment is based on org mode 9.3.6, GNU Emacs
28.2. I haven't tested with the latest org mode, so I'm not sure if these
issues still exist.)

1. Dynamic Block Parsing Issue

   #+begin_example
#+begin
#+end:
   #+end_example

   The above is a Dynamic Block, and the parsed syntax tree is:

   (org-data nil (section (:begin 1 :end 17 :contents-begin 1 :contents-end
17 :post-blank 0 :post-affiliated 1 :parent #0) (dynamic-block (:begin 1
:end 17 :block-name nil :arguments nil :contents-begin nil :contents-end
nil :post-blank 0 :post-affiliated 1 :parent #1))))

   However, the following is not recognized as a Dynamic Block:

   #+begin_example
a
#+begin
#+end:
   #+end_example

   the parsed syntax tree is:

(org-data nil (section (:begin 1 :end 18 :contents-begin 1 :contents-end 18
:post-blank 0 :post-affiliated 1 :parent #0) (paragraph (:begin 1 :end 11
:contents-begin 1 :contents-end 11 :post-blank 0 :post-affiliated 1 :parent
#1)) (keyword (:key "END" :value "" :begin 11 :end 18 :post-blank 0
:post-affiliated 11 :parent #1))))

2. About Paragraph

   #+begin_example
+--+
+--+
+this is a paragraph+
+this is a paragraph+
   #+end_example

There should be only one paragraph above, but it's parsed as two:

(org-data nil (section (:begin 1 :end 55 :contents-begin 1 :contents-end 55
:post-blank 0 :post-affiliated 1 :parent #0) (paragraph (:begin 1 :end 6
:contents-begin 1 :contents-end 6 :post-blank 0 :post-affiliated 1 :parent
#1)) (paragraph (:begin 6 :end 55 :contents-begin 6 :contents-end 55
:post-blank 0 :post-affiliated 6 :parent #1))))

3. About Pre-blank

   The first blank line in a Greater Element is parsed as a paragraph.
However, this behavior is inconsistent in Items and Footnote definitions.
In Items and Footnote definitions, a blank line is parsed as a Pre-blank.
In Footnote definitions, especially, this blank line appears only once
because having more than one would break the Footnote definitions.

#+begin_example
[fn:a]

bbb
#+end_example

(org-data nil (section (:begin 1 :end 13 :contents-begin 1 :contents-end 13
:post-blank 0 :post-affiliated 1 :parent #0) (footnote-definition (:label
"a" :begin 1 :end 13 :contents-begin 9 :contents-end 13 :pre-blank 2
:post-blank 0 :post-affiliated 1 :parent #1) (paragraph (:begin 9 :end 13
:contents-begin 9 :contents-end 13 :post-blank 0 :post-affiliated 9 :parent
#2)))))


I'm a big fan of org mode. A few years ago, I couldn't find an app that
supported org mode on my phone, so I created one myself. I hope that
through my app, I can introduce org mode to more people, even those who are
not familiar with Emacs. I also hope to make some contributions to org mode.

Additionally, is it possible to include Metanote in the 'Tools that work
with Org' section at https://orgmode.org/tools.html ?  Metanote website is
https://metanote-dev.github.io/.

Best regards,
Zhen

[-- Attachment #2: Type: text/html, Size: 3877 bytes --]

Re: Some Issues I Encountered While Developing Metanote

[Ihor Radchenko]

note meta <metanote.team@gmail.com> writes:

> 1. Dynamic Block Parsing Issue
>
>    #+begin_example
> #+begin
> #+end:
>    #+end_example
>
>    The above is a Dynamic Block, and the parsed syntax tree is:
>
>    (org-data nil (section (:begin 1 :end 17 :contents-begin 1 :contents-end
> 17 :post-blank 0 :post-affiliated 1 :parent #0) (dynamic-block (:begin 1
> :end 17 :block-name nil :arguments nil :contents-begin nil :contents-end
> nil :post-blank 0 :post-affiliated 1 :parent #1))))

Are you sure that you meant

#+begin
#+end:

Maybe

#+begin:
#+end:
?

Either way, there is no dynamic block in both cases - dynamic blocks
must have non-empty NAME. See
https://orgmode.org/worg/org-syntax.html#Dynamic_Blocks

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[BUG] org-element-paragraph-separate is not accurate when detecting table.el tables (was: Some Issues I Encountered While Developing Metanote)

[Ihor Radchenko]

note meta <metanote.team@gmail.com> writes:

> 2. About Paragraph
>
>    #+begin_example
> +--+
> +--+
> +this is a paragraph+
> +this is a paragraph+
>    #+end_example
>
> There should be only one paragraph above, but it's parsed as two:
>
> (org-data nil (section (:begin 1 :end 55 :contents-begin 1 :contents-end 55
> :post-blank 0 :post-affiliated 1 :parent #0) (paragraph (:begin 1 :end 6
> :contents-begin 1 :contents-end 6 :post-blank 0 :post-affiliated 1 :parent
> #1)) (paragraph (:begin 6 :end 55 :contents-begin 6 :contents-end 55
> :post-blank 0 :post-affiliated 6 :parent #1))))

Confirmed.
`org-element-paragraph-separate', which is used to determine paragraph
boundaries incorrectly detects the above +--+ as table.el table
boundary. It matches

^[ \t]*\+\(?:-+\+)+[ \t]*$

while the actual criteria to match table.el tables is much more complex:

   ;; There is no strict definition of a table.el
   ;; table.  Try to prevent false positive while being
   ;; quick.
   (let ((rule-regexp
	  (rx (zero-or-more (any " \t"))
	      "+"
	      (one-or-more (one-or-more "-") "+")
	      (zero-or-more (any " \t"))
	      eol))
	 (non-table.el-line
	  (rx bol
	      (zero-or-more (any " \t"))
	      (or eol (not (any "+| \t")))))
	 (next (line-beginning-position 2)))
     ;; Start with a full rule.
     (and
      (looking-at-p rule-regexp)
      (< next limit) ;no room for a table.el table
      (save-excursion
	(end-of-line)
	(cond
	 ;; Must end with a full rule.
	 ((not (re-search-forward non-table.el-line limit 'move))
	  (if (bolp) (forward-line -1) (forward-line 0))
	  (looking-at-p rule-regexp))
	 ;; Ignore pseudo-tables with a single
	 ;; rule.
	 ((= next (line-beginning-position))
	  nil)
	 ;; Must end with a full rule.
	 (t
	  (forward-line -1)
	  (looking-at-p rule-regexp))))))

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Some Issues I Encountered While Developing Metanote

[Ihor Radchenko]

note meta <metanote.team@gmail.com> writes:

> 3. About Pre-blank
>
>    The first blank line in a Greater Element is parsed as a paragraph.
> However, this behavior is inconsistent in Items and Footnote definitions.
> In Items and Footnote definitions, a blank line is parsed as a Pre-blank.
> In Footnote definitions, especially, this blank line appears only once
> because having more than one would break the Footnote definitions.
>
> #+begin_example
> [fn:a]
>
> bbb
> #+end_example

Not just items and footnotes, but also headings.

AFAIK, the original motivations to separate :pre-blank from contents was
to avoid problems during export to LaTeX, where blank lines can
sometimes break document structure because blank lines serve as
paragraph separators.

In practice, it does not matter much though - ox-latex cleans up blank
lines in paragraphs.

The only place where :pre-blank is actually utilized is headings.

That said, removing :pre-blank will be a breaking change in the parser.
And I do not see any clear practical benefit dropping it.
So, we should probably leave the current idiosyncrasy for items,
footnote-definitions, and headlines as is.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Some Issues I Encountered While Developing Metanote

[Ihor Radchenko]

note meta <metanote.team@gmail.com> writes:

> I'm a big fan of org mode. A few years ago, I couldn't find an app that
> supported org mode on my phone, so I created one myself. I hope that
> through my app, I can introduce org mode to more people, even those who are
> not familiar with Emacs. I also hope to make some contributions to org mode.

Contributions benefiting both you and Org mode are certainly welcome.

> Additionally, is it possible to include Metanote in the 'Tools that work
> with Org' section at https://orgmode.org/tools.html ?  Metanote website is
> https://metanote-dev.github.io/.

Unfortunately, we cannot recommend Metanote on
https://orgmode.org/tools.html page.

This page, unlike WORG wiki, is the official GNU Org mode web-page.

We, as a GNU project, are obliged to follow FSF guidelines, including
https://www.gnu.org/prep/maintain/maintain.html#Ethical-and-Philosophical-Consideration
that explicitly prohibit recommending non-Libre software.

Metanote is a closed source, non-Libre software and thus cannot be
recommended by us officially.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Some Issues I Encountered While Developing Metanote

[Ihor Radchenko]

[ Adding Org mailing list back to CC. Please use "Reply All" or "Wide
  reply" to keep the discussion public ]

note meta <metanote.team@gmail.com> writes:

> Ihor Radchenko <yantar92@posteo.net> 于2023年9月12日周二 16:15写道：
>>
>> ...
>> Are you sure that you meant
>>
>> #+begin
>> #+end:
>>
>> Maybe
>>
>> #+begin:
>> #+end:
>> ?
>>
>> Either way, there is no dynamic block in both cases - dynamic blocks
>> must have non-empty NAME. See
>> https://orgmode.org/worg/org-syntax.html#Dynamic_Blocks
>>
> Indeed, according to the org mode syntax documentation, neither of
> these examples should be a Dynamic block.
>
> #+begin (here is a space)
> #+end:
>
> I double-checked, and from the output of the org mode parser, it
> indeed get parsed as a Dynamic block, this is the output:

Not on the latest Org.
I was also not able to reproduce using built-in Org versions in Emacs
29, 28, and 27. In Emacs 26, I can reproduce.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>