About 'inline special blocks'

About 'inline special blocks'

[Juan Manuel Macías]

Hi all,

I think this idea was suggested by Ihor in a thread from a few months
ago (I don't remember which one), but since other topics were discussed,
the idea remained a bit in limbo. I still find the idea very
interesting, and I think it would be very productive for Org to have a
multipurpose inline container, so it occurred to me to open this thread
to invite a possible discussion on the subject.

I suppose it would have been more practical to start the thread directly
proposing a patch, but since it is about adding a new element to Org,
which is not trivial, I thought that maybe it would be more convenient
to have a previous discussion and poll the users' and developers opinion.

The question is: Does Org Mode need inline special blocks? On the one
hand, it seems that we can live without them. Macros and links can work
competently for this purpose. But macros have the drawback of the comma
as an escape character; and links also have its drawbacks, although the
org-link-set-parameters function is very versatile. And even a huge fan
of links like me can recognize these limitations :-). For example, we
cannot put a footnote inside a link.

Therefore, I think that inline special blocks would fill an important
gap. They could be translated into HTML as a <span></span> container; to
odt as character styles or to LaTeX as commands with arguments. And they
would open the doors to a real solution for multilingual support in Org.

Perhaps the syntax could be a continuation of that of inline code
blocks. Something like:

<name>_[options]{text}

And in options include some 'jibarized' version of attr_latex,
attr_html, etc.

Well, I don't know if I have managed to sell the product well or if I
have been too abstract :-)

Best regards,

Juan Manuel 

Re: About 'inline special blocks'

[Kaushal Modi]

On Mon, May 23, 2022 at 10:33 AM Juan Manuel Macías
<maciaschain@posteo.net> wrote:

> I think this idea was suggested by Ihor in a thread from a few months
> ago (I don't remember which one), but since other topics were discussed,
> the idea remained a bit in limbo. I still find the idea very
> interesting, and I think it would be very productive for Org to have a
> multipurpose inline container, so it occurred to me to open this thread
> to invite a possible discussion on the subject.

Thanks for doing this. I missed that thread. I would welcome this
feature addition too.

If I understand correctly, this will mean adding a new element type
that all the Org exporters can then support. Right?

> The question is: Does Org Mode need inline special blocks?

Yes.

> On the one hand, it seems that we can live without them.

Not quite. I developed few hacks in ox-hugo to make regular special
blocks act like special inline blocks :D

Example:

=====
More than the visual inaccuracy of seeing curved quoted where straight
quotes should be,
#+begin_mark
if someone copies that code to try it out, it will
not work
#+end_mark
!
=====

Another example:

=====
By the way, I submitted a patch for fixing the escaping of straight
quotes in ~shortdoc-add-function~ documentation string
#+begin_sidenote
I planned to fix just this straight quote escaping issue, but then I
also ended up slightly improving the documentation of the ~(FUNC :eval
EVAL)~ and other forms used for adding a function's documentation to
~shortdoc~.
#+end_sidenote
in ..
=====

ox-hugo does the job of deleting the newlines and white-space (leaving
just 1) before and after few "special" special Org blocks.


> Therefore, I think that inline special blocks would fill an important
> gap. They could be translated into HTML as a <span></span> container;

+1

> Perhaps the syntax could be a continuation of that of inline code
> blocks. Something like:
>
> <name>_[options]{text}

The challenging part will be deciding the syntax so that there are no
false matches.

May be reserve "inline_" for inline blocks?

e.g. inline_<name>[options]{text}  ?

Using my example above, if I want the <mark> elements in HTML, I would do

abc inline_mark{some text} def

and that would export to below for an HTML based exporter:

abc <mark>some text</mark> def

Re: About 'inline special blocks'

[Juan Manuel Macías]

Hi, Kaushal, thanks for all your interesting comments,

Kaushal Modi writes:

> The challenging part will be deciding the syntax so that there are no
> false matches.
>
> May be reserve "inline_" for inline blocks?
>
> e.g. inline_<name>[options]{text}  ?

It seems to me the most consistent option, if we continue in some way
the syntax of the inline code blocks, which would be the close relatives
of the inline special blocks. Perhaps (to shorten the syntax a bit)
'inline' could be replaced by some reserved symbol. Something like:

&_<name>[options]{text}

I think a major issue would also be how to properly compact <[options]>
so as not to result in too overloaded syntax. Maybe something like:

[latex(list of attributes) html(list of attributes)...]

?

But that is an abuse of direct formatting, which I think should always be
avoided, especially in a format-agnostic environment like Org, which is
more of a logician than a typesetter :-)

And, in any case, it is to be expected that the user will not need to
overload that part, since these hypothetical inline blocks would be
intended for short segments of text within the paragraph. I think the
most typical use case would be something like your 'mark' example.

Best regards,

Juan Manuel 

Re: About 'inline special blocks'

[Tim Cross]

I've not got a lot to add here. I'm not against the idea, but Juan makes
some points below which I think are particularly important and wanted to
add weight to them!

Juan Manuel Macías <maciaschain@posteo.net> writes:

> Hi, Kaushal, thanks for all your interesting comments,
>
> Kaushal Modi writes:
>
>> The challenging part will be deciding the syntax so that there are no
>> false matches.
>>
>> May be reserve "inline_" for inline blocks?
>>
>> e.g. inline_<name>[options]{text}  ?
>
> It seems to me the most consistent option, if we continue in some way
> the syntax of the inline code blocks, which would be the close relatives
> of the inline special blocks. Perhaps (to shorten the syntax a bit)
> 'inline' could be replaced by some reserved symbol. Something like:
>
> &_<name>[options]{text}
>

I agree. Selection of the 'symbol' might be tricky, but the idea is
sound.


> I think a major issue would also be how to properly compact <[options]>
> so as not to result in too overloaded syntax. Maybe something like:
>
> [latex(list of attributes) html(list of attributes)...]
>
> ?
>
> But that is an abuse of direct formatting, which I think should always be
> avoided, especially in a format-agnostic environment like Org, which is
> more of a logician than a typesetter :-)

I think this is a really important point. Whenever we add formatting
specific directives, we always end up in a somewhat uncertain situation
with respect to the other back ends. I also feel that in-line blocks
which support large and complex formatting configuration really defeat
the purpose of an in=line block (which I feel should be kept relatively
simple). I also find complex constructs of this form really degrades the
readability of source documents. 

>
> And, in any case, it is to be expected that the user will not need to
> overload that part, since these hypothetical inline blocks would be
> intended for short segments of text within the paragraph. I think the
> most typical use case would be something like your 'mark' example.
>

Yes, that was my thinking as well. 

Re: About 'inline special blocks'

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]

Hi Tim,

Tim Cross <theophilusx@gmail.com> writes:

>> But that is an abuse of direct formatting, which I think should always be
>> avoided, especially in a format-agnostic environment like Org, which is
>> more of a logician than a typesetter :-)
>
> I think this is a really important point. Whenever we add formatting
> specific directives, we always end up in a somewhat uncertain situation
> with respect to the other back ends. I also feel that in-line blocks
> which support large and complex formatting configuration really defeat
> the purpose of an in=line block (which I feel should be kept relatively
> simple). I also find complex constructs of this form really degrades the
> readability of source documents.

To me, this is another reason for comment and #+attr_X lines not to break
paragraphs, as then we can just re-use #+attr_X lines.

E.g.
┌────
│ Hi there look at
│ #+attr_X: :prop val
│ inline_mark{some content}
│ and now continuing the paragraph...
└────

As mentioned previously, this would also be rather nice for comments in
paragraphs and also applying attributes to a link in a paragraph, i.e.
┌────
│ I'm a big fan of the
│ #+attr_html: :title Visit the Org homepage
│ [[https://orgmode.org/][Org]]
│ project.
└────

All the best,
Timothy

Re: About 'inline special blocks'

[Eric S Fraga]

On Tuesday, 24 May 2022 at 10:51, Timothy wrote:
> To me, this is another reason for comment and #+attr_X lines not to
> break paragraphs [...].

And, in fact, if this were true (which I would like), I personally would
see no reason for having inline special blocks.

Just my 2¢.

-- 
: Eric S Fraga, with org release_9.5.3-511-g8e69ad in Emacs 29.0.50

Re: About 'inline special blocks'

[Christian Moe]

+1

Eric S Fraga writes:

> On Tuesday, 24 May 2022 at 10:51, Timothy wrote:
>> To me, this is another reason for comment and #+attr_X lines not to
>> break paragraphs [...].
>
> And, in fact, if this were true (which I would like), I personally would
> see no reason for having inline special blocks.
>
> Just my 2¢.

Re: About 'inline special blocks'

[Max Nikulin]

On 24/05/2022 09:51, Timothy wrote:
> 
> To me, this is another reason for comment and #+attr_X lines not to break
> paragraphs, as then we can just re-use #+attr_X lines.

I like the idea that comments and attribute lines should not be 
paragraph separators. I expect, it should alleviate the issue that LaTeX 
and Org paragraphs may significantly differ. Do somebody has examples 
when such change will cause negative effects (besides broken 
compatibility, of course)?

> ┌────
> │ Hi there look at
> │ #+attr_X: :prop val
> │ inline_mark{some content}
> │ and now continuing the paragraph...
> └────

However I am afraid that using the same construct for block-level 
elements and inline object will cause confusion. Consider a paragraph 
starting from a link. Which attributes belongs to the whole paragraph 
and which ones should affect the starting link only?

I consider attributes specific to an inline object as a great feature, 
but I am unsure if it requires special inline object. Would not it be 
enough to allow attributes for already existing objects (emphasis, 
links, citations)? It is feasible to require from external tools such as 
pandoc to support special blocks (likely implemented in lisp code)?

Concerning fear that complicated attributes makes text hardly readable, 
macros might be a rescue, but it would depend on implementation.

I had an idea to implement proof-of-concept for inline attributes using 
a special link type and a parse tree filter that transfers attributes to 
the next object. Unfortunately time related bugs in Emacs appeared to be 
rather time consuming.

---- >8 ----
#+macro: nofollow [[attr:(:html (:rel "nofollow noopener"))]]

An {{{nofollow}}[[attr:(:html (:title "be 
careful!"))]][[http://unsafe.com][unsafe link]].
---- 8< ----

Such implementation would allow to test if it convenient enough and 
whether special blocks are really necessary.

Re: About 'inline special blocks'

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> On 24/05/2022 09:51, Timothy wrote:
>> 
>> To me, this is another reason for comment and #+attr_X lines not to break
>> paragraphs, as then we can just re-use #+attr_X lines.
>
> I like the idea that comments and attribute lines should not be 
> paragraph separators. I expect, it should alleviate the issue that LaTeX 
> and Org paragraphs may significantly differ. Do somebody has examples 
> when such change will cause negative effects (besides broken 
> compatibility, of course)?

I will raise a compatibility issue, but it is bad enough to not think
about other things.
AFAIU, the proposed change will break whole export system? How would you
represent the AST of

First line
# comment
Second line

?

Currently, the above is parsed as 

(org-data
    (section
	(paragraph "First line\n")
	(comment (... :value "comment" ))
	(paragraph "Second line\n"))))

Should we consider a comment as inline object? I suspect that such
change will cause unpredictable breakages all around Org and third-party
packages.

> I had an idea to implement proof-of-concept for inline attributes using 
> a special link type and a parse tree filter that transfers attributes to 
> the next object. Unfortunately time related bugs in Emacs appeared to be 
> rather time consuming.
>
> ---- >8 ----
> #+macro: nofollow [[attr:(:html (:rel "nofollow noopener"))]]
>
> An {{{nofollow}}[[attr:(:html (:title "be 
> careful!"))]][[http://unsafe.com][unsafe link]].
> ---- 8< ----
>
> Such implementation would allow to test if it convenient enough and 
> whether special blocks are really necessary.

I am not sure if I like this idea. It seems fine, but I afraid that it
will complicate parser at some point. We may want to assign such inline
properties to the following object, which is already a pain for
affiliated keywords.

Best,
Ihor

Re: About 'inline special blocks'

[Max Nikulin]

On 25/05/2022 14:22, Ihor Radchenko wrote:
> Max Nikulin writes:
>> On 24/05/2022 09:51, Timothy wrote:
>>>
>>> To me, this is another reason for comment and #+attr_X lines not to break
>>> paragraphs, as then we can just re-use #+attr_X lines.
> 
> I will raise a compatibility issue, but it is bad enough to not think
> about other things.
> AFAIU, the proposed change will break whole export system? How would you
> represent the AST of
> 
> First line
> # comment
> Second line
> 
> Should we consider a comment as inline object? I suspect that such
> change will cause unpredictable breakages all around Org and third-party
> packages.

I believed that comments are stripped before passing to export backend, 
so I did not expect any problem with inline comments.

>> ---- >8 ----
>> #+macro: nofollow [[attr:(:html (:rel "nofollow noopener"))]]
>>
>> An {{{nofollow}}[[attr:(:html (:title "be
>> careful!"))]][[http://unsafe.com][unsafe link]].
>> ---- 8< ----
> 
> I am not sure if I like this idea. It seems fine, but I afraid that it
> will complicate parser at some point. We may want to assign such inline
> properties to the following object, which is already a pain for
> affiliated keywords.

Certainly parser should recognize inline attributes, but I do not expect 
real complications here. Assigning attributes may be performed when AST 
is ready. Just collect attributes and put them to the following 
non-attribute object during breadth-first tree transversal. Attributes 
as the last child is a reason for a warning.

Merging paragraphs separated by comment lines during export (was: About 'inline special blocks')

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> First line
>> # comment
>> Second line
>> 
>> Should we consider a comment as inline object? I suspect that such
>> change will cause unpredictable breakages all around Org and third-party
>> packages.
>
> I believed that comments are stripped before passing to export backend, 
> so I did not expect any problem with inline comments.

You are right. After examining org-export--skip-p and
org-export--prune-tree, I see that all the comments are removed
unconditionally. But then, all we need to do is simply merging
paragraphs separated by a comment with :post-blank = 0 and the first
paragraph having :post-blank = 0. As long as export is concerned it is
simply a question of writing an export filter (Timothy, did you have
anything other than export in mind?)

>>> ---- >8 ----
>>> #+macro: nofollow [[attr:(:html (:rel "nofollow noopener"))]]
>>>
>>> An {{{nofollow}}[[attr:(:html (:title "be
>>> careful!"))]][[http://unsafe.com][unsafe link]].
>>> ---- 8< ----
>> 
>> I am not sure if I like this idea. It seems fine, but I afraid that it
>> will complicate parser at some point. We may want to assign such inline
>> properties to the following object, which is already a pain for
>> affiliated keywords.
>
> Certainly parser should recognize inline attributes, but I do not expect 
> real complications here. Assigning attributes may be performed when AST 
> is ready. Just collect attributes and put them to the following 
> non-attribute object during breadth-first tree transversal. Attributes 
> as the last child is a reason for a warning.

I was mostly thinking about element cache. Dealing with affiliated
keywords caused a lot of pain when I was working on cache. On the other
hand, inline objects are currently parsed together - org-element-context
always parses everything starting from the parent object
:contents-begin. So, maybe it is not going to be as much problem as I
thought.

Another concern about inline attributes is plain-text objects. Consider
the following paragraph:

Some text *bold* plain text. _Underline {{{attribute}}} more text
/italics/ end of underline_.

The object structure will be:
(paragraph
  (plain-text "Some text ")
  (bold
    (plain-text "bold"))
  (plain-text " plain text. ")
  (underline
    (plain-text "Underline ")
    (attribute)
    (plain-text " more text ")
    (italics
      (plain-text "italics"))
    (plain-text " end of underline"))
  (plain-text ".\n"))

So, should the attribute be assigned to " more text "? Just the next
word? What if we have something like

Text {{{attribute to be assigned to "two words"}}}two words but
plain-text element still continues.

Best,
Ihor

Re: About 'inline special blocks'

[Ihor Radchenko]

Tim Cross <theophilusx@gmail.com> writes:

>> I think a major issue would also be how to properly compact <[options]>
>> so as not to result in too overloaded syntax. Maybe something like:
>>
>> [latex(list of attributes) html(list of attributes)...]
>>
>> ?
>>
>> But that is an abuse of direct formatting, which I think should always be
>> avoided, especially in a format-agnostic environment like Org, which is
>> more of a logician than a typesetter :-)
>
> I think this is a really important point. Whenever we add formatting
> specific directives, we always end up in a somewhat uncertain situation
> with respect to the other back ends. I also feel that in-line blocks
> which support large and complex formatting configuration really defeat
> the purpose of an in=line block (which I feel should be kept relatively
> simple). I also find complex constructs of this form really degrades the
> readability of source documents. 

I think that we might simply allow to define complex configuration
before the containing paragraph. Something like:

#+attr_latex[name]: <complex config goes here>
Vestibulum convallis, lorem blockname_[<<name>>]{text} a tempus semper, dui
dui euismod elit, vitae placerat urna tortor vitae lacus.

"<<name>>" will be treated as "<complex config goes here>" during
export/parsing.

I am purposely reusing #+keyword[secondary] and <<name>> syntax to keep
things similar to other existing elements.

Best,
Ihor

Re: About 'inline special blocks'

[João Pedro]

On Tue, May 24 2022 11:56, Ihor Radchenko <yantar92@gmail.com> wrote:

> I think that we might simply allow to define complex configuration
> before the containing paragraph. Something like:

On that note, I think that allowing for inline special blocks would make
that more readable. It would work wonderfully with
org-special-block-extras [1], where the user could define complex blocks
with formatting configuration and what-not, and just use that as an
inline block.

> #+attr_latex[name]: <complex config goes here>
> Vestibulum convallis, lorem blockname_[<<name>>]{text} a tempus semper, dui
> dui euismod elit, vitae placerat urna tortor vitae lacus.
>
> "<<name>>" will be treated as "<complex config goes here>" during
> export/parsing.

Although I do agree that this sort of solves the same problem as the
other approach, but I, personally, find that defining new special blocks
is not only easier to reuse, but more readable as well. But I'm thinking
in terms of org-special-block-extras here, so take my 2 cents with a
grain of salt.


[1] https://github.com/alhassy/org-special-block-extras

Best,

-- 
João Pedro de Amorim Paula
IT undergraduate at Universidade Federal do Rio Grande do Norte (UFRN)

Re: About 'inline special blocks'

[Ihor Radchenko]

João Pedro <jpedrodeamorim@gmail.com> writes:

>> #+attr_latex[name]: <complex config goes here>
>> Vestibulum convallis, lorem blockname_[<<name>>]{text} a tempus semper, dui
>> dui euismod elit, vitae placerat urna tortor vitae lacus.
>>
>> "<<name>>" will be treated as "<complex config goes here>" during
>> export/parsing.
>
> Although I do agree that this sort of solves the same problem as the
> other approach, but I, personally, find that defining new special blocks
> is not only easier to reuse, but more readable as well. But I'm thinking
> in terms of org-special-block-extras here, so take my 2 cents with a
> grain of salt.

I agree. But it is a known problem on defining new specific command vs.
running a new generic command with arguments. You can indeed define a
new command (block in our case), but if you just need to adjust some
parameter once in the whole document, there is no point creating a whole
new block type just for that purpose. Think about defun vs. lambda.

> [1] https://github.com/alhassy/org-special-block-extras

I am not sure if I mentioned this earlier, but org-special-block-extras
could be a good addition to Org core.

Best,
Ihor

Re: About 'inline special blocks'

[João Pedro]

On Thu, May 26 2022 12:56, Ihor Radchenko <yantar92@gmail.com> wrote:

> I agree. But it is a known problem on defining new specific command vs.
> running a new generic command with arguments. You can indeed define a
> new command (block in our case), but if you just need to adjust some
> parameter once in the whole document, there is no point creating a whole
> new block type just for that purpose. Think about defun vs. lambda.

Oh, got it. Well, as I said, I'd be more than happy using the #+attr_X
solution though!

> I am not sure if I mentioned this earlier, but org-special-block-extras
> could be a good addition to Org core.

Has anyone tried reaching out to the author? I think it would be a great
addition! It is a seemingly simple idea that opens up some many
possibilities in Org-mode.

Best,

-- 
João Pedro de Amorim Paula
IT undergraduate at Universidade Federal do Rio Grande do Norte (UFRN)

Re: About 'inline special blocks'

[Ihor Radchenko]

João Pedro <jpedrodeamorim@gmail.com> writes:

>> I am not sure if I mentioned this earlier, but org-special-block-extras
>> could be a good addition to Org core.
>
> Has anyone tried reaching out to the author? I think it would be a great
> addition! It is a seemingly simple idea that opens up some many
> possibilities in Org-mode.

If I recall correctly, it was one of the comments (or questions) when he
presented during Emacsconf. However, I do not recall him asking about
anything on Org mailing list.

I think that you can simply open an issue in his Github repo.

Best,
Ihor

Re: About 'inline special blocks'

[João Pedro]

On Thu, May 26 2022 20:20, Ihor Radchenko <yantar92@gmail.com> wrote:

> I think that you can simply open an issue in his Github repo.

I reached out to him on e-mail, if he doesn't reply there I'll create
the issue. Thanks!

-- 
João Pedro de Amorim Paula
IT undergraduate at Universidade Federal do Rio Grande do Norte (UFRN)

About opening issues vs email [Was: About 'inline special blocks']

[Kaushal Modi]

On Thu, May 26, 2022 at 1:36 PM João Pedro <jpedrodeamorim@gmail.com> wrote:
>
> On Thu, May 26 2022 20:20, Ihor Radchenko <yantar92@gmail.com> wrote:
>
> > I think that you can simply open an issue in his Github repo.
>
> I reached out to him on e-mail, if he doesn't reply there I'll create
> the issue. Thanks!

Just saying this based on my personal preference. I would rather
prefer that people directly open an issue on Github than emailing me
personally.

Reasons:

- Issues section is there for a reason. If the repo owner did not like
people opening Issues, they would disable that section.
- Conversations and discussions are better formatted and readable in
the issue threads.
- If the feature is implemented, I like to link that commit or PR to
that issue so that the entire history and reasoning behind a feature
addition (esp. if it's a breaking one) can be followed through
hyperlinks from the commit log to the issue thread.

Re: About opening issues vs email [Was: About 'inline special blocks']

[Ihor Radchenko]

Kaushal Modi <kaushal.modi@gmail.com> writes:

>> I reached out to him on e-mail, if he doesn't reply there I'll create
>> the issue. Thanks!
>
> Just saying this based on my personal preference. I would rather
> prefer that people directly open an issue on Github than emailing me
> personally.
>
> Reasons:
>
> - Issues section is there for a reason. If the repo owner did not like
> people opening Issues, they would disable that section.

I guess it depends. Some people may not prefer public discussions. At
least not always. The question here is not exactly an issue to be fixed,
but something much more complex.

> - Conversations and discussions are better formatted and readable in
> the issue threads.

How so? No branching of replies (fixed linear layout). No attachments.
Often yet another flavor of markdown.

> - If the feature is implemented, I like to link that commit or PR to
> that issue so that the entire history and reasoning behind a feature
> addition (esp. if it's a breaking one) can be followed through
> hyperlinks from the commit log to the issue thread.

You can equally link a public email exchange.

Best,
Ihor

Re: About opening issues vs email [Was: About 'inline special blocks']

[João Pedro]

Hey Kaushal! Thanks for your insight.

On Thu, May 26 2022 17:22, Kaushal Modi <kaushal.modi@gmail.com> wrote:

> - Issues section is there for a reason. If the repo owner did not like
> people opening Issues, they would disable that section.

I agree with Ihor's response for this point.

> - Conversations and discussions are better formatted and readable in
> the issue threads.

I disagree with this one, I find mailing lists much more well-formatted,
readable and conversation/discussion friendly than PRs on GitHub (it
mostly has to do it the web UI, TBH [1]).

> - If the feature is implemented, I like to link that commit or PR to
> that issue so that the entire history and reasoning behind a feature
> addition (esp. if it's a breaking one) can be followed through
> hyperlinks from the commit log to the issue thread.

But it isn't this particular case. I only emailed asking if he would
have interest on merging his work on Org-mode core, which isn't really a
feature or a modification on the codebase. Nonetheless, one of the first
things I said was that I could open an issue on the public repo if he'd
rather have it documented there as well. I just felt more inclined to
send him an e-mail because 1. he made it public, and 2. I feel more
comfortable using e-mails for communicating such things.

[1] https://asylum.madhouse-project.org/blog/2018/07/24/on-git-github-and-email/

Regards,

-- 
João Pedro de Amorim Paula
IT undergraduate at Universidade Federal do Rio Grande do Norte (UFRN)

Re: About 'inline special blocks'

[Juan Manuel Macías]

Ihor Radchenko writes:

> I think that we might simply allow to define complex configuration
> before the containing paragraph. Something like:
>
> #+attr_latex[name]: <complex config goes here>
> Vestibulum convallis, lorem blockname_[<<name>>]{text} a tempus semper, dui
> dui euismod elit, vitae placerat urna tortor vitae lacus.
>
> "<<name>>" will be treated as "<complex config goes here>" during
> export/parsing.

I really like this idea of taking the complex configuration (in case it
is needed) out of the paragraph. I vote for a procedure of this style.
That rows in favor of legibility and lightness. Of course, the blocks
that need an /ad hoc/ configuration represent, in my opinion, an extreme
use case; and, as I mentioned before, I think that it should be avoided
as much as possible. I also fully agree with Tim's comments on this.
Ideally, any format settings for LaTeX, odt, html, etc. must be done
globally, outside the body.

Best regards,

Juan Manuel 

Re: About 'inline special blocks'

[Ihor Radchenko]

Ihor Radchenko <yantar92@gmail.com> writes:

> Vestibulum convallis, lorem blockname_[<<name>>]{text} a tempus semper, dui
> dui euismod elit, vitae placerat urna tortor vitae lacus.

This thread is possible relevant to the ongoing emacs-devel discussion
where RMS requested support/integration of Org and texinfo.

Richard Stallman wrote:
https://yhetil.org/emacs-devel/E1o1cEe-0006Cj-CT@fencepost.gnu.org

> I don't know for certain that every possible nesting "does the right
> thing".  I do know that @var{} is used inside many other constructs.
> By contrast, @dfn{} would not be nested inside or around other
> contructs very much.  @key can be nested inside @kbd, and it behaves
> a little differently when nested.

He mentioned an important point that could make the idea of inline
special blocks more appealing.

While arbitrary markup can indeed be introduced using our current link
syntax, there is one important limitation of links:

 *** link description cannot contain other links ***

If one seriously tries to extend Org syntax with custom markup elements,
nested markup will not be possible. And we do not want to change Org
links to allow other links inside.

Inline special blocks may not have such limitation if we allow special
blocks to be nested.

Best,
Ihor

Re: About 'inline special blocks'

[Juan Manuel Macías]

Ihor Radchenko <yantar92@gmail.com> writes:

> While arbitrary markup can indeed be introduced using our current link
> syntax, there is one important limitation of links:
>
>  *** link description cannot contain other links ***
>
> If one seriously tries to extend Org syntax with custom markup elements,
> nested markup will not be possible. And we do not want to change Org
> links to allow other links inside.
>
> Inline special blocks may not have such limitation if we allow special
> blocks to be nested.

  +1.

This seems to me a *very* important point.

Best regards,

Juan Manuel 

Re: About 'inline special blocks'

[Juan Manuel Macías]

To add some ideas that have been occurring to me these days...

I am more and more convinced that inline special blocks, by their
nature, should not support fine tune options or anything like
attr_latex, attr_html, etc. like its older brothers, as it would produce
an overly complicated syntax. Big brothers are independent of the
paragraph and there it makes sense to add lines with attr_latex, etc.,
since it is a line-oriented syntax. Bringing that into the paragraph is
unnecessarily overloading the paragraph and breaking the social contract
of lightweight markup, where paragraphs should still look like
paragraphs.

Another argument against possible fine-tuning within inline special
blocks, for export purposes, is that (in my opinion) direct formatting
is a practice that should be avoided as much as possible in a document.
A document with a lot of direct formatting is an inconsistent document.
In html, all possible formatting should be controlled by style sheets;
in LaTeX, by (re)defining macros, commands and environments in the
preamble or in a .sty file; in odt using character styles.

Perhaps if we detach special blocks from fine-tuning possibilities we
lose some (export) flexibility, but we gain in a clearer implementation
of these elements and keep Org aseptic about the output format. And in
any case, if someone needs a fine-tuning in a certain case, there are
always the export filters. Or it can be implemented in a similar way to
inline tasks, with a default format function (for html, latex, etc),
which can be changed via a defcustom.

Starting from that, a syntax like this in Org:

%[name]{contents}

Would produce in LaTeX, by default:

\name{contents}

in html:

<name>contents></name>

in odt:

<text:span text:style-name="name">contents</text:span>

and so on.

In short, I think it would be enough to simply implement something like
this.

Best regards,

Juan Manuel 

Re: About 'inline special blocks'

[Christian Moe]

Juan Manuel Macías writes:

> To add some ideas that have been occurring to me these days...
>

Hi,

This makes sense to me.

Note: For the html output in your example, I expect you don't mean
<name>contents></name>, but <span class="name">contents</span>. That
would give the desired custom style controle of the output, and would
parallel the behavior of special blocks.

If "inline special blocks" will be able to nest, they will have an
advantage over org macros, which cannot.

Apart from nesting, an org macro could do the same job, but less
elegantly. The suggested inline syntax would not require commas to be
escaped in the contents. And it would be somewhat more concise and far
more legible, as illustrated in the below example (with working macros,
imagined inline special blocks, and a CSS implementation):

#+begin_example
#+macro: fmt @@html:<span class="$1">$2</span>@@@@latex:\$1{$2}@@@@odt:<text:span text:style-name="$1">$2</text:span>@@
#+html_head: <style>.highlight {background-color: yellow;}
#+html_head:        .smallcaps {font-variant: small-caps;}</style>

This is some {{{fmt(highlight, highlighted text)}}} and this is some
{{{fmt(smallcaps, text in small caps)}}}.

This is some %[highlight]{highlighted text} and this is some
%[smallcaps]{text in small caps}.
#+end_example

Yours,
Christian

> I am more and more convinced that inline special blocks, by their
> nature, should not support fine tune options or anything like
> attr_latex, attr_html, etc. like its older brothers, as it would produce
> an overly complicated syntax. Big brothers are independent of the
> paragraph and there it makes sense to add lines with attr_latex, etc.,
> since it is a line-oriented syntax. Bringing that into the paragraph is
> unnecessarily overloading the paragraph and breaking the social contract
> of lightweight markup, where paragraphs should still look like
> paragraphs.
>
> Another argument against possible fine-tuning within inline special
> blocks, for export purposes, is that (in my opinion) direct formatting
> is a practice that should be avoided as much as possible in a document.
> A document with a lot of direct formatting is an inconsistent document.
> In html, all possible formatting should be controlled by style sheets;
> in LaTeX, by (re)defining macros, commands and environments in the
> preamble or in a .sty file; in odt using character styles.
>
> Perhaps if we detach special blocks from fine-tuning possibilities we
> lose some (export) flexibility, but we gain in a clearer implementation
> of these elements and keep Org aseptic about the output format. And in
> any case, if someone needs a fine-tuning in a certain case, there are
> always the export filters. Or it can be implemented in a similar way to
> inline tasks, with a default format function (for html, latex, etc),
> which can be changed via a defcustom.
>
> Starting from that, a syntax like this in Org:
>
> %[name]{contents}
>
> Would produce in LaTeX, by default:
>
> \name{contents}
>
> in html:
>
> <name>contents></name>
>
> in odt:
>
> <text:span text:style-name="name">contents</text:span>
>
> and so on.
>
> In short, I think it would be enough to simply implement something like
> this.
>
> Best regards,
>
> Juan Manuel

Re: About 'inline special blocks'

[Juan Manuel Macías]

Hi, Christian,

Thanks for your comments.

Christian Moe writes:

> Hi,
>
> This makes sense to me.
>
> Note: For the html output in your example, I expect you don't mean
> <name>contents></name>, but <span class="name">contents</span>. That
> would give the desired custom style controle of the output, and would
> parallel the behavior of special blocks.

You are absolutely right, it is my fault. These days I'm doing a work
with a lot of xml, and I've mixed things up in my head :-). In html the
expected form is as you say. Apologize for the confusion.

> If "inline special blocks" will be able to nest, they will have an
> advantage over org macros, which cannot.
>
> Apart from nesting, an org macro could do the same job, but less
> elegantly. The suggested inline syntax would not require commas to be
> escaped in the contents. And it would be somewhat more concise and far
> more legible, as illustrated in the below example (with working macros,
> imagined inline special blocks, and a CSS implementation):
>
> #+begin_example
> #+macro: fmt @@html:<span class="$1">$2</span>@@@@latex:\$1{$2}@@@@odt:<text:span text:style-name="$1">$2</text:span>@@
> #+html_head: <style>.highlight {background-color: yellow;}
> #+html_head:        .smallcaps {font-variant: small-caps;}</style>
>
> This is some {{{fmt(highlight, highlighted text)}}} and this is some
> {{{fmt(smallcaps, text in small caps)}}}.
>
> This is some %[highlight]{highlighted text} and this is some
> %[smallcaps]{text in small caps}.
> #+end_example

I have used macros a lot in the past for these purposes. But the problem
of having to escape commas and the somewhat confusing (and ugly) syntax
of macros has led me to rarely use them now. Links have been a good
replacement for me, but they still have their limitations (the most
important, as Ihor commented, not being able to include a link within
the description. But we can't put footnotes either). I actually think
that inline special blocks could be tremendously useful and versatile.
And, in syntactic terms, an important point in favor of Org against
Markdown, which if I'm not mistaken does not have anything similar (I
hardly use md, so I'm not very aware).

Best regards,

Juan Manuel

Re: About 'inline special blocks'

[Tim Cross]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> To add some ideas that have been occurring to me these days...
>
> I am more and more convinced that inline special blocks, by their
> nature, should not support fine tune options or anything like
> attr_latex, attr_html, etc. like its older brothers, as it would produce
> an overly complicated syntax. Big brothers are independent of the
> paragraph and there it makes sense to add lines with attr_latex, etc.,
> since it is a line-oriented syntax. Bringing that into the paragraph is
> unnecessarily overloading the paragraph and breaking the social contract
> of lightweight markup, where paragraphs should still look like
> paragraphs.
>

Agree. I think your reasoning here is spot on. 

> Another argument against possible fine-tuning within inline special
> blocks, for export purposes, is that (in my opinion) direct formatting
> is a practice that should be avoided as much as possible in a document.
> A document with a lot of direct formatting is an inconsistent document.
> In html, all possible formatting should be controlled by style sheets;
> in LaTeX, by (re)defining macros, commands and environments in the
> preamble or in a .sty file; in odt using character styles.
>

Agreed. In fact, I use in-line blocks so rarely that at first I wasn't
going to respond as I really didn't have much skin in the game when it
comes to inline blocks. The reason I dond't use them much is pretty much
the same as your reasoning above.

> Perhaps if we detach special blocks from fine-tuning possibilities we
> lose some (export) flexibility, but we gain in a clearer implementation
> of these elements and keep Org aseptic about the output format. And in
> any case, if someone needs a fine-tuning in a certain case, there are
> always the export filters. Or it can be implemented in a similar way to
> inline tasks, with a default format function (for html, latex, etc),
> which can be changed via a defcustom.
>

I also like this approach. We need some form of escape hatch. However,
for uncommon edge cases, I would rather have a slightly less convenient
escape hatch and a simple consistent general syntax than a more complex
syntax which is difficult to maintain and keep consistent and reliable. 

> Starting from that, a syntax like this in Org:
>
> %[name]{contents}
>
> Would produce in LaTeX, by default:
>
> \name{contents}
>
> in html:
>
> <name>contents></name>

or should that be <span class="name">contents</name>?


>
> in odt:
>
> <text:span text:style-name="name">contents</text:span>
>
> and so on.
>
> In short, I think it would be enough to simply implement something like
> this.
>

Sound reasoning IMO. 

Re: About 'inline special blocks'

[Max Nikulin]

On 19/06/2022 19:47, Juan Manuel Macías wrote:
> 
> I am more and more convinced that inline special blocks, by their
> nature, should not support fine tune options or anything like
> attr_latex, attr_html, etc. like its older brothers, as it would produce
> an overly complicated syntax.

I would like to stress that styles can not be a rescue in some important 
cases. Let's leave aside ad hoc final tuning of formatting. In the case 
of HTML export there are still <img alt="Description"> and <a href="..." 
title="Description"> attributes that are namely per-object, not part of 
style.

I was going to raise this issue for several months, so I just have sent 
to following bug report:
Max Nikulin to emacs-orgmode. [BUG] manual: confusing example of adding 
attributes to a link (affiliated keywords) Mon, 20 Jun 2022 23:25:29 
+0700. https://list.orgmode.org/t8q71r$mgv$1@ciao.gmane.io

I have not heard that PDF offers something similar, but e.g. link with 
title may be exported as footnote with title text and URL instead of 
inline link. However to handle such cases generic attributes available 
to all export backends should be introduced.

Even when styles are enough in principle, attributes may be more 
convenient since the latter may be composable, so making unnecessary 
defining every possible (or used) combination of styles.

> in html:
> 
> <name>contents></name>

Concerning <name> vs. <span class="name">, is it the same for assistive 
technologies like screen readers to add <strong>text</strong> (or 
<b>text</b>) and <span class="strong">text</span> with "font-weight: 
bolder;" in CSS?

Re: About 'inline special blocks'

[Juan Manuel Macías]

Max Nikulin writes:

Hi Maxim,

Max Nikulin writes:

> I would like to stress that styles can not be a rescue in some
> important cases. Let's leave aside ad hoc final tuning of formatting.
> In the case of HTML export there are still <img alt="Description"> and
> <a href="..." title="Description"> attributes that are namely
> per-object, not part of style.

You are right, but my question is: Could there be a similar use case
within inline special blocks? Keep in mind that this (for now,
hypothetical) element type would be intended only for very short
segments of text within the paragraph. I don't find a scenario where
it's worth overloading that with options and attributes, IMHO.

I believe that direct formatting (as a rule of composition and not as an
exception), which comes ---I suppose--- from the use and abuse of word
processors, is the great cancer for the consistency of the documents,
where a guiding style and a series of constants must prevail. Of course,
I do not deny that it is often necessary to do a post-process and adjust
exceptions. There are always exceptions. In the case of LaTeX and
ConTeXt, TeX is powerful enough to deal with exceptions also at a high
level, due to its high degree of automation. And LuaTeX, even more so. A
simple example to automatically adjust the width of the caption in
figures with a simple lua function in LuaLaTeX:

#+begin_src latex
\begin{luacode*}
  function caption_width ( text )
  text = string.gsub ( text, "(\\includegraphics.+)", "\\sbox0{%1}")
  text = string.gsub ( text, "(\\caption{.+})", "\\begin{minipage}{\\wd0}\\usebox0%1\\end{minipage}")
  return text
  end
\end{luacode*}
    \newcommand\CaptionAutoWidth{\directlua{luatexbase.add_to_callback
	( "process_input_buffer" , caption_width , "caption_width" )}}
\AtBeginDocument{\CaptionAutoWidth}
#+end_src


However, note that I speak in general terms. It is difficult to get rid
of manual intervention one hundred percent. But the question is whether
it's worth adding fine-tuning options to an element as "specialized" as
inline special blocks. Of course, LaTeX is more flexible and you can
always change a variable on the fly. You can do something like:

#+begin_src latex
\definecolor{foo}{HTML}{FF0000}
\definecolor{var}{HTML}{7CFC00}
\def\mycolor{foo}
\newcommand\mytextcolor[1]{%
  \textcolor{\mycolor}{#1}}
\begin{document}
lorem \mytextcolor{ipsum dolor}
\def\mycolor{var}
lorem \mytextcolor{ipsum dolor}
#+end_src

html/css seems more rigid and I'm not that familiar with it. Perhaps
there is more uses case where the existence of ad hoc attributes and
options would be justified? And, in any case, how to implement it
without the paragraph becoming unreadable? The solution that Ihor
commented on in a past post of using identifiers for each inline block
would be fine (and maybe it could be used also for the attributes of the
links within the paragraph).

>> in html:
>> <name>contents></name>
>
> Concerning <name> vs. <span class="name">, is it the same for
> assistive technologies like screen readers to add
> <strong>text</strong> (or <b>text</b>) and <span
> class="strong">text</span> with "font-weight: bolder;" in CSS?

"<name>contents></name>": it was my confusion, sorry. I already explained
it here: https://list.orgmode.org/8735g0tnoh.fsf@posteo.net/

Best regards,

Juan Manuel 

Re: About 'inline special blocks'

[Max Nikulin]

On 21/06/2022 02:06, Juan Manuel Macías wrote:
> Max Nikulin writes:
> 
>> I would like to stress that styles can not be a rescue in some
>> important cases. Let's leave aside ad hoc final tuning of formatting.
>> In the case of HTML export there are still <img alt="Description"> and
>> <a href="..." title="Description"> attributes that are namely
>> per-object, not part of style.
> 
> You are right, but my question is: Could there be a similar use case
> within inline special blocks? Keep in mind that this (for now,
> hypothetical) element type would be intended only for very short
> segments of text within the paragraph. I don't find a scenario where
> it's worth overloading that with options and attributes, IMHO.

Juan Manuel, your answer is not clear for me. Direct formatting is 
another issue. There are cases when attributes are part of *content*, 
not formatting. If alternative text for images and description of links 
are not convincing, there is e.g.

     Org document may be exported as
     <abbr title="HyperText Markup Language">HTML</abbr> file.

Do you consider inline special blocks solely for formatting and only for 
the kind of it that may be expressed through styles? Then attributes for 
inline objects should be another feature with its own syntax.

>>> in html:
>>> <name>contents></name>
>>
>> Concerning <name> vs. <span class="name">, is it the same for
>> assistive technologies like screen readers to add
>> <strong>text</strong> (or <b>text</b>) and <span
>> class="strong">text</span> with "font-weight: bolder;" in CSS?
> 
> "<name>contents></name>": it was my confusion, sorry. I already explained
> it here: https://list.orgmode.org/8735g0tnoh.fsf@posteo.net/

I am unsure that <name>content</name> should not be supported at all. 
However I admit that difference of the following code may be 
insignificant in reality:

     <strong class="alert">Do not do it!</strong>
vs.
     <span role="strong" class="alert">Do not do it!</span>

Re: About 'inline special blocks'

[Juan Manuel Macías]

Max Nikulin writes:

> If alternative text for images and description of
> links are not convincing [...]

It does convince me, Maxim, that's why I told you in my previous message
that you were right in that example you had put about the alternate
text. And my question was (and still is) if you consider that a scenario
of this type, where the attributes are part of the content and not the
output format (and the latter happens in 90% of the cases) could occur
in the inline special blocks. If so, then I'm fine with inline special
blocks supporting attributes. But in my opinion this data should somehow
go outside the paragraph. Or be hidden as in the links.

I still think, however, that the attributes are unnecessary for inline
special blocks. I can't find any examples where they might be needed and
not something that is resolved at the global style level[1] or via
export filter. But I'm open to considering examples and use cases.

[1] There are cases in LaTeX of commands with more than one argument,
e.g. \foreignlanguage{lang}{content}, \textcolor{color}{content} and
the like. But even those can be simplified from the preamble by defining
macros with a single argument. And in Babel you can also do something
like \babeltags{de = german} and write \textde{text in german}.

The csquotes package is an extreme case, where there are intra-paragraph
commands that take optional arguments to \cite and punctuation, but I
think org-cite would be more appropriate in these cases:

\foreigntextquote{lang}[cite][punct]{text}

Best regards,

Juan Manuel 

Re: About 'inline special blocks'

[Tim Cross]

Max Nikulin <manikulin@gmail.com> writes:

> On 19/06/2022 19:47, Juan Manuel Macías wrote:
> Concerning <name> vs. <span class="name">, is it the same for assistive
> technologies like screen readers to add <strong>text</strong> (or <b>text</b>)
> and <span class="strong">text</span> with "font-weight: bolder;" in CSS?

First, never use <b></b> or <it></it>, only use semantic tags for
accessibility. 

The question unfortunately has a complicated answer. Basically, <div>
and <span> are the two tags which have no semantic meaning. So, from an
accessibility perspective, they don't convey anything. They are
basically a presentation layout tgag. 

However, this is not a bad thing, but rather a very good thing. This
touches on the area where far too many people get accessibility wrong.
It is like the very misguided rule which says all images must have an
alt tag. The key point to consider is whether what your communicating
via layout has any real use for someone using a screen reader. Consider
something like 

#+being_src
<section aria-label="section label">
    <h3>Section Title</h3>
    <section class="fancy-css-class">
      <section class="some-css-class">
         Some Content wrapped within multiple section elements.
       </section>
      </section>
     ...
</section>
#+end_src

The inner <section> is being used to avoid using a <div> in the mistaken
belief that using a <div> (or <span>) would be bad for accessibility.
Unfortunately, the above wil often result in the screen reader reading
out "Seciton section section SOme content" (some screen readers would
ignore the inner section as it has no aria tag). 

Same sort of problem occurred with the rule about images must have an
'alt' tag. I cannot count the number of pages I visit where the screen
reader says "logo logo filler righ pad left pad filler logo brand". 

So, the span tag is great for accessibility when what the author is
trying to convey is layout information or styling information which is
of no use to blind or VI users. Often such style emphasis is used to
assist sighted users who can quickly scan the page. Blind and VI users
cannot scan in the same way. What is more important for us is the
ability to get an overview of the semantic content of the page -
sections, table, lists etc. 

Sadly, org isn't great from an accessibility perspective. This is
something I would like to see improved, but it is a huge and complex
task. There are some 'easy' winds we could try. For example, org still
defaults to using the <b></b> and <i></i> tags instead of
<strong></strong> and <em></em>. Likewise, we should move to html5 as
the default, not xhtml, but last time I raised that, there was
considerable push back to stick with xhtml. We also need complete
overhaul of the use of aria tags and numerous other areas. As I said, a
very large job which is complex and extremely time consuming. 

Sadly, I'm not sure there is a lot we can do with accessibility and PDFs
in org mode. This is the one area where TeX/LaTeX does a poor job. Last
time I looked, there was considerable discussion about what to do from
an accessibility standpoint in the TeX community, but seemed to be
little or very slow progress (not a criticism of the efforts of members
of that community, but rather a reflection of how complicated this stuff
is). 

Org mode export accessibility (was: About 'inline special blocks')

[Ihor Radchenko]

Tim Cross <theophilusx@gmail.com> writes:

> Sadly, org isn't great from an accessibility perspective. This is
> something I would like to see improved, but it is a huge and complex
> task. There are some 'easy' winds we could try. For example, org still
> defaults to using the <b></b> and <i></i> tags instead of
> <strong></strong> and <em></em>. Likewise, we should move to html5 as
> the default, not xhtml, but last time I raised that, there was
> considerable push back to stick with xhtml. We also need complete
> overhaul of the use of aria tags and numerous other areas. As I said, a
> very large job which is complex and extremely time consuming. 

I will not argue about html5 switch - I don't have enough knowledge to
weigh on this.

However, can't we at least address accessibility issues with the
existing HTML export? A good starting point could be identifying what
can be improved in ox-html.el.

> Sadly, I'm not sure there is a lot we can do with accessibility and PDFs
> in org mode. This is the one area where TeX/LaTeX does a poor job. Last
> time I looked, there was considerable discussion about what to do from
> an accessibility standpoint in the TeX community, but seemed to be
> little or very slow progress (not a criticism of the efforts of members
> of that community, but rather a reflection of how complicated this stuff
> is).

From Org perspective, we can do what is available in the exported
format. If LaTeX is not great from accessibility point of view, is there
a better format? Or are there things we can do to improve situation in
ox-latex.el?

What about other export backends?

Best,
Ihor

Re: Org mode export accessibility (was: About 'inline special blocks')

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>> Sadly, org isn't great from an accessibility perspective. This is
>> something I would like to see improved, but it is a huge and complex
>> task. There are some 'easy' winds we could try. For example, org still
>> defaults to using the <b></b> and <i></i> tags instead of
>> <strong></strong> and <em></em>. Likewise, we should move to html5 as
>> the default, not xhtml, but last time I raised that, there was
>> considerable push back to stick with xhtml. We also need complete
>> overhaul of the use of aria tags and numerous other areas. As I said, a
>> very large job which is complex and extremely time consuming. 
>
> I will not argue about html5 switch - I don't have enough knowledge to
> weigh on this.
>
> However, can't we at least address accessibility issues with the
> existing HTML export? A good starting point could be identifying what
> can be improved in ox-html.el.
>

Yes, we can probably make some incremental improvements. However, it is
a complex and difficult area and I suspect to really improve the
situation, we likely need a major re-design. A big part of the challenge
is how to enable authors to add the right level of accessibility
'tagging', but at the same time, not lose one of the main advantages of
org mode i.e. simplicity and ease of syntax. 

>> Sadly, I'm not sure there is a lot we can do with accessibility and PDFs
>> in org mode. This is the one area where TeX/LaTeX does a poor job. Last
>> time I looked, there was considerable discussion about what to do from
>> an accessibility standpoint in the TeX community, but seemed to be
>> little or very slow progress (not a criticism of the efforts of members
>> of that community, but rather a reflection of how complicated this stuff
>> is).
>
> From Org perspective, we can do what is available in the exported
> format. If LaTeX is not great from accessibility point of view, is there
> a better format? Or are there things we can do to improve situation in
> ox-latex.el?
>

As I understand it (which isn't brilliant), the core problem is more to
do with how the LaTeX/TeX engine processes the input to generate the
postscript and pdf output. Modern PDFs have a wealth of internal tagging
which simply sin't supported via the tex -> pdf pathway. The matter is
made slightly worse by a lack of built-in support within latex/tex for
accessibility 'tags' (similar to the aria tags for web content). 

> What about other export backends?
>

To be honest, no idea. I'm certainly not an expert in these areas. While
I am impacted more by lack of accessibility, unfortunately, that doesn't
make you an expert.

I do feel that in order to get reasonable accessibility levels, it is
probably something which needs to be baked in as part of the overall
design and not something which can be added later. This isn't really
feasible. Things can probably be slightly improved, but I doubt org mode
and the documents it produces will ever be particularly good from an
accessibility perspective. 

Re: Org mode export accessibility

[Juan Manuel Macías]

Tim Cross writes:

> As I understand it (which isn't brilliant), the core problem is more to
> do with how the LaTeX/TeX engine processes the input to generate the
> postscript and pdf output. Modern PDFs have a wealth of internal tagging
> which simply sin't supported via the tex -> pdf pathway. The matter is
> made slightly worse by a lack of built-in support within latex/tex for
> accessibility 'tags' (similar to the aria tags for web content). 

There is a relatively recent experimental package for LaTeX that may be
of interest to you:

CTAN: https://www.ctan.org/pkg/tagpdf

GitHub: https://github.com/u-fischer/tagpdf

The package is maintained by Ulrike Fischer, who is very active in the
TeX community. However, the package description says:

> The package offers tools to experiment with tagging and accessibility
> using pdfLaTeX and LuaTeX. It isn't meant for production but allows
> the user to try out how difficult it is to tag some structures; to try
> out how much tagging is really needed; to test what else is needed so
> that a pdf works e.g. with a screen reader. Its goal is to get a
> feeling for what has to be done, which kernel changes are needed, how
> packages should be adapted.

Best regards,

Juan Manuel 

Re: Org mode export accessibility

[Ihor Radchenko]

Let me take a freedom to add T.V Raman to the discussion. This thread
might be of interest for him and he probably knows a lot more about
accessibility options.

This thread starts at
https://list.orgmode.org/87v8sn3obd.fsf@gmail.com/T/#u

Juan Manuel Macías <maciaschain@posteo.net> writes:

> Tim Cross writes:
>
>> As I understand it (which isn't brilliant), the core problem is more to
>> do with how the LaTeX/TeX engine processes the input to generate the
>> postscript and pdf output. Modern PDFs have a wealth of internal tagging
>> which simply sin't supported via the tex -> pdf pathway. The matter is
>> made slightly worse by a lack of built-in support within latex/tex for
>> accessibility 'tags' (similar to the aria tags for web content). 
>
> There is a relatively recent experimental package for LaTeX that may be
> of interest to you:
>
> CTAN: https://www.ctan.org/pkg/tagpdf
>
> GitHub: https://github.com/u-fischer/tagpdf
>
> The package is maintained by Ulrike Fischer, who is very active in the
> TeX community. However, the package description says:
>
>> The package offers tools to experiment with tagging and accessibility
>> using pdfLaTeX and LuaTeX. It isn't meant for production but allows
>> the user to try out how difficult it is to tag some structures; to try
>> out how much tagging is really needed; to test what else is needed so
>> that a pdf works e.g. with a screen reader. Its goal is to get a
>> feeling for what has to be done, which kernel changes are needed, how
>> packages should be adapted.
>
> Best regards,
>
> Juan Manuel 

Re: Org mode export accessibility

[T.V Raman]

Thanks for looping me in.
I'm not subscribed to emacs-orgmode --- so feel free to forward if you
find the thoughts below materially useful.

As  a long-term org-mode user --- and an even  longer  term TeX
user: here are some thoughts:

1. Accessibility as word used in isolation has now become mostly
   meaningless, to be concrete one has to ask "Accessibility to whom"? 

2. So in the following, everything I say is with respect to users with
   visual impairments.

3. It's incorrect to define "Accessibility" in terms of a specific
   user access tool or technology -- that usage is marketing jargon
   for a specific Access Solution like a screenreader --- so I refrain in general from
   defining this in terms of Screenreaders.

With those meta-thoughts out of the way:

A: Org-generated documents are mostly well-structured documents, and
not in general a user-interface e.g. (WebApp); so  with regard to export --
either HTML or LaTeX/PDF ---  ARIA is mostly irrelevant.

B: The LaTeX->PDF pipeline *can* produce tagged PDF with respect to
document structure eg. Sectioning, but only if you use pdflatex or
pdftex i.e. LaTeX/Tex->dvi->[ps]->pdf is lossy with respect to
structure present in the markup; this is a short-coming of DVI which
predates  the thought of document structure making it through to the
output.

C: pdftex and pdflatex were built in the late 90's by a student in
Prague (Hanu Than? from memory) --- only reason I know this is that I
got Adobe to fund that project when at Adobe in the 90's. It's a very
good piece of work that essentially uses PDF directly as the "Device
Independent" format rather than the original dvi. DVI as designed in
the 70's was device-independent for the time, ie it did not hardwire
printer controls and could be mapped to various print mechanisms. For
the 90s, by which time Document Structure meant a lot more than being
some version of inkjet printer driver independent, the afore mentioned
project used PDF as the Device-Independent format --- and
leveraged the Tagged PDF bits from PDF 1.4 to achieve the result.

D: All that said, it is likely still easier to go from org->HTML
directly and produce content that is easier to machine-process  ---
rather than go through one more level of indirection via LaTeX and
PDF; however there may well be additional constraints in a publication
workflow, e.g. publisher wants to only publish final-form -- and yes,
in this case, HTML and PDF are both final-form.

E: Finally, note that in (D) I said "machine processable" not
"Accessible"; machine-processable is a pre-requisite to "repurpose "
what you publish, and making  that result usable by different user
communities is a direct consequence of suche machine-processability.


Hope this helps.

-- 
--Raman 
Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

--

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

Re: Org mode export accessibility

[Ihor Radchenko]

"T.V Raman" <raman@google.com> writes:

> 1. Accessibility as word used in isolation has now become mostly
>    meaningless, to be concrete one has to ask "Accessibility to whom"? 
>
> 2. So in the following, everything I say is with respect to users with
>    visual impairments.

This is exactly the perspective I was hoping to hear from you. Though
this thread is not dedicated to visual impairments. (I guess you also
did not touch the question of color blindness).

> 3. It's incorrect to define "Accessibility" in terms of a specific
>    user access tool or technology -- that usage is marketing jargon
>    for a specific Access Solution like a screenreader --- so I refrain in general from
>    defining this in terms of Screenreaders.

Yet, in order to simplify the efforts needed to read a document exported
from Org mode one needs to use some kind of tool/technology. Unless a
common standard exist in this area, we have to support at least the most
common Access Solutions (prioritizing Free software, if possible).

From you message, it does not look like there is any common standard.

> With those meta-thoughts out of the way:
>
> A: Org-generated documents are mostly well-structured documents, and ...
> B: The LaTeX->PDF pipeline *can* produce tagged PDF with respect to ...
> C: pdftex and pdflatex were built in the late 90's by a student in ...
> D: All that said, it is likely still easier to go from org->HTML ...

Do I understand correctly that you have no issues with reading documents
exported using current version of Org?

> E: Finally, note that in (D) I said "machine processable" not
> "Accessible"; machine-processable is a pre-requisite to "repurpose "
> what you publish, and making  that result usable by different user
> communities is a direct consequence of suche machine-processability.

I understand. But one can similarly say that .org files are "machine
processable" and Org export code is not strictly necessary. Yet, it ends
up extremely useful in practice.

I suspect that the exported documents can similarly be improved to
reduce the amount of efforts required from visually impair users to read
such documents. The question is what kinds improvements can be made on
Org side.

Best,
Ihor

Re: Org mode export accessibility

[briangpowell]

[-- Attachment #1: Type: text/plain, Size: 5452 bytes --]

"[I suspect that the exported documents can similarly be improved to
reduce the amount of effort required from visually impaired users to read
such documents. The question is what improvements can be made on
Org side.]

Best,
Ihor"

Very glad to hear from TV Raman, the creator of EmacSpeak,

I'm not blind like TV but I was motivated to turn my a main OrgMode buffer
into an audio desktop like TV's

But now back to the topic; much agree with Ihor, we should focus on "what
improvements can be made on OrgMode side"

& TV's points are well made too: "pdftex and pdflatex were built in the
late 90's"--very true & they were rarely useful

Suggest OrgMode make changes aimed at the "Lowest Common Denominator" of
accessibility--accessibility in the visual sense AND in the machine or
program processable sense or more exactly the "document convertible
sense"--I mean documents should be made firstly in a form that all
computers can easily navigate & present on computer screens and/or audio
desktops in addition to being readily able to print out

TV's right, the usual pipeline of LaTeX->PDF can produce tagged & useful
documents but can an end user easily copy and paste the document? How
useful are pretty documents that run on proprietary systems? Many PDF's can
make simple processes like this very hard or impossible--the documents can
be very pretty but they can contain control characters & special characters
& even malicious code

Suggest OrgMode outputs focus on creating "Lowest Common Denominator"
documents as output:
TeXinfo docs should be used as the LCD doctype--suggest you focus on
creating 1 document in Texinfo that you use to create all other sorts of
documents, when possible:

Pipeline should be more like
OrgMode->Texinfo->TROFF||DTD/XML/HTML/XHTML->LaTeX/TeX->DVI||SVG->PS->PDF

* TeXinfo: https://savannah.gnu.org/projects/texinfo
https://www.gnu.org/software/texinfo

** "Texinfo uses a single source file to produce output in a number of
formats, both online and printed (dvi, html, info, pdf, xml, etc.). This
means that instead of writing different documents for online information
and another for a printed manual, you need write only one document.  And
when the work is revised, you need revise only that one document.  The
Texinfo system is integrated well with GNU Emacs.

*** Texinfo docs can also be viewed & used by ALL end-users without any
issues--regardless of the power of their computer or monitor or even if
they're blind like TV Raman--he uses an audio desktop or EmacSpeak--and the
same docs can be printed on any printer & remain navigable with "rn" &
other simple news-reading software--or the "info" program

* Output formats currently supported by Texinfo:
https://www.gnu.org/software/texinfo/manual/texinfo/html_node/Output-Formats.html
<=> Info,Text,HTML,DVI,PostScript{PS},PDF,DocBook,XML

** Related/useful may be: "latex2nemeth"--a LATEX to Braille/Nemeth,
approach "Simple pictures in PSTricks are also supported in order to
produce tactile graphics": https://ctan.org/pkg/latex2nemeth

On Thu, Jun 30, 2022 at 3:53 AM Ihor Radchenko <yantar92@gmail.com> wrote:

> "T.V Raman" <raman@google.com> writes:
>
> > 1. Accessibility as word used in isolation has now become mostly
> >    meaningless, to be concrete one has to ask "Accessibility to whom"?
> >
> > 2. So in the following, everything I say is with respect to users with
> >    visual impairments.
>
> This is exactly the perspective I was hoping to hear from you. Though
> this thread is not dedicated to visual impairments. (I guess you also
> did not touch the question of color blindness).
>
> > 3. It's incorrect to define "Accessibility" in terms of a specific
> >    user access tool or technology -- that usage is marketing jargon
> >    for a specific Access Solution like a screenreader --- so I refrain
> in general from
> >    defining this in terms of Screenreaders.
>
> Yet, in order to simplify the efforts needed to read a document exported
> from Org mode one needs to use some kind of tool/technology. Unless a
> common standard exist in this area, we have to support at least the most
> common Access Solutions (prioritizing Free software, if possible).
>
> From you message, it does not look like there is any common standard.
>
> > With those meta-thoughts out of the way:
> >
> > A: Org-generated documents are mostly well-structured documents, and ...
> > B: The LaTeX->PDF pipeline *can* produce tagged PDF with respect to ...
> > C: pdftex and pdflatex were built in the late 90's by a student in ...
> > D: All that said, it is likely still easier to go from org->HTML ...
>
> Do I understand correctly that you have no issues with reading documents
> exported using current version of Org?
>
> > E: Finally, note that in (D) I said "machine processable" not
> > "Accessible"; machine-processable is a pre-requisite to "repurpose "
> > what you publish, and making  that result usable by different user
> > communities is a direct consequence of suche machine-processability.
>
> I understand. But one can similarly say that .org files are "machine
> processable" and Org export code is not strictly necessary. Yet, it ends
> up extremely useful in practice.
>
> I suspect that the exported documents can similarly be improved to
> reduce the amount of efforts required from visually impair users to read
> such documents. The question is what kinds improvements can be made on
> Org side.
>
> Best,
> Ihor
>
>

[-- Attachment #2: Type: text/html, Size: 6907 bytes --]

Re: Org mode export accessibility

[T.V Raman]

On org side:

1. During authoring, ensure that authors have the ability to label
   images, drawings and math content.
   2. When exporting, make sure that that information gets through to
      the exported format.
      3. For math especially, make sure the TeX/LaTeX is preserved one
         way  or the other in the export

That's just a few initial thoughts, am sure there could be morebriangpowell writes:
 > "[I suspect that the exported documents can similarly be improved to
 > reduce the amount of effort required from visually impaired users to read
 > such documents. The question is what improvements can be made on
 > Org side.]
 > 
 > Best,
 > Ihor"
 > 
 > Very glad to hear from TV Raman, the creator of EmacSpeak,
 > 
 > I'm not blind like TV but I was motivated to turn my a main OrgMode buffer
 > into an audio desktop like TV's
 > 
 > But now back to the topic; much agree with Ihor, we should focus on "what
 > improvements can be made on OrgMode side"
 > 
 > & TV's points are well made too: "pdftex and pdflatex were built in the
 > late 90's"--very true & they were rarely useful
 > 
 > Suggest OrgMode make changes aimed at the "Lowest Common Denominator" of
 > accessibility--accessibility in the visual sense AND in the machine or
 > program processable sense or more exactly the "document convertible
 > sense"--I mean documents should be made firstly in a form that all
 > computers can easily navigate & present on computer screens and/or audio
 > desktops in addition to being readily able to print out
 > 
 > TV's right, the usual pipeline of LaTeX->PDF can produce tagged & useful
 > documents but can an end user easily copy and paste the document? How
 > useful are pretty documents that run on proprietary systems? Many PDF's can
 > make simple processes like this very hard or impossible--the documents can
 > be very pretty but they can contain control characters & special characters
 > & even malicious code
 > 
 > Suggest OrgMode outputs focus on creating "Lowest Common Denominator"
 > documents as output:
 > TeXinfo docs should be used as the LCD doctype--suggest you focus on
 > creating 1 document in Texinfo that you use to create all other sorts of
 > documents, when possible:
 > 
 > Pipeline should be more like
 > OrgMode->Texinfo->TROFF||DTD/XML/HTML/XHTML->LaTeX/TeX->DVI||SVG->PS->PDF
 > 
 > * TeXinfo: https://savannah.gnu.org/projects/texinfo
 > https://www.gnu.org/software/texinfo
 > 
 > ** "Texinfo uses a single source file to produce output in a number of
 > formats, both online and printed (dvi, html, info, pdf, xml, etc.). This
 > means that instead of writing different documents for online information
 > and another for a printed manual, you need write only one document.  And
 > when the work is revised, you need revise only that one document.  The
 > Texinfo system is integrated well with GNU Emacs.
 > 
 > *** Texinfo docs can also be viewed & used by ALL end-users without any
 > issues--regardless of the power of their computer or monitor or even if
 > they're blind like TV Raman--he uses an audio desktop or EmacSpeak--and the
 > same docs can be printed on any printer & remain navigable with "rn" &
 > other simple news-reading software--or the "info" program
 > 
 > * Output formats currently supported by Texinfo:
 > https://www.gnu.org/software/texinfo/manual/texinfo/html_node/Output-Formats.html
 > <=> Info,Text,HTML,DVI,PostScript{PS},PDF,DocBook,XML
 > 
 > ** Related/useful may be: "latex2nemeth"--a LATEX to Braille/Nemeth,
 > approach "Simple pictures in PSTricks are also supported in order to
 > produce tactile graphics": https://ctan.org/pkg/latex2nemeth
 > 
 > On Thu, Jun 30, 2022 at 3:53 AM Ihor Radchenko <yantar92@gmail.com> wrote:
 > 
 > > "T.V Raman" <raman@google.com> writes:
 > >
 > > > 1. Accessibility as word used in isolation has now become mostly
 > > >    meaningless, to be concrete one has to ask "Accessibility to whom"?
 > > >
 > > > 2. So in the following, everything I say is with respect to users with
 > > >    visual impairments.
 > >
 > > This is exactly the perspective I was hoping to hear from you. Though
 > > this thread is not dedicated to visual impairments. (I guess you also
 > > did not touch the question of color blindness).
 > >
 > > > 3. It's incorrect to define "Accessibility" in terms of a specific
 > > >    user access tool or technology -- that usage is marketing jargon
 > > >    for a specific Access Solution like a screenreader --- so I refrain
 > > in general from
 > > >    defining this in terms of Screenreaders.
 > >
 > > Yet, in order to simplify the efforts needed to read a document exported
 > > from Org mode one needs to use some kind of tool/technology. Unless a
 > > common standard exist in this area, we have to support at least the most
 > > common Access Solutions (prioritizing Free software, if possible).
 > >
 > > From you message, it does not look like there is any common standard.
 > >
 > > > With those meta-thoughts out of the way:
 > > >
 > > > A: Org-generated documents are mostly well-structured documents, and ...
 > > > B: The LaTeX->PDF pipeline *can* produce tagged PDF with respect to ...
 > > > C: pdftex and pdflatex were built in the late 90's by a student in ...
 > > > D: All that said, it is likely still easier to go from org->HTML ...
 > >
 > > Do I understand correctly that you have no issues with reading documents
 > > exported using current version of Org?
 > >
 > > > E: Finally, note that in (D) I said "machine processable" not
 > > > "Accessible"; machine-processable is a pre-requisite to "repurpose "
 > > > what you publish, and making  that result usable by different user
 > > > communities is a direct consequence of suche machine-processability.
 > >
 > > I understand. But one can similarly say that .org files are "machine
 > > processable" and Org export code is not strictly necessary. Yet, it ends
 > > up extremely useful in practice.
 > >
 > > I suspect that the exported documents can similarly be improved to
 > > reduce the amount of efforts required from visually impair users to read
 > > such documents. The question is what kinds improvements can be made on
 > > Org side.
 > >
 > > Best,
 > > Ihor
 > >
 > >

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

--

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

Re: Org mode export accessibility

[Ihor Radchenko]

"T.V Raman" <raman@google.com> writes:

> On org side:

Thanks for the feedback!

> 1. During authoring, ensure that authors have the ability to label
>    images, drawings and math content.
>    2. When exporting, make sure that that information gets through to
>       the exported format.

Could you please elaborate what you mean by "label"?

Org generally supports figures and equations in export, including adding
captions - all specific to the corresponding export backends (when the
backends support figures/equations).

>       3. For math especially, make sure the TeX/LaTeX is preserved one
>          way  or the other in the export

Do you refer to the TeX source? To any specific export format?

Best,
Ihor

Re: Org mode export accessibility

[T.V Raman]

1. Labels: yes -- figure captions etc. are examples of "labels" in
   general.

2. Math: Yes -- I meant the TeX/LaTeX representation of a math
   expression.
   Ihor Radchenko writes:
 > "T.V Raman" <raman@google.com> writes:
 > 
 > > On org side:
 > 
 > Thanks for the feedback!
 > 
 > > 1. During authoring, ensure that authors have the ability to label
 > >    images, drawings and math content.
 > >    2. When exporting, make sure that that information gets through to
 > >       the exported format.
 > 
 > Could you please elaborate what you mean by "label"?
 > 
 > Org generally supports figures and equations in export, including adding
 > captions - all specific to the corresponding export backends (when the
 > backends support figures/equations).
 > 
 > >       3. For math especially, make sure the TeX/LaTeX is preserved one
 > >          way  or the other in the export
 > 
 > Do you refer to the TeX source? To any specific export format?
 > 
 > Best,
 > Ihor

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

--

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

Re: Org mode export accessibility

[Ihor Radchenko]

"T.V Raman" <raman@google.com> writes:

>  > >       3. For math especially, make sure the TeX/LaTeX is preserved one
>  > >          way  or the other in the export
>  > 
>  > Do you refer to the TeX source? To any specific export format?
> 2. Math: Yes -- I meant the TeX/LaTeX representation of a math
>    expression.
>    Ihor Radchenko writes:

For html export we do preserve TeX/LaTeX representation because we use
Mathjax that directly supports such representation.

However, I am not sure how to preserve the original representation, say,
in LaTeX pdf export.

Best,
Ihor

Re: Org mode export accessibility

[T.V Raman]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=gb18030, Size: 1229 bytes --]

Ihor Radchenko <yantar92@gmail.com> writes:

What can or cannot be preserved is a function of the export format.

MathJax is a wonderful thing and the LaTeX expression embedded in the
HTML is the best one can do -- MathML loses semantics -- which is why I
always recommend preserving the LaTeX when going to HTML.

PDF as a format doesn't have the affordance to preserve the LaTeX and
even if you kluged up something --- the tools to extract that out of the
PDF would need to be invented.

> "T.V Raman" <raman@google.com> writes:
>
>>  > >       3. For math especially, make sure the TeX/LaTeX is preserved one
>>  > >          way  or the other in the export
>>  > 
>>  > Do you refer to the TeX source? To any specific export format?
>> 2. Math: Yes -- I meant the TeX/LaTeX representation of a math
>>    expression.
>>    Ihor Radchenko writes:
>
> For html export we do preserve TeX/LaTeX representation because we use
> Mathjax that directly supports such representation.
>
> However, I am not sure how to preserve the original representation, say,
> in LaTeX pdf export.
>
> Best,
> Ihor

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
7©4 Id: kg:/m/0285kf1  •0Ü8

Re: Org mode export accessibility

[Max Nikulin]

Accessibility issues of exported files have been raised on the Org mode 
mail list again. I decided to revive this thread because I noticed a 
LaTeX package that may be related to the following suggestion (I am not 
sure that I got it right though).

On 07/07/2022 21:42, T.V Raman wrote:
> 
> 3. For math especially, make sure the TeX/LaTeX is preserved one
>    way  or the other in the export

Math can be extracted from PDF files as TeX commands using e.g. 
pdftotext if LaTeX source file contains \usepackage{mmap}. Is it what 
you were writing about?

https://ctan.org/pkg/mmap
https://mirrors.ctan.org/macros/latex/contrib/mmap/README

The latest thread with discussion of accessibility:

https://list.orgmode.org/th4dth$5df$1@ciao.gmane.io
Max Nikulin Re: Add \usepackage{cmap} as default LaTeX class in ox-latex 
(was: org exported pdf files) Thu, 29 Sep 2022 22:34:07 +0700.

Re: Org mode export accessibility

[T.V Raman]

As one example, yes.

Here is hopefully a more detailed explanation:

1. First off, here Accessibility == Specifically, Accessibility to the
   blind.
2. For blind and low vision users, you may need to "re-render" the
   math, either via magnification, speech or Braille.
3. For doing this, you need the original "generating markup" MathML,
   or better yet LaTeX (those distinctions  require more  explanation,
   but this should suffice for this thread)
4. When things are rendered to PDF, a derivative format that traces
   back to the print world and traditionally didn't feel the need to
   recreate the original, all that info from (3) is lost.
5. For regular document structure, e.g. headings etc, PDF now supports
   this "natively" via something called "marked content" introduced in
   the late 90's; to leverage this, publishing tools exporting to PDF
   can leverage that facility to enable downstream tools re-create or
   extract an approximation of the original structure. 
6. The above left out Math --- mostly because authoring tools did not
   go as far as doing that for Math.
7. Then you got  pdftex that replaced DVI (which was essentially pure
   visual layout) with PDF -- also pure visual layout -- but a format
   that has continued to evolve.
8. Now, connect the dots, newer LaTeX packages like the one mentioned
   (there may well be others now or in the future) can inprinciple
   ensure that the required "back pointers to regenerate the original
   markup" can make it all the way through  to the generated PDF.
9. Note that this is not the end of the trail; for such exports to
   make a difference to the end-user, user-facing tools still need to
   know "how" to leverage these facilities.


Max Nikulin writes:
 > Accessibility issues of exported files have been raised on the Org mode 
 > mail list again. I decided to revive this thread because I noticed a 
 > LaTeX package that may be related to the following suggestion (I am not 
 > sure that I got it right though).
 > 
 > On 07/07/2022 21:42, T.V Raman wrote:
 > > 
 > > 3. For math especially, make sure the TeX/LaTeX is preserved one
 > >    way  or the other in the export
 > 
 > Math can be extracted from PDF files as TeX commands using e.g. 
 > pdftotext if LaTeX source file contains \usepackage{mmap}. Is it what 
 > you were writing about?
 > 
 > https://ctan.org/pkg/mmap
 > https://mirrors.ctan.org/macros/latex/contrib/mmap/README
 > 
 > The latest thread with discussion of accessibility:
 > 
 > https://list.orgmode.org/th4dth$5df$1@ciao.gmane.io
 > Max Nikulin Re: Add \usepackage{cmap} as default LaTeX class in ox-latex 
 > (was: org exported pdf files) Thu, 29 Sep 2022 22:34:07 +0700.

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

--

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

Re: Org mode export accessibility

[Max Nikulin]

On 30/09/2022 20:29, T.V Raman wrote:
> 
> 9. Note that this is not the end of the trail; for such exports to
>     make a difference to the end-user, user-facing tools still need to
>     know "how" to leverage these facilities.

Do you mean that there are no tools yet that may take advantage of math 
expressions embedded into PDF as LaTeX source markup? I am just trying 
to figure out current state of affairs. I suppose, it is unlikely that 
someone from Org mailing list subscribers is ready to invest significant 
amount of time into development of LaTeX packages. However if there were 
ready to use and safe packages improving accessibility then changing 
export templates would be more real goal.

In the meanwhile I have realized that \usepackage{mmap} has a little use 
in isolation. Fractions, indices, etc. are lost. Maybe some other 
packages should be loaded to provide more informative text equivalent 
for math.

Some details concerning what I have tried is below.

Original LaTeX expression:

Text \[ \int_{\alpha}^{\beta} \frac{dx}{x^2 + \varepsilon} \]

pdftotext -layout output for \usepackage{mmap}:
Text   \int     \beta
                            dx
            \alpha        x2 + \varepsilon

end of output

pdftotext output for \usepackage{cmap} and \usepackage[noTeX]{mmap}:
Text   ∫︁   𝛽
                   𝑑𝑥
         𝛼       𝑥2 + 𝜀

end of output

pdftotext output when neither cmap nor mmap is loaded:
Text   Z   β
                  dx
        α       x2 + ε

end of output

For this dumb example effect of mmap is hardly noticeable. Integral 
symbol is not replaced by "Z", but it is hard to guess sub- and 
superscripts and presence of the \frac command. Without "-layout" order 
of symbols is mostly preserved, but structure is lost

Text
∫︁

𝛽

𝛼

𝑑𝑥
𝑥2 + 𝜀

Re: Org mode export accessibility

[T.V Raman]

correct; user-facing tools are still lacking but that shouldn't cause
us to wait since that creates a chicken-and-egg problem.

Max Nikulin writes:
 > On 30/09/2022 20:29, T.V Raman wrote:
 > > 
 > > 9. Note that this is not the end of the trail; for such exports to
 > >     make a difference to the end-user, user-facing tools still need to
 > >     know "how" to leverage these facilities.
 > 
 > Do you mean that there are no tools yet that may take advantage of math 
 > expressions embedded into PDF as LaTeX source markup? I am just trying 
 > to figure out current state of affairs. I suppose, it is unlikely that 
 > someone from Org mailing list subscribers is ready to invest significant 
 > amount of time into development of LaTeX packages. However if there were 
 > ready to use and safe packages improving accessibility then changing 
 > export templates would be more real goal.
 > 
 > In the meanwhile I have realized that \usepackage{mmap} has a little use 
 > in isolation. Fractions, indices, etc. are lost. Maybe some other 
 > packages should be loaded to provide more informative text equivalent 
 > for math.
 > 
 > Some details concerning what I have tried is below.
 > 
 > Original LaTeX expression:
 > 
 > Text \[ \int_{\alpha}^{\beta} \frac{dx}{x^2 + \varepsilon} \]
 > 
 > pdftotext -layout output for \usepackage{mmap}:
 > Text   \int     \beta
 >                             dx
 >             \alpha        x2 + \varepsilon
 > 
 > end of output
 > 
 > pdftotext output for \usepackage{cmap} and \usepackage[noTeX]{mmap}:
 > Text   ∫︁   𝛽
 >                    𝑑𝑥
 >          𝛼       𝑥2 + 𝜀
 > 
 > end of output
 > 
 > pdftotext output when neither cmap nor mmap is loaded:
 > Text   Z   β
 >                   dx
 >         α       x2 + ε
 > 
 > end of output
 > 
 > For this dumb example effect of mmap is hardly noticeable. Integral 
 > symbol is not replaced by "Z", but it is hard to guess sub- and 
 > superscripts and presence of the \frac command. Without "-layout" order 
 > of symbols is mostly preserved, but structure is lost
 > 
 > Text
 > ∫︁
 > 
 > 𝛽
 > 
 > 𝛼
 > 
 > 𝑑𝑥
 > 𝑥2 + 𝜀

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

--

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

Re: Org mode export accessibility

[Ihor Radchenko]

"T.V Raman" <raman@google.com> writes:

> 8. Now, connect the dots, newer LaTeX packages like the one mentioned
>    (there may well be others now or in the future) can inprinciple
>    ensure that the required "back pointers to regenerate the original
>    markup" can make it all the way through  to the generated PDF.

Are you aware of such other available LaTeX packages?

Also, would it help to embed an Org source into the generated PDF? If it
is, is there a preferred way to indicate that the generated PDF contains
Org source?

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92

Re: Org mode export accessibility

[T.V Raman]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=gb18030, Size: 736 bytes --]

Ihor Radchenko <yantar92@gmail.com> writes:


No on both counts.
> "T.V Raman" <raman@google.com> writes:
>
>> 8. Now, connect the dots, newer LaTeX packages like the one mentioned
>>    (there may well be others now or in the future) can inprinciple
>>    ensure that the required "back pointers to regenerate the original
>>    markup" can make it all the way through  to the generated PDF.
>
> Are you aware of such other available LaTeX packages?
>
> Also, would it help to embed an Org source into the generated PDF? If it
> is, is there a preferred way to indicate that the generated PDF contains
> Org source?

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
7©4 Id: kg:/m/0285kf1  •0Ü8

Org source in PDF (Re: Org mode export accessibility)

[Max Nikulin]

On 01/10/2022 11:36, Ihor Radchenko wrote:
> 
> Also, would it help to embed an Org source into the generated PDF?

It is unrelated to accessibility, but I have heard that OpenOffice may 
embed original document into generated PDF to allow "edit PDF" feature. 
Instead of attempting to map PDF content onto ODT structures it just 
extracts and opens for edit document source. My curiosity has been never 
strong enough to find details related to the implementation.

Re: Org source in PDF (Re: Org mode export accessibility)

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

Hi Max,

Just for reference, I (automatically) embed the `.tex' and `.org' source files in PDFs I
produce.

Max Nikulin <manikulin@gmail.com> writes:

> It is unrelated to accessibility, but I have heard that OpenOffice may embed
> original document into generated PDF to allow “edit PDF” feature. Instead of
> attempting to map PDF content onto ODT structures it just extracts and opens for
> edit document source. My curiosity has been never strong enough to find details
> related to the implementation.

All the best,
Timothy

Re: Org mode export accessibility

[T.V Raman]

P.S. Emacspeak is not camel-cased -- please say Emacspeak -- and not
with the 's' capitalized.
-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

--

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

Re: Org mode export accessibility

[T.V Raman]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=gb18030, Size: 6299 bytes --]

briangpowell <briangpowellms@gmail.com> writes:


P.S. Please dont quote me out of context. I did not say pdftex and
pdflatex were not useful, I still rely on them heavily.
> "[I suspect that the exported documents can similarly be improved to
> reduce the amount of effort required from visually impaired users to
> read
> such documents. The question is what improvements can be made on
> Org side.]
>
> Best,
> Ihor"
>
> Very glad to hear from TV Raman, the creator of EmacSpeak,
>
> I'm not blind like TV but I was motivated to turn my a main OrgMode
> buffer into an audio desktop like TV's
>
> But now back to the topic; much agree with Ihor, we should focus on
> "what improvements can be made on OrgMode side"
>
> & TV's points are well made too: "pdftex and pdflatex were built in
> the late 90's"--very true & they were rarely useful
>
> Suggest OrgMode make changes aimed at the "Lowest Common Denominator"
> of accessibility--accessibility in the visual sense AND in the machine
> or program processable sense or more exactly the "document convertible
> sense"--I mean documents should be made firstly in a form that all
> computers can easily navigate & present on computer screens and/or
> audio desktops in addition to being readily able to print out
>
> TV's right, the usual pipeline of LaTeX->PDF can produce tagged &
> useful documents but can an end user easily copy and paste the
> document? How useful are pretty documents that run on proprietary
> systems? Many PDF's can make simple processes like this very hard or
> impossible--the documents can be very pretty but they can contain
> control characters & special characters & even malicious code
>
> Suggest OrgMode outputs focus on creating "Lowest Common Denominator"
> documents as output:
> TeXinfo docs should be used as the LCD doctype--suggest you focus on
> creating 1 document in Texinfo that you use to create all other sorts
> of documents, when possible:
>
> Pipeline should be more like
> OrgMode->Texinfo->TROFF||DTD/XML/HTML/XHTML->LaTeX/TeX->DVI||SVG->PS->PDF
>
>
> * TeXinfo: https://savannah.gnu.org/projects/texinfo
> https://www.gnu.org/software/texinfo
>
> ** "Texinfo uses a single source file to produce output in a number of
> formats, both online and printed (dvi, html, info, pdf, xml, etc.).
> This means that instead of writing different documents for online
> information and another for a printed manual, you need write only one
> document.  And when the work is revised, you need revise only that one
> document.  The Texinfo system is integrated well with GNU Emacs. 
>
> *** Texinfo docs can also be viewed & used by ALL end-users without
> any issues--regardless of the power of their computer or monitor or
> even if they're blind like TV Raman--he uses an audio desktop or
> EmacSpeak--and the same docs can be printed on any printer & remain
> navigable with "rn" & other simple news-reading software--or the
> "info" program
>
> * Output formats currently supported by Texinfo:
> https://www.gnu.org/software/texinfo/manual/texinfo/html_node/Output-Formats.html
> <=> Info,Text,HTML,DVI,PostScript{PS},PDF,DocBook,XML
>
> ** Related/useful may be: "latex2nemeth"--a LATEX to Braille/Nemeth,
> approach "Simple pictures in PSTricks are also supported in order to
> produce tactile graphics": https://ctan.org/pkg/latex2nemeth
>
> On Thu, Jun 30, 2022 at 3:53 AM Ihor Radchenko <yantar92@gmail.com>
> wrote:
>
>     "T.V Raman" <raman@google.com> writes:
>
>     > 1. Accessibility as word used in isolation has now become mostly
>     >    meaningless, to be concrete one has to ask "Accessibility to
>     whom"? 
>     >
>     > 2. So in the following, everything I say is with respect to
>     users with
>     >    visual impairments.
>
>     This is exactly the perspective I was hoping to hear from you.
>     Though
>     this thread is not dedicated to visual impairments. (I guess you
>     also
>     did not touch the question of color blindness).
>
>     > 3. It's incorrect to define "Accessibility" in terms of a
>     specific
>     >    user access tool or technology -- that usage is marketing
>     jargon
>     >    for a specific Access Solution like a screenreader --- so I
>     refrain in general from
>     >    defining this in terms of Screenreaders.
>
>     Yet, in order to simplify the efforts needed to read a document
>     exported
>     from Org mode one needs to use some kind of tool/technology.
>     Unless a
>     common standard exist in this area, we have to support at least
>     the most
>     common Access Solutions (prioritizing Free software, if possible).
>
>     From you message, it does not look like there is any common
>     standard.
>
>     > With those meta-thoughts out of the way:
>     >
>     > A: Org-generated documents are mostly well-structured documents,
>     and ...
>     > B: The LaTeX->PDF pipeline *can* produce tagged PDF with respect
>     to ...
>     > C: pdftex and pdflatex were built in the late 90's by a student
>     in ...
>     > D: All that said, it is likely still easier to go from org->HTML
>     ...
>
>     Do I understand correctly that you have no issues with reading
>     documents
>     exported using current version of Org?
>
>     > E: Finally, note that in (D) I said "machine processable" not
>     > "Accessible"; machine-processable is a pre-requisite to
>     "repurpose "
>     > what you publish, and making  that result usable by different
>     user
>     > communities is a direct consequence of suche
>     machine-processability.
>
>     I understand. But one can similarly say that .org files are
>     "machine
>     processable" and Org export code is not strictly necessary. Yet,
>     it ends
>     up extremely useful in practice.
>
>     I suspect that the exported documents can similarly be improved to
>     reduce the amount of efforts required from visually impair users
>     to read
>     such documents. The question is what kinds improvements can be
>     made on
>     Org side.
>
>     Best,
>     Ihor
>

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
7©4 Id: kg:/m/0285kf1  •0Ü8

Re: Org mode export accessibility

[Ihor Radchenko]

briangpowell <briangpowellms@gmail.com> writes:

> Suggest OrgMode outputs focus on creating "Lowest Common Denominator"
> documents as output:
> TeXinfo docs should be used as the LCD doctype--suggest you focus on
> creating 1 document in Texinfo that you use to create all other sorts of
> documents, when possible:
>
> Pipeline should be more like
> OrgMode->Texinfo->TROFF||DTD/XML/HTML/XHTML->LaTeX/TeX->DVI||SVG->PS->PDF
>
> * TeXinfo: https://savannah.gnu.org/projects/texinfo
> https://www.gnu.org/software/texinfo

I do not think that using Texinfo as intermediate format is useful.
Texinfo does not support many of the available Org syntax structures
like, for example, backend-specific export blocks. We will inevitably
lose some document structure information when exporting to Texinfo.
Please remember that Texinfo is by no means a generic export engine - it
is tailored to produce software documentation specifically and may not
be suitable for more generic authored documents.

Note that we already have a much more feature-full export functionality.
RMS even suggested that Org might be used as a replacement for Texinfo:
https://yhetil.org/emacs-devel/E1nzQh5-0001OB-22@fencepost.gnu.org

Best,
Ihor

Re: Org mode export accessibility

[T.V Raman]

Correct.
1. Org should be the truth, not some export format that is picked
   based on political opinion.

2. Lowest Common denominator in my vocabulary is a "bad word" that LCD
   is something that makes everyone equally unhappy.

   So Take-Away:

   1. Make org the best format for holding the authored content.
      2. Consider all exports to be either modality or use-case
         specific and produce the  best version that that export
         format can support.

         3. But then dont fall into the delusion that the export
            format is the truth --
            
   

Ihor Radchenko writes:
 > briangpowell <briangpowellms@gmail.com> writes:
 > 
 > > Suggest OrgMode outputs focus on creating "Lowest Common Denominator"
 > > documents as output:
 > > TeXinfo docs should be used as the LCD doctype--suggest you focus on
 > > creating 1 document in Texinfo that you use to create all other sorts of
 > > documents, when possible:
 > >
 > > Pipeline should be more like
 > > OrgMode->Texinfo->TROFF||DTD/XML/HTML/XHTML->LaTeX/TeX->DVI||SVG->PS->PDF
 > >
 > > * TeXinfo: https://savannah.gnu.org/projects/texinfo
 > > https://www.gnu.org/software/texinfo
 > 
 > I do not think that using Texinfo as intermediate format is useful.
 > Texinfo does not support many of the available Org syntax structures
 > like, for example, backend-specific export blocks. We will inevitably
 > lose some document structure information when exporting to Texinfo.
 > Please remember that Texinfo is by no means a generic export engine - it
 > is tailored to produce software documentation specifically and may not
 > be suitable for more generic authored documents.
 > 
 > Note that we already have a much more feature-full export functionality.
 > RMS even suggested that Org might be used as a replacement for Texinfo:
 > https://yhetil.org/emacs-devel/E1nzQh5-0001OB-22@fencepost.gnu.org
 > 
 > Best,
 > Ihor

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮

--

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮