Case insensitivity of simple [[links]]

Case insensitivity of simple [[links]]

[Rudolf Adamkovič]

Howdy!

A while ago, I asked about case insensitivity of the syntactically
simplest "[[links]]" in Org.  I am interested in using these links
because they are the most practical with the "literal" view.

In 'emacs -Q' Org 9.6.9, "[[links]]" are case _sensitive_ when looking
for headings.  This was originally not the case, and case _insensitive_
linking was broken in Org 7 and then fixed in Org 8, as per [1] and [2].
Is it broken again?  I ask because 'org-link-search' documentation says
that search is "case-insensitive and ignores white spaces".

On top of the above, yesterday I noticed that

  (setq org-link-search-must-match-exact-headline nil)

makes "[[link]]" find "* Link" when clicked.  My joy was short-lived,
however, as I soon found that I cannot export anything due to

  org-export-data: Unable to resolve link: "link".

As a user, I am a bit lost here.  I cannot figure out what is a bug and
what is a feature in all of the above.  In a nutshell, all I want is to
make "[[link]]" find "* Link" while editing, after exporting, and on any
Emacs, that is 'emacs -Q'.  I have been struggling with this problem for
a while now, down-casing my headings as a workaround.  Any ideas?

Thank you!

Rudy

[1]
https://stackoverflow.com/questions/35542208/how-to-make-internal-link-in-org-mode-case-insensitive

[2] https://emacs.stackexchange.com/questions/20441/how-to-make-internal-links-in-org-mode-case-insensitive

-- 
"All you have to do is write one true sentence.  Write the truest sentence that
you know."
--- Ernest Miller Hemingway (1899-1961)

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: Case insensitivity of simple [[links]]

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

> A while ago, I asked about case insensitivity of the syntactically
> simplest "[[links]]" in Org.  I am interested in using these links
> because they are the most practical with the "literal" view.
>
> In 'emacs -Q' Org 9.6.9, "[[links]]" are case _sensitive_ when looking
> for headings.  This was originally not the case, and case _insensitive_
> linking was broken in Org 7 and then fixed in Org 8, as per [1] and [2].
> Is it broken again?  I ask because 'org-link-search' documentation says
> that search is "case-insensitive and ignores white spaces".

Currently, headline matching in `org-link-search' and
`org-export-resolve-fuzzy-link' is case-sensitive.

> On top of the above, yesterday I noticed that
>
>   (setq org-link-search-must-match-exact-headline nil)
>
> makes "[[link]]" find "* Link" when clicked.  My joy was short-lived,
> however, as I soon found that I cannot export anything due to
>
>   org-export-data: Unable to resolve link: "link".

This is because of different reason - Org export is not able to export
fuzzy links that do not point to a named target:

    (defun org-export-resolve-fuzzy-link (link info &rest pseudo-types)
      "Return LINK destination.
    
    INFO is a plist holding contextual information.
    
    Return value can be an object or an element:
    
    - If LINK path matches a target object (i.e. <<path>>) return it.
    
    - If LINK path exactly matches the name or results affiliated keyword
      (i.e. #+NAME: path or #+RESULTS: name) of an element, return that
      element.
    
    - If LINK path exactly matches any headline name, return that
      element.
    
    - Otherwise, throw an error.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Case insensitivity of simple [[links]]

[Rudolf Adamkovič]

Ihor Radchenko <yantar92@posteo.net> writes:

> [...]

So, if I understand correctly, there is no way to make "[[link]]" find
"* Link" while editing, after exporting, and on any Emacs, that is
'emacs -Q'.  In other words, writing "* link" in lowercase, which is my
workaround, is "the solution".  Do I have that right?

Rudy
-- 
"It is no paradox to say that in our most theoretical moods we may be
nearest to our most practical applications."
-- Alfred North Whitehead, 1861-1947

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: Case insensitivity of simple [[links]]

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> [...]
>
> So, if I understand correctly, there is no way to make "[[link]]" find
> "* Link" while editing, after exporting, and on any Emacs, that is
> 'emacs -Q'.  In other words, writing "* link" in lowercase, which is my
> workaround, is "the solution".  Do I have that right?

You might as well use [[Link]].

As for case sensitivity, it is something we might discuss. I just
outlined the current state of the Org code.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Case insensitivity of simple [[links]]

[Rudolf Adamkovič]

Ihor Radchenko <yantar92@posteo.net> writes:

> You might as well use [[Link]].

For context, I use '#+OPTIONS: num:nil', which means that '[[link]]'
exports as 'link' and not a section number.  I thought about using
'[[Link]]' as you suggest, but it would make almost all sentences I
write ungrammatical.  For comparison, consider:

Properly capitalized:

  The set of cellular membrane-bound organelles inside of the eukaryotic
  cell that work together on the synthesis and distribution of proteins
  and lipids.

Sentence-capitalized to make Org find the respective headings:

  The set of Cellular membrane-bound Organelles inside of the Eukaryotic
  cell that work together on the synthesis and distribution of Proteins
  and Lipids.

> As for case sensitivity, it is something we might discuss. I just
> outlined the current state of the Org code.

Thank you for outlining the current state.  I asked about it because I
have been struggling with this problem for a long time, and I was not
sure whether Org can or cannot do it, particularly given the fact that
my use of literal links is as simple as it gets.

If someone could fix this problem, say by adding link case-insensitivity
to '#+OPTIONS', I am willing to sponsor it financially, as the current
behavior is my biggest problem with the Org mode.

Rudy
-- 
"'Obvious' is all too often a synonym for 'wrong'."
-- Jeff Erickson, Algorithms, 2019

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: Case insensitivity of simple [[links]]

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> You might as well use [[Link]].
>
> For context, I use '#+OPTIONS: num:nil', which means that '[[link]]'
> exports as 'link' and not a section number.

I am confused. num:nil just means that section titles will not be
numbered. It has nothing to do with links, AFAIK.

> ... I thought about using
> '[[Link]]' as you suggest, but it would make almost all sentences I
> write ungrammatical.  For comparison, consider:
>
> Properly capitalized:
>
>   The set of cellular membrane-bound organelles inside of the eukaryotic
>   cell that work together on the synthesis and distribution of proteins
>   and lipids.
>
> Sentence-capitalized to make Org find the respective headings:
>
>   The set of Cellular membrane-bound Organelles inside of the Eukaryotic
>   cell that work together on the synthesis and distribution of Proteins
>   and Lipids.

What about [[Link][link]]? Check out how we do it in
https://orgmode.org/worg/org-syntax.org (rendered as
https://orgmode.org/worg/org-syntax.html).

>> As for case sensitivity, it is something we might discuss. I just
>> outlined the current state of the Org code.
>
> Thank you for outlining the current state.  I asked about it because I
> have been struggling with this problem for a long time, and I was not
> sure whether Org can or cannot do it, particularly given the fact that
> my use of literal links is as simple as it gets.
>
> If someone could fix this problem, say by adding link case-insensitivity
> to '#+OPTIONS', I am willing to sponsor it financially, as the current
> behavior is my biggest problem with the Org mode.

Judging from your example, I am not sure if it is an actual problem with
Org mode. I am still to see a use case when case-insensitivity would be
useful. Now, I am inclined to fix the docstring (and possibly the
manual), clarifying about case-sensitivity of the internal links.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Case insensitivity of simple [[links]]

[Rudolf Adamkovič]

Ihor Radchenko <yantar92@posteo.net> writes:

>> For context, I use '#+OPTIONS: num:nil', which means that '[[link]]'
>> exports as 'link' and not a section number.
>
> I am confused. num:nil just means that section titles will not be
> numbered. It has nothing to do with links, AFAIK.

My sentence says it all, but if that is not clear, consider:

EXAMPLE 1:

  This is a [[test]] link.
  
  * test

EXPORT OF EXAMPLE 1:

  This is a 1 link.

EXAMPLE 2:

  #+OPTIONS: num:nil
  
  This is a [[test]] link.
  
  * test

EXPORT OF EXAMPLE 2:

  This is a test link.

> What about [[Link][link]]?

That makes Org documents hard to read in *plain text*, that is without
the "descriptive links" fontification.  Here is my note again, compare:

EXAMPLE 1: Hard-to-read, unhelpful markup:

  The set of [[Cellular membrane][cellular membrane]]-bound
  [[Organelle][organelle]]s inside of the [[Eukaryotic cell][eukaryotic
  cell]] that work together on the synthesis and distribution of
  [[Protein][protein]]s and [[Lipid][lipid]]s.

EXAMPLE 2: Lightweight, helpful markup:

  The set of [[cellular membrane]]-bound [[organelle]]s inside of the
  [[eukaryotic cell]] that work together on the synthesis and
  distribution of [[protein]]s and [[lipid]]s.

> Judging from your example, I am not sure if it is an actual problem
> with Org mode. I am still to see a use case when case-insensitivity
> would be useful. Now, I am inclined to fix the docstring (and possibly
> the manual), clarifying about case-sensitivity of the internal links.

For me, this is not only "an actual problem with Org" but the biggest
problem, and the only big problem, I have with Org.

As a markup language, Org is "99% there" in having some of the *most
practical* plain text [[link]]s available, but then falls short on case
sensitivity, ending up with the exact opposite.

Now, I could use descriptive links, which hide the [[Wart][wart]] behind
fontification tricks, but that would not solve my problem.  Why?  I need
to use other plain text tools, such as Git, with my densely linked Org
documents.

Rudy
-- 
"Simplicity is complexity resolved."
-- Constantin Brâncuși, 1876-1957

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: Case insensitivity of simple [[links]]

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

>> What about [[Link][link]]?
>
> That makes Org documents hard to read in *plain text*, that is without
> the "descriptive links" fontification.  Here is my note again, compare:
>
> EXAMPLE 1: Hard-to-read, unhelpful markup:
>
>   The set of [[Cellular membrane][cellular membrane]]-bound
>   [[Organelle][organelle]]s inside of the [[Eukaryotic cell][eukaryotic
>   cell]] that work together on the synthesis and distribution of
>   [[Protein][protein]]s and [[Lipid][lipid]]s.
>
> EXAMPLE 2: Lightweight, helpful markup:
>
>   The set of [[cellular membrane]]-bound [[organelle]]s inside of the
>   [[eukaryotic cell]] that work together on the synthesis and
>   distribution of [[protein]]s and [[lipid]]s.

Note that there is generally no guarantee that [[name]] link will be
exported as "name" by any particular backend. Your workaround with
num:nil is just a workaround that may or may not work in future.

>> Judging from your example, I am not sure if it is an actual problem
>> with Org mode. I am still to see a use case when case-insensitivity
>> would be useful. Now, I am inclined to fix the docstring (and possibly
>> the manual), clarifying about case-sensitivity of the internal links.
>
> For me, this is not only "an actual problem with Org" but the biggest
> problem, and the only big problem, I have with Org.
>
> As a markup language, Org is "99% there" in having some of the *most
> practical* plain text [[link]]s available, but then falls short on case
> sensitivity, ending up with the exact opposite.
>
> Now, I could use descriptive links, which hide the [[Wart][wart]] behind
> fontification tricks, but that would not solve my problem.  Why?  I need
> to use other plain text tools, such as Git, with my densely linked Org
> documents.

I can see how this can be helpful in certain scenarios.
I made headline and radio target link matching case-insensitive on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=ed42dc34a
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=ec2399330

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Case insensitivity of simple [[links]]

[Rudolf Adamkovič]

Ihor Radchenko <yantar92@posteo.net> writes:

> Note that there is generally no guarantee that [[name]] link will be
> exported as "name" by any particular backend. Your workaround with
> num:nil is just a workaround that may or may not work in future.

I did not know this is considered a workaround!  I have always thought
this is how Org is specified to work (as in tests).

> I can see how this can be helpful in certain scenarios.
> I made headline and radio target link matching case-insensitive on main.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=ed42dc34a
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=ec2399330

Wait, did you just solve my *biggest* problem with Org?  OMG!

One more question before I declare a full victory: I want my notebook to
show literal links by default, but the following does not seem to work:

# -*- fill-column: 79; org-link-descriptive: nil -*-

The line seems to confuse Org in that not only it shows descriptive
links but the first call to 'org-toggle-link-display' does nothing.

Rudy
-- 
"'Obvious' is all too often a synonym for 'wrong'."
-- Jeff Erickson, Algorithms, 2019

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: Case insensitivity of simple [[links]]

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> Note that there is generally no guarantee that [[name]] link will be
>> exported as "name" by any particular backend. Your workaround with
>> num:nil is just a workaround that may or may not work in future.
>
> I did not know this is considered a workaround!  I have always thought
> this is how Org is specified to work (as in tests).

Tests only cover (some) built-in backends. And they do not always test
documented features. AFAIK, text used when exporting fuzzy links without
description is not something we document. Individual backends are not
constrained to follow any specific convention in such scenario, although
built-in backends generally handle certain things similarly (often,
because they are used as a template for each other).

> One more question before I declare a full victory: I want my notebook to
> show literal links by default, but the following does not seem to work:
>
> # -*- fill-column: 79; org-link-descriptive: nil -*-
>
> The line seems to confuse Org in that not only it shows descriptive
> links but the first call to 'org-toggle-link-display' does nothing.

It does nothing because it is one of the options that must be set before
Org mode is loaded. Resolving buffer-local variables happens after the
major mode is loaded.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Case insensitivity of simple [[links]]

[Rudolf Adamkovič]

Ihor Radchenko <yantar92@posteo.net> writes:

>>> Note that there is generally no guarantee that [[name]] link will be
>>> exported as "name" by any particular backend. Your workaround with
>>> num:nil is just a workaround that may or may not work in future.
>>
>> I did not know this is considered a workaround!  I have always thought
>> this is how Org is specified to work (as in tests).

Actually, dang!  I spoke too soon.  While links now work consistently
before and after exports, their description matches the destination
heading, not the source link.  But if we case-fold now, it would make a
lot more sense to match the case of the source link, as it enables one
to link [[capitalized heading]]s.  Otherwise, we case-fold with no
significant benefits, as far as I can see.

Without also fixing this part, my problem with Org remains. :(

>> [...] call to 'org-toggle-link-display' does nothing.

> It does nothing because it is one of the options that must be set before
> Org mode is loaded. Resolving buffer-local variables happens after the
> major mode is loaded.

I have noticed that 'org-use-extra-keys', for example, is documented as
"must set it before loading Org", but there is nothing similar in the
'org-link-descriptive' docstring.  So perhaps this is a bug, not a
feature?

Upon a quick look, 'org-toggle-link-display' does exactly two things:

  1. (setq org-link-descriptive (not org-link-descriptive)) and
  2. (org-link-descriptive-ensure),

where 'org-link-descriptive-ensure' wraps exactly one expression:

  (org-fold-core-set-folding-spec-property
     (car org-link--link-folding-spec)
     :visible (not org-link-descriptive))

Could Org execute this expression after loading a document, given the
buffer-local value of 'org-link-descriptive' necessitates it?

Rudy
-- 
"It is far better to have a question that can't be answered than an answer that
can't be questioned."
--- Carl Sagan

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: Case insensitivity of simple [[links]]

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

>>> [...] call to 'org-toggle-link-display' does nothing.
>
>> It does nothing because it is one of the options that must be set before
>> Org mode is loaded. Resolving buffer-local variables happens after the
>> major mode is loaded.
>
> I have noticed that 'org-use-extra-keys', for example, is documented as
> "must set it before loading Org", but there is nothing similar in the
> 'org-link-descriptive' docstring.  So perhaps this is a bug, not a
> feature?

I was not precise. `org-link-descriptive' must be set before Org major
mode is loaded in current buffer. In general, all the Emacs
customization are not guaranteed to do anything when the major mode is
already loaded. There is a mechanism with :set function for defcustom,
but most people use setq that ignores this mechanism.

Anyway, I just added a :set function for `org-link-descriptive' on main.
It will be used when customization interface, `setopt', or
`custom-set-variables' is used.

> Upon a quick look, 'org-toggle-link-display' does exactly two things:
>
>   1. (setq org-link-descriptive (not org-link-descriptive)) and
>   2. (org-link-descriptive-ensure),
>
> where 'org-link-descriptive-ensure' wraps exactly one expression:
>
>   (org-fold-core-set-folding-spec-property
>      (car org-link--link-folding-spec)
>      :visible (not org-link-descriptive))
>
> Could Org execute this expression after loading a document, given the
> buffer-local value of 'org-link-descriptive' necessitates it?

No. Because the code execution during Org loading happens before Emacs
loads file-local variables. This is by major mode design and we cannot
do much about this. See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=57003

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Case insensitivity of simple [[links]]

[Rudolf Adamkovič]

Ihor Radchenko <yantar92@posteo.net> writes:

> [...] loading happens before Emacs loads file-local variables. This is
> by major mode design and we cannot do much about this. See
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=57003

I see.  So, that is by design of Emacs, and Org has no built-in
mechanism, akin to the existing #+OPTIONS, where I could say "show
literal links", right?

Rudy
-- 
"I love deadlines.  I love the whooshing noise they make as they go by."
-- Douglas Adams, The Salmon of Doubt, 2002

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: Case insensitivity of simple [[links]]

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> [...] loading happens before Emacs loads file-local variables. This is
>> by major mode design and we cannot do much about this. See
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=57003
>
> I see.  So, that is by design of Emacs, and Org has no built-in
> mechanism, akin to the existing #+OPTIONS, where I could say "show
> literal links", right?

We can add this to #+STARTUP options, if necessary.
It is similar to #+OPTIONS.

#+STARTUP is the way we currently work around the file-local variable
 problem.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Case insensitivity of simple [[links]]

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

> Actually, dang!  I spoke too soon.  While links now work consistently
> before and after exports, their description matches the destination
> heading, not the source link.  But if we case-fold now, it would make a
> lot more sense to match the case of the source link, as it enables one
> to link [[capitalized heading]]s.  Otherwise, we case-fold with no
> significant benefits, as far as I can see.
>
> Without also fixing this part, my problem with Org remains. :(

What you want is very hard to implement in Org mode for all the backends
- the export API, when querying a link target, returns that target as
AST element. Individual backends then decide what to do with that target
in order to extract the actual exported link text. And we have no
control over the third-party export backends.

What you may do instead is to create a custom element in
`org-export-filter-parse-tree-functions' that will force description in
the fuzzy links.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>