[BUG] Incorrect fontification of src blocks for indentation sensitive language modes

[BUG] Incorrect fontification of src blocks for indentation sensitive language modes

[Bhavin Gandhi]

In the following Org mode file, the Ledger mode source block is not
highlighted correctly.

How to reproduce:
1. Clone Ledger mode: git clone
https://github.com/ledger/ledger-mode.git /tmp/ledger-mode
2. emacs -Q -L ~/src/org-mode/lisp
3. Evaluate the following in the *scratch* buffer.

(add-to-list 'load-path "/tmp/ledger-mode")
(require 'ledger-mode)

4. Create an Org mode file test.org with following content
--8<---------------cut here---------------start------------->8---
* Some heading
- Option one
  #+begin_src ledger
  2022/08/16 Buy books
      Expenses:Books                              ₹299
      Assets:Cash
  #+end_src
--8<---------------cut here---------------end--------------->8---

5. The block is not fontified.
6. Now go to the src block and do C-c ', you will see the block
   correctly fontified, and after doing C-c ' again, we are back to
   incorrect fontification.

I tried to debug this and found the function
org-src-font-lock-fontify-block. Turns out that the code block is
actually being fontified, but Ledger mode seems to get the block with
indentation. I found this by adding a (message string) in this function.

--8<---------------cut here---------------start------------->8---
  2022/08/16 Buy books
      Expenses:Books                              ₹299
      Assets:Cash
--8<---------------cut here---------------end--------------->8---

Notice the extra indentation at the beginning of year 2022. This block
is not valid Ledger mode content, so it just keeps it grayed out,
instead of having colors. If we remove the indentation from the src block in
the Org mode file, it gets fontified correctly.

Setting org-adapt-indentation to t didn't make any difference. Value of
org-src-fontify-natively is t by default. I even tried
https://github.com/yantar92/org/tree/feature/org-font-lock-element, it
had the same issue.
I think the language mode should get the block without the extra
indentation. Or am I missing something here?

PS: I haven't looked at the code of org-src-font-lock-fontify-block in
detail, so I could be wrong with this analysis.

-- 
Bhavin Gandhi (bhavin192) | https://geeksocket.in

Re: [BUG] Incorrect fontification of src blocks for indentation sensitive language modes

[Ihor Radchenko]

Bhavin Gandhi <bhavin7392@gmail.com> writes:

> In the following Org mode file, the Ledger mode source block is not
> highlighted correctly.
>
> How to reproduce:
> ...

Confirmed.
Another example is
https://www.reddit.com/r/orgmode/comments/wi0jq9/highlight_doesnt_work_for_src_block_of_diff_with/

> PS: I haven't looked at the code of org-src-font-lock-fontify-block in
> detail, so I could be wrong with this analysis.

`org-src-font-lock-fontify-block' is indeed the culprit. It copies the
text between begin/end markers verbatim without adjusting the
indentation. The proper way would be using an approach similar to
`org-edit-src-code', but it may be tricky to map the fontification back
into the original block without a need to delete/re-insert the code into
Org buffer. Editing during fontification is a bad idea, so some kind of
smarter approach should be used in order to fix this.

P.S. I am genuinely surprised that this bug is only revealed now.

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92

Re: [BUG] Incorrect fontification of src blocks for indentation sensitive language modes

[Fraga, Eric]

On Wednesday, 17 Aug 2022 at 17:39, Ihor Radchenko wrote:
> P.S. I am genuinely surprised that this bug is only revealed now.

I'm not.  Is there any other language whose fontification is indentation
dependent?  Python is indentation based but the fontification doesn't
get affected (generally speaking) as it's keyword based.

-- 
: Eric S Fraga, with org release_9.5.4-737-gd3a9c4 in Emacs 29.0.50