[BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[wolf]

[-- Attachment #1: Type: text/plain, Size: 5849 bytes --]

Hello,

when editing a source block of yaml type, the indentation does not work properly
(tabs are used instead of spaces).  I am not sure if org or yaml-mode are the
cause, however since RET is bound to org-return, I am reporting it here first.

I did test it inside a container with nothing but emacs, so no user
configuration is involved at all.

Reproduction steps:
1. Have org-mode and yaml-mode installed.
2. C-x C-f /tmp/x.org RET
3. Type in:

#+begin_src yaml
  a:
    b:
      c:
	d:
#+end_src

After each line, press RET to go to the next one.  RET is bound to org-return.
Notice that when you press RET after `c:', emacs will insert a TAB character,
instead of expected 8 spaces.

When editing a yaml document in yaml-mode (either a separate document, or using
the C-c '), the indentation works properly.



yaml-mode: version 0.0.15 (but I did test the latest from melpa with same result)

Emacs  : GNU Emacs 28.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.37, cairo version 1.16.0)
Package: Org mode version 9.6.6 ( @ /home/wolf/.emacs.d/elpa/org-9.6.6/)

current state:
==============
(setq
 org-link-elisp-confirm-function 'yes-or-no-p
 org-bibtex-headline-format-function #[257 "\300^A\236A\207" [:title] 3 "\n\n(fn ENTRY)"]
 org-persist-after-read-hook '(org-element--cache-persist-after-read)
 org-export-before-parsing-hook '(org-attach-expand-links)
 org-cycle-tab-first-hook '(org-babel-hide-result-toggle-maybe org-babel-header-arg-expand)
 org-archive-hook '(org-attach-archive-delete-maybe)
 org-cycle-hook '(org-cycle-hide-archived-subtrees org-cycle-show-empty-lines
                  org-cycle-optimize-window-after-visibility-change
                  org-cycle-display-inline-images)
 org-persist-before-read-hook '(org-element--cache-persist-before-read)
 org-mode-hook '(#[0 "\300\301\302\303\304$\207"
                   [add-hook change-major-mode-hook org-fold-show-all append local] 5]
                 #[0 "\300\301\302\303\304$\207"
                   [add-hook change-major-mode-hook org-babel-show-result-all append local] 5]
                 org-babel-result-hide-spec org-babel-hide-all-hashes)
 org-confirm-shell-link-function 'yes-or-no-p
 outline-isearch-open-invisible-function 'outline-isearch-open-invisible
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-src-mode-hook '(org-src-babel-configure-edit-buffer org-src-mode-configure-edit-buffer)
 org-confirm-elisp-link-function 'yes-or-no-p
 org-speed-command-hook '(org-speed-command-activate org-babel-speed-command-activate)
 org-fold-core-isearch-open-function 'org-fold-core--isearch-reveal
 org-persist-before-write-hook '(org-element--cache-persist-before-write)
 org-tab-first-hook '(org-babel-hide-result-toggle-maybe org-babel-header-arg-expand)
 org-link-shell-confirm-function 'yes-or-no-p
 org-babel-pre-tangle-hook '(save-buffer)
 org-agenda-loop-over-headlines-in-active-region nil
 org-occur-hook '(org-first-headline-recenter)
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-link-parameters '(("attachment" :follow org-attach-follow :complete
                        org-attach-complete-link)
                       ("id" :follow org-id-open)
                       ("eww" :follow org-eww-open :store org-eww-store-link)
                       ("rmail" :follow org-rmail-open :store org-rmail-store-link)
                       ("mhe" :follow org-mhe-open :store org-mhe-store-link)
                       ("irc" :follow org-irc-visit :store org-irc-store-link :export
                        org-irc-export)
		       ("info" :follow org-info-open :export org-info-export :store
                        org-info-store-link :insert-description
                        org-info-description-as-command)
		       ("gnus" :follow org-gnus-open :store org-gnus-store-link)
		       ("docview" :follow org-docview-open :export org-docview-export :store
			org-docview-store-link)
                       ("bibtex" :follow org-bibtex-open :store org-bibtex-store-link)
                       ("bbdb" :follow org-bbdb-open :export org-bbdb-export :complete
			org-bbdb-complete-link :store org-bbdb-store-link)
                       ("w3m" :store org-w3m-store-link)
                       ("doi" :follow org-link-doi-open :export org-link-doi-export)
                       ("file+sys") ("file+emacs") ("shell" :follow org-link--open-shell)
                       ("news" :follow
                        #[514 "\301\300\302^DQ^B\"\207" ["news" browse-url ":"] 6
                          "\n\n(fn URL ARG)"]
                        )
                       ("mailto" :follow
                        #[514 "\301\300\302^DQ^B\"\207" ["mailto" browse-url ":"] 6
                          "\n\n(fn URL ARG)"]
                        )
                       ("https" :follow
                        #[514 "\301\300\302^DQ^B\"\207" ["https" browse-url ":"] 6
                          "\n\n(fn URL ARG)"]
                        )
                       ("http" :follow
                        #[514 "\301\300\302^DQ^B\"\207" ["http" browse-url ":"] 6
                          "\n\n(fn URL ARG)"]
                        )
                       ("ftp" :follow
                        #[514 "\301\300\302^DQ^B\"\207" ["ftp" browse-url ":"] 6
                          "\n\n(fn URL ARG)"]
                        )
                       ("help" :follow org-link--open-help :store org-link--store-help)
                       ("file" :complete org-link-complete-file)
                       ("elisp" :follow org-link--open-elisp))
 org-metaup-hook '(org-babel-load-in-session-maybe)
 )


-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 1634 bytes --]

wolf <wolf@wolfsden.cz> writes:

> Reproduction steps:
> 1. Have org-mode and yaml-mode installed.
> 2. C-x C-f /tmp/x.org RET
> 3. Type in:
>
> #+begin_src yaml
>   a:
>     b:
>       c:
> 	d:
> #+end_src
>
> After each line, press RET to go to the next one.  RET is bound to org-return.
> Notice that when you press RET after `c:', emacs will insert a TAB character,
> instead of expected 8 spaces.

Confirmed.
This is caused by `org-src--contents-for-write-back' not adjusting
blank line indentation in some cases.

The attached diff will fix the issue, but git log reveals that the whole
thing is fragile - we adjusted code in this area multiple times with
a number of tricky special cases:

https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=e1c49af76
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=70e65a202
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=ee0fd1ec3
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=70e65a202
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=857ae366b
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=91236a3db

https://orgmode.org/list/0f961d3b-a4ef-fbbf-ab16-7ff1af8b2070@gmail.com
https://orgmode.org/list/5DCBAF63-0E88-44AC-B892-1260F37E7E00@manicmind.earth

CCing Bastien and Sébastien, the authors of the previous commits in this
area.

I feel that the whole approach we use now with preserve-fl, use-tabs?,
and preserve-blank-line is overcomplicated. Maybe someone can explain
why we need all these special cases? The code does not reveal a whole lot.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test.diff --]
[-- Type: text/x-patch, Size: 837 bytes --]

diff --git a/lisp/org-src.el b/lisp/org-src.el
index 317682844..876310867 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -499,12 +499,9 @@ (defun org-src--contents-for-write-back (write-back-buf)
 	(when preserve-fl (forward-line))
         (while (not (eobp))
 	  (skip-chars-forward " \t")
-          (when (or (not (eolp))                               ; not a blank line
-                    (and (eq (point) (marker-position marker)) ; current line
-                         preserve-blank-line))
-	    (let ((i (current-column)))
-	      (delete-region (line-beginning-position) (point))
-	      (indent-to (+ i indentation-offset))))
+	  (let ((i (current-column)))
+	    (delete-region (line-beginning-position) (point))
+	    (indent-to (+ i indentation-offset)))
 	  (forward-line)))
       (set-marker marker nil))))
 

[-- Attachment #3: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 2618 bytes --]

Hi Ihor, wolf,

Ihor Radchenko writes:
> Confirmed.
> This is caused by `org-src--contents-for-write-back' not adjusting
> blank line indentation in some cases.

I don't think that's the issue. In fact, applying your diff didn't
seem to solve the issue on my end.

Generally, if you edit the given yaml in a =C-c C-'= buffer and go
back to the org buffer with the default configuration, spaces will be
converted to tabs, because =indent-tabs-mode= is =t= in the org buffer.

I don't think there's much that we can do about it. We could try to
read the value of =indent-tabs-mode= in the native buffer and preserve
it in the org buffer, but then the org buffer would have mixed
indentation all over, and that's exactly what the current code tries to
avoid.

To fix the original issue, you can set =indent-tabs-mode= to =nil= in
your org files, or possibly set =org-preserve-indentation= to =t=
(untested).

> I feel that the whole approach we use now with preserve-fl, use-tabs?,
> and preserve-blank-line is overcomplicated. Maybe someone can explain
> why we need all these special cases? The code does not reveal a whole lot.

  - =use-tabs?= seems pretty straightforward, its purpose is to respect
    the value of =indent-tabs-mode= in the org buffer.
  - =preserve-fl= is an isolated issue, and only concerns LaTeX
    fragments. I will attach a test with the issue it solves with
    multiline LaTeX fragments. I think LaTeX fragments are particular
    because in the org buffer they do not need to start at the
    beginning of a line.
  - The =preserve-blank-line= part (committed by me) is quite abstruse.
    Its name does not make any sense !

    Originally, we did not try to reindent blank lines when writing
    back to the org buffer. I changed it so that when using
    =org-return=, the newline would get correctly indented, I think.
    Then I changed it again so that only the current blank line might
    get indented, see
    : https://list.orgmode.org/725763.1632663635@apollo2.minshall.org/T/
    The variable =preserve-blank-line= decides whether we should indent
    the current blank line (if it is empty, we should maybe not).


Here are three patches attached.
  1. Two tests : about editing LaTeX fragments, and preserving empty
     lines.
  2. Renaming of =preserve-blank-line=, for clarity.
  3. Two more tests testing the behaviour of =org-return= and
     indentation, with the default configuration. When writing these, I
     found the behaviour was buggy in one case, and modified
     =org-indent-line= to fix it.

Does that look alright to you ?

Regards,
-- 
Sébastien Miquel

[-- Attachment #2: 0001-test-org-src.el-Add-two-tests.patch --]
[-- Type: text/x-patch, Size: 2516 bytes --]

From 9613a54d20883e56301f987f5495b962f3763cad Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Sat, 17 Jun 2023 17:00:51 +0200
Subject: [PATCH] test-org-src.el: Add two tests

* testing/lisp/test-org-src.el (test-org-src/preserve-empty-lines):
Test that empty lines are not indented.
(test-org-src/indented-latex-fragments): Test special edit of
multiline indented LaTeX fragment.
---
 testing/lisp/test-org-src.el | 52 ++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/testing/lisp/test-org-src.el b/testing/lisp/test-org-src.el
index 2a45ba66e..42edb364a 100644
--- a/testing/lisp/test-org-src.el
+++ b/testing/lisp/test-org-src.el
@@ -144,6 +144,47 @@ This is a tab:\t.
               (org-edit-src-exit)
               (buffer-string))))))
 
+(ert-deftest test-org-src/preserve-empty-lines ()
+  "Editing block preserves empty lines."
+  (should
+   (equal "
+#+begin_src emacs-lisp
+  The following line is empty
+
+  abc
+#+end_src"
+          (org-test-with-temp-text
+           "
+#+begin_src emacs-lisp
+  The following line is empty
+
+  abc<point>
+#+end_src"
+           (let ((org-edit-src-content-indentation 2)
+                 (org-src-preserve-indentation nil))
+             (org-edit-special)
+             (org-edit-src-exit)
+             (buffer-string)))))
+  (should
+   (equal "
+#+begin_src emacs-lisp
+  The following line is empty
+
+  abc
+#+end_src"
+          (org-test-with-temp-text
+           "
+#+begin_src emacs-lisp
+  The following line is empty
+<point>
+  abc
+#+end_src"
+           (let ((org-edit-src-content-indentation 2)
+                 (org-src-preserve-indentation nil))
+             (org-edit-special)
+             (org-edit-src-exit)
+             (buffer-string)))))  )
+
 (ert-deftest test-org-src/coderef-format ()
   "Test `org-src-coderef-format' specifications."
   ;; Regular tests in a src block, an example block and an edit
@@ -376,6 +417,17 @@ This is a tab:\t.
 	(org-edit-src-exit)
 	(buffer-string))))))
 
+(ert-deftest test-org-src/indented-latex-fragments ()
+  "Test editing multiline indented LaTeX fragment."
+  (should
+   (equal
+    "- Item $abc\n  efg$"
+    (org-test-with-temp-text
+     "- Item $abc<point>\n  efg$"
+     (org-edit-special)
+     (org-edit-src-exit)
+     (buffer-string)))))
+
 (ert-deftest test-org-src/footnote-references ()
   "Test editing footnote references."
   ;; Error when there is no definition to edit.
-- 
2.41.0


[-- Attachment #3: 0001-org-src.el-Rename-internal-variable-for-clarity.patch --]
[-- Type: text/x-patch, Size: 3384 bytes --]

From cb7e3fb59109d2e3ea4fab72ae2cd69f5b1efa08 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Sat, 17 Jun 2023 17:10:53 +0200
Subject: [PATCH] org-src.el: Rename internal variable for clarity

* lisp/org-src.el (org-src--contents-for-write-back):
(org-src--edit-element): Rename `preserve-blank-line' to
`indent-current-blank-line'. It is used to decide whether we should
indent the current blank line after a special edit.
---
 lisp/org-src.el | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index f15ba8e99..f0982ea12 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -318,8 +318,8 @@ is 0.")
   "File name associated to Org source buffer, or nil.")
 (put 'org-src-source-file-name 'permanent-local t)
 
-(defvar-local org-src--preserve-blank-line nil)
-(put 'org-src--preserve-blank-line 'permanent-local t)
+(defvar-local org-src--indent-current-blank-line nil)
+(put 'org-src--indent-current-blank-line 'permanent-local t)
 
 (defun org-src--construct-edit-buffer-name (org-buffer-name lang)
   "Construct the buffer name for a source editing buffer.
@@ -473,7 +473,7 @@ Assume point is in the corresponding edit buffer."
                      (list (buffer-substring (point-min) eol)
                            (buffer-substring eol (point-max))))))
 	(write-back org-src--allow-write-back)
-        (preserve-blank-line org-src--preserve-blank-line)
+        (indent-current-blank-line org-src--indent-current-blank-line)
         marker)
     (with-current-buffer write-back-buf
       ;; Reproduce indentation parameters from source buffer.
@@ -493,7 +493,7 @@ Assume point is in the corresponding edit buffer."
 	  (skip-chars-forward " \t")
           (when (or (not (eolp))                               ; not a blank line
                     (and (eq (point) (marker-position marker)) ; current line
-                         preserve-blank-line))
+                         indent-current-blank-line))
 	    (let ((i (current-column)))
 	      (delete-region (line-beginning-position) (point))
 	      (indent-to (+ i indentation-offset))))
@@ -552,8 +552,9 @@ Leave point in edit buffer."
              (blank-line (save-excursion (beginning-of-line)
                                          (looking-at-p "^[[:space:]]*$")))
              (empty-line (and blank-line (looking-at-p "^$")))
-             (preserve-blank-line (or (and blank-line (not empty-line))
-                                      (and empty-line (= (+ block-ind content-ind) 0))))
+             (indent-current-blank-line (or (and blank-line (not empty-line))
+                                            (and empty-line
+                                                 (= (+ block-ind content-ind) 0))))
 	     (preserve-ind
 	      (and (memq type '(example-block src-block))
 		   (or (org-element-property :preserve-indent datum)
@@ -603,7 +604,7 @@ Leave point in edit buffer."
 	(setq org-src--overlay overlay)
 	(setq org-src--allow-write-back write-back)
 	(setq org-src-source-file-name source-file-name)
-        (setq org-src--preserve-blank-line preserve-blank-line)
+        (setq org-src--indent-current-blank-line indent-current-blank-line)
 	;; Start minor mode.
 	(org-src-mode)
 	;; Clear undo information so we cannot undo back to the
-- 
2.41.0


[-- Attachment #4: 0001-org.el-org-indent-line-Fix-to-the-indentation-inside.patch --]
[-- Type: text/x-patch, Size: 4240 bytes --]

From c5d9197700257f1505a70c0cfee0aa83619a4399 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Sat, 17 Jun 2023 19:07:50 +0200
Subject: [PATCH] org.el (org-indent-line): Fix to the indentation inside src
 blocks

* lisp/org.el (org-indent-line): When indenting a line inside a src
block, preindent it with the common block indentation before calling
native indentation.
* testing/lisp/test-org-src.el (test-org-src/org-return-indents): Test
indentation inside src blocks, when calling `org-return'.
---
 lisp/org.el                  | 19 +++++++++++++------
 testing/lisp/test-org-src.el | 34 ++++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+), 6 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 4273385bf..38d13f111 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -19083,20 +19083,27 @@ Also align node properties according to `org-property-format'."
 		     (org-with-point-at (org-element-property :end element)
 		       (skip-chars-backward " \t\n")
 		       (line-beginning-position))))
-             ;; At the beginning of a blank line, do some preindentation.  This
-             ;; signals org-src--edit-element to preserve the indentation on exit
-             (when (and (looking-at-p "^[[:space:]]*$")
-                        (not org-src-preserve-indentation))
-               (let (block-content-ind some-ind)
+             ;; Do some preindentation, to add the common block
+             ;; indentation to the current line.
+             (when (not org-src-preserve-indentation)
+               (let ((current-ind (org-current-text-indentation))
+                     block-content-ind some-ind)
                  (org-with-point-at (org-element-property :begin element)
                    (setq block-content-ind (+ (org-current-text-indentation)
                                               org-edit-src-content-indentation))
+                   ;; Check that the first line of the block has the
+                   ;; minimal indentation
                    (forward-line)
 		   (save-match-data (re-search-forward "^[ \t]*\\S-" nil t))
                    (backward-char)
                    (setq some-ind (if (looking-at-p "#\\+end_src")
                                       block-content-ind (org-current-text-indentation))))
-                 (indent-line-to (min block-content-ind some-ind))))
+                 ;; If the current line is the first one and does not
+                 ;; have the minimal indentation, we do not preindent,
+                 ;; since it could break the relative indentation with
+                 ;; respect to the following lines
+                 (when (< current-ind (min block-content-ind some-ind))
+                   (indent-line-to (min block-content-ind some-ind)))))
 	     (org-babel-do-key-sequence-in-edit-buffer (kbd "TAB")))
 	    (t
 	     (let ((column (org--get-expected-indentation element nil)))
diff --git a/testing/lisp/test-org-src.el b/testing/lisp/test-org-src.el
index 42edb364a..4d943927d 100644
--- a/testing/lisp/test-org-src.el
+++ b/testing/lisp/test-org-src.el
@@ -144,6 +144,40 @@ This is a tab:\t.
               (org-edit-src-exit)
               (buffer-string))))))
 
+(ert-deftest test-org-src/org-return-indents ()
+  "Calling `org-return' indents newline."
+  (should
+   (equal "
+#+begin_src emacs-lisp
+  (or c1\n      \n      c2)
+#+end_src"
+          (org-test-with-temp-text
+           "
+#+begin_src emacs-lisp
+  (or c1<point>
+      c2)
+#+end_src"
+           (let ((org-edit-src-content-indentation 2)
+                 (org-src-tab-acts-natively t)
+                 (org-src-preserve-indentation nil))
+             (org-return t)
+             (buffer-string)))))
+  (should
+   (equal "
+#+begin_src emacs-lisp
+  (or c1
+      c2)
+#+end_src"
+          (org-test-with-temp-text
+           "
+#+begin_src emacs-lisp
+  (or c1<point> c2)
+#+end_src"
+           (let ((org-edit-src-content-indentation 2)
+                 (org-src-preserve-indentation nil))
+             (org-return t)
+             (buffer-string))))))
+
 (ert-deftest test-org-src/preserve-empty-lines ()
   "Editing block preserves empty lines."
   (should
-- 
2.41.0

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> Ihor Radchenko writes:
>> Confirmed.
>> This is caused by `org-src--contents-for-write-back' not adjusting
>> blank line indentation in some cases.
>
> I don't think that's the issue. In fact, applying your diff didn't
> seem to solve the issue on my end.

You are right.

> To fix the original issue, you can set =indent-tabs-mode= to =nil= in
> your org files, or possibly set =org-preserve-indentation= to =t=
> (untested).

Sure. Or passing -i switch. But that's wrong, IMHO.
Org mode files should be portable. We are aiming for universal
Emacs-independent markup. The value of `indent-tabs-mode' must not
affect Org markup - third party apps should be able to read src block
code without knowledge about Emacs minor modes being active when the Org
file is created. And Org files should not be broken for other users with
different settings for indent-tabs-mode.

> Generally, if you edit the given yaml in a =C-c C-'= buffer and go
> back to the org buffer with the default configuration, spaces will be
> converted to tabs, because =indent-tabs-mode= is =t= in the org buffer.
>
> I don't think there's much that we can do about it. We could try to
> read the value of =indent-tabs-mode= in the native buffer and preserve
> it in the org buffer, but then the org buffer would have mixed
> indentation all over, and that's exactly what the current code tries to
> avoid.

I think that mixed indentation in the Org buffer is fine. In particular,
in verbatim-type environments.

Let me demonstrate another related problem that manifests more clearly:

1. emacs -Q /tmp/bug.org (new file)
2. C-c C-, s python :results output <RET>
3. M-x whitespace-mode <RET>
4. if True:<RET>
5. if True:<RET>
6. print("Yes")
7. Observe tab in front of print("Yes")
8. M-: (require 'ob-python) <RET>
9. C-c C-c yes <RET>
10. Observe python error

  File "<stdin>", line 3
    print("yes")
TabError: inconsistent use of tabs and spaces in indentation
[ Babel evaluation exited with code 1 ]    

We can, indeed, fix this problem as well by re-indenting the src code
body. But that is (1) awkward; (2) will not work if there happen to be a
language that actually uses mixed indentation - by converting tabs to
spaces we are ultimately using the original info about src body.

I feel that we will be getting various edge cases like the original
report and like the one I made up above if we keep trying to convert
tabs/spaces. Just retaining the original code indentation will be much
more robust, IMHO.

>   - =preserve-fl= is an isolated issue, and only concerns LaTeX
>     fragments. I will attach a test with the issue it solves with
>     multiline LaTeX fragments. I think LaTeX fragments are particular
>     because in the org buffer they do not need to start at the
>     beginning of a line.

> ... "- Item $abc<point>\n  efg$"

Shouldn't newlines be removed completely before editing the body here?
Just like what we do for inline src blocks. See `org-babel--normalize-body'.

> Here are three patches attached.
>   1. Two tests : about editing LaTeX fragments, and preserving empty
>      lines.

LGTM.

>   2. Renaming of =preserve-blank-line=, for clarity.

My concern is for `newline-and-indent'. Current line is _previous_ line
in such scenario, not the newly inserted line.

>   3. Two more tests testing the behaviour of =org-return= and
>      indentation, with the default configuration. When writing these, I
>      found the behaviour was buggy in one case, and modified
>      =org-indent-line= to fix it.

Let's discuss indent-tabs-mode first. I do not yet have a clear
picture about the best Org behaviour.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

Ihor Radchenko writes:
> I feel that we will be getting various edge cases like the original
> report and like the one I made up above if we keep trying to convert
> tabs/spaces. Just retaining the original code indentation will be much
> more robust, IMHO.

Python code being broken with the default configuration is
problematic, and fixing that seems worth the downsides (the
indentation in an org file will now partially depend on the
=indent-tabs-mode= settings of other modes, which cannot be set using
buffer local variables).

>>    - =preserve-fl= is an isolated issue, and only concerns LaTeX
>>      fragments. I will attach a test with the issue it solves with
>>      multiline LaTeX fragments. I think LaTeX fragments are particular
>>      because in the org buffer they do not need to start at the
>>      beginning of a line.
>> ... "- Item $abc<point>\n  efg$"
> Shouldn't newlines be removed completely before editing the body here?
> Just like what we do for inline src blocks. See `org-babel--normalize-body'.

I was not aware of how we treated inline src blocks, but I don't think
so. LaTeX fragments, in particular $$…$$ fragments, can have
significant (for the user) newlines.

>> Here are three patches attached.
>>    1. Two tests : about editing LaTeX fragments, and preserving empty
>>       lines.
> LGTM.
>
>>    2. Renaming of =preserve-blank-line=, for clarity.
> My concern is for `newline-and-indent'. Current line is _previous_  line
> in such scenario, not the newly inserted line.

The way =newline-and-indent= works, I think, is that a newline is
inserted, then =org-indent-line= is called, which preindents the line
to the common org indentation, then calls =TAB= in a native edit
buffer that does the rest of the indentation. The "current" line I
refer to in the code is the new line (the "current" line is the one
from which the native edit was called).

-- 
Sébastien Miquel

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> Ihor Radchenko writes:
>> I feel that we will be getting various edge cases like the original
>> report and like the one I made up above if we keep trying to convert
>> tabs/spaces. Just retaining the original code indentation will be much
>> more robust, IMHO.
>
> Python code being broken with the default configuration is
> problematic, and fixing that seems worth the downsides (the
> indentation in an org file will now partially depend on the
> =indent-tabs-mode= settings of other modes, which cannot be set using
> buffer local variables).

What about the following approach:

When converting from org-src buffer back to Org,

1. We do not touch the original indentation, except minimal common
   indentation of the whole src code, respecting the src mode value of
   `indent-tabs-mode'.
2. Minimal common indentation is treated according to
   `org-src-preserve-indentation'.
3. `org-src-preserve-indentation', when in effect, will add extra
   indentation of #+begin indentation + `org-src-preserve-indentation',
   now honouring `indent-tabs-mode' in Org buffer.

When converting from Org to org-src buffer,

1. When `org-src-preserve-indentation' is in effect, remove the common
   `org-src-preserve-indentation' + #+begin indentation from the body.

>>> ... "- Item $abc<point>\n  efg$"
>> Shouldn't newlines be removed completely before editing the body here?
>> Just like what we do for inline src blocks. See `org-babel--normalize-body'.
>
> I was not aware of how we treated inline src blocks, but I don't think
> so. LaTeX fragments, in particular $$…$$ fragments, can have
> significant (for the user) newlines.

May you provide an example?
AFAIK, LaTeX usually treats newlines as whitespace, same with " ".

>>>    2. Renaming of =preserve-blank-line=, for clarity.
>> My concern is for `newline-and-indent'. Current line is _previous_  line
>> in such scenario, not the newly inserted line.
>
> The way =newline-and-indent= works, I think, is that a newline is
> inserted, then =org-indent-line= is called, which preindents the line
> to the common org indentation, then calls =TAB= in a native edit
> buffer that does the rest of the indentation. The "current" line I
> refer to in the code is the new line (the "current" line is the one
> from which the native edit was called).

I think I stumbled upon related problem in my testing, but can no longer
reproduce. Your explanation indeed makes sense for my example.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

Ihor Radchenko writes:

> What about the following approach:
>
> When converting from org-src buffer back to Org,
>
> 1. We do not touch the original indentation, except minimal common
>     indentation of the whole src code, respecting the src mode value of
>     `indent-tabs-mode'.
> 2. Minimal common indentation is treated according to
>     `org-src-preserve-indentation'.
> 3. `org-src-preserve-indentation', when in effect, will add extra
>     indentation of #+begin indentation + `org-src-preserve-indentation',
>     now honouring `indent-tabs-mode' in Org buffer.
>
> When converting from Org to org-src buffer,
>
> 1. When `org-src-preserve-indentation' is in effect, remove the common
>     `org-src-preserve-indentation' + #+begin indentation from the body.

You've mixed up =org-src-preserve-indentation= and
=org-edit-src-content-indentation= so I may misunderstand. But I guess
what you propose amounts to

  1. When =org-src-preserve-indentation= is =t=, do not touch
     indentation one way or the other (same as now).
  2. Otherwise, do what we do now, but for the common indentation in
     the org buffer, use the org value of =indent-tabs-mode=, and for
     the rest of the indentation, use the native value of
     =indent-tabs-mode=. In this case, instead of trying to read this
     value, we might as well just blindly add the common indentation,
     to every non empty line.



>>>> ... "- Item $abc<point>\n  efg$"
>>> Shouldn't newlines be removed completely before editing the body here?
>>> Just like what we do for inline src blocks. See `org-babel--normalize-body'.
>>
>> I was not aware of how we treated inline src blocks, but I don't think
>> so. LaTeX fragments, in particular $$…$$ fragments, can have
>> significant (for the user) newlines.
>
> May you provide an example?
> AFAIK, LaTeX usually treats newlines as whitespace, same with " ".

When I say significant, I don't mean for compilation. When editing an
array of equations for example, one might want to keep one equation
per line in the buffer.


-- 
Sébastien Miquel

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

>> 1. When `org-src-preserve-indentation' is in effect, remove the common
>>     `org-src-preserve-indentation' + #+begin indentation from the body.
>
> You've mixed up =org-src-preserve-indentation= and
> =org-edit-src-content-indentation= so I may misunderstand.

You are right. I was referring to the value of
`org-edit-src-content-indentation' when talking about the indentation
offset.

> ...But I guess
> what you propose amounts to
> ...

You are right.

>>> I was not aware of how we treated inline src blocks, but I don't think
>>> so. LaTeX fragments, in particular $$…$$ fragments, can have
>>> significant (for the user) newlines.
>>
>> May you provide an example?
>> AFAIK, LaTeX usually treats newlines as whitespace, same with " ".
>
> When I say significant, I don't mean for compilation. When editing an
> array of equations for example, one might want to keep one equation
> per line in the buffer.

Then, may latex-fragment-specific logic be moved to
`org-edit-latex-fragment'? It already provides WRITE-BACK in
the `org-src--edit-element' call. We may as well take care about
clearing up indentation of the first line and other things there.

Side note: It looks like `org-edit-special' docstring does not mention
all the cases it considers. In particular, latex fragments are not
mentioned.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

Ihor Radchenko writes:
>> ...But I guess
>> what you propose amounts to
>> ...
> You are right.

I'll try to write a patch, this week-end.

>>>> I was not aware of how we treated inline src blocks, but I don't think
>>>> so. LaTeX fragments, in particular $$…$$ fragments, can have
>>>> significant (for the user) newlines.
>>> May you provide an example?
>>> AFAIK, LaTeX usually treats newlines as whitespace, same with " ".
>> When I say significant, I don't mean for compilation. When editing an
>> array of equations for example, one might want to keep one equation
>> per line in the buffer.
> Then, may latex-fragment-specific logic be moved to
> `org-edit-latex-fragment'? It already provides WRITE-BACK in
> the `org-src--edit-element' call. We may as well take care about
> clearing up indentation of the first line and other things there.

Looks like it could, yes : call the WRITE-BACK after indentation. This
doesn't seem to conflict for other org elements.

-- 
Sébastien Miquel

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> Side note: It looks like `org-edit-special' docstring does not mention
> all the cases it considers. In particular, latex fragments are not
> mentioned.

Fixed on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f8b0b2bab

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

Hi Ihor,

Ihor Radchenko writes:
>> ...But I guess
>> what you propose amounts to
>> ...
> You are right.

On second thought, I don't think that's a good idea. I did not
understand how tab characters worked: they do not have a fixed width,
but align to the next tab column. This means that if we add a couple
of spaces in front of a tab indented piece of code, we can break
vertical alignment in the org buffer, which I think is pretty bad.

Should we use native buffer's value of =indent-tabs-mode= to set
=use-tabs?= ? I think this trivial change should work.

-- 
Sébastien Miquel

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

Sébastien Miquel writes:
> Should we use native buffer's value of =indent-tabs-mode= to set
> =use-tabs?= ? I think this trivial change should work.

It doesn't seem quite that easy. If we want to add 4 columns to a tab
indented line (and tab width is 8), we can either call =indent-to= to
indent with a tab and 4 spaces, or add the spaces at the beginning.
With the second option, we risk breaking vertical alignement in the
org buffer. With the first option, on a subsequent edit, the current
=org-do-remove-indentation= will break the tab character into 4
spaces, making the indentation 8 spaces instead of a tab. You need to
have =org-do-remove-indentation= re-indent the whole line, with the
correct value of =indent-tabs-mode=.

-- 
Sébastien Miquel

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> On second thought, I don't think that's a good idea. I did not
> understand how tab characters worked: they do not have a fixed width,
> but align to the next tab column. This means that if we add a couple
> of spaces in front of a tab indented piece of code, we can break
> vertical alignment in the org buffer, which I think is pretty bad.

May you please provide an example for such breakage?
From my point of view, alignment is far lesser concern compared to
indentation breaking code execution/tangling/other functional parts of Org.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

Ihor Radchenko writes:
> May you please provide an example for such breakage?
>  From my point of view, alignment is far lesser concern compared to
> indentation breaking code execution/tangling/other functional parts of Org.

Let's say tab-width is 8 and elisp-mode uses tabs for indentation. In
the following snippet, arg2 should be indented with a tab.

#+BEGIN_SRC emacs-lisp
(some-f arg1
         arg2) ;; arg2 is indented at column 8
#+END_SRC

If I add two spaces at the beginning of the block. It should visually
look like this:

#+BEGIN_SRC emacs-lisp
   (some-f arg1
         arg2) ;; arg2 is indented at column 8
#+END_SRC

Calling =indent-to= would rather put the two spaces after the tab
character. But then we lose this idea of clean separation between the
org indentation and the native indentation, and we will need to
convert the indent of every line on subsequent edits (org -> elisp),
as well as for tangling, or code execution. If we have to do this, we
might as well just respect the org buffer =indent-tabs-mode=, and redo
every indentation with each conversion. This could possibly be costly
when tangling a large buffer.

The visual breakage would be much more common than those more serious
issues, which had gone unnoticed so far.

-- 
Sébastien Miquel

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> Ihor Radchenko writes:
>> May you please provide an example for such breakage?
>>  From my point of view, alignment is far lesser concern compared to
>> indentation breaking code execution/tangling/other functional parts of Org.
>
> Let's say tab-width is 8 and elisp-mode uses tabs for indentation. In
> the following snippet, arg2 should be indented with a tab.
>
> #+BEGIN_SRC emacs-lisp
> (some-f arg1
>          arg2) ;; arg2 is indented at column 8
> #+END_SRC
>
> If I add two spaces at the beginning of the block. It should visually
> look like this:
>
> #+BEGIN_SRC emacs-lisp
>    (some-f arg1
>          arg2) ;; arg2 is indented at column 8
> #+END_SRC

This is not a problem. We can just apply appropriate 'display property
in `org-src-font-lock-fontify-block', manually replacing the tab with
appropriate number of spaces (as in the origin buffer).

> Calling =indent-to= would rather put the two spaces after the tab
> character.

Then, we should not use `indent-to' and instead put spaces manually.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 601 bytes --]


Ihor Radchenko writes:
> This is not a problem. We can just apply appropriate 'display property
> in `org-src-font-lock-fontify-block', manually replacing the tab with
> appropriate number of spaces (as in the origin buffer).

Ok, that works, thanks.

Here are two patches, the first that removes the special case logic
for LaTeX fragments, and the second that implements what you suggest,
and applies on top of the first one. Does that look ok to you ?

NB: Calling newline-and-indent from the middle of a line is currently
broken, but that has nothing to do with this change.

-- 
Sébastien Miquel

[-- Attachment #2: 0001-org-src-contents-for-write-back-simplify-the-case-of.patch --]
[-- Type: text/x-patch, Size: 3051 bytes --]

From d2a86d9011455172e5990149f844031f534e65f2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Sun, 25 Jun 2023 13:38:21 +0200
Subject: [PATCH] org-src--contents-for-write-back: simplify the case of LaTeX
 fragments

* lisp/org-src.el (org-src--contents-for-write-back): Extract special
case logic for LaTeX fragments.
(org-edit-latex-fragment): Trim the contents to be inserted in the
original buffer.
---
 lisp/org-src.el | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index f15ba8e99..5c272c7f5 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -466,7 +466,6 @@ Assume point is in the corresponding edit buffer."
 		  org-src--content-indentation
 		0))))
 	(use-tabs? (and (> org-src--tab-width 0) t))
-        (preserve-fl (eq org-src--source-type 'latex-fragment))
 	(source-tab-width org-src--tab-width)
 	(contents (org-with-wide-buffer
                    (let ((eol (line-end-position)))
@@ -479,16 +478,13 @@ Assume point is in the corresponding edit buffer."
       ;; Reproduce indentation parameters from source buffer.
       (setq indent-tabs-mode use-tabs?)
       (when (> source-tab-width 0) (setq tab-width source-tab-width))
-      ;; Apply WRITE-BACK function on edit buffer contents.
       (insert (org-no-properties (car contents)))
       (setq marker (point-marker))
       (insert (org-no-properties (car (cdr contents))))
       (goto-char (point-min))
-      (when (functionp write-back) (save-excursion (funcall write-back)))
       ;; Add INDENTATION-OFFSET to every line in buffer,
       ;; unless indentation is meant to be preserved.
       (when (> indentation-offset 0)
-	(when preserve-fl (forward-line))
         (while (not (eobp))
 	  (skip-chars-forward " \t")
           (when (or (not (eolp))                               ; not a blank line
@@ -498,6 +494,9 @@ Assume point is in the corresponding edit buffer."
 	      (delete-region (line-beginning-position) (point))
 	      (indent-to (+ i indentation-offset))))
 	  (forward-line)))
+      ;; Apply WRITE-BACK function on edit buffer contents.
+      (goto-char (point-min))
+      (when (functionp write-back) (save-excursion (funcall write-back)))
       (set-marker marker nil))))
 
 (defun org-src--edit-element
@@ -1150,7 +1149,14 @@ Throw an error when not at such a table."
        (lambda ()
 	 ;; Blank lines break things, replace with a single newline.
 	 (while (re-search-forward "\n[ \t]*\n" nil t) (replace-match "\n"))
-	 ;; If within a table a newline would disrupt the structure,
+         ;; Trim contents.
+	 (goto-char (point-min))
+         (skip-chars-forward " \t")
+	 (delete-region (point-min) (point))
+         (goto-char (point-max))
+	 (skip-chars-backward " \t")
+	 (delete-region (point) (point-max))
+         ;; If within a table a newline would disrupt the structure,
 	 ;; so remove newlines.
 	 (goto-char (point-min))
 	 (when (org-element-lineage context '(table-cell))
-- 
2.41.0


[-- Attachment #3: 0001-org-src.el-Use-native-value-of-indent-tabs-mode-for-.patch --]
[-- Type: text/x-patch, Size: 8665 bytes --]

From ab48e9671efdaba6566f406b1df81a441c72252c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Tue, 27 Jun 2023 09:23:01 +0200
Subject: [PATCH] org-src.el: Use native value of `indent-tabs-mode' for
 indentation

* lisp/org-macs.el (org-do-remove-indentation): Preserve
indentation (spaces vs tabs) past the common indentation to remove.
* lisp/org-src.el (org-src--contents-for-write-back): Preserve the
native indentation (spaces vs tabs).  If necessary, add a common org
indentation to the block according to org's `indent-tabs-mode'.
(org-src-font-lock-fontify-block): In case of mixed indentation,
display the tab characters with a fixed width, according to the native
tab width value.
* testing/lisp/test-org-src.el (test-org-src/indented-blocks): Update
tests.  Indentation no longer obeys `indent-tabs-mode' from the org
buffer, but is separated in two parts.
---
 lisp/org-macs.el             |  4 ++-
 lisp/org-src.el              | 36 ++++++++++++++-------
 testing/lisp/test-org-src.el | 62 +++++++++++++++++++-----------------
 3 files changed, 60 insertions(+), 42 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 51dbfe118..f42e6b14b 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -485,7 +485,9 @@ line.  Return nil if it fails."
 	  (let ((ind (progn (skip-chars-forward " \t") (current-column))))
 	    (cond ((eolp) (delete-region (line-beginning-position) (point)))
 		  ((< ind n) (throw :exit nil))
-		  (t (indent-line-to (- ind n))))
+		  (t (delete-region (line-beginning-position)
+                                    (progn (move-to-column n t)
+                                           (point)))))
 	    (forward-line)))
 	;; Signal success.
 	t))))
diff --git a/lisp/org-src.el b/lisp/org-src.el
index 5c272c7f5..5a1030c42 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -473,11 +473,15 @@ Assume point is in the corresponding edit buffer."
                            (buffer-substring eol (point-max))))))
 	(write-back org-src--allow-write-back)
         (preserve-blank-line org-src--preserve-blank-line)
-        marker)
+        marker indent-str)
+    (setq indent-str
+          (with-temp-buffer
+            ;; Reproduce indentation parameters from org buffer.
+            (setq indent-tabs-mode use-tabs?)
+            (when (> source-tab-width 0) (setq tab-width source-tab-width))
+            (indent-to indentation-offset)
+            (buffer-string)))
     (with-current-buffer write-back-buf
-      ;; Reproduce indentation parameters from source buffer.
-      (setq indent-tabs-mode use-tabs?)
-      (when (> source-tab-width 0) (setq tab-width source-tab-width))
       (insert (org-no-properties (car contents)))
       (setq marker (point-marker))
       (insert (org-no-properties (car (cdr contents))))
@@ -486,13 +490,12 @@ Assume point is in the corresponding edit buffer."
       ;; unless indentation is meant to be preserved.
       (when (> indentation-offset 0)
         (while (not (eobp))
-	  (skip-chars-forward " \t")
-          (when (or (not (eolp))                               ; not a blank line
-                    (and (eq (point) (marker-position marker)) ; current line
+          (when (or (not (eolp)) ; not an empty line
+                    ;; If the current line is empty, we may
+                    ;; want to indent it.
+                    (and (eq (point) (marker-position marker))
                          preserve-blank-line))
-	    (let ((i (current-column)))
-	      (delete-region (line-beginning-position) (point))
-	      (indent-to (+ i indentation-offset))))
+            (insert indent-str))
 	  (forward-line)))
       ;; Apply WRITE-BACK function on edit buffer contents.
       (goto-char (point-min))
@@ -636,7 +639,8 @@ Leave point in edit buffer."
   "Fontify code block between START and END using LANG's syntax.
 This function is called by Emacs' automatic fontification, as long
 as `org-src-fontify-natively' is non-nil."
-  (let ((modified (buffer-modified-p)))
+  (let ((modified (buffer-modified-p))
+        native-tab-width)
     (remove-text-properties start end '(face nil))
     (let ((lang-mode (org-src-get-lang-mode lang)))
       (when (fboundp lang-mode)
@@ -650,6 +654,7 @@ as `org-src-fontify-natively' is non-nil."
 	      ;; Add string and a final space to ensure property change.
 	      (insert string " "))
 	    (unless (eq major-mode lang-mode) (funcall lang-mode))
+            (setq native-tab-width tab-width)
             (font-lock-ensure)
 	    (let ((pos (point-min)) next)
 	      (while (setq next (next-property-change pos))
@@ -707,6 +712,15 @@ as `org-src-fontify-natively' is non-nil."
       (when (or (facep src-face) (listp src-face))
         (font-lock-append-text-property start end 'face src-face))
       (font-lock-append-text-property start end 'face 'org-block))
+    ;; Display tab indentation characters preceded by spaces as spaces
+    (unless org-src-preserve-indentation
+      (save-excursion
+        (goto-char start)
+        (while (re-search-forward "^[ ]+\\([\t]+\\)" end t)
+          (let* ((b (match-beginning 1))
+                 (e (match-end 1))
+                 (s (make-string (* (- e b) native-tab-width) ? )))
+            (add-text-properties b e `(display ,s))))))
     ;; Clear abbreviated link folding.
     (org-fold-region start end nil 'org-link)
     (add-text-properties
diff --git a/testing/lisp/test-org-src.el b/testing/lisp/test-org-src.el
index 2a45ba66e..11d56209d 100644
--- a/testing/lisp/test-org-src.el
+++ b/testing/lisp/test-org-src.el
@@ -304,11 +304,11 @@ This is a tab:\t.
 	(insert " Foo")
 	(org-edit-src-exit)
 	(buffer-string)))))
-  ;; Global indentation obeys `indent-tabs-mode' from the original
-  ;; buffer.
-  (should
+  ;; Global indentation does not obey `indent-tabs-mode' from the
+  ;; original buffer.
+  (should-not
    (string-match-p
-    "^\t+\s*argument2"
+    "\t"
     (org-test-with-temp-text
 	"
 - Item
@@ -318,19 +318,21 @@ This is a tab:\t.
               argument2))
   #+END_SRC"
       (setq-local indent-tabs-mode t)
-      (let ((org-edit-src-content-indentation 2)
+      (let ((tab-width 8)
+            (org-edit-src-content-indentation 2)
 	    (org-src-preserve-indentation nil))
 	(org-edit-special)
 	(org-edit-src-exit)
 	(buffer-string)))))
+  ;; Tab character is preserved
   (should
    (string-match-p
-    "^\s+argument2"
+    "\targument2"
     (org-test-with-temp-text
 	"
 - Item
   #+BEGIN_SRC emacs-lisp<point>
-    (progn\n      (function argument1\n\t\targument2))
+    (progn\n      (function argument1\n    \targument2))
   #+END_SRC"
       (setq-local indent-tabs-mode nil)
       (let ((org-edit-src-content-indentation 2)
@@ -338,43 +340,43 @@ This is a tab:\t.
 	(org-edit-special)
 	(org-edit-src-exit)
 	(buffer-string)))))
-  ;; Global indentation also obeys `tab-width' from original buffer.
+  ;; Indentation does not obey `tab-width' from org buffer.
   (should
    (string-match-p
-    "^\t\\{3\\}\s\\{2\\}argument2"
+    "^  \targument2"
     (org-test-with-temp-text
 	"
-- Item
-  #+BEGIN_SRC emacs-lisp<point>
+#+BEGIN_SRC emacs-lisp
   (progn
-    (function argument1
-              argument2))
-  #+END_SRC"
+    (list argument1\n  \t<point>argument2))
+#+END_SRC"
       (setq-local indent-tabs-mode t)
       (setq-local tab-width 4)
-      (let ((org-edit-src-content-indentation 0)
+      (let ((org-edit-src-content-indentation 2)
 	    (org-src-preserve-indentation nil))
 	(org-edit-special)
+        (setq-local indent-tabs-mode t)
+        (setq-local tab-width 8)
+        (lisp-indent-line)
 	(org-edit-src-exit)
 	(buffer-string)))))
+  ;; Tab characters are displayed with `tab-width' from the native
+  ;; edit buffer.
   (should
-   (string-match-p
-    "^\t\s\\{6\\}argument2"
+   (equal
+    10
     (org-test-with-temp-text
-	"
-- Item
-  #+BEGIN_SRC emacs-lisp<point>
+     "
+#+BEGIN_SRC emacs-lisp
   (progn
-    (function argument1
-              argument2))
-  #+END_SRC"
-      (setq-local indent-tabs-mode t)
-      (setq-local tab-width 8)
-      (let ((org-edit-src-content-indentation 0)
-	    (org-src-preserve-indentation nil))
-	(org-edit-special)
-	(org-edit-src-exit)
-	(buffer-string))))))
+    (list argument1\n  \t<point>argument2))
+#+END_SRC"
+     (setq-local indent-tabs-mode t)
+     (setq-local tab-width 4)
+     (let ((org-edit-src-content-indentation 2)
+	   (org-src-preserve-indentation nil))
+       (font-lock-ensure)
+       (current-column))))))
 
 (ert-deftest test-org-src/footnote-references ()
   "Test editing footnote references."
-- 
2.41.0

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> Here are two patches, the first that removes the special case logic
> for LaTeX fragments, and the second that implements what you suggest,
> and applies on top of the first one. Does that look ok to you ?

Thanks!

> +      ;; Apply WRITE-BACK function on edit buffer contents.
> +      (goto-char (point-min))
> +      (when (functionp write-back) (save-excursion (funcall write-back)))
>        (set-marker marker nil))))

`save-excursion' is no longer necessary here.

>  (defun org-src--edit-element
> @@ -1150,7 +1149,14 @@ Throw an error when not at such a table."
>         (lambda ()
>  	 ;; Blank lines break things, replace with a single newline.
>  	 (while (re-search-forward "\n[ \t]*\n" nil t) (replace-match "\n"))
> -	 ;; If within a table a newline would disrupt the structure,
> +         ;; Trim contents.

It would be nice to have a bit more context about the purpose in the
comment here.

> Subject: [PATCH] org-src.el: Use native value of `indent-tabs-mode' for
>  indentation
>
> * lisp/org-src.el (org-src--contents-for-write-back): Preserve the
> native indentation (spaces vs tabs).  If necessary, add a common org
> indentation to the block according to org's `indent-tabs-mode'.
> (org-src-font-lock-fontify-block): In case of mixed indentation,
> display the tab characters with a fixed width, according to the native
> tab width value.
> * testing/lisp/test-org-src.el (test-org-src/indented-blocks): Update
> tests.  Indentation no longer obeys `indent-tabs-mode' from the org
> buffer, but is separated in two parts.

> diff --git a/lisp/org-src.el b/lisp/org-src.el
> index 5c272c7f5..5a1030c42 100644
> --- a/lisp/org-src.el
> +++ b/lisp/org-src.el
>  ...
> -	  (skip-chars-forward " \t")
> -          (when (or (not (eolp))                               ; not a blank line
> -                    (and (eq (point) (marker-position marker)) ; current line
> +          (when (or (not (eolp)) ; not an empty line
> +                    ;; If the current line is empty, we may
> +                    ;; want to indent it.
> +                    (and (eq (point) (marker-position marker))
>                           preserve-blank-line))

Do we still need this dance with special case for current line?

> +    ;; Display tab indentation characters preceded by spaces as spaces
> +    (unless org-src-preserve-indentation

unless? Don't we rather want to preserve the original indentation
alignment when `org-src-preserve-indentation' is t?

> +      (save-excursion
> +        (goto-char start)
> +        (while (re-search-forward "^[ ]+\\([\t]+\\)" end t)

Why just tabs at indentation? It would make sense to preserve width of
all the tabs.

> +          (let* ((b (match-beginning 1))
> +                 (e (match-end 1))
> +                 (s (make-string (* (- e b) native-tab-width) ? )))
> +            (add-text-properties b e `(display ,s))))))

Will the actual tab width be always equal to native-tab-width? What
about tab stops? May it be more reliable to use different of column
numbers in the src mode buffer after/before the tab?

> @@ -318,19 +318,21 @@ This is a tab:\t.
>                argument2))
>    #+END_SRC"
>        (setq-local indent-tabs-mode t)
> -      (let ((org-edit-src-content-indentation 2)
> +      (let ((tab-width 8)
> +            (org-edit-src-content-indentation 2)

Why is setting tab-width necessary? 8 is the default value.

>    #+BEGIN_SRC emacs-lisp<point>
> -    (progn\n      (function argument1\n\t\targument2))
> +    (progn\n      (function argument1\n    \targument2))

I think it would be a bit more readable to convert this string into
actual multi-line, where the alignment is visible when reading the test
source.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 4476 bytes --]

Hi Ihor,

Thank you for the feedback.

Ihor Radchenko writes:
>> +      ;; Apply WRITE-BACK function on edit buffer contents.
>> +      (goto-char (point-min))
>> +      (when (functionp write-back) (save-excursion (funcall write-back)))
>>         (set-marker marker nil))))
> `save-excursion' is no longer necessary here.

Done.

>>   (defun org-src--edit-element
>> @@ -1150,7 +1149,14 @@ Throw an error when not at such a table."
>>          (lambda ()
>>   	 ;; Blank lines break things, replace with a single newline.
>>   	 (while (re-search-forward "\n[ \t]*\n" nil t) (replace-match "\n"))
>> -	 ;; If within a table a newline would disrupt the structure,
>> +         ;; Trim contents.
> It would be nice to have a bit more context about the purpose in the
> comment here.

I was confused. We need only trim the beginning of the result, for the
indentation added by `org-src--contents-for-write-back'. I've added an
explanation.

>>   ...
>> -	  (skip-chars-forward " \t")
>> -          (when (or (not (eolp))                               ; not a blank line
>> -                    (and (eq (point) (marker-position marker)) ; current line
>> +          (when (or (not (eolp)) ; not an empty line
>> +                    ;; If the current line is empty, we may
>> +                    ;; want to indent it.
>> +                    (and (eq (point) (marker-position marker))
>>                            preserve-blank-line))
> Do we still need this dance with special case for current line?

Yes. This is fragile, but what it does is, if the line from which the
original edit was called was blank and not empty, and the line from
which the edit is ended is empty, we assume these two lines are the
same, and we add the common block indentation to this empty line.
This, and the pre-indentation in =org-indent-line=, make `TAB` work
to indent an empty line.

The logic in =org-edit-element= can now be simplified though, see the
third patch attached.

>> +    ;; Display tab indentation characters preceded by spaces as spaces
>> +    (unless org-src-preserve-indentation
> unless? Don't we rather want to preserve the original indentation
> alignment when `org-src-preserve-indentation' is t?
>> +      (save-excursion
>> +        (goto-char start)
>> +        (while (re-search-forward "^[ ]+\\([\t]+\\)" end t)
> Why just tabs at indentation? It would make sense to preserve width of
> all the tabs.
>> +          (let* ((b (match-beginning 1))
>> +                 (e (match-end 1))
>> +                 (s (make-string (* (- e b) native-tab-width) ? )))
>> +            (add-text-properties b e `(display ,s))))))
> Will the actual tab width be always equal to native-tab-width? What
> about tab stops? May it be more reliable to use different of column
> numbers in the src mode buffer after/before the tab?

I was trying to be conservative. When =org-src-preserve-indentation=
is t, I guess we can do the same, for consistency.

As you say, the tabs at the beginning of indentation are the only ones
we can be somewhat sure of the width, if we assume the native
indentation was done sanely (using tabs then spaces). I'm inclined to
only deal with those.

To get the true tab widths, we'd need to get the columns numbers in
yet a third different buffer, with the common indentation removed. I
don't think that's worth it.

Anyway, what I wrote doesn't deal correctly with the case where the
org indentation may use tabs. I've updated the patch: I use the value
of what should be the org indentation, and only change the display of
the following consecutive tabs.

>> @@ -318,19 +318,21 @@ This is a tab:\t.
>>                 argument2))
>>     #+END_SRC"
>>         (setq-local indent-tabs-mode t)
>> -      (let ((org-edit-src-content-indentation 2)
>> +      (let ((tab-width 8)
>> +            (org-edit-src-content-indentation 2)
> Why is setting tab-width necessary? 8 is the default value.

Removed, though I have left an instance of =(setq-local tab-width 8)=,
for clarity.

>>     #+BEGIN_SRC emacs-lisp<point>
>> -    (progn\n      (function argument1\n\t\targument2))
>> +    (progn\n      (function argument1\n    \targument2))
> I think it would be a bit more readable to convert this string into
> actual multi-line, where the alignment is visible when reading the test
> source.

I've left it this way. I think the tests using tab characters are
often written this way, so as to be understandable without whitespace
mode.

-- 
Sébastien Miquel

[-- Attachment #2: 0001-org-src-contents-for-write-back-simplify-the-case-of.patch --]
[-- Type: text/x-patch, Size: 3037 bytes --]

From d024fe96ad889097d025a87dae3316acc44299f6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Sun, 25 Jun 2023 13:38:21 +0200
Subject: [PATCH] org-src--contents-for-write-back: simplify the case of LaTeX
 fragments

* lisp/org-src.el (org-src--contents-for-write-back): Extract special
case logic for LaTeX fragments.
(org-edit-latex-fragment): Trim the contents to be inserted in the
original buffer.
---
 lisp/org-src.el | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index f15ba8e99..121e59241 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -466,7 +466,6 @@ Assume point is in the corresponding edit buffer."
 		  org-src--content-indentation
 		0))))
 	(use-tabs? (and (> org-src--tab-width 0) t))
-        (preserve-fl (eq org-src--source-type 'latex-fragment))
 	(source-tab-width org-src--tab-width)
 	(contents (org-with-wide-buffer
                    (let ((eol (line-end-position)))
@@ -479,16 +478,13 @@ Assume point is in the corresponding edit buffer."
       ;; Reproduce indentation parameters from source buffer.
       (setq indent-tabs-mode use-tabs?)
       (when (> source-tab-width 0) (setq tab-width source-tab-width))
-      ;; Apply WRITE-BACK function on edit buffer contents.
       (insert (org-no-properties (car contents)))
       (setq marker (point-marker))
       (insert (org-no-properties (car (cdr contents))))
       (goto-char (point-min))
-      (when (functionp write-back) (save-excursion (funcall write-back)))
       ;; Add INDENTATION-OFFSET to every line in buffer,
       ;; unless indentation is meant to be preserved.
       (when (> indentation-offset 0)
-	(when preserve-fl (forward-line))
         (while (not (eobp))
 	  (skip-chars-forward " \t")
           (when (or (not (eolp))                               ; not a blank line
@@ -498,6 +494,9 @@ Assume point is in the corresponding edit buffer."
 	      (delete-region (line-beginning-position) (point))
 	      (indent-to (+ i indentation-offset))))
 	  (forward-line)))
+      ;; Apply WRITE-BACK function on edit buffer contents.
+      (goto-char (point-min))
+      (when (functionp write-back) (funcall write-back))
       (set-marker marker nil))))
 
 (defun org-src--edit-element
@@ -1150,7 +1149,12 @@ Throw an error when not at such a table."
        (lambda ()
 	 ;; Blank lines break things, replace with a single newline.
 	 (while (re-search-forward "\n[ \t]*\n" nil t) (replace-match "\n"))
-	 ;; If within a table a newline would disrupt the structure,
+         ;; Trim contents: `org-src--contents-for-write-back' may have
+         ;; added indentation at the beginning, which we remove.
+	 (goto-char (point-min))
+         (skip-chars-forward " \t")
+	 (delete-region (point-min) (point))
+         ;; If within a table a newline would disrupt the structure,
 	 ;; so remove newlines.
 	 (goto-char (point-min))
 	 (when (org-element-lineage context '(table-cell))
-- 
2.41.0


[-- Attachment #3: 0001-org-src.el-Use-native-value-of-indent-tabs-mode-for-.patch --]
[-- Type: text/x-patch, Size: 9141 bytes --]

From f6ed74b536723e2c888b51f8d0ecc84e104411fc Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Tue, 27 Jun 2023 09:23:01 +0200
Subject: [PATCH] org-src.el: Use native value of `indent-tabs-mode' for
 indentation

* lisp/org-macs.el (org-do-remove-indentation): Preserve
indentation (spaces vs tabs) past the common indentation to remove.
* lisp/org-src.el (org-src--contents-for-write-back): Preserve the
native indentation (spaces vs tabs).  If necessary, add a common org
indentation to the block according to org's `indent-tabs-mode'.
(org-src-font-lock-fontify-block): In case of mixed indentation,
display the tab characters with a fixed width, according to the native
tab width value.
* testing/lisp/test-org-src.el (test-org-src/indented-blocks): Update
tests.  Indentation no longer obeys `indent-tabs-mode' from the org
buffer, but is separated in two parts.
---
 lisp/org-macs.el             |  4 +-
 lisp/org-src.el              | 41 ++++++++++++++------
 testing/lisp/test-org-src.el | 75 ++++++++++++++++++++++--------------
 3 files changed, 79 insertions(+), 41 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 51dbfe118..f42e6b14b 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -485,7 +485,9 @@ line.  Return nil if it fails."
 	  (let ((ind (progn (skip-chars-forward " \t") (current-column))))
 	    (cond ((eolp) (delete-region (line-beginning-position) (point)))
 		  ((< ind n) (throw :exit nil))
-		  (t (indent-line-to (- ind n))))
+		  (t (delete-region (line-beginning-position)
+                                    (progn (move-to-column n t)
+                                           (point)))))
 	    (forward-line)))
 	;; Signal success.
 	t))))
diff --git a/lisp/org-src.el b/lisp/org-src.el
index 121e59241..a8f076b21 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -473,11 +473,15 @@ Assume point is in the corresponding edit buffer."
                            (buffer-substring eol (point-max))))))
 	(write-back org-src--allow-write-back)
         (preserve-blank-line org-src--preserve-blank-line)
-        marker)
+        marker indent-str)
+    (setq indent-str
+          (with-temp-buffer
+            ;; Reproduce indentation parameters from org buffer.
+            (setq indent-tabs-mode use-tabs?)
+            (when (> source-tab-width 0) (setq tab-width source-tab-width))
+            (indent-to indentation-offset)
+            (buffer-string)))
     (with-current-buffer write-back-buf
-      ;; Reproduce indentation parameters from source buffer.
-      (setq indent-tabs-mode use-tabs?)
-      (when (> source-tab-width 0) (setq tab-width source-tab-width))
       (insert (org-no-properties (car contents)))
       (setq marker (point-marker))
       (insert (org-no-properties (car (cdr contents))))
@@ -486,13 +490,12 @@ Assume point is in the corresponding edit buffer."
       ;; unless indentation is meant to be preserved.
       (when (> indentation-offset 0)
         (while (not (eobp))
-	  (skip-chars-forward " \t")
-          (when (or (not (eolp))                               ; not a blank line
-                    (and (eq (point) (marker-position marker)) ; current line
+          (when (or (not (eolp)) ; not an empty line
+                    ;; If the current line is empty, we may
+                    ;; want to indent it.
+                    (and (eq (point) (marker-position marker))
                          preserve-blank-line))
-	    (let ((i (current-column)))
-	      (delete-region (line-beginning-position) (point))
-	      (indent-to (+ i indentation-offset))))
+            (insert indent-str))
 	  (forward-line)))
       ;; Apply WRITE-BACK function on edit buffer contents.
       (goto-char (point-min))
@@ -636,7 +639,7 @@ Leave point in edit buffer."
   "Fontify code block between START and END using LANG's syntax.
 This function is called by Emacs' automatic fontification, as long
 as `org-src-fontify-natively' is non-nil."
-  (let ((modified (buffer-modified-p)))
+  (let ((modified (buffer-modified-p)) native-tab-width)
     (remove-text-properties start end '(face nil))
     (let ((lang-mode (org-src-get-lang-mode lang)))
       (when (fboundp lang-mode)
@@ -650,6 +653,7 @@ as `org-src-fontify-natively' is non-nil."
 	      ;; Add string and a final space to ensure property change.
 	      (insert string " "))
 	    (unless (eq major-mode lang-mode) (funcall lang-mode))
+            (setq native-tab-width tab-width)
             (font-lock-ensure)
 	    (let ((pos (point-min)) next)
 	      (while (setq next (next-property-change pos))
@@ -707,6 +711,21 @@ as `org-src-fontify-natively' is non-nil."
       (when (or (facep src-face) (listp src-face))
         (font-lock-append-text-property start end 'face src-face))
       (font-lock-append-text-property start end 'face 'org-block))
+    ;; Display native tab indentation characters as spaces
+    (save-excursion
+      (goto-char start)
+      (let ((indent-offset
+	     (if org-src-preserve-indentation 0
+	       (+ (progn (backward-char)
+                         (org-current-text-indentation))
+	          org-edit-src-content-indentation))))
+        (while (re-search-forward "^[ ]*\t" end t)
+          (let* ((b (and (eq indent-offset (move-to-column indent-offset))
+                         (point)))
+                 (e (progn (skip-chars-forward "\t") (point)))
+                 (s (and b (make-string (* (- e b) native-tab-width) ? ))))
+            (when (and b (< b e)) (add-text-properties b e `(display ,s)))
+            (forward-char)))))
     ;; Clear abbreviated link folding.
     (org-fold-region start end nil 'org-link)
     (add-text-properties
diff --git a/testing/lisp/test-org-src.el b/testing/lisp/test-org-src.el
index 2a45ba66e..c4309ccfc 100644
--- a/testing/lisp/test-org-src.el
+++ b/testing/lisp/test-org-src.el
@@ -304,11 +304,11 @@ This is a tab:\t.
 	(insert " Foo")
 	(org-edit-src-exit)
 	(buffer-string)))))
-  ;; Global indentation obeys `indent-tabs-mode' from the original
-  ;; buffer.
-  (should
+  ;; Global indentation does not obey `indent-tabs-mode' from the
+  ;; original buffer.
+  (should-not
    (string-match-p
-    "^\t+\s*argument2"
+    "\t"
     (org-test-with-temp-text
 	"
 - Item
@@ -323,14 +323,15 @@ This is a tab:\t.
 	(org-edit-special)
 	(org-edit-src-exit)
 	(buffer-string)))))
+  ;; Tab character is preserved
   (should
    (string-match-p
-    "^\s+argument2"
+    "\targument2"
     (org-test-with-temp-text
 	"
 - Item
   #+BEGIN_SRC emacs-lisp<point>
-    (progn\n      (function argument1\n\t\targument2))
+    (progn\n      (function argument1\n    \targument2))
   #+END_SRC"
       (setq-local indent-tabs-mode nil)
       (let ((org-edit-src-content-indentation 2)
@@ -338,43 +339,59 @@ This is a tab:\t.
 	(org-edit-special)
 	(org-edit-src-exit)
 	(buffer-string)))))
-  ;; Global indentation also obeys `tab-width' from original buffer.
+  ;; Indentation does not obey `tab-width' from org buffer.
   (should
    (string-match-p
-    "^\t\\{3\\}\s\\{2\\}argument2"
+    "^  \targument2"
     (org-test-with-temp-text
 	"
-- Item
-  #+BEGIN_SRC emacs-lisp<point>
+#+BEGIN_SRC emacs-lisp
   (progn
-    (function argument1
-              argument2))
-  #+END_SRC"
+    (list argument1\n  \t<point>argument2))
+#+END_SRC"
       (setq-local indent-tabs-mode t)
       (setq-local tab-width 4)
-      (let ((org-edit-src-content-indentation 0)
+      (let ((org-edit-src-content-indentation 2)
 	    (org-src-preserve-indentation nil))
 	(org-edit-special)
+        (setq-local indent-tabs-mode t)
+        (setq-local tab-width 8)
+        (lisp-indent-line)
 	(org-edit-src-exit)
 	(buffer-string)))))
+  ;; Tab characters are displayed with `tab-width' from the native
+  ;; edit buffer.
   (should
-   (string-match-p
-    "^\t\s\\{6\\}argument2"
+   (equal
+    10
     (org-test-with-temp-text
-	"
-- Item
-  #+BEGIN_SRC emacs-lisp<point>
+     "
+#+BEGIN_SRC emacs-lisp
   (progn
-    (function argument1
-              argument2))
-  #+END_SRC"
-      (setq-local indent-tabs-mode t)
-      (setq-local tab-width 8)
-      (let ((org-edit-src-content-indentation 0)
-	    (org-src-preserve-indentation nil))
-	(org-edit-special)
-	(org-edit-src-exit)
-	(buffer-string))))))
+    (list argument1\n  \t<point>argument2))
+#+END_SRC"
+     (setq-local indent-tabs-mode t)
+     (setq-local tab-width 4)
+     (let ((org-edit-src-content-indentation 2)
+	   (org-src-preserve-indentation nil))
+       (font-lock-ensure)
+       (current-column)))))
+  ;; The initial tab characters respect org's `tab-width'.
+  (should
+   (equal
+    10
+    (org-test-with-temp-text
+     "
+#+BEGIN_SRC emacs-lisp
+\t(progn
+\t  (list argument1\n\t\t<point>argument2))
+#+END_SRC"
+     (setq-local indent-tabs-mode t)
+     (setq-local tab-width 2)
+     (let ((org-edit-src-content-indentation 2)
+	   (org-src-preserve-indentation nil))
+       (font-lock-ensure)
+       (current-column))))))
 
 (ert-deftest test-org-src/footnote-references ()
   "Test editing footnote references."
-- 
2.41.0


[-- Attachment #4: 0001-org-src.el-Rename-internal-variable-for-clarity.patch --]
[-- Type: text/x-patch, Size: 3131 bytes --]

From 71e7a746c7d39b88038060c5b035f1c9e6f35b13 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Thu, 29 Jun 2023 14:37:09 +0200
Subject: [PATCH] org-src.el: Rename internal variable for clarity

* lisp/org-src.el (org-src--contents-for-write-back):
(org-src--edit-element): Rename `preserve-blank-line' to
`indent-current-empty-line'.  It is used to decide whether we should
indent the current empty line after a special edit.
---
 lisp/org-src.el | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index ac201da44..13b7f7e20 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -318,8 +318,8 @@ is 0.")
   "File name associated to Org source buffer, or nil.")
 (put 'org-src-source-file-name 'permanent-local t)
 
-(defvar-local org-src--preserve-blank-line nil)
-(put 'org-src--preserve-blank-line 'permanent-local t)
+(defvar-local org-src--indent-current-empty-line nil)
+(put 'org-src--indent-current-empty-line 'permanent-local t)
 
 (defun org-src--construct-edit-buffer-name (org-buffer-name lang)
   "Construct the buffer name for a source editing buffer.
@@ -472,7 +472,7 @@ Assume point is in the corresponding edit buffer."
                      (list (buffer-substring (point-min) eol)
                            (buffer-substring eol (point-max))))))
 	(write-back org-src--allow-write-back)
-        (preserve-blank-line org-src--preserve-blank-line)
+        (indent-current-empty-line org-src--indent-current-empty-line)
         marker indent-str)
     (setq indent-str
           (with-temp-buffer
@@ -494,7 +494,7 @@ Assume point is in the corresponding edit buffer."
                     ;; If the current line is empty, we may
                     ;; want to indent it.
                     (and (eq (point) (marker-position marker))
-                         preserve-blank-line))
+                         indent-current-empty-line))
             (insert indent-str))
 	  (forward-line)))
       ;; Apply WRITE-BACK function on edit buffer contents.
@@ -554,8 +554,6 @@ Leave point in edit buffer."
              (blank-line (save-excursion (beginning-of-line)
                                          (looking-at-p "^[[:space:]]*$")))
              (empty-line (and blank-line (looking-at-p "^$")))
-             (preserve-blank-line (or (and blank-line (not empty-line))
-                                      (and empty-line (= (+ block-ind content-ind) 0))))
 	     (preserve-ind
 	      (and (memq type '(example-block src-block))
 		   (or (org-element-property :preserve-indent datum)
@@ -605,7 +603,8 @@ Leave point in edit buffer."
 	(setq org-src--overlay overlay)
 	(setq org-src--allow-write-back write-back)
 	(setq org-src-source-file-name source-file-name)
-        (setq org-src--preserve-blank-line preserve-blank-line)
+        (setq org-src--indent-current-empty-line (and blank-line
+                                                      (not empty-line)))
 	;; Start minor mode.
 	(org-src-mode)
 	;; Clear undo information so we cannot undo back to the
-- 
2.41.0

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> +         ;; Trim contents: `org-src--contents-for-write-back' may have
> +         ;; added indentation at the beginning, which we remove.

May you also mention that we remove the indentation to avoid adding
spaces to latex fragments in the middle of a paragraph?

>>> +          (when (or (not (eolp)) ; not an empty line
>>> +                    ;; If the current line is empty, we may
>>> +                    ;; want to indent it.
>>> +                    (and (eq (point) (marker-position marker))
>>>                            preserve-blank-line))
>> Do we still need this dance with special case for current line?
>
> Yes. This is fragile, but what it does is, if the line from which the
> original edit was called was blank and not empty, and the line from
> which the edit is ended is empty, we assume these two lines are the
> same, and we add the common block indentation to this empty line.
> This, and the pre-indentation in =org-indent-line=, make `TAB` work
> to indent an empty line.

But why do we need to avoid indenting empty lines?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

Ihor Radchenko writes:
> But why do we need to avoid indenting empty lines?

In the following link, Greg Minshal argues for preserving empty lines:
https://list.orgmode.org/725763.1632663635@apollo2.minshall.org/T/

(CCing the mailing list)

-- 
Sébastien Miquel

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> Ihor Radchenko writes:
>> But why do we need to avoid indenting empty lines?
>
> In the following link, Greg Minshal argues for preserving empty lines:
> https://list.orgmode.org/725763.1632663635@apollo2.minshall.org/T/

Thanks for the reference!
I can see that non-empty blank line below might be annoying for some users.

\t#+begin_src lang
\t  line1
\t  line2
^\t  $
\t#+end_src

However, I do not see why Org should clear blank lines created by the
lang-mode itself

\t#+begin_src lang
\t  line1:
\t  \tline2:
^\t \t\t$
\t#+end_src

If source code in the edit buffer contains non-empty blank lines, it is
not Org's responsibility to clear them. In fact, it will go against
possible user settings!

So, I agree that we should not indent empty lines. However, I do not
agree that we should not indent non-empty blank lines.

---

The code in your patch is confusing, considering the above
considerations.

> -        (setq org-src--preserve-blank-line preserve-blank-line)
> +        (setq org-src--indent-current-empty-line (and blank-line
> +                                                      (not empty-line)))

Here, you have a variable named "empty-line" set when (not empty-line). ??

Also,

>       (while (not (eobp))
> -	  (skip-chars-forward " \t")
> -          (when (or (not (eolp))                               ; not a blank line
> -                    (and (eq (point) (marker-position marker)) ; current line
> +          (when (or (not (eolp)) ; not an empty line
> +                    ;; If the current line is empty, we may
> +                    ;; want to indent it.
> +                    (and (eq (point) (marker-position marker))
>                           preserve-blank-line))
>            (insert indent-str))
>	  (forward-line)))

removed `skip-chars-forward' call, so the loop will always check every
bol and (not (eolp)) will be t for every line, except ^$.
Then, considering that preserve-blank-line is set when (not empty-line),
your second condition will never trigger.

I feel that something is fishy in the logic.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 2921 bytes --]


Ihor Radchenko writes:
>> +         ;; Trim contents: `org-src--contents-for-write-back' may have
>> +         ;; added indentation at the beginning, which we remove.
>
> May you also mention that we remove the indentation to avoid adding
> spaces to latex fragments in the middle of a paragraph?

On second thought, I don't think moving the LaTeX fragment logic away
from `org-src--contents-for-write-back` makes sense. This part of the
function does the opposite of `org-do-remove-indentation`, and the
latter has a boolean argument `skip-fl`, so it makes sense to keep it
the same here. It is simple enough.

If you're worried about consistency with inline src blocks, I find it
weird for them to have newlines, let alone newlines mixed with org's
indentation. But if we do want to treat them the same, then we also
need to modify `org-do-remove-indentation` to skip the first line for
them as well.

I've taken this part off the patch for now.

> If source code in the edit buffer contains non-empty blank lines, it is
> not Org's responsibility to clear them. In fact, it will go against
> possible user settings!
>
> So, I agree that we should not indent empty lines. However, I do not
> agree that we should not indent non-empty blank lines.

The patch I propose does indent non-empty blank lines. The issue is
with =org-do-remove-indentation= which empties blank lines. I've added
a fix to this.

>> -        (setq org-src--preserve-blank-line preserve-blank-line)
>> +        (setq org-src--indent-current-empty-line (and blank-line
>> +                                                      (not empty-line)))
>
> Here, you have a variable named "empty-line" set when (not empty-line). ??

I've renamed it yet again!

>
>>       (while (not (eobp))
>> -	  (skip-chars-forward " \t")
>> -          (when (or (not (eolp))                               ; not a blank line
>> -                    (and (eq (point) (marker-position marker)) ; current line
>> +          (when (or (not (eolp)) ; not an empty line
>> +                    ;; If the current line is empty, we may
>> +                    ;; want to indent it.
>> +                    (and (eq (point) (marker-position marker))
>>                           preserve-blank-line))
>>            (insert indent-str))
>> 	  (forward-line)))
>
> removed `skip-chars-forward' call, so the loop will always check every
> bol and (not (eolp)) will be t for every line, except ^$.
> Then, considering that preserve-blank-line is set when (not empty-line),
> your second condition will never trigger.
>
> I feel that something is fishy in the logic.

What happens is: in the org buffer, the line is not empty, because it
has the org indentation (which was possibly just added by
org-indent-line), but in the edit buffer, the line is empty, because
the common org indentation was removed. In that case, we want to add
back the org indentation.

--
Sébastien Miquel

[-- Attachment #2: 0001-org-src.el-Use-native-value-of-indent-tabs-mode-for-.patch --]
[-- Type: text/x-patch, Size: 9561 bytes --]

From ecc87b4a75dec7ff8fba4c86635ba3cdb43444ff Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Tue, 27 Jun 2023 09:23:01 +0200
Subject: [PATCH] org-src.el: Use native value of `indent-tabs-mode' for
 indentation

* lisp/org-macs.el (org-do-remove-indentation): Preserve
indentation (spaces vs tabs) past the common indentation to remove.
Do not empty blank lines.
* lisp/org-src.el (org-src--contents-for-write-back): Preserve the
native indentation (spaces vs tabs).  If necessary, add a common org
indentation to the block according to org's `indent-tabs-mode'.
(org-src-font-lock-fontify-block): In case of mixed indentation,
display the tab characters with a fixed width, according to the native
tab width value.
* testing/lisp/test-org-src.el (test-org-src/indented-blocks): Update
tests.  Indentation no longer obeys `indent-tabs-mode' from the org
buffer, but is separated in two parts.
---
 lisp/org-macs.el             |  9 +++--
 lisp/org-src.el              | 45 ++++++++++++++++------
 testing/lisp/test-org-src.el | 75 ++++++++++++++++++++++--------------
 3 files changed, 85 insertions(+), 44 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 51dbfe118..ea210dc60 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -483,9 +483,12 @@ line.  Return nil if it fails."
         (when skip-fl (forward-line))
 	(while (not (eobp))
 	  (let ((ind (progn (skip-chars-forward " \t") (current-column))))
-	    (cond ((eolp) (delete-region (line-beginning-position) (point)))
-		  ((< ind n) (throw :exit nil))
-		  (t (indent-line-to (- ind n))))
+	    (cond ((< ind n)
+                   (if (eolp) (delete-region (line-beginning-position) (point))
+                     (throw :exit nil)))
+		  (t (delete-region (line-beginning-position)
+                                    (progn (move-to-column n t)
+                                           (point)))))
 	    (forward-line)))
 	;; Signal success.
 	t))))
diff --git a/lisp/org-src.el b/lisp/org-src.el
index f15ba8e99..9e6031016 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -474,11 +474,15 @@ Assume point is in the corresponding edit buffer."
                            (buffer-substring eol (point-max))))))
 	(write-back org-src--allow-write-back)
         (preserve-blank-line org-src--preserve-blank-line)
-        marker)
+        marker indent-str)
+    (setq indent-str
+          (with-temp-buffer
+            ;; Reproduce indentation parameters from org buffer.
+            (setq indent-tabs-mode use-tabs?)
+            (when (> source-tab-width 0) (setq tab-width source-tab-width))
+            (indent-to indentation-offset)
+            (buffer-string)))
     (with-current-buffer write-back-buf
-      ;; Reproduce indentation parameters from source buffer.
-      (setq indent-tabs-mode use-tabs?)
-      (when (> source-tab-width 0) (setq tab-width source-tab-width))
       ;; Apply WRITE-BACK function on edit buffer contents.
       (insert (org-no-properties (car contents)))
       (setq marker (point-marker))
@@ -488,15 +492,16 @@ Assume point is in the corresponding edit buffer."
       ;; Add INDENTATION-OFFSET to every line in buffer,
       ;; unless indentation is meant to be preserved.
       (when (> indentation-offset 0)
-	(when preserve-fl (forward-line))
+        ;; LaTeX-fragments are inline. Do not add indentation to their
+        ;; first line.
+        (when preserve-fl (forward-line))
         (while (not (eobp))
-	  (skip-chars-forward " \t")
-          (when (or (not (eolp))                               ; not a blank line
-                    (and (eq (point) (marker-position marker)) ; current line
+          (when (or (not (eolp)) ; not an empty line
+                    ;; If the current line is empty, we may
+                    ;; want to indent it.
+                    (and (eq (point) (marker-position marker))
                          preserve-blank-line))
-	    (let ((i (current-column)))
-	      (delete-region (line-beginning-position) (point))
-	      (indent-to (+ i indentation-offset))))
+            (insert indent-str))
 	  (forward-line)))
       (set-marker marker nil))))
 
@@ -637,7 +642,7 @@ Leave point in edit buffer."
   "Fontify code block between START and END using LANG's syntax.
 This function is called by Emacs' automatic fontification, as long
 as `org-src-fontify-natively' is non-nil."
-  (let ((modified (buffer-modified-p)))
+  (let ((modified (buffer-modified-p)) native-tab-width)
     (remove-text-properties start end '(face nil))
     (let ((lang-mode (org-src-get-lang-mode lang)))
       (when (fboundp lang-mode)
@@ -651,6 +656,7 @@ as `org-src-fontify-natively' is non-nil."
 	      ;; Add string and a final space to ensure property change.
 	      (insert string " "))
 	    (unless (eq major-mode lang-mode) (funcall lang-mode))
+            (setq native-tab-width tab-width)
             (font-lock-ensure)
 	    (let ((pos (point-min)) next)
 	      (while (setq next (next-property-change pos))
@@ -708,6 +714,21 @@ as `org-src-fontify-natively' is non-nil."
       (when (or (facep src-face) (listp src-face))
         (font-lock-append-text-property start end 'face src-face))
       (font-lock-append-text-property start end 'face 'org-block))
+    ;; Display native tab indentation characters as spaces
+    (save-excursion
+      (goto-char start)
+      (let ((indent-offset
+	     (if org-src-preserve-indentation 0
+	       (+ (progn (backward-char)
+                         (org-current-text-indentation))
+	          org-edit-src-content-indentation))))
+        (while (re-search-forward "^[ ]*\t" end t)
+          (let* ((b (and (eq indent-offset (move-to-column indent-offset))
+                         (point)))
+                 (e (progn (skip-chars-forward "\t") (point)))
+                 (s (and b (make-string (* (- e b) native-tab-width) ? ))))
+            (when (and b (< b e)) (add-text-properties b e `(display ,s)))
+            (forward-char)))))
     ;; Clear abbreviated link folding.
     (org-fold-region start end nil 'org-link)
     (add-text-properties
diff --git a/testing/lisp/test-org-src.el b/testing/lisp/test-org-src.el
index 2a45ba66e..c4309ccfc 100644
--- a/testing/lisp/test-org-src.el
+++ b/testing/lisp/test-org-src.el
@@ -304,11 +304,11 @@ This is a tab:\t.
 	(insert " Foo")
 	(org-edit-src-exit)
 	(buffer-string)))))
-  ;; Global indentation obeys `indent-tabs-mode' from the original
-  ;; buffer.
-  (should
+  ;; Global indentation does not obey `indent-tabs-mode' from the
+  ;; original buffer.
+  (should-not
    (string-match-p
-    "^\t+\s*argument2"
+    "\t"
     (org-test-with-temp-text
 	"
 - Item
@@ -323,14 +323,15 @@ This is a tab:\t.
 	(org-edit-special)
 	(org-edit-src-exit)
 	(buffer-string)))))
+  ;; Tab character is preserved
   (should
    (string-match-p
-    "^\s+argument2"
+    "\targument2"
     (org-test-with-temp-text
 	"
 - Item
   #+BEGIN_SRC emacs-lisp<point>
-    (progn\n      (function argument1\n\t\targument2))
+    (progn\n      (function argument1\n    \targument2))
   #+END_SRC"
       (setq-local indent-tabs-mode nil)
       (let ((org-edit-src-content-indentation 2)
@@ -338,43 +339,59 @@ This is a tab:\t.
 	(org-edit-special)
 	(org-edit-src-exit)
 	(buffer-string)))))
-  ;; Global indentation also obeys `tab-width' from original buffer.
+  ;; Indentation does not obey `tab-width' from org buffer.
   (should
    (string-match-p
-    "^\t\\{3\\}\s\\{2\\}argument2"
+    "^  \targument2"
     (org-test-with-temp-text
 	"
-- Item
-  #+BEGIN_SRC emacs-lisp<point>
+#+BEGIN_SRC emacs-lisp
   (progn
-    (function argument1
-              argument2))
-  #+END_SRC"
+    (list argument1\n  \t<point>argument2))
+#+END_SRC"
       (setq-local indent-tabs-mode t)
       (setq-local tab-width 4)
-      (let ((org-edit-src-content-indentation 0)
+      (let ((org-edit-src-content-indentation 2)
 	    (org-src-preserve-indentation nil))
 	(org-edit-special)
+        (setq-local indent-tabs-mode t)
+        (setq-local tab-width 8)
+        (lisp-indent-line)
 	(org-edit-src-exit)
 	(buffer-string)))))
+  ;; Tab characters are displayed with `tab-width' from the native
+  ;; edit buffer.
   (should
-   (string-match-p
-    "^\t\s\\{6\\}argument2"
+   (equal
+    10
     (org-test-with-temp-text
-	"
-- Item
-  #+BEGIN_SRC emacs-lisp<point>
+     "
+#+BEGIN_SRC emacs-lisp
   (progn
-    (function argument1
-              argument2))
-  #+END_SRC"
-      (setq-local indent-tabs-mode t)
-      (setq-local tab-width 8)
-      (let ((org-edit-src-content-indentation 0)
-	    (org-src-preserve-indentation nil))
-	(org-edit-special)
-	(org-edit-src-exit)
-	(buffer-string))))))
+    (list argument1\n  \t<point>argument2))
+#+END_SRC"
+     (setq-local indent-tabs-mode t)
+     (setq-local tab-width 4)
+     (let ((org-edit-src-content-indentation 2)
+	   (org-src-preserve-indentation nil))
+       (font-lock-ensure)
+       (current-column)))))
+  ;; The initial tab characters respect org's `tab-width'.
+  (should
+   (equal
+    10
+    (org-test-with-temp-text
+     "
+#+BEGIN_SRC emacs-lisp
+\t(progn
+\t  (list argument1\n\t\t<point>argument2))
+#+END_SRC"
+     (setq-local indent-tabs-mode t)
+     (setq-local tab-width 2)
+     (let ((org-edit-src-content-indentation 2)
+	   (org-src-preserve-indentation nil))
+       (font-lock-ensure)
+       (current-column))))))
 
 (ert-deftest test-org-src/footnote-references ()
   "Test editing footnote references."
-- 
2.41.0


[-- Attachment #3: 0001-org-src.el-Rename-internal-variable-for-clarity.patch --]
[-- Type: text/x-patch, Size: 3099 bytes --]

From bb16c48b8482104d9ea409e4260076ae08f42dc1 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Thu, 29 Jun 2023 14:37:09 +0200
Subject: [PATCH] org-src.el: Rename internal variable for clarity

* lisp/org-src.el (org-src--contents-for-write-back):
(org-src--edit-element): Rename `preserve-blank-line' to
`indent-current-empty-line'.  It is used to decide whether we should
indent the current empty line after a special edit.
---
 lisp/org-src.el | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index 9e6031016..6fbb0b1c2 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -318,8 +318,8 @@ is 0.")
   "File name associated to Org source buffer, or nil.")
 (put 'org-src-source-file-name 'permanent-local t)
 
-(defvar-local org-src--preserve-blank-line nil)
-(put 'org-src--preserve-blank-line 'permanent-local t)
+(defvar-local org-src--curr-line-blank-indented nil)
+(put 'org-src--curr-line-blank-indented 'permanent-local t)
 
 (defun org-src--construct-edit-buffer-name (org-buffer-name lang)
   "Construct the buffer name for a source editing buffer.
@@ -473,7 +473,7 @@ Assume point is in the corresponding edit buffer."
                      (list (buffer-substring (point-min) eol)
                            (buffer-substring eol (point-max))))))
 	(write-back org-src--allow-write-back)
-        (preserve-blank-line org-src--preserve-blank-line)
+        (indent-current-empty-line org-src--curr-line-blank-indented)
         marker indent-str)
     (setq indent-str
           (with-temp-buffer
@@ -500,7 +500,7 @@ Assume point is in the corresponding edit buffer."
                     ;; If the current line is empty, we may
                     ;; want to indent it.
                     (and (eq (point) (marker-position marker))
-                         preserve-blank-line))
+                         indent-current-empty-line))
             (insert indent-str))
 	  (forward-line)))
       (set-marker marker nil))))
@@ -557,8 +557,6 @@ Leave point in edit buffer."
              (blank-line (save-excursion (beginning-of-line)
                                          (looking-at-p "^[[:space:]]*$")))
              (empty-line (and blank-line (looking-at-p "^$")))
-             (preserve-blank-line (or (and blank-line (not empty-line))
-                                      (and empty-line (= (+ block-ind content-ind) 0))))
 	     (preserve-ind
 	      (and (memq type '(example-block src-block))
 		   (or (org-element-property :preserve-indent datum)
@@ -608,7 +606,8 @@ Leave point in edit buffer."
 	(setq org-src--overlay overlay)
 	(setq org-src--allow-write-back write-back)
 	(setq org-src-source-file-name source-file-name)
-        (setq org-src--preserve-blank-line preserve-blank-line)
+        (setq org-src--curr-line-blank-indented (and blank-line
+                                                     (not empty-line)))
 	;; Start minor mode.
 	(org-src-mode)
 	;; Clear undo information so we cannot undo back to the
-- 
2.41.0

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> On second thought, I don't think moving the LaTeX fragment logic away
> from `org-src--contents-for-write-back` makes sense. This part of the
> function does the opposite of `org-do-remove-indentation`, and the
> latter has a boolean argument `skip-fl`, so it makes sense to keep it
> the same here. It is simple enough.
>
> If you're worried about consistency with inline src blocks, I find it
> weird for them to have newlines, let alone newlines mixed with org's
> indentation. But if we do want to treat them the same, then we also
> need to modify `org-do-remove-indentation` to skip the first line for
> them as well.
>
> I've taken this part off the patch for now.

Ok.

I am not that much worried about consistency with inline src blocks.
Rather do not like the mix of org-src buffer local variables and
checking the block type. But we can leave this refactoring to another
day. It is not just about preserve-fl. (also indentation-offset)

> What happens is: in the org buffer, the line is not empty, because it
> has the org indentation (which was possibly just added by
> org-indent-line), but in the edit buffer, the line is empty, because
> the common org indentation was removed. In that case, we want to add
> back the org indentation.

May you please provide an example when it is necessary?
`org-indent-line' will run `org-babel-do-key-sequence-in-edit-buffer', so
it should still use `org-src--contents-for-write-back' and will not
modify the org buffer text directly.

> --- a/lisp/org-macs.el
> +++ b/lisp/org-macs.el
> @@ -483,9 +483,12 @@ line.  Return nil if it fails."
>          (when skip-fl (forward-line))
>  	(while (not (eobp))
>  	  (let ((ind (progn (skip-chars-forward " \t") (current-column))))
> -	    (cond ((eolp) (delete-region (line-beginning-position) (point)))
> -		  ((< ind n) (throw :exit nil))
> -		  (t (indent-line-to (- ind n))))
> +	    (cond ((< ind n)
> +                   (if (eolp) (delete-region (line-beginning-position) (point))
> +                     (throw :exit nil)))

This function is actually confusing both before and after the change.
According to the docstring:

    When optional argument N is a positive integer, remove exactly
    that much characters from indentation, if possible.

But the function can actually remove less than N characters.

Before your change, all the blank non-empty lines were unconditionally
removed. After your change, the first such line is removed and the
function returns nil without continuing.

> * lisp/org-macs.el (org-do-remove-indentation): Preserve
> indentation (spaces vs tabs) past the common indentation to remove.
> Do not empty blank lines.

Since not removing blank lines is intentional after the change, why
doing it on a single line that is indented less than N?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

Ihor Radchenko writes:
>> What happens is: in the org buffer, the line is not empty, because it
>> has the org indentation (which was possibly just added by
>> org-indent-line), but in the edit buffer, the line is empty, because
>> the common org indentation was removed. In that case, we want to add
>> back the org indentation.
>
> May you please provide an example when it is necessary?
> `org-indent-line' will run `org-babel-do-key-sequence-in-edit-buffer', so
> it should still use `org-src--contents-for-write-back' and will not
> modify the org buffer text directly.

You're at the end of a line, you press =org-return-and-indent=.
  1. It adds a newline character.
  2. =org-indent-line= adds the org indentation, _before_ calling
     =org-babel-do-key-sequence-in-edit-buffer=
  3. The native edit call removes the common indentation, before
     calling tab in the native buffer.
  4. Calling tab in the native buffer possibly does nothing.
  5. =org-src--contents-for-write-back= sees the current line is empty,
     but it should indent it (with org indentation) nonetheless.


>> --- a/lisp/org-macs.el
>> +++ b/lisp/org-macs.el
>> @@ -483,9 +483,12 @@ line.  Return nil if it fails."
>>           (when skip-fl (forward-line))
>>   	(while (not (eobp))
>>   	  (let ((ind (progn (skip-chars-forward " \t") (current-column))))
>> -	    (cond ((eolp) (delete-region (line-beginning-position) (point)))
>> -		  ((< ind n) (throw :exit nil))
>> -		  (t (indent-line-to (- ind n))))
>> +	    (cond ((< ind n)
>> +                   (if (eolp) (delete-region (line-beginning-position) (point))
>> +                     (throw :exit nil)))
>
> This function is actually confusing both before and after the change.
> According to the docstring:
>
>      When optional argument N is a positive integer, remove exactly
>      that much characters from indentation, if possible.
>
> But the function can actually remove less than N characters.
>
> Before your change, all the blank non-empty lines were unconditionally
> removed. After your change, the first such line is removed and the
> function returns nil without continuing.

I don't understand. With this change, the function only stops if it
finds a non blank line with less than n indentation (same as before).
When a blank line with less than n indentation is found, it is emptied
(same as before), and execution continues.

>> * lisp/org-macs.el (org-do-remove-indentation): Preserve
>> indentation (spaces vs tabs) past the common indentation to remove.
>> Do not empty blank lines.
>
> Since not removing blank lines is intentional after the change, why
> doing it on a single line that is indented less than N?

Are you advocating for computing N using blank lines as well ?

  1. It isn't consistent with the previous behaviour.
  2. If I mistakenly add a space to an empty line in a src block, an
     edit-special round trip will add indentation to every line.

Is there any benefit ?

-- 
Sébastien Miquel

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

>> May you please provide an example when it is necessary?
>> `org-indent-line' will run `org-babel-do-key-sequence-in-edit-buffer', so
>> it should still use `org-src--contents-for-write-back' and will not
>> modify the org buffer text directly.
>
> You're at the end of a line, you press =org-return-and-indent=.
>   1. It adds a newline character.
>   2. =org-indent-line= adds the org indentation, _before_ calling
>      =org-babel-do-key-sequence-in-edit-buffer=

I missed this:

             ;; At the beginning of a blank line, do some preindentation.  This
             ;; signals org-src--edit-element to preserve the indentation on exit

>   3. The native edit call removes the common indentation, before
>      calling tab in the native buffer.
>   4. Calling tab in the native buffer possibly does nothing.
>   5. =org-src--contents-for-write-back= sees the current line is empty,
>      but it should indent it (with org indentation) nonetheless.

Ok. I understand better now (I think).

We are talking about

\t#+begin_src bash
\tcd foo<point>
\t#+end_src

and user pressing <RET>
Then, the expected result is

\t#+begin_src bash
\tcd foo
\t<point>
\t#+end_src

with <point> being aligned with "cd foo" above.

Alternatively,

\t#+begin_src emacs-lisp
\t(when t<point>
\t#+end_src

<RET> should yield indentation in src block as well

\t#+begin_src emacs-lisp
\t(when t
\t  <point>
\t#+end_src

-------------

For the second scenario, no special treatment of current line is needed.
For the first scenario, why do we need to do it all the way in
`org-src--contents-for-write-back'? Why not directly in
`org-indent-line'?

>> Before your change, all the blank non-empty lines were unconditionally
>> removed. After your change, the first such line is removed and the
>> function returns nil without continuing.
>
> I don't understand. With this change, the function only stops if it
> finds a non blank line with less than n indentation (same as before).
> When a blank line with less than n indentation is found, it is emptied
> (same as before), and execution continues.

Never mind this. I misread the code. Thought that `throw' is called on
blank lines.

>> Since not removing blank lines is intentional after the change, why
>> doing it on a single line that is indented less than N?
>
> Are you advocating for computing N using blank lines as well ?

No. It was a misunderstanding.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 296 bytes --]


Ihor Radchenko writes:
> For the second scenario, no special treatment of current line is needed.
> For the first scenario, why do we need to do it all the way in
> `org-src--contents-for-write-back'? Why not directly in
> `org-indent-line'?

Ah, yes, that is much better.

-- 
Sébastien Miquel

[-- Attachment #2: 0001-org-src.el-Use-native-value-of-indent-tabs-mode-for-.patch --]
[-- Type: text/x-patch, Size: 13075 bytes --]

From 9d31a71bc0ab7cfd466ecad60037d00c62bdd9f6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Tue, 27 Jun 2023 09:23:01 +0200
Subject: [PATCH] org-src.el: Use native value of `indent-tabs-mode' for
 indentation

* lisp/org.el (org-indent-line): Simplify native indentation inside
src block.  Ensure we add the org indentation if the line is empty.
* lisp/org-macs.el (org-do-remove-indentation): Preserve
indentation (spaces vs tabs) past the common indentation to remove.
Do not empty blank lines.
* lisp/org-src.el (org-src--contents-for-write-back): Preserve the
native indentation (spaces vs tabs).  If necessary, add a common org
indentation to the block according to org's `indent-tabs-mode'.
(org-src-font-lock-fontify-block): In case of mixed indentation,
display the tab characters with a fixed width, according to the native
tab width value.
* testing/lisp/test-org-src.el (test-org-src/indented-blocks): Update
tests.  Indentation no longer obeys `indent-tabs-mode' from the org
buffer, but is separated in two parts.
---
 lisp/org-macs.el             |  9 +++--
 lisp/org-src.el              | 52 ++++++++++++++-----------
 lisp/org.el                  | 23 ++++-------
 testing/lisp/test-org-src.el | 75 ++++++++++++++++++++++--------------
 4 files changed, 89 insertions(+), 70 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 51dbfe118..ea210dc60 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -483,9 +483,12 @@ line.  Return nil if it fails."
         (when skip-fl (forward-line))
 	(while (not (eobp))
 	  (let ((ind (progn (skip-chars-forward " \t") (current-column))))
-	    (cond ((eolp) (delete-region (line-beginning-position) (point)))
-		  ((< ind n) (throw :exit nil))
-		  (t (indent-line-to (- ind n))))
+	    (cond ((< ind n)
+                   (if (eolp) (delete-region (line-beginning-position) (point))
+                     (throw :exit nil)))
+		  (t (delete-region (line-beginning-position)
+                                    (progn (move-to-column n t)
+                                           (point)))))
 	    (forward-line)))
 	;; Signal success.
 	t))))
diff --git a/lisp/org-src.el b/lisp/org-src.el
index f15ba8e99..2beb49e63 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -318,9 +318,6 @@ is 0.")
   "File name associated to Org source buffer, or nil.")
 (put 'org-src-source-file-name 'permanent-local t)
 
-(defvar-local org-src--preserve-blank-line nil)
-(put 'org-src--preserve-blank-line 'permanent-local t)
-
 (defun org-src--construct-edit-buffer-name (org-buffer-name lang)
   "Construct the buffer name for a source editing buffer.
 Format is \"*Org Src ORG-BUFFER-NAME[ LANG ]*\"."
@@ -473,12 +470,15 @@ Assume point is in the corresponding edit buffer."
                      (list (buffer-substring (point-min) eol)
                            (buffer-substring eol (point-max))))))
 	(write-back org-src--allow-write-back)
-        (preserve-blank-line org-src--preserve-blank-line)
-        marker)
+        marker indent-str)
+    (setq indent-str
+          (with-temp-buffer
+            ;; Reproduce indentation parameters from org buffer.
+            (setq indent-tabs-mode use-tabs?)
+            (when (> source-tab-width 0) (setq tab-width source-tab-width))
+            (indent-to indentation-offset)
+            (buffer-string)))
     (with-current-buffer write-back-buf
-      ;; Reproduce indentation parameters from source buffer.
-      (setq indent-tabs-mode use-tabs?)
-      (when (> source-tab-width 0) (setq tab-width source-tab-width))
       ;; Apply WRITE-BACK function on edit buffer contents.
       (insert (org-no-properties (car contents)))
       (setq marker (point-marker))
@@ -488,15 +488,11 @@ Assume point is in the corresponding edit buffer."
       ;; Add INDENTATION-OFFSET to every line in buffer,
       ;; unless indentation is meant to be preserved.
       (when (> indentation-offset 0)
-	(when preserve-fl (forward-line))
+        ;; LaTeX-fragments are inline. Do not add indentation to their
+        ;; first line.
+        (when preserve-fl (forward-line))
         (while (not (eobp))
-	  (skip-chars-forward " \t")
-          (when (or (not (eolp))                               ; not a blank line
-                    (and (eq (point) (marker-position marker)) ; current line
-                         preserve-blank-line))
-	    (let ((i (current-column)))
-	      (delete-region (line-beginning-position) (point))
-	      (indent-to (+ i indentation-offset))))
+          (when (not (eolp)) (insert indent-str)) ; not an empty line
 	  (forward-line)))
       (set-marker marker nil))))
 
@@ -549,11 +545,6 @@ Leave point in edit buffer."
                              (org-element-property :parent datum) nil))
                            (t (org-current-text-indentation)))))
 	     (content-ind org-edit-src-content-indentation)
-             (blank-line (save-excursion (beginning-of-line)
-                                         (looking-at-p "^[[:space:]]*$")))
-             (empty-line (and blank-line (looking-at-p "^$")))
-             (preserve-blank-line (or (and blank-line (not empty-line))
-                                      (and empty-line (= (+ block-ind content-ind) 0))))
 	     (preserve-ind
 	      (and (memq type '(example-block src-block))
 		   (or (org-element-property :preserve-indent datum)
@@ -603,7 +594,6 @@ Leave point in edit buffer."
 	(setq org-src--overlay overlay)
 	(setq org-src--allow-write-back write-back)
 	(setq org-src-source-file-name source-file-name)
-        (setq org-src--preserve-blank-line preserve-blank-line)
 	;; Start minor mode.
 	(org-src-mode)
 	;; Clear undo information so we cannot undo back to the
@@ -637,7 +627,7 @@ Leave point in edit buffer."
   "Fontify code block between START and END using LANG's syntax.
 This function is called by Emacs' automatic fontification, as long
 as `org-src-fontify-natively' is non-nil."
-  (let ((modified (buffer-modified-p)))
+  (let ((modified (buffer-modified-p)) native-tab-width)
     (remove-text-properties start end '(face nil))
     (let ((lang-mode (org-src-get-lang-mode lang)))
       (when (fboundp lang-mode)
@@ -651,6 +641,7 @@ as `org-src-fontify-natively' is non-nil."
 	      ;; Add string and a final space to ensure property change.
 	      (insert string " "))
 	    (unless (eq major-mode lang-mode) (funcall lang-mode))
+            (setq native-tab-width tab-width)
             (font-lock-ensure)
 	    (let ((pos (point-min)) next)
 	      (while (setq next (next-property-change pos))
@@ -708,6 +699,21 @@ as `org-src-fontify-natively' is non-nil."
       (when (or (facep src-face) (listp src-face))
         (font-lock-append-text-property start end 'face src-face))
       (font-lock-append-text-property start end 'face 'org-block))
+    ;; Display native tab indentation characters as spaces
+    (save-excursion
+      (goto-char start)
+      (let ((indent-offset
+	     (if org-src-preserve-indentation 0
+	       (+ (progn (backward-char)
+                         (org-current-text-indentation))
+	          org-edit-src-content-indentation))))
+        (while (re-search-forward "^[ ]*\t" end t)
+          (let* ((b (and (eq indent-offset (move-to-column indent-offset))
+                         (point)))
+                 (e (progn (skip-chars-forward "\t") (point)))
+                 (s (and b (make-string (* (- e b) native-tab-width) ? ))))
+            (when (and b (< b e)) (add-text-properties b e `(display ,s)))
+            (forward-char)))))
     ;; Clear abbreviated link folding.
     (org-fold-region start end nil 'org-link)
     (add-text-properties
diff --git a/lisp/org.el b/lisp/org.el
index 4063ba98f..cda674919 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -19086,21 +19086,14 @@ Also align node properties according to `org-property-format'."
 		     (org-with-point-at (org-element-property :end element)
 		       (skip-chars-backward " \t\n")
 		       (line-beginning-position))))
-             ;; At the beginning of a blank line, do some preindentation.  This
-             ;; signals org-src--edit-element to preserve the indentation on exit
-             (when (and (looking-at-p "^[[:space:]]*$")
-                        (not org-src-preserve-indentation))
-               (let (block-content-ind some-ind)
-                 (org-with-point-at (org-element-property :begin element)
-                   (setq block-content-ind (+ (org-current-text-indentation)
-                                              org-edit-src-content-indentation))
-                   (forward-line)
-		   (save-match-data (re-search-forward "^[ \t]*\\S-" nil t))
-                   (backward-char)
-                   (setq some-ind (if (looking-at-p "#\\+end_src")
-                                      block-content-ind (org-current-text-indentation))))
-                 (indent-line-to (min block-content-ind some-ind))))
-	     (org-babel-do-key-sequence-in-edit-buffer (kbd "TAB")))
+             (let ((block-content-ind
+                    (when (not org-src-preserve-indentation)
+                      (org-with-point-at (org-element-property :begin element)
+                        (+ (org-current-text-indentation)
+                           org-edit-src-content-indentation)))))
+               (org-babel-do-key-sequence-in-edit-buffer (kbd "TAB"))
+               (when (and block-content-ind (looking-at-p "^$"))
+                 (indent-line-to block-content-ind))))
 	    (t
 	     (let ((column (org--get-expected-indentation element nil)))
 	       ;; Preserve current column.
diff --git a/testing/lisp/test-org-src.el b/testing/lisp/test-org-src.el
index 2a45ba66e..c4309ccfc 100644
--- a/testing/lisp/test-org-src.el
+++ b/testing/lisp/test-org-src.el
@@ -304,11 +304,11 @@ This is a tab:\t.
 	(insert " Foo")
 	(org-edit-src-exit)
 	(buffer-string)))))
-  ;; Global indentation obeys `indent-tabs-mode' from the original
-  ;; buffer.
-  (should
+  ;; Global indentation does not obey `indent-tabs-mode' from the
+  ;; original buffer.
+  (should-not
    (string-match-p
-    "^\t+\s*argument2"
+    "\t"
     (org-test-with-temp-text
 	"
 - Item
@@ -323,14 +323,15 @@ This is a tab:\t.
 	(org-edit-special)
 	(org-edit-src-exit)
 	(buffer-string)))))
+  ;; Tab character is preserved
   (should
    (string-match-p
-    "^\s+argument2"
+    "\targument2"
     (org-test-with-temp-text
 	"
 - Item
   #+BEGIN_SRC emacs-lisp<point>
-    (progn\n      (function argument1\n\t\targument2))
+    (progn\n      (function argument1\n    \targument2))
   #+END_SRC"
       (setq-local indent-tabs-mode nil)
       (let ((org-edit-src-content-indentation 2)
@@ -338,43 +339,59 @@ This is a tab:\t.
 	(org-edit-special)
 	(org-edit-src-exit)
 	(buffer-string)))))
-  ;; Global indentation also obeys `tab-width' from original buffer.
+  ;; Indentation does not obey `tab-width' from org buffer.
   (should
    (string-match-p
-    "^\t\\{3\\}\s\\{2\\}argument2"
+    "^  \targument2"
     (org-test-with-temp-text
 	"
-- Item
-  #+BEGIN_SRC emacs-lisp<point>
+#+BEGIN_SRC emacs-lisp
   (progn
-    (function argument1
-              argument2))
-  #+END_SRC"
+    (list argument1\n  \t<point>argument2))
+#+END_SRC"
       (setq-local indent-tabs-mode t)
       (setq-local tab-width 4)
-      (let ((org-edit-src-content-indentation 0)
+      (let ((org-edit-src-content-indentation 2)
 	    (org-src-preserve-indentation nil))
 	(org-edit-special)
+        (setq-local indent-tabs-mode t)
+        (setq-local tab-width 8)
+        (lisp-indent-line)
 	(org-edit-src-exit)
 	(buffer-string)))))
+  ;; Tab characters are displayed with `tab-width' from the native
+  ;; edit buffer.
   (should
-   (string-match-p
-    "^\t\s\\{6\\}argument2"
+   (equal
+    10
     (org-test-with-temp-text
-	"
-- Item
-  #+BEGIN_SRC emacs-lisp<point>
+     "
+#+BEGIN_SRC emacs-lisp
   (progn
-    (function argument1
-              argument2))
-  #+END_SRC"
-      (setq-local indent-tabs-mode t)
-      (setq-local tab-width 8)
-      (let ((org-edit-src-content-indentation 0)
-	    (org-src-preserve-indentation nil))
-	(org-edit-special)
-	(org-edit-src-exit)
-	(buffer-string))))))
+    (list argument1\n  \t<point>argument2))
+#+END_SRC"
+     (setq-local indent-tabs-mode t)
+     (setq-local tab-width 4)
+     (let ((org-edit-src-content-indentation 2)
+	   (org-src-preserve-indentation nil))
+       (font-lock-ensure)
+       (current-column)))))
+  ;; The initial tab characters respect org's `tab-width'.
+  (should
+   (equal
+    10
+    (org-test-with-temp-text
+     "
+#+BEGIN_SRC emacs-lisp
+\t(progn
+\t  (list argument1\n\t\t<point>argument2))
+#+END_SRC"
+     (setq-local indent-tabs-mode t)
+     (setq-local tab-width 2)
+     (let ((org-edit-src-content-indentation 2)
+	   (org-src-preserve-indentation nil))
+       (font-lock-ensure)
+       (current-column))))))
 
 (ert-deftest test-org-src/footnote-references ()
   "Test editing footnote references."
-- 
2.41.0

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 573 bytes --]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> From 9d31a71bc0ab7cfd466ecad60037d00c62bdd9f6 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
> Date: Tue, 27 Jun 2023 09:23:01 +0200
> Subject: [PATCH] org-src.el: Use native value of `indent-tabs-mode' for
>  indentation

Thanks! This looks good and also fixes the original issue raised in this
thread.

May you now rebase the patch onto the latest main, add a Link: to this
discussion to the commit message, and apply the attached extra comments?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: comments.diff --]
[-- Type: text/x-patch, Size: 1063 bytes --]

diff --git a/lisp/org-src.el b/lisp/org-src.el
index 20e0b598c..e1f7d50dc 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -479,6 +479,8 @@ (defun org-src--contents-for-write-back (write-back-buf)
                            (buffer-substring eol (point-max))))))
 	(write-back org-src--allow-write-back)
         marker indent-str)
+    ;; Compute the exact sequence of tabs and spaces used to indent up
+    ;; to `indentation-offset' in the Org buffer.
     (setq indent-str
           (with-temp-buffer
             ;; Reproduce indentation parameters from org buffer.
@@ -500,6 +502,9 @@ (defun org-src--contents-for-write-back (write-back-buf)
         ;; first line.
         (when preserve-fl (forward-line))
         (while (not (eobp))
+          ;; Keep empty src lines empty, even when src block is
+          ;; indented on Org side.
+          ;; See https://list.orgmode.org/725763.1632663635@apollo2.minshall.org/T/
           (when (not (eolp)) (insert indent-str)) ; not an empty line
 	  (forward-line)))
       (set-marker marker nil))))

[-- Attachment #3: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 237 bytes --]


Ihor Radchenko writes:
> May you now rebase the patch onto the latest main, add a Link: to this
> discussion to the commit message, and apply the attached extra comments?

Here it is. Thanks for helping with this.

-- 
Sébastien Miquel

[-- Attachment #2: 0001-org-src.el-Use-native-value-of-indent-tabs-mode-for-.patch --]
[-- Type: text/x-patch, Size: 13506 bytes --]

From 7906d7b7fa2d376e95156ab7177494f2cececaff Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Tue, 27 Jun 2023 09:23:01 +0200
Subject: [PATCH] org-src.el: Use native value of `indent-tabs-mode' for
 indentation

* lisp/org.el (org-indent-line): Simplify native indentation inside
src block.  Ensure we add the org indentation if the line is empty.
* lisp/org-macs.el (org-do-remove-indentation): Preserve
indentation (spaces vs tabs) past the common indentation to remove.
Do not empty blank lines.
* lisp/org-src.el (org-src--contents-for-write-back): Preserve the
native indentation (spaces vs tabs).  If necessary, add a common org
indentation to the block according to org's `indent-tabs-mode'.
(org-src-font-lock-fontify-block): Display the native indentation tab
characters with a fixed width, according to the native tab width
value, to preserve vertical alignement in the org buffer.
* testing/lisp/test-org-src.el (test-org-src/indented-blocks): Update
tests.  Indentation no longer obeys `indent-tabs-mode' from the org
buffer, but is separated in an eventual org part, and the native part.

Link: https://list.orgmode.org/87a5wcez7e.fsf@localhost/T/#t
---
 lisp/org-macs.el             |  9 +++--
 lisp/org-src.el              | 57 ++++++++++++++++-----------
 lisp/org.el                  | 23 ++++-------
 testing/lisp/test-org-src.el | 75 ++++++++++++++++++++++--------------
 4 files changed, 94 insertions(+), 70 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index d6879e8cf..aa5c4845e 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -402,9 +402,12 @@ line.  Return nil if it fails."
         (when skip-fl (forward-line))
 	(while (not (eobp))
 	  (let ((ind (progn (skip-chars-forward " \t") (current-column))))
-	    (cond ((eolp) (delete-region (line-beginning-position) (point)))
-		  ((< ind n) (throw :exit nil))
-		  (t (indent-line-to (- ind n))))
+	    (cond ((< ind n)
+                   (if (eolp) (delete-region (line-beginning-position) (point))
+                     (throw :exit nil)))
+		  (t (delete-region (line-beginning-position)
+                                    (progn (move-to-column n t)
+                                           (point)))))
 	    (forward-line)))
 	;; Signal success.
 	t))))
diff --git a/lisp/org-src.el b/lisp/org-src.el
index 317682844..e1f7d50dc 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -326,9 +326,6 @@ is 0.")
   "File name associated to Org source buffer, or nil.")
 (put 'org-src-source-file-name 'permanent-local t)
 
-(defvar-local org-src--preserve-blank-line nil)
-(put 'org-src--preserve-blank-line 'permanent-local t)
-
 (defun org-src--construct-edit-buffer-name (org-buffer-name lang)
   "Construct the buffer name for a source editing buffer.
 Format is \"*Org Src ORG-BUFFER-NAME[ LANG ]*\"."
@@ -481,12 +478,17 @@ Assume point is in the corresponding edit buffer."
                      (list (buffer-substring (point-min) eol)
                            (buffer-substring eol (point-max))))))
 	(write-back org-src--allow-write-back)
-        (preserve-blank-line org-src--preserve-blank-line)
-        marker)
+        marker indent-str)
+    ;; Compute the exact sequence of tabs and spaces used to indent up
+    ;; to `indentation-offset' in the Org buffer.
+    (setq indent-str
+          (with-temp-buffer
+            ;; Reproduce indentation parameters from org buffer.
+            (setq indent-tabs-mode use-tabs?)
+            (when (> source-tab-width 0) (setq tab-width source-tab-width))
+            (indent-to indentation-offset)
+            (buffer-string)))
     (with-current-buffer write-back-buf
-      ;; Reproduce indentation parameters from source buffer.
-      (setq indent-tabs-mode use-tabs?)
-      (when (> source-tab-width 0) (setq tab-width source-tab-width))
       ;; Apply WRITE-BACK function on edit buffer contents.
       (insert (org-no-properties (car contents)))
       (setq marker (point-marker))
@@ -496,15 +498,14 @@ Assume point is in the corresponding edit buffer."
       ;; Add INDENTATION-OFFSET to every line in buffer,
       ;; unless indentation is meant to be preserved.
       (when (> indentation-offset 0)
-	(when preserve-fl (forward-line))
+        ;; LaTeX-fragments are inline. Do not add indentation to their
+        ;; first line.
+        (when preserve-fl (forward-line))
         (while (not (eobp))
-	  (skip-chars-forward " \t")
-          (when (or (not (eolp))                               ; not a blank line
-                    (and (eq (point) (marker-position marker)) ; current line
-                         preserve-blank-line))
-	    (let ((i (current-column)))
-	      (delete-region (line-beginning-position) (point))
-	      (indent-to (+ i indentation-offset))))
+          ;; Keep empty src lines empty, even when src block is
+          ;; indented on Org side.
+          ;; See https://list.orgmode.org/725763.1632663635@apollo2.minshall.org/T/
+          (when (not (eolp)) (insert indent-str)) ; not an empty line
 	  (forward-line)))
       (set-marker marker nil))))
 
@@ -557,11 +558,6 @@ Leave point in edit buffer."
                              (org-element-parent datum) nil))
                            (t (org-current-text-indentation)))))
 	     (content-ind org-edit-src-content-indentation)
-             (blank-line (save-excursion (forward-line 0)
-                                         (looking-at-p "^[[:space:]]*$")))
-             (empty-line (and blank-line (looking-at-p "^$")))
-             (preserve-blank-line (or (and blank-line (not empty-line))
-                                      (and empty-line (= (+ block-ind content-ind) 0))))
 	     (preserve-ind
 	      (and (memq type '(example-block src-block))
 		   (or (org-element-property :preserve-indent datum)
@@ -611,7 +607,6 @@ Leave point in edit buffer."
 	(setq org-src--overlay overlay)
 	(setq org-src--allow-write-back write-back)
 	(setq org-src-source-file-name source-file-name)
-        (setq org-src--preserve-blank-line preserve-blank-line)
 	;; Start minor mode.
 	(org-src-mode)
 	;; Clear undo information so we cannot undo back to the
@@ -645,7 +640,7 @@ Leave point in edit buffer."
   "Fontify code block between START and END using LANG's syntax.
 This function is called by Emacs' automatic fontification, as long
 as `org-src-fontify-natively' is non-nil."
-  (let ((modified (buffer-modified-p)))
+  (let ((modified (buffer-modified-p)) native-tab-width)
     (remove-text-properties start end '(face nil))
     (let ((lang-mode (org-src-get-lang-mode lang)))
       (when (fboundp lang-mode)
@@ -659,6 +654,7 @@ as `org-src-fontify-natively' is non-nil."
 	      ;; Add string and a final space to ensure property change.
 	      (insert string " "))
 	    (unless (eq major-mode lang-mode) (funcall lang-mode))
+            (setq native-tab-width tab-width)
             (font-lock-ensure)
 	    (let ((pos (point-min)) next)
 	      (while (setq next (next-property-change pos))
@@ -716,6 +712,21 @@ as `org-src-fontify-natively' is non-nil."
       (when (or (facep src-face) (listp src-face))
         (font-lock-append-text-property start end 'face src-face))
       (font-lock-append-text-property start end 'face 'org-block))
+    ;; Display native tab indentation characters as spaces
+    (save-excursion
+      (goto-char start)
+      (let ((indent-offset
+	     (if org-src-preserve-indentation 0
+	       (+ (progn (backward-char)
+                         (org-current-text-indentation))
+	          org-edit-src-content-indentation))))
+        (while (re-search-forward "^[ ]*\t" end t)
+          (let* ((b (and (eq indent-offset (move-to-column indent-offset))
+                         (point)))
+                 (e (progn (skip-chars-forward "\t") (point)))
+                 (s (and b (make-string (* (- e b) native-tab-width) ? ))))
+            (when (and b (< b e)) (add-text-properties b e `(display ,s)))
+            (forward-char)))))
     ;; Clear abbreviated link folding.
     (org-fold-region start end nil 'org-link)
     (add-text-properties
diff --git a/lisp/org.el b/lisp/org.el
index 57d8082e5..62278ec77 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -19153,21 +19153,14 @@ Also align node properties according to `org-property-format'."
 		     (org-with-point-at (org-element-end element)
 		       (skip-chars-backward " \t\n")
 		       (line-beginning-position))))
-             ;; At the beginning of a blank line, do some preindentation.  This
-             ;; signals org-src--edit-element to preserve the indentation on exit
-             (when (and (looking-at-p "^[[:space:]]*$")
-                        (not org-src-preserve-indentation))
-               (let (block-content-ind some-ind)
-                 (org-with-point-at (org-element-begin element)
-                   (setq block-content-ind (+ (org-current-text-indentation)
-                                              org-edit-src-content-indentation))
-                   (forward-line)
-		   (save-match-data (re-search-forward "^[ \t]*\\S-" nil t))
-                   (backward-char)
-                   (setq some-ind (if (looking-at-p "#\\+end_src")
-                                      block-content-ind (org-current-text-indentation))))
-                 (indent-line-to (min block-content-ind some-ind))))
-	     (org-babel-do-key-sequence-in-edit-buffer (kbd "TAB")))
+             (let ((block-content-ind
+                    (when (not org-src-preserve-indentation)
+                      (org-with-point-at (org-element-property :begin element)
+                        (+ (org-current-text-indentation)
+                           org-edit-src-content-indentation)))))
+               (org-babel-do-key-sequence-in-edit-buffer (kbd "TAB"))
+               (when (and block-content-ind (looking-at-p "^$"))
+                 (indent-line-to block-content-ind))))
 	    (t
 	     (let ((column (org--get-expected-indentation element nil)))
 	       ;; Preserve current column.
diff --git a/testing/lisp/test-org-src.el b/testing/lisp/test-org-src.el
index e31fcb946..a634d85e8 100644
--- a/testing/lisp/test-org-src.el
+++ b/testing/lisp/test-org-src.el
@@ -345,11 +345,11 @@ This is a tab:\t.
 	(insert " Foo")
 	(org-edit-src-exit)
 	(buffer-string)))))
-  ;; Global indentation obeys `indent-tabs-mode' from the original
-  ;; buffer.
-  (should
+  ;; Global indentation does not obey `indent-tabs-mode' from the
+  ;; original buffer.
+  (should-not
    (string-match-p
-    "^\t+\s*argument2"
+    "\t"
     (org-test-with-temp-text
 	"
 - Item
@@ -364,14 +364,15 @@ This is a tab:\t.
 	(org-edit-special)
 	(org-edit-src-exit)
 	(buffer-string)))))
+  ;; Tab character is preserved
   (should
    (string-match-p
-    "^\s+argument2"
+    "\targument2"
     (org-test-with-temp-text
 	"
 - Item
   #+BEGIN_SRC emacs-lisp<point>
-    (progn\n      (function argument1\n\t\targument2))
+    (progn\n      (function argument1\n    \targument2))
   #+END_SRC"
       (setq-local indent-tabs-mode nil)
       (let ((org-edit-src-content-indentation 2)
@@ -379,43 +380,59 @@ This is a tab:\t.
 	(org-edit-special)
 	(org-edit-src-exit)
 	(buffer-string)))))
-  ;; Global indentation also obeys `tab-width' from original buffer.
+  ;; Indentation does not obey `tab-width' from org buffer.
   (should
    (string-match-p
-    "^\t\\{3\\}\s\\{2\\}argument2"
+    "^  \targument2"
     (org-test-with-temp-text
 	"
-- Item
-  #+BEGIN_SRC emacs-lisp<point>
+#+BEGIN_SRC emacs-lisp
   (progn
-    (function argument1
-              argument2))
-  #+END_SRC"
+    (list argument1\n  \t<point>argument2))
+#+END_SRC"
       (setq-local indent-tabs-mode t)
       (setq-local tab-width 4)
-      (let ((org-edit-src-content-indentation 0)
+      (let ((org-edit-src-content-indentation 2)
 	    (org-src-preserve-indentation nil))
 	(org-edit-special)
+        (setq-local indent-tabs-mode t)
+        (setq-local tab-width 8)
+        (lisp-indent-line)
 	(org-edit-src-exit)
 	(buffer-string)))))
+  ;; Tab characters are displayed with `tab-width' from the native
+  ;; edit buffer.
   (should
-   (string-match-p
-    "^\t\s\\{6\\}argument2"
+   (equal
+    10
     (org-test-with-temp-text
-	"
-- Item
-  #+BEGIN_SRC emacs-lisp<point>
+     "
+#+BEGIN_SRC emacs-lisp
   (progn
-    (function argument1
-              argument2))
-  #+END_SRC"
-      (setq-local indent-tabs-mode t)
-      (setq-local tab-width 8)
-      (let ((org-edit-src-content-indentation 0)
-	    (org-src-preserve-indentation nil))
-	(org-edit-special)
-	(org-edit-src-exit)
-	(buffer-string))))))
+    (list argument1\n  \t<point>argument2))
+#+END_SRC"
+     (setq-local indent-tabs-mode t)
+     (setq-local tab-width 4)
+     (let ((org-edit-src-content-indentation 2)
+	   (org-src-preserve-indentation nil))
+       (font-lock-ensure)
+       (current-column)))))
+  ;; The initial tab characters respect org's `tab-width'.
+  (should
+   (equal
+    10
+    (org-test-with-temp-text
+     "
+#+BEGIN_SRC emacs-lisp
+\t(progn
+\t  (list argument1\n\t\t<point>argument2))
+#+END_SRC"
+     (setq-local indent-tabs-mode t)
+     (setq-local tab-width 2)
+     (let ((org-edit-src-content-indentation 2)
+	   (org-src-preserve-indentation nil))
+       (font-lock-ensure)
+       (current-column))))))
 
 (ert-deftest test-org-src/indented-latex-fragments ()
   "Test editing multiline indented LaTeX fragment."
-- 
2.41.0

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> Ihor Radchenko writes:
>> May you now rebase the patch onto the latest main, add a Link: to this
>> discussion to the commit message, and apply the attached extra comments?
>
> Here it is. Thanks for helping with this.

Applied, onto main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=2e2ed4055
The original bug can no longer be reproduced.
Fixed.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> Applied, onto main.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=2e2ed4055

Sebastien, it looks like one of the tests is failing on the older Emacs:
https://builds.sr.ht/~bzg/job/1020247

Most likely, because `current-column' did not take into account 'display
property until recently.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 463 bytes --]


Ihor Radchenko writes:
> Sebastien, it looks like one of the tests is failing on the older Emacs:
> https://builds.sr.ht/~bzg/job/1020247

Does this specify anywhere what version of emacs it is using ?

> Most likely, because `current-column' did not take into account 'display
> property until recently.

Here's a workaround. I think I found what emacs commit makes it work,
but I haven't been able to test it with an older emacs version.

-- 
Sébastien Miquel

[-- Attachment #2: 0001-test-org-src.el-Work-around-current-column-bug-in-ol.patch --]
[-- Type: text/x-patch, Size: 1697 bytes --]

From 81b33f16ad2e2b2cff20110ff1dafefbc348f9dc Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Fri, 7 Jul 2023 13:58:17 +0200
Subject: [PATCH] test-org-src.el: Work around `current-column' bug in older
 emacs

* testing/lisp/test-org-src.el (test-org-src/indented-blocks): Work
around `current-column' not working in the presence of display strings
in older emacs.
---
 testing/lisp/test-org-src.el | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/testing/lisp/test-org-src.el b/testing/lisp/test-org-src.el
index a634d85e8..5a3af8bb3 100644
--- a/testing/lisp/test-org-src.el
+++ b/testing/lisp/test-org-src.el
@@ -416,7 +416,11 @@ This is a tab:\t.
      (let ((org-edit-src-content-indentation 2)
 	   (org-src-preserve-indentation nil))
        (font-lock-ensure)
-       (current-column)))))
+       ;; Replacement for `current-column', since it doesn't work with
+       ;; older versions of emacs before commit 4243747b1b8: Fix
+       ;; 'current-column' in the presence of display strings
+       (+ (progn (backward-char) (length (get-text-property (point) 'display)))
+          (current-column))))))
   ;; The initial tab characters respect org's `tab-width'.
   (should
    (equal
@@ -432,7 +436,8 @@ This is a tab:\t.
      (let ((org-edit-src-content-indentation 2)
 	   (org-src-preserve-indentation nil))
        (font-lock-ensure)
-       (current-column))))))
+       (+ (progn (backward-char) (length (get-text-property (point) 'display)))
+          (current-column)))))))
 
 (ert-deftest test-org-src/indented-latex-fragments ()
   "Test editing multiline indented LaTeX fragment."
-- 
2.41.0

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> Ihor Radchenko writes:
>> Sebastien, it looks like one of the tests is failing on the older Emacs:
>> https://builds.sr.ht/~bzg/job/1020247
>
> Does this specify anywhere what version of emacs it is using ?

Yup. In the full log https://builds.sr.ht/query/log/1020247/build/log
It is Emacs 28.2. Older Emacs versions are also failing.

>> Most likely, because `current-column' did not take into account 'display
>> property until recently.
>
> Here's a workaround. I think I found what emacs commit makes it work,
> but I haven't been able to test it with an older emacs version.

We should probably reserve the workaround to Emacs 28 and older and
eventually remove it when Org drops Emacs 28 support.

> Subject: [PATCH] test-org-src.el: Work around `current-column' bug in older
>  emacs
>
> * testing/lisp/test-org-src.el (test-org-src/indented-blocks): Work
> around `current-column' not working in the presence of display strings
> in older emacs.

I tested using Emacs 28 and 27 and your patch is passing all the tests.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 262 bytes --]


Ihor Radchenko writes:
> We should probably reserve the workaround to Emacs 28 and older and
> eventually remove it when Org drops Emacs 28 support.

Ok.

> I tested using Emacs 28 and 27 and your patch is passing all the tests.

Thanks.

-- 
Sébastien Miquel

[-- Attachment #2: 0001-test-org-src.el-Work-around-current-column-bug-in-ol.patch --]
[-- Type: text/x-patch, Size: 1832 bytes --]

From 6ea37a3041fb3266e94af0bfce29aa76f6c4439d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Fri, 7 Jul 2023 13:58:17 +0200
Subject: [PATCH] test-org-src.el: Work around `current-column' bug in older
 emacs

* testing/lisp/test-org-src.el (test-org-src/indented-blocks): Work
around `current-column' not working in the presence of display strings
in older emacs.
---
 testing/lisp/test-org-src.el | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/testing/lisp/test-org-src.el b/testing/lisp/test-org-src.el
index a634d85e8..ebf8d8569 100644
--- a/testing/lisp/test-org-src.el
+++ b/testing/lisp/test-org-src.el
@@ -416,7 +416,13 @@ This is a tab:\t.
      (let ((org-edit-src-content-indentation 2)
 	   (org-src-preserve-indentation nil))
        (font-lock-ensure)
-       (current-column)))))
+       ;;  `current-column' will not work with older versions of emacs
+       ;; before commit 4243747b1b8: Fix 'current-column' in the
+       ;; presence of display strings
+       (if (<= emacs-major-version 28)
+           (+ (progn (backward-char) (length (get-text-property (point) 'display)))
+              (current-column))
+         (current-column))))))
   ;; The initial tab characters respect org's `tab-width'.
   (should
    (equal
@@ -432,7 +438,10 @@ This is a tab:\t.
      (let ((org-edit-src-content-indentation 2)
 	   (org-src-preserve-indentation nil))
        (font-lock-ensure)
-       (current-column))))))
+       (if (<= emacs-major-version 28)
+           (+ (progn (backward-char) (length (get-text-property (point) 'display)))
+              (current-column))
+         (current-column)))))))
 
 (ert-deftest test-org-src/indented-latex-fragments ()
   "Test editing multiline indented LaTeX fragment."
-- 
2.41.0

Re: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)]

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> Subject: [PATCH] test-org-src.el: Work around `current-column' bug in older
>  emacs
>
> * testing/lisp/test-org-src.el (test-org-src/indented-blocks): Work
> around `current-column' not working in the presence of display strings
> in older emacs.

Applied, onto main. Thanks!
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=67e819d6e

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[BUG] org-list-struct-apply-struct overrides src block indentation (was: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)])

[Ihor Radchenko]

Using a similar example, another indentation bug has been revealed:

1. emacs -Q
2. Create an Org file with the following contents:

- 1
  - 2
    - 3
      - 4
- 5
  #+begin_src yaml
    a:
      b:
        c:
          d:
  #+end_src

Everything indented using spaces, including "d:" that uses 8 spaces,
which should remain spaces since yaml-mode uses spaces for indentation.

3. M-x whitespace-mode
4. Move point to item 5 and press M-<right> 3 times
5. Observe "d:" indented using two tabs with all the spaces replaced by tabs.

The cause is the following line in `org-list-struct-apply-struct'
[[file:~/Git/org-mode/lisp/org-list.el::indent-line-to (+ (org-current-text-indentation) delta)))]]

It calls `indent-line-to' that replaces spaces with tabs according to
current value of indent-tabs-mode in Org buffer.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] org-list-struct-apply-struct overrides src block indentation (was: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)])

[Sébastien Miquel]

Ihor Radchenko writes:
> The cause is the following line in `org-list-struct-apply-struct'
> [[file:~/Git/org-mode/lisp/org-list.el::indent-line-to  (+ (org-current-text-indentation) delta)))]]
>
> It calls `indent-line-to' that replaces spaces with tabs according to
> current value of indent-tabs-mode in Org buffer.

There are a few other instances of `indent-line-to` in the code. I
guess the right fix is to un-obsolete `org-indent-line-to`, use it and
make a special case if point is in a src block. This use case is
uncommon and not really compatible with `org-src-preserve-indentation`
though.

Somewhat more common possibility: say one has a src block at 0
indentation, and wants to make it part of an org list. Is there any
proper org way to do this ? I can use `indent-rigidly`, but again,
this might break an org-src indentation. No easy fix to this beside
providing a simple org version of `indent-rigidly`.

The issues above do not seem too bad. They are uncommon, and an
indent-region call should fix the indentation.

-- 
Sébastien Miquel

Re: [BUG] org-list-struct-apply-struct overrides src block indentation (was: [BUG] Source block indentation does not work properly for yaml-mode [9.6.6 ( @ /home/user/.emacs.d/elpa/org-9.6.6/)])

[Ihor Radchenko]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> Ihor Radchenko writes:
>> The cause is the following line in `org-list-struct-apply-struct'
>> [[file:~/Git/org-mode/lisp/org-list.el::indent-line-to  (+ (org-current-text-indentation) delta)))]]
>>
>> It calls `indent-line-to' that replaces spaces with tabs according to
>> current value of indent-tabs-mode in Org buffer.
>
> There are a few other instances of `indent-line-to` in the code. I
> guess the right fix is to un-obsolete `org-indent-line-to`, use it and
> make a special case if point is in a src block.

Yup. Sounds reasonable. Although we should not use it blindly; just
where necessary. Checking for element at point will create overheads.

> This use case is
> uncommon and not really compatible with `org-src-preserve-indentation`
> though.

May you please elaborate?

> Somewhat more common possibility: say one has a src block at 0
> indentation, and wants to make it part of an org list. Is there any
> proper org way to do this ? I can use `indent-rigidly`, but again,
> this might break an org-src indentation. No easy fix to this beside
> providing a simple org version of `indent-rigidly`.

I recall struggling with it and ending up with adding spaces manually to
the first src line and then using indent-region.

`indent-line-function' technically allows force indent, if
`tab-always-indent' is set to t, but we have <TAB> bound to org-cycle,
which will take precedence when at #+begin_src.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>