[BUG] Inconsistent org-babel tangle behaviour

[BUG] Inconsistent org-babel tangle behaviour

[Thibaut Meyer via General discussions about Org-mode.]

Hi everyone,

setting a property in an org file like

'#+PROPERTY: header-args    :tangle "file.ext"'

with a source block of the form:
(note no language is specified for the block)

#+begin_src
<1: this block won't get tangled>
#+end_src

won't tangle the block

Adding any string as a language to the code block, like

#+begin_src random_string
<2: this block will get tangled>
#+end_src

tangles the code block

what's weirder is, just adding to a block the ":tangle" header without
or with any value, even with the value no, makes the tangling process
respect the header-args property value and thus the block will tangle
successfully.
(note no language is specified for the block)

#+begin_src :tangle no
<3: this block will get tangled!!>
#+end_src

Adding again a language value make the tangle header act normally

#+begin_src C :tangle no
<4: this block won't get tangled>
#+end_src

The behaviour doesn't seem super consistent. Ideally, I think having a
source block like the first one gettin tangled would be good, and having
the third one not tangle would be better as well.

The documentation states:
[Working with source code > Structure of Code Blocks]
"When ‘<language>’ identifier is omitted, the block also cannot have
‘<switches>’ and ‘<header arguments>’."

but, to my knowledge, there is no mention regarding the fact that the
headers assigned through the "header-args" property should not be
applied if a block has no language identifier.

I tested using org-mode pulled from the repo
"GNU Emacs 29.4 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.43, cairo version 1.18.2)"
"Org mode version 9.8-pre (release_9.7.16-174-gb9637e @ /home/thibaut/tmp/org-mode/lisp/)"

Cheers

Re: [BUG] Inconsistent org-babel tangle behaviour

[Ihor Radchenko]

Thibaut Meyer via "General discussions about Org-mode."
<emacs-orgmode@gnu.org> writes:

> setting a property in an org file like
>
> '#+PROPERTY: header-args    :tangle "file.ext"'
>
> with a source block of the form:
> (note no language is specified for the block)
>
> #+begin_src
> <1: this block won't get tangled>
> #+end_src
>
> won't tangle the block

Currently, source blocks without language are never tangled.

> what's weirder is, just adding to a block the ":tangle" header without
> or with any value, even with the value no, makes the tangling process
> respect the header-args property value and thus the block will tangle
> successfully.
> (note no language is specified for the block)

> #+begin_src :tangle no
> <3: this block will get tangled!!>
> #+end_src

Here, a language is actually specified. The language is ":tangle" (there
is nothing stopping anyone from creating a language name starting from
colon).

> The behaviour doesn't seem super consistent. Ideally, I think having a
> source block like the first one gettin tangled would be good, and having
> the third one not tangle would be better as well.

Src block without language could indeed be tangled in your case.
However, if the :tangle value is set to something like "yes", there is
no way or can deduce the tangle file name.

Note that M-x org-lint warns about src blocks without language. While
src blocks without language are allowed syntactically, many Org features
will not work with such blocks.

Usually, you can simply use "conf" or "text" language name as a
placeholder.

I guess we can support the scenario you described, but I am not sure if
we should.

> The documentation states:
> [Working with source code > Structure of Code Blocks]
> "When ‘<language>’ identifier is omitted, the block also cannot have
> ‘<switches>’ and ‘<header arguments>’."
>
> but, to my knowledge, there is no mention regarding the fact that the
> headers assigned through the "header-args" property should not be
> applied if a block has no language identifier.

That part of the manual can be improved. Maybe something like below.

  When =<language>= identifier is omitted, the block also should not
  have =<switches>= and =<header arguments>=.  Otherwise, the first
  switch/argument will be treated as =<language>=.

Let me know if it clarifies things.

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Inconsistent org-babel tangle behaviour

[Thibaut Meyer via General discussions about Org-mode.]

> Here, a language is actually specified. The language is ":tangle" (there
> is nothing stopping anyone from creating a language name starting from
> colon).

I see. I just assumed that header arguments and switches were
"recognized" and not considered as the language if placed positionally
first.

> Src block without language could indeed be tangled in your case.

That would be indeed nice...

> However, if the :tangle value is set to something like "yes", there is
> no way or can deduce the tangle file name.

.. but I guess that this prevents us from supporting this use case.

> Usually, you can simply use "conf" or "text" language name as a
> placeholder.

It's true that once you know the behaviour, it's not that big of a deal to
do so.

>> The documentation states:
>> [Working with source code > Structure of Code Blocks]
>> "When ‘<language>’ identifier is omitted, the block also cannot have
>> ‘<switches>’ and ‘<header arguments>’."
>>
>> but, to my knowledge, there is no mention regarding the fact that the
>> headers assigned through the "header-args" property should not be
>> applied if a block has no language identifier.
>
> That part of the manual can be improved. Maybe something like below.
>
>   When =<language>= identifier is omitted, the block also should not
>   have =<switches>= and =<header arguments>=.  Otherwise, the first
>   switch/argument will be treated as =<language>=.
>
> Let me know if it clarifies things.

It does clarify things. In addition, I think that a small mention of
that "quirk" in the "Extracting Source Code" part of the manual would
also be in order. I'll send a patch shortly.

Thibaut

Re: [BUG] Inconsistent org-babel tangle behaviour

[Ihor Radchenko]

Thibaut Meyer <thibaut@thibaut.dev> writes:

> I see. I just assumed that header arguments and switches were
> "recognized" and not considered as the language if placed positionally
> first.
>
>> Src block without language could indeed be tangled in your case.
>
> That would be indeed nice...
>
>> However, if the :tangle value is set to something like "yes", there is
>> no way or can deduce the tangle file name.
>
> .. but I guess that this prevents us from supporting this use case.

Not necessarily.
The reason why src blocks without language are not tangled is simply
because of the way I fixed
https://list.orgmode.org/orgmode/AS8P193MB241369D9F0701F57E293429288D3A@AS8P193MB2413.EURP193.PROD.OUTLOOK.COM/T/#u

I think that `org-babel-tangle-collect-blocks' can be changed to allow
tangling src blocks without language with explicitly provided tangle
target.

It might be a good exercise if you want to submit a patch.

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[PATCH] ob-tangle: source blocks without language can be tangled (was: [BUG] Inconsistent org-babel tangle behaviour)

[Thibaut Meyer via General discussions about Org-mode.]

* lisp/ob-tangle.el: (org-babel-tangle-collect-blocks): source blocks
without a language are not not ignored anymore, if they inherit a
src-tfile other than "yes" or "no".
* lisp/ob-core.el: adapts the org-babel-src-block-regexp regular
expression to make the language part of a source block optional.
* testing/lisp/test-ob-tangle.el: (ob-tangle/collect-blocks): Augment
the test with source blocks without language specifiers.
* etc/ORG-NEWS: (=ob-tangle= now tangles source blocks that do not
specify a =language= if an inherited property sets a tangle filename):
Document the new behaviour.
* doc/org-manual.org: (Structure of Code Blocks) Clarify what happens
when the language identifier is omitted.
---
Here is my attempt. Modifying the regex in ob-core.el felt scary, but
it seems fine since this didn't break any tests and manually testing the
new regex seemed to work fine. I've added the small change to the
documentation as you suggested as I think it clarifies things regarding
how the source block structure is interpreted.
Cheers.

 doc/org-manual.org             |  3 ++-
 etc/ORG-NEWS                   |  8 ++++++++
 lisp/ob-core.el                |  4 ++--
 lisp/ob-tangle.el              |  8 ++++++--
 testing/lisp/test-ob-tangle.el | 16 ++++++++++++----
 5 files changed, 30 insertions(+), 9 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 2d5fd57b3..f0c53265a 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -18028,7 +18028,8 @@ or
   block.  See [[*Languages]], for identifiers of supported languages.
 
   When =<language>= identifier is omitted, the block also cannot
-  have =<switches>= and =<header arguments>=.
+  have =<switches>= and =<header arguments>=.  Otherwise, the first
+  switch/argument will be treated as =<language>=.
 
   Language identifier is also used to fontify code blocks in Org
   buffers, when ~org-src-fontify-natively~ is set to non-~nil~.  See
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 2d4616fab..4aa628c7e 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -437,6 +437,14 @@ For example, given =H:3= and =toc:2= in =#+OPTIONS:=, all headings at
 the 1st and 2nd level appear in the table of contents and those at the
 3rd level do not.
 
+*** =ob-tangle= now tangles source blocks that do not specify a =language= if an inherited property sets a tangle filename
+
+Previously, all source blocks that did not specify a =language= where
+ignored by ~org-babel-tangle-collect-blocks~. Now, if it inherits a
+:tangle header argument with a value other than =no= or =yes= (that is, a
+filename), a source block without =language= will get tangled to that
+file.
+
 * Version 9.7
 
 ** Important announcements and breaking changes
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index ff021cbd3..cf56367f8 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -216,8 +216,8 @@ When matching, reference is stored in match group 1."
 
 (defvar org-babel-src-block-regexp
   (concat
-   ;; (1) indentation                 (2) lang
-   "^\\([ \t]*\\)#\\+begin_src[ \t]+\\([^ \f\t\n\r\v]+\\)[ \t]*"
+   ;; (1) indentation                      (2) lang
+   "^\\([ \t]*\\)#\\+begin_src\\(?:[ \t]+\\([^ \f\t\n\r\v]*\\)\\)?[ \t]*"
    ;; (3) switches
    "\\([^\":\n]*\"[^\"\n*]*\"[^\":\n]*\\|[^\":\n]*\\)"
    ;; (4) header arguments
diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index 044212da2..d37d3c67f 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -508,9 +508,13 @@ code blocks by target file."
 	       (src-lang (nth 0 info))
 	       (src-tfile (cdr (assq :tangle (nth 2 info)))))
 	  (unless (or (string= src-tfile "no")
-                      (not src-lang) ;; src block without lang
+                      ;; src block without lang
+                      (and (not src-lang) (string= src-tfile "yes"))
 		      (and tangle-file (not (equal tangle-file src-tfile)))
-		      (and lang-re (not (string-match-p lang-re src-lang))))
+                      ;; lang-re but either no lang or lang doesn't match
+		      (and lang-re
+                           (or (not src-lang)
+                               (not (string-match-p lang-re src-lang)))))
 	    ;; Add the spec for this block to blocks under its tangled
 	    ;; file name.
 	    (let* ((block (org-babel-tangle-single-block counter))
diff --git a/testing/lisp/test-ob-tangle.el b/testing/lisp/test-ob-tangle.el
index e13bca0cb..6e95c4c91 100644
--- a/testing/lisp/test-ob-tangle.el
+++ b/testing/lisp/test-ob-tangle.el
@@ -628,6 +628,10 @@ another block
 \"H1: :tangle ~/../../tmp/absolute.el\"
 #+end_src
 
+#+begin_src
+\"H1: no language and inherited :tangle relative.el in properties\"
+#+end_src
+
 * H2 without :tangle in properties
 
 #+begin_src emacs-lisp
@@ -660,6 +664,10 @@ another block
 
 #+begin_src emacs-lisp :tangle ~/../../tmp/absolute.el
 \"H2: :tangle ~/../../tmp/absolute.el\"
+#+end_src
+
+#+begin_src
+\"H2: without language and thus without :tangle\"
 #+end_src"
                     `((?a . ,el-file-abs)
                       (?r . ,el-file-rel))))
@@ -684,7 +692,7 @@ another block
                                               collected-blocks)))))
         (should (equal (funcall normalize-expected-targets-alist
                                 `(("/tmp/absolute.el" . 4)
-                                  ("relative.el" . 5)
+                                  ("relative.el" . 6)
                                   ;; file name differs between tests
                                   (,el-file-abs . 4)))
                        (funcall count-blocks-in-target-files
@@ -699,11 +707,11 @@ another block
           (should (equal
                    (funcall normalize-expected-targets-alist
                             `(("/tmp/absolute.el" . 4)
-                              ("relative.el" . 5)
+                              ("relative.el" . 6)
                               ;; Default :tangle header now also
                               ;; points to the file name derived from the name of
-                              ;; the Org file, so 5 blocks should go there.
-                              (,el-file-abs . 5)))
+                              ;; the Org file, so 6 blocks should go there.
+                              (,el-file-abs . 6)))
                    (funcall count-blocks-in-target-files
                             (org-babel-tangle-collect-blocks)))))))))
 
-- 
2.47.1