bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič via Bug reports for GNU Emacs, the Swiss army knife of text editors]

I have a .org file with two kinds of src blocks:

1. sqlite blocks
2. bibtex blocks

I want to execute all sqlite blocks with org-babel-execute-buffer.

When I try to do so, org-mode complains:

org-babel-execute-src-block: No org-babel-execute function for bibtex!

I know I can work around the problem (as one does in org-mode) with:

#+property: header-args:bibtex+ :eval no

Then, why not skip the irrelevant blocks, the ones with no execute functions, by default?

Rudy


In GNU Emacs 29.0.50 (build 10, x86_64-apple-darwin21.1.0, NS appkit-2113.00 Version 12.0.1 (Build 21A559))
 of 2021-12-15 built on Workstation.local
Repository revision: 5a057f6716e6991a60968e48f849da57a0339735
Repository branch: master
Windowing system distributor 'Apple', version 10.3.2113
System Description:  macOS 12.1

Configured using:
 'configure --with-json --with-xwidgets --with-native-compilation'

Configured features:
ACL DBUS GIF GLIB GMP GNUTLS JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY KQUEUE NS PDUMPER PNG RSVG SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS WEBP XIM XWIDGETS ZLIB

Important settings:
  value of $LC_ALL: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Org

Minor modes in effect:
  shell-dirtrack-mode: t
  TeX-PDF-mode: t
  flymake-mode: t
  display-fill-column-indicator-mode: t
  corfu-global-mode: t
  corfu-mode: t
  vertico-mode: t
  marginalia-mode: t
  global-diff-hl-mode: t
  diff-hl-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  global-hl-todo-mode: t
  global-subword-mode: t
  subword-mode: t
  save-place-mode: t
  global-auto-revert-mode: t
  delete-selection-mode: t
  savehist-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/Users/salutis/.emacs.d/elpa/transient-20211208.1819/transient hides /Users/salutis/src/emacs/nextstep/Emacs.app/Contents/Resources/lisp/transient
/Users/salutis/src/emacs/nextstep/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-compat hides /Users/salutis/src/emacs/nextstep/Emacs.app/Contents/Resources/lisp/obsolete/eieio-compat

Features:
(shadow sort bbdb-message fortune mail-extr emacsbug sendmail tramp-cmds
goto-addr mhtml-mode css-mode smie js sgml-mode facemenu htmlize preview
tex-buf font-latex tex-mode pulse color consult-xref xref shortdoc
ob-ditaa ob-plantuml org-crypt org-habit ess-tracebug ess-utils
ess-custom org-colview org-plot ob-awk ob-calc calc-store calc-trail
calc-ext calc calc-loaddefs rect calc-macs ob-clojure ob-css ob-dot
ob-eshell eshell esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg
esh-module esh-groups esh-util ob-forth ob-fortran ob-gnuplot ob-groovy
ob-haskell ob-java ob-js ob-julia ob-latex ob-lilypond ob-lisp ob-lua
ob-makefile ob-maxima ob-ocaml ob-octave ob-org ob-perl ob-processing
ob-python python tramp-sh tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat ob-ruby ob-sass ob-scheme
ob-screen ob-sed ob-shell shell embark ffap helpful imenu trace edebug
backtrace info-look help-fns radix-tree elisp-refs sql latex
latex-flymake tex-ispell tex-style tex texmathp ox-md ox-odt rng-loc
rng-uri rng-parse rng-match rng-dt rng-util rng-pttrn nxml-parse nxml-ns
nxml-enc xmltok nxml-util ox-latex ox-icalendar ox-html table ox-ascii
ox-publish ox citar-org oc-csl citeproc citeproc-itemgetters
citeproc-biblatex citeproc-bibtex citeproc-cite citeproc-subbibs
citeproc-sort citeproc-name citeproc-formatters citeproc-number rst
citeproc-proc citeproc-disamb citeproc-itemdata
citeproc-generic-elements citeproc-macro citeproc-choose citeproc-date
citeproc-context citeproc-prange citeproc-style citeproc-locale
citeproc-term f citeproc-rt citeproc-lib citeproc-s let-alist queue dash
org-id citar crm s parsebib citar-file vc-mtn vc-hg vc-bzr vc-src
vc-sccs vc-svn vc-cvs vc-rcs consult-vertico consult recentf tree-widget
view org-pomodoro alert log4e notifications gntp org-timer org-agenda
org-refile org-duration bug-reference flymake-proc flymake project
compile image-file image-converter disp-table char-fold orderless
cursor-sensor paredit edmacro kmacro display-fill-column-indicator bbdb
bbdb-site timezone modus-vivendi-theme modus-operandi-theme modus-themes
corfu vertico marginalia pdf-loader diff-hl log-view pcvs-util vc-dir
ewoc vc diminish yasnippet hl-todo finder-inf ob-sqlite ob-sql ob-C
cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs ob-R org-clock cl ls-lisp cap-words superword subword
saveplace autorevert filenotify comp comp-cstr warnings delsel savehist
elfeed-link elfeed-show elfeed-search elfeed-csv elfeed elfeed-curl
elfeed-log xml-query bookmark pp elfeed-db elfeed-lib vc-git diff-mode
vc-dispatcher org-element avl-tree generator ol-eww eww xdg url-queue
thingatpt mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect
gnus-search eieio-opt speedbar ezimage dframe gnus-art mm-uu mml2015
mm-view mml-smime smime dig gnus-sum shr pixel-fill kinsoku svg dom
gnus-group gnus-undo gnus-start gnus-dbus dbus xml gnus-cloud nnimap
nnmail mail-source utf7 netrc nnoo parse-time gnus-spec gnus-int
gnus-range message yank-media rmc puny rfc822 mml mml-sec epa derived
epg rfc6068 epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader gnus-win gnus nnheader gnus-util rmail
rmail-loaddefs rfc2047 rfc2045 ietf-drums text-property-search
mail-utils mm-util mail-prsvr wid-edit ol-docview doc-view jka-compr
image-mode exif dired dired-loaddefs ol-bibtex ol-bbdb ol-w3m ol-doi
org-link-doi cl-extra help-mode org ob ob-tangle ob-ref ob-lob ob-table
ob-exp org-macro org-footnote org-src ob-comint org-pcomplete pcomplete
comint ansi-color ring org-list org-faces org-entities noutline outline
easy-mmode org-version ob-emacs-lisp ob-core ob-eval org-table oc-basic
bibtex iso8601 time-date ol rx org-keys oc org-compat advice org-macs
org-loaddefs format-spec find-func cal-menu calendar cal-loaddefs
tex-site info package browse-url url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map url-vars seq gv subr-x byte-opt
bytecomp byte-compile cconv cl-loaddefs cl-lib iso-transl tooltip eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget keymap hashtable-print-readable backquote threads
xwidget-internal dbusbind kqueue cocoa ns lcms2 multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 2797544 115059)
 (symbols 48 57535 17)
 (strings 32 307749 28260)
 (string-bytes 1 9616525)
 (vectors 16 123840)
 (vector-slots 8 4329893 151750)
 (floats 8 797 616)
 (intervals 56 15721 1087)
 (buffers 992 148))

-- 
"Programming reliably --- must be an activity of an undeniably mathematical nature […] You see, mathematics is about thinking, and doing mathematics is always trying to think as well as possible." -- Edsger W. Dijkstra (1981)

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Kyle Meyer]

Rudolf Adamkovič:

> I have a .org file with two kinds of src blocks:
>
> 1. sqlite blocks
> 2. bibtex blocks
>
> I want to execute all sqlite blocks with org-babel-execute-buffer.
>
> When I try to do so, org-mode complains:
>
> org-babel-execute-src-block: No org-babel-execute function for bibtex!
>
> I know I can work around the problem (as one does in org-mode) with:
>
> #+property: header-args:bibtex+ :eval no
>
> Then, why not skip the irrelevant blocks, the ones with no execute functions, by default?

I don't know, but I've reassigned this bug in order to redirect this to
the Org list, which serves as the primary place for Org development and
discussion.

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Berry, Charles]

> On Dec 16, 2021, at 8:51 PM, Kyle Meyer <kyle@kyleam.com> wrote:
> 
> Rudolf Adamkovič:
> 
>> I have a .org file with two kinds of src blocks:
>> 
>> 1. sqlite blocks
>> 2. bibtex blocks
>> 
>> I want to execute all sqlite blocks with org-babel-execute-buffer.
>> 
>> When I try to do so, org-mode complains:
>> 
>> org-babel-execute-src-block: No org-babel-execute function for bibtex!
>> 
>> I know I can work around the problem (as one does in org-mode) with:
>> 
>> #+property: header-args:bibtex+ :eval no
>> 
>> Then, why not skip the irrelevant blocks, the ones with no execute functions, by default?


If I have a typo in the name of a language, the error message you quote tells me what my mistake was. 

I'd say that is a feature, not a bug.

HTH,
Chuck

> 
> I don't know, but I've reassigned this bug in order to redirect this to
> the Org list, which serves as the primary place for Org development and
> discussion.
> 

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Ihor Radchenko]

"Berry, Charles" <ccberry@health.ucsd.edu> writes:

> If I have a typo in the name of a language, the error message you quote tells me what my mistake was. 
>
> I'd say that is a feature, not a bug.

Agree. However, some languages simply do not define babel execute
function. The error is same regardless whether a language backend is not
available/not loaded or the backend is loaded, but does not provide
babel execute function.

Maybe we can provide a custom list of languages where we do not throw
the error? If a language is in the list and there is no babel execute
function, we can simply ignore the source block. If a language is in the
list, but there is babel execute function, we throw another error.

Though I am not a big fan of introducing yet another customisation.
Maybe someone has better ideas?

Best,
Ihor

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Berry, Charles]

> On Dec 18, 2021, at 1:49 AM, Ihor Radchenko <yantar92@gmail.com> wrote:
> 
> "Berry, Charles" <ccberry@health.ucsd.edu> writes:
> 
>> If I have a typo in the name of a language, the error message you quote tells me what my mistake was. 
>> 
>> I'd say that is a feature, not a bug.
> 
> Agree. However, some languages simply do not define babel execute
> function. The error is same regardless whether a language backend is not
> available/not loaded or the backend is loaded, but does not provide
> babel execute function.
> 
> Maybe we can provide a custom list of languages where we do not throw
> the error? If a language is in the list and there is no babel execute
> function, we can simply ignore the source block. If a language is in the
> list, but there is babel execute function, we throw another error.
> 
> Though I am not a big fan of introducing yet another customisation.
> Maybe someone has better ideas?
> 

I believe there is a feeling that org-babel is already so complicated that adding features should be avoided. 

There are workable approaches under the current setup. 

---

For one, you can define lang specific header-args, so just define :eval no for the lang's that you want to use, but that do not have org-babel-execute:<lang> defined: 

#+property: header-args:nada :eval no
#+property: header-args :exports both :eval yes

The code here will be rendered, but the block will not execute:

#+begin_src nada
  "haha"
#+end_src

---

Another is to use a custom :eval arg like this:

#+property: header-args :eval (my-eval-p) :exports both

#+begin_src not_defined
  "haha"
#+end_src

Where `my-eval-p' is defined as:

#+begin_src emacs-lisp
  (defun my-eval-p ()
    (let ((lang (car (org-babel-get-src-block-info 'light))))
      (if (fboundp (intern (concat "org-babel-execute:" lang)))
	  "yes"
	"no")))
#+end_src

Obviously, you can elaborate `my-eval-p' to satisfy individual needs.

Best,
Chuck

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Berry, Charles]

> On Dec 18, 2021, at 11:57 AM, Charles Berry <ccberry@health.ucsd.edu> wrote:
> 
> There are workable approaches under the current setup. 
> 
> -


Also, when exporting it looks `org-babel-exp-results' does not attempt to run src blocks for which

(fboundp (intern (concat "org-babel-execute:" lang)))

is nil.

So doing an export should `just work'.

HTH,

Chuck

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič via General discussions about Org-mode.]

Ihor Radchenko <yantar92@gmail.com> writes:

> Though I am not a big fan of introducing yet another customisation.
> Maybe someone has better ideas?

I struggle to understand.  Why do we need a customization?  If Org knows
that some backend exists but has no execute function, why does it even
try to execute it?  It cannot.  Do I miss something?

Rudy
-- 
"Logic is a science of the necessary laws of thought, without which no employment of the understanding and the reason takes place." -- Immanuel Kant, 1785

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Berry, Charles]

> On Dec 21, 2021, at 2:53 PM, Rudolf Adamkovič <salutis@me.com> wrote:
> 
> I struggle to understand.  Why do we need a customization?  If Org knows
> that some backend exists but has no execute function, why does it even
> try to execute it?  It cannot.  Do I miss something?


Sorry if my prior posts were garbled. To clarify:

When exporting src blocks for which there is no execute function for the lang, the block will not be executed and the export will proceed.

When evaluating src blocks in other circumstances, e.g. M-x org-babel-execute-buffer, the process will terminate when org-babel-execute-src-block issues `(error "No org-babel-execute function for %s!" lang))'

The latter can be a clue that the name of a lang is misspelled or that org-babel-load-languages needs to be customized to provide the execute function.

IMO, nothing needs fixing. The ability to handle OP's original issue is already in place.

HTH,
Chuck

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

>> Though I am not a big fan of introducing yet another customisation.
>> Maybe someone has better ideas?
>
> I struggle to understand.  Why do we need a customization?  If Org knows
> that some backend exists but has no execute function, why does it even
> try to execute it?  It cannot.  Do I miss something?

There no notion of backend in Org babel. Babel backends are merely
defining a set of functions with special function symbols
(org-babel-*:lang_name). So, Org cannot distinguish between language
backends that are simply not loaded and the ones that do not define
org-babel-execute:lang.

Best,
Ihor

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič via General discussions about Org-mode.]

Ihor Radchenko <yantar92@gmail.com> writes:

> So, Org cannot distinguish between language backends that are simply
> not loaded and the ones that do not define org-babel-execute:lang.

Oh, if we have this architectural limitation in place, then Org cannot
help the user, and every user will have "explain" to Org that executing
BibTeX makes no sense.  I guess we can then close this bug then!

Rudy
-- 
"'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and
if it were so, it would be; but as it isn't, it ain't. That's logic.'"
-- Lewis Carroll, Through the Looking Glass

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

>> So, Org cannot distinguish between language backends that are simply
>> not loaded and the ones that do not define org-babel-execute:lang.
>
> Oh, if we have this architectural limitation in place, then Org cannot
> help the user, and every user will have "explain" to Org that executing
> BibTeX makes no sense.  I guess we can then close this bug then!

Not necessarily close. As I proposed above, one way is adding a user
customisation. And we do not have to force the users adding trivial
things like bibtex. Bibtex may be one of the defaults.

Best,
Ihor

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Berry, Charles]

> On Dec 23, 2021, at 8:09 PM, Ihor Radchenko <yantar92@gmail.com> wrote:
> 
> Rudolf Adamkovič <salutis@me.com> writes:
> 
>>> So, Org cannot distinguish between language backends that are simply
>>> not loaded and the ones that do not define org-babel-execute:lang.
>> 
>> Oh, if we have this architectural limitation in place, then Org cannot
>> help the user, and every user will have "explain" to Org that executing
>> BibTeX makes no sense.  I guess we can then close this bug then!
> 
> Not necessarily close. As I proposed above, one way is adding a user
> customisation. And we do not have to force the users adding trivial
> things like bibtex. Bibtex may be one of the defaults.

Honestly, I think there is no problem here to solve. 

If there is a use case that you think I didn't cover below, please provide an ECM and directions for triggering the unwanted behavior.

Details:

---

Exports do not eval blocks for which there is no org-babel-execute:<lang>.

So, `C-c C-e l L' (and so on) will *not eval* bibtex src blocks.

The problem cited here *only* exists for execution using org-babel-execute-buffer or similar functions.

For that case, setting buffer or heading properties, such as:

#+begin_src org
  :PROPERTIES:
  :header-args: :eval yes :exports results
  :header-args:bibtex: :eval no
  :END:
#+end_src

resolves the issue.

So the user needs to add just one-line per language to set this up.

Following (info "(org) Property Syntax") the user can set this behavior for all Org files by customize-ing `org-global-properties', whose docstring gives plentiful details.

---



HTH,
Chuck

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič via General discussions about Org-mode.]

"Berry, Charles" <ccberry@health.ucsd.edu> writes:

> The problem cited here *only* exists for execution using
> org-babel-execute-buffer or similar functions.
>
> For that case, setting buffer or heading properties, such as […]
> resolves the issue.
>
> So the user needs to add just one-line per language to set this up.

I think we look at the problem from two different perspectives.  You
look at the problem from the "how" perspective, whereas I look at it
from the "why" perspective.  Sure, we can work around everything in Org.
Now, does that mean that Org should try to execute BibTeX blocks when
executing a buffer?  I do not think so.

Rudy

-- 
"'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and
if it were so, it would be; but as it isn't, it ain't. That's logic.'"
-- Lewis Carroll, Through the Looking Glass

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Berry, Charles]

Rudy,

Thanks for the comment, but ...

> On Dec 25, 2021, at 1:37 PM, Rudolf Adamkovič <salutis@me.com> wrote:
> 
> I think we look at the problem from two different perspectives.  You
> look at the problem from the "how" perspective, whereas I look at it
> from the "why" perspective.  Sure, we can work around everything in Org.
> Now, does that mean that Org should try to execute BibTeX blocks when
> executing a buffer?  I do not think so.

Using standard features (properties, header-args) in a standard way does not strike me as a `work around'.

You have not made a clear case that src blocks are even needed to support your use case. It may well be that export blocks (which never execute) would work just as well.  But with no ECM, one can only guess.

I'm still inclined to call this a feature request, not a bug. And decisions about adding complexity to an already complicated code base should take that point of view, IMO.

Best,

Chuck

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič via General discussions about Org-mode.]

"Berry, Charles" <ccberry@health.ucsd.edu> writes:

> Rudy,
>
> Thanks for the comment, but ...
>
>> On Dec 25, 2021, at 1:37 PM, Rudolf Adamkovič <salutis@me.com> wrote:
>> 
>> I think we look at the problem from two different perspectives.  You
>> look at the problem from the "how" perspective, whereas I look at it
>> from the "why" perspective.  Sure, we can work around everything in
>> Org.  Now, does that mean that Org should try to execute BibTeX
>> blocks when executing a buffer?  I do not think so.
>
> Using standard features (properties, header-args) in a standard way
> does not strike me as a `work around'.

I apologize for the lack of clarity.  I used the term "workaround" from
the perspective of a user, who probably does not expect Org to try
"execute BibTeX" ever.

> You have not made a clear case that src blocks are even needed to
> support your use case. It may well be that export blocks (which never
> execute) would work just as well.  But with no ECM, one can only
> guess.

True that, I did not include an example!  Below, I include a typical use
case: Org notes about an article with a corresponding BibTeX entry.

(Note: I do not know what ECM stands for.)

[…]

#+property: header-args+ :comments link
#+cite_export: csl apa.csl

* Notes

[…]

* References

#+print_bibliography:

* Sources

#+begin_src bibtex :tangle knuth+2021.bib
@Article{knuth+2021,
  author       = {Knuth, Donald E. and Shustek, Len},
  title        = {Let's Not Dumb down the History of Computer Science},
  journal      = {Communications of the {ACM}},
  year         = 2021,
  volume       = 64,
  number       = 2,
  pages        = {33–35},
  month        = {jan},
  issue_date   = {February 2021},
  publisher    = {Association for Computing Machinery, {ACM}},
  issn         = {0001-0782},
  doi          = {10.1145/3442377},
  abstract     = {Donald Knuth on the best way to recognize the history
                  of computer science.},
  numpages     = 3
}
#+end_src

> I'm still inclined to call this a feature request, not a bug. And
> decisions about adding complexity to an already complicated code base
> should take that point of view, IMO.

Sure.  As a user, I can see it as both a bug, where Org tries to execute
BibTeX, but also as a feature, where Org needs to become smarter and
differentiate BibTeX from, say, Scheme or C.

Rudy
-- 
"Logic is a science of the necessary laws of thought, without which no
employment of the understanding and the reason takes place." -- Immanuel
Kant, 1785

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič via General discussions about Org-mode.]

Rudolf Adamkovič via "General discussions about Org-mode."
<emacs-orgmode@gnu.org> writes:

> Below, I include a typical use case: Org notes about an article with a
> corresponding BibTeX entry.

Please, ignore the example I gave, as it does not show anything.  I
apologize it.  Below, I provide a good example, one that made me stumble
upon the problem.

I have a paper that includes BibTeX blocks for references and SQLite
blocks to compute results.  I expect Org to execute all SQLite without
aborting to inform me that it could not execute BibTeX. Further, I
expect Org to it without me instructing it, for executing BibTeX does
not make sense.

#+title: …
#+author: Rudolf Adamkovič
#+email: salutis@me.com
#+property: header-args+ :comments link
#+cite_export: csl apa.csl
#+property: header-args:bibtex+ :exports none
#+property: header-args:bibtex+ :eval yes
#+options: toc:nil
#+latex_header: \usepackage{setspace}
#+latex_header: \doublespacing

* Abstract […]

* Introduction

[…] [cite/text:@beall-2019] explains […]

#+begin_src bibtex :tangle ~/bib/beall-2019.bib
[…]
#+end_src

[… 10 more BibTeX blocks used for the literature review]

* Methods […]

* Results

[…]

#+begin_src sqlite :db ~/tmp/test.db :exports none
[…]
#+end_src

[… 10 more SQLite blocks used to compute the results …]

[…]

Rudy
-- 
"'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and
if it were so, it would be; but as it isn't, it ain't. That's logic.'"
-- Lewis Carroll, Through the Looking Glass

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Max Nikulin]

On 30/12/2021 15:00, Rudolf Adamkovič wrote:
> 
> #+property: header-args:bibtex+ :eval yes

Why do you set ":eval yes" explicitly for bibtex if you believe that it 
should not be executed?

I suppose, the following may be considered as a complete example 
suitable to compare behavior of `org-babel-execute-buffer' and export:

---- >8 ----
#+property: header-args:bibtex+ :exports none

Test file with =bibtex= (to tangle)
and =elisp= (to execute) code blocks.

#+begin_src bibtex :tangle yes
   @misc{ex1,
     title = "Example"
   }
#+end_src

#+begin_src elisp
   (message "From elisp")
#+end_src
---- 8< ----

To solve the problem avoiding per-file configuration you can try

    (setq org-babel-default-header-args:bibtex '((:eval . "no")))

Unsure if it should be default. Since LaTeX export is important part of 
Org it is reasonable to expect that Org have some notion of BibTeX 
allowing to avoid execution of such source blocks.

Currently there is no BibTeX babel backend, but some user might create 
one that e.g. formats entries as some alternative to org-cite. Can 
default header arguments cause a problem in such case?

Should some function a macro be provided to facilitate declaring 
languages as data format (config files, JSON, YAML, etc.) rather than 
executable source code?

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Berry, Charles]

Max,

> On Dec 31, 2021, at 4:05 AM, Max Nikulin <manikulin@gmail.com> wrote:
> 
> 
> Should some function a macro be provided to facilitate declaring languages as data format (config files, JSON, YAML, etc.) rather than executable source code?

I think we already have this in the form of export blocks, viz.

#+name: yaml_header_1
#+begin_export yaml
output:
  html_document:
    toc: true
    toc_depth: 2
#+end_export

will be ignored by babel and by exporters that do not define a transcoder for `yaml' blocks.

The data can be used in transcoders for derived backends or in src blocks like this one:

#+begin_src emacs-lisp :var yaml_dat_1=yaml_header_1
  yaml_dat_1
#+end_src

And =C-c '= will invoke an edit buffer in yaml mode (if available).

And it seems that fontifying natively works just as with src blocks.

Best,
Chuck

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič via General discussions about Org-mode.]

"Berry, Charles" <ccberry@health.ucsd.edu> writes:

> Max,
>
>> On Dec 31, 2021, at 4:05 AM, Max Nikulin <manikulin@gmail.com> wrote:
>> 
>> 
>> Should some function a macro be provided to facilitate declaring
>> languages as data format (config files, JSON, YAML, etc.) rather than
>> executable source code?
>
> I think we already have this in the form of export blocks, viz.

While "export" blocks do support "org-edit-special" and more, they do
not support ":tangle", right?  For BibTeX, one needs to tangle.

Rudy
-- 
"I love deadlines. I love the whooshing noise they make as they go by."
-- Douglas Adams, The Salmon of Doubt

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič via General discussions about Org-mode.]

Max Nikulin <manikulin@gmail.com> writes:

> On 30/12/2021 15:00, Rudolf Adamkovič wrote:
>> 
>> #+property: header-args:bibtex+ :eval yes
>
> Why do you set ":eval yes" explicitly for bibtex if you believe that
> it should not be executed?

Plain old stupidity.  :)

Rudy
-- 
"Programming reliably --- must be an activity of an undeniably
mathematical nature […] You see, mathematics is about thinking, and
doing mathematics is always trying to think as well as possible." --
Edsger W. Dijkstra (1981)

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Berry, Charles]

Rudy,

> On Dec 29, 2021, at 11:26 PM, Rudolf Adamkovič <salutis@me.com> wrote:
> 
> (Note: I do not know what ECM stands for.)


ECM stands for =Exemple Complet Minimal=, per https://orgmode.org/worg/org-faq.html#ecm, which says "The term refers to test files that can reliably reproduce a bug with the minimal amount of code."

It also references https://list.orgmode.org/orgmode/80oc3s13rg.fsf@somewhere.org/ for more details.

Roughly, an ECM is something a reader should be able to copy into a file and run "as is" on his/her setup. 

HTH,
Chuck

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Max Nikulin]

On 25/12/2021 02:52, Berry, Charles wrote:
> 
> For that case, setting buffer or heading properties, such as:
> 
> #+begin_src org
>    :PROPERTIES:
>    :header-args: :eval yes :exports results
>    :header-args:bibtex: :eval no
>    :END:
> #+end_src
> 
> resolves the issue.

Chuck, is it expected that the following form to specify properties 
suppresses of evaluation during export but allows execution during 
processing of `org-babel-execute-buffer'? I am puzzled that behavior for 
property drawer is different.

---- >8 ----
#+property: header-args :eval yes :exports both
#+property: header-args:elisp :eval no

#+begin_src elisp
   (message "From elisp")
#+end_src
---- 8< ----

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Berry, Charles]

Max,

> On Dec 29, 2021, at 4:53 AM, Max Nikulin <manikulin@gmail.com> wrote:
> 
> On 25/12/2021 02:52, Berry, Charles wrote:
>> For that case, setting buffer or heading properties, such as:
>> #+begin_src org
>>   :PROPERTIES:
>>   :header-args: :eval yes :exports results
>>   :header-args:bibtex: :eval no
>>   :END:
>> #+end_src
>> resolves the issue.
> 
> Chuck, is it expected that the following form to specify properties suppresses of evaluation during export but allows execution during processing of `org-babel-execute-buffer'? I am puzzled that behavior for property drawer is different.
> 

It isn't different when you have `:header-args:elisp: :eval no' in the drawer.

I get the message `Evaluation of this elisp code block is disabled.' either way.

Caveat: I am running release_9.5.1-233-ged5335.

But a couple of things:

1) Be sure to refresh when introducing `#+property' lines. If you paste in a property line and then org-babel-execute-src-block, the property will not be acknowledged. AFAICS, property drawers do not suffer from this. 

2) Property drawers only affect the heading under which they are placed unless placed at the top of the file. And either way, it is an error to insert a blank line above a property drawer. org-lint will complain although the message may be cryptic.

3) You probably know this, but exports use `org-babel-exp-process-buffer' which may perform differently than `org-babel-execute-buffer'. 




> ---- >8 ----
> #+property: header-args :eval yes :exports both
> #+property: header-args:elisp :eval no
> 
> #+begin_src elisp
>  (message "From elisp")
> #+end_src
> ---- 8< ----


HTH,
Chuck

Re: bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Max Nikulin]

On 30/12/2021 00:47, Berry, Charles wrote:
> 
> 1) Be sure to refresh when introducing `#+property' lines. If you
> paste in a property line and then org-babel-execute-src-block, the
> property will not be acknowledged. AFAICS, property drawers do not
> suffer from this.

Thank you, after C-c on the property line it works as expected. I was 
confused that export worked as expected and forgot that execute buffer 
may require activating just added lines.

I was lucky enough to avoid a pitfall with a blank line above property 
drawer. Usually I add empty line at the beginning of a file unless a 
shebang is required.

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič via General discussions about Org-mode.]

Ihor Radchenko <yantar92@gmail.com> writes:

> And we do not have to force the users adding trivial things like
> bibtex. Bibtex may be one of the defaults.

I see. In effect, Org would not try to "execute BibTeX" when executing a
buffer. That makes sense!

Rudy
-- 
"I love deadlines. I love the whooshing noise they make as they go by."
-- Douglas Adams, The Salmon of Doubt

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Max Nikulin]

On 24/12/2021 05:00, Rudolf Adamkovič wrote:
> Ihor Radchenko writes:
> 
>> So, Org cannot distinguish between language backends that are simply
>> not loaded and the ones that do not define org-babel-execute:lang.
> 
> Oh, if we have this architectural limitation in place, then Org cannot
> help the user, and every user will have "explain" to Org that executing
> BibTeX makes no sense.  I guess we can then close this bug then!

Rudolf, do have support BibTeX as babel language loaded to Org (by 
customization of `org-babel-load-languages' or by explicit `require')?

Re: bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> Rudolf, do have support BibTeX as babel language loaded to Org (by 
> customization of `org-babel-load-languages' or by explicit `require')?

Thanks for reminding about `org-babel-load-languages'.
Note that `org-babel-execute-src-block' (called by
org-babel-execute-buffer) does not really use it. It just checks for
(intern (concat "org-babel-execute:" lang)).

Maybe we just should not throw an error when lang is not in
`org-babel-load-languages'? Or maybe we can check for this in
`org-babel-confirm-evaluate'.

Best,
Ihor

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Max Nikulin]

Since debbugs is not the primary bug tracker, it is better to close this 
issue on debbugs and maybe confirm it for https://updates.orgmode.org/

 From my point of view, making babel aware that some widely used 
languages are data-only (so evaluation does not make sense) improves 
user experience.

On the other hand there is no support of BibTeX in babel, so current 
behavior is correct: attempt to execute a block of a language that is 
not enabled causes an error. (While I was reading first messages in this 
thread, it was not obvious for me that there in no ob-bibtex.el package.)

Anyway emacs requires enough customization, so
     (setq org-babel-default-header-args:bibtex '((:eval . "no")))
globally in inti.el or per-file setting
     #+property: header-args:bibtex :eval no
should not be a problem. User can achieve desired behavior with no 
changes in org code.

On 26/12/2021 20:27, Ihor Radchenko wrote:
> 
> Thanks for reminding about `org-babel-load-languages'.
> Note that `org-babel-execute-src-block' (called by
> org-babel-execute-buffer) does not really use it. It just checks for
> (intern (concat "org-babel-execute:" lang)).
> 
> Maybe we just should not throw an error when lang is not in
> `org-babel-load-languages'? Or maybe we can check for this in
> `org-babel-confirm-evaluate'.

Berry, Charles. Fri, 17 Dec 2021 19:25:52 +0000
https://list.orgmode.org/A4FB4F42-64A8-4D6D-A621-D621E1AA1F04@health.ucsd.edu

> If I have a typo in the name of a language, the error message you
> quote tells me what my mistake was.
Certainly, if some language is mentioned in `org-babel-load-languages' 
than it is not a typo, but as I wrote earlier, I do not think, nil 
should work as ":eval no".

Side note: to notify users about a typo, it is better to check all 
source blocks in advance before starting to execute them in 
`org-babel-execute-buffer'. Export may issue warnings as well even when 
evaluation is disabled. See P.S. below for one more note.

I do not mind using `org-babel-load-languages' more widely. However 
massive clean up of worg is required to change abundant examples that 
replace `org-babel-load-languages' by calling 
`org-babel-do-load-languages'. The latter is intended for `defcustom'. 
It is better to provide incremental functions to load and unload 
languages for usage in init files or source blocks. Direct (require 
'ob-something) becomes less favorable as well.

Some symbol like "noeval" may be added to nit and t as values of 
`org-babel-load-languages' pairs.

Berry, Charles. Fri, 31 Dec 2021 19:11:45 +0000.
https://list.orgmode.org/F641D224-52FC-45D8-95F0-9E99F824E1F7@health.ucsd.edu

> #+name: yaml_header_1
> #+begin_export yaml

Rudy wrote that it prevents tangling. I can add that it prevents export 
to e.g. HTML for documentation how to configure something that has two 
representations: executable script and steps that can be followed by 
manually typing commands and editing configuration files.

So I consider data-only languages as a convenient feature.

 From security reasons I would like to have possibility to disable 
evaluation in a such way that it is impossible to override through 
per-file variables or properties.

Are we going to change anything related to `org-babel-load-language' and 
`org-babel-execute-buffer' or maybe just add a set 
`org-babel-default-header-args:something variables'?


P.S.

Berry, Charles. Sat, 18 Dec 2021 20:13:15 +0000.
https://list.orgmode.org/C61FBF93-064B-4FB2-9378-39D3078F108E@health.ucsd.edu

> Also, when exporting it looks `org-babel-exp-results' does not
> attempt  to run src blocks for which
> 
> (fboundp (intern (concat "org-babel-execute:" lang)))
> 
> is nil.
> 
> So doing an export should `just work'.

I would say that it is hardly consistent with mistake detection as a 
feature.

Re: bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič]

Max Nikulin <manikulin@gmail.com> writes:

> Rudolf, do have support BibTeX as babel language loaded to Org (by
> customization of `org-babel-load-languages' or by explicit `require')?

I have it listed in org-babel-do-load-languages as (bibtex . nil).

Rudy
-- 
"'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and
if it were so, it would be; but as it isn't, it ain't. That's logic.'"
-- Lewis Carroll, Through the Looking Glass

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Max Nikulin]

I am sorry that I sent previous message to emacs-orgmode list only, not 
to debbugs.

On 27/12/2021 02:51, Rudolf Adamkovič wrote:
> Max Nikulin writes:
> 
>> Rudolf, do have support BibTeX as babel language loaded to Org (by
>> customization of `org-babel-load-languages' or by explicit `require')?
> 
> I have it listed in org-babel-do-load-languages as (bibtex . nil).

Let's consider the following example:

---- >8 ----
Test

#+begin_src elisp
   (message "Test")
#+end_src
---- 8< -----

emacs -Q -L ~/src/org-mode/lisp/ --eval "(custom-set-variables 
'(org-babel-load-languages '((emacs-lisp . nil))))" babel-exec.org

Do you agree that "No org-babel-execute function for elisp!" is a 
reasonable reaction to `org-babel-execute-buffer`?

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič via General discussions about Org-mode.]

Max Nikulin <manikulin@gmail.com> writes:

> Let's consider the following example:
>
> ---- >8 ----
> Test
>
> #+begin_src elisp
>    (message "Test")
> #+end_src
> ---- 8< -----
>
> emacs -Q -L ~/src/org-mode/lisp/ --eval "(custom-set-variables 
> '(org-babel-load-languages '((emacs-lisp . nil))))" babel-exec.org
>
> Do you agree that "No org-babel-execute function for elisp!" is a 
> reasonable reaction to `org-babel-execute-buffer`?

The user did ask Org to load "emacs-lisp" but to never execute it.
Thus, I would expect Org to do just that.  What do you think?

-- 
"Logic is a science of the necessary laws of thought, without which no
employment of the understanding and the reason takes place." -- Immanuel
Kant, 1785

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Max Nikulin]

On 28/12/2021 04:37, Rudolf Adamkovič wrote:
> Max Nikulin writes:
> 
>> Let's consider the following example:
>>
>> ---- >8 ----
>> Test
>>
>> #+begin_src elisp
>>     (message "Test")
>> #+end_src
>> ---- 8< -----
>>
>> emacs -Q -L ~/src/org-mode/lisp/ --eval "(custom-set-variables
>> '(org-babel-load-languages '((emacs-lisp . nil))))" babel-exec.org
>>
>> Do you agree that "No org-babel-execute function for elisp!" is a
>> reasonable reaction to `org-babel-execute-buffer`?
> 
> The user did ask Org to load "emacs-lisp" but to never execute it.
> Thus, I would expect Org to do just that.  What do you think?

No, it means instruction to unload support of emacs-lisp even it was 
loaded before.

See info "(org) Languages" https://orgmode.org/manual/Languages.html and 
`org-babel-do-load-languages' function. The latter may be considered too 
technical for ordinary user, but examples of its usage are excessively 
proliferated, e.g. it is mentioned for every supported language on Worg 
https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-octave.html

The function is called when the value of `org-babel-load-languages' is 
changed through easy customization interface, so (emacs-lisp . nil) is a 
command rather than a declaration.

The use case is to temporary enable support of some language and later 
to disable it again without terminating of emacs session. Just to avoid 
accidental execution of some snippet.

I can not say that your expectation of the effect of nil is a nonsense, 
but changing its interpretation may cause silent failures and as a 
result upset users. Consider that a file that includes source code 
blocks for different languages is moved to another machine (or 
configuration changed since previous execution on the same machine). It 
is better to get explicit error than to copy result in a hurry and to 
discover that it is incomplete some time later. Per-file and per-block 
configuration is safer against such pitfalls.

I do not insist that current behavior is the best possible. There are 
still some reasons behind such choice.

bug#52545: 29.0.50; Make org-babel-execute-buffer ignore irrelevant src blocks

[Rudolf Adamkovič via General discussions about Org-mode.]

Max Nikulin <manikulin@gmail.com> writes:

> No, it means instruction to unload support of emacs-lisp even it was 
> loaded before.

I see.  Thank you for the explanation!  I misunderstood the (rather
unusual) "org-babel-do-load-languages" API despite having read the
documentation.

> I do not insist that current behavior is the best possible. There are 
> still some reasons behind such choice.

Then, perhaps Ihor had the best idea, where Org would become smart
enough to avoid "executing BibTeX" and similar?

Rudy
-- 
"I love deadlines. I love the whooshing noise they make as they go by."
-- Douglas Adams, The Salmon of Doubt

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia