Template for ob- packages?

Template for ob- packages?

[Galaxy Being]

[-- Attachment #1: Type: text/plain, Size: 161 bytes --]

Is there a generic starter template for writing an ob- package, some sort
of example code?

⨽
Lawrence Bottorff
Grand Marais, MN, USA
borgauf@gmail.com

[-- Attachment #2: Type: text/html, Size: 394 bytes --]

Re: Template for ob- packages?

[Matt]

 ---- On Thu, 16 Feb 2023 16:10:51 -0500  Galaxy Being  wrote --- 
 > Is there a generic starter template for writing an ob- package, some sort of example code?

Yes: https://git.sr.ht/~bzg/worg/tree/master/item/org-contrib/babel/ob-template.el

When I started learned Babel, I took notes with the hope of expanding it into a Worg article.  Here are the notes: https://excalamus.com/2021-11-03-org_babel.html

If you're interested in fleshing out the notes into proper documentation, I'd be happy to help.

Re: Template for ob- packages?

[Thomas S. Dye]

Aloha Lawrence,

Galaxy Being <borgauf@gmail.com> writes:

> Is there a generic starter template for writing an ob- package, 
> some sort of example code?
>
> ⨽
> Lawrence Bottorff
> Grand Marais, MN, USA
> borgauf@gmail.com

Yes, see org-contrib/babel/ob-template.el on Worg.

All the best,
Tom

-- 
Thomas S. Dye
https://tsdye.online/tsdye

Re: Template for ob- packages?

[Galaxy Being]

[-- Attachment #1: Type: text/plain, Size: 885 bytes --]

Yes, I'll try to have a look. My Emacs gig so far has been user/power user
and "investigator" of Babel languages. Right now I'm on Prolog. I'll learn
some of ob-prolog's capabilities first.

On Thu, Feb 16, 2023 at 3:16 PM Matt <matt@excalamus.com> wrote:

>
>  ---- On Thu, 16 Feb 2023 16:10:51 -0500  Galaxy Being  wrote ---
>  > Is there a generic starter template for writing an ob- package, some
> sort of example code?
>
> Yes:
> https://git.sr.ht/~bzg/worg/tree/master/item/org-contrib/babel/ob-template.el
>
> When I started learned Babel, I took notes with the hope of expanding it
> into a Worg article.  Here are the notes:
> https://excalamus.com/2021-11-03-org_babel.html
>
> If you're interested in fleshing out the notes into proper documentation,
> I'd be happy to help.
>


-- 
⨽
Lawrence Bottorff
Grand Marais, MN, USA
borgauf@gmail.com

[-- Attachment #2: Type: text/html, Size: 1605 bytes --]

Re: Template for ob- packages?

[Leo Butler]

On Thu, Feb 16 2023, Matt <matt@excalamus.com> wrote:

>  ---- On Thu, 16 Feb 2023 16:10:51 -0500  Galaxy Being  wrote --- 
>  > Is there a generic starter template for writing an ob-
> package, some sort of example code?
>
> Yes: https://git.sr.ht/~bzg/worg/tree/master/item/org-contrib/babel/ob-template.el
>
> When I started learned Babel, I took notes with the hope of expanding
> it into a Worg article. Here are the notes:
> https://excalamus.com/2021-11-03-org_babel.html
>
> If you're interested in fleshing out the notes into proper
> documentation, I'd be happy to help.

Matt, thanks for sharing those notes. I would suggest that they be added
to worg in their current state.

Leo

Re: Template for ob- packages?

[Matt]

[-- Attachment #1: Type: text/plain, Size: 314 bytes --]


 ---- On Fri, 17 Feb 2023 17:32:18 -0500  Leo Butler  wrote --- 

 > Matt, thanks for sharing those notes. I would suggest that they be added
 > to worg in their current state.

Thanks for your vote of confidence.

I've attached the source, in case someone wants to do that.  Otherwise, I'll get to it when I can.

[-- Attachment #2: 2021-11-03-org_babel.org --]
[-- Type: application/octet-stream, Size: 13343 bytes --]

#+TITLE: Org Babel API
#+DATE: 2021-11-03
#+TYPE: post
#+TOC: t

* TODO COMMENT
- [ ] explain each function
- [ ] explain terms
- [ ] remove cruft functions from =ob-template.el= (or just refactor
  that example)

* COMMENT Setup
Provide the Org source directory path in order to make this file
portable.

#+begin_src emacs-lisp :results none
(setq-local org-dir "/home/ahab/.emacs.d/straight/repos/org/lisp")
#+end_src

#+begin_src sh :var ORGDIR=(identity org-dir) :results output
echo "$ORGDIR"
#+end_src

#+RESULTS:
: /home/ahab/.emacs.d/straight/repos/org/lisp

* Overview
This document explains the Org Babel API with a focus on maintenance
and extension.  Babel is part of the Org package for Emacs.  It allows
for embedded code blocks within a text document[fn:1].

At the time of writing, the Org Babel API is only described in
[[https://git.sr.ht/~bzg/worg/tree/master/item/org-contrib/babel/ob-template.el][=ob-template.el=]], the [[https://lists.gnu.org/archive/html/emacs-orgmode/2015-09/msg00487.html][email thread]] linked to in [[https://orgmode.org/worg/org-contrib/babel/languages/index.html#develop][the Worg docs]], and, of
course, the source code itself.

The functions that need to be (or even can be) defined aren't really
laid out anywhere, nor are the ways they interact explained in one
place. This document aims to change that.

*NOTE* At some point I'll merge this into Worg[fn:2].  However, this
is still a draft and I don't feel it's right to host it there
yet. These are my active notes and people probably expect Worg to be
somewhat more complete.

* The Babel Codebase
Org Babel is not a single file or package. It's a suite of
files. Babel files begin with "ob-", for "Org Babel", and are found in
the =lisp/= directory of the Org project[fn:3].  The primary file is
=ob-core.el=.

#+begin_src sh :var ORGDIR=(identity org-dir) :results output
ls $ORGDIR | grep "ob-" | grep "el$" | head -n 10 && echo "..."
#+end_src

#+RESULTS:
#+begin_example
ob-awk.el
ob-calc.el
ob-C.el
ob-clojure.el
ob-comint.el
ob-core.el
ob-css.el
ob-ditaa.el
ob-dot.el
ob-emacs-lisp.el
...
#+end_example

At the time of writing, Babel is useable with more than 40 different
languages.

#+begin_src sh :var ORGDIR=(identity org-dir) :results output
ls $ORGDIR | grep "ob-" | grep "el$" | wc -l
#+end_src

#+RESULTS:
: 48

Babel is no longer maintained by the original author.  Instead,
extensions have individual maintainers. Each extension varies in
quality and consistency. Some see regular updates whereas others don't
seem to have changed much.

It's [[https://www.gnu.org/software/emacs/manual/html_node/elisp/Coding-Conventions.html][convention]] in Elisp to choose a prefix to distinguish functions
and variables between packages.  The name "Org Babel" implies the
package prefix "org-babel-"[fn:4].  This is the prefix used within
Babel *as well as other Org extensions (e.g. Org Links, Org Export
Framework, etc.)*. That is, symbols defined in =ob-<lang>.el= don't
use a "ob-<lang>-" prefix–they use "org-babel-".  It's impossible to
distinguish Org Babel symbols from other Org symbols by name alone.

* Babel Basics
Babel works by defining certain functions that =ob-core.el= expects.

If we compare =ob-template.el= and =ob-core.el=, we get the following
in no particular order:

| function name                             | ob-core.el | ob-template.el |
|-------------------------------------------+------------+----------------|
| [[org-babel-execute:template][=org-babel-execute:template=]]              | X          | X              |
| [[org-babel-template-initiate-session][=org-babel-template-initiate-session=]]     | X          | X              |
| [[org-babel-expand-body:template][=org-babel-expand-body:template=]]          | X          | X              |
| [[org-babel-prep-session:template][=org-babel-prep-session:template=]]         | X          | X              |
| [[=org-babel-template-var-to-template=][=org-babel-template-var-to-template=]]      |            | X              |
| [[=org-babel-template-table-or-string=][=org-babel-template-table-or-string=]]      |            | X              |
| [[org-babel-variable-assignments:template][=org-babel-variable-assignments:template=]] | X          |                |
| [[org-babel-process-params][=org-babel-process-params=]]                | X          |                |

| variable name                            | ob-core.el | ob-template.el |
|------------------------------------------+------------+----------------|
| [[=org-babel-tangle-lang-exts=][=org-babel-tangle-lang-exts=]]             |            | X              |
| [[org-babel-default-header-args:template][=org-babel-default-header-args:template=]] | X          | X              |

Functions from =ob-core.el= not in =ob-template.el=, but still used in
some =ob-template.el= derived languages (e.g. =ob-shell.el=):

- [[org-babel-header-args:template][=org-babel-header-args:template=]]
- [[org-babel-load-session:template][=org-babel-load-session:template=]]
- [[org-babel-load-session:template][=org-babel-variable-assignments:template=]]

Important functions to understand for any extension that uses a comint
(all of them?):

- [[org-babel-comint-wait-for-output][=org-babel-comint-wait-for-output=]]

* Function: =org-babel-execute:template= <<org-babel-execute:template>>
*Arguments*: /body/, /params/

*Returns*: emacs-lisp containing the results of evaluating /body/ in
/template/ language.

The function called to actually evaluate a code block.  It is defined
in an "ob-" file and gets called by =org-babel-execute-src-block=.

The =org-babel-execute:template= function will evaluate the body of
the source code and return the results as emacs-lisp depending on the
value of the =:results header= argument:

- =:results output= means that the output to STDOUT will be captured
  and returned
- =:results value= means that the value of the last statement in the
  source code block will be returned

The [[https://git.sr.ht/~bzg/worg/tree/master/item/org-contrib/babel/ob-template.el][ob-template.el]] file claims that "the most common first step in
this function is the expansion of the PARAMS argument using
=org-babel-process-params=."  This was probably true when that was
written. At the time of writing, however, only four of the files in the
org-babel suite use it.

#+begin_quote
Please feel free to not implement options which aren't appropriate
for your language (e.g. not all languages support interactive
"session" evaluation).  Also you are free to define any new header
arguments which you feel may be useful -- all header arguments
specified by the user will be available in the PARAMS variable.
#+end_quote

* TODO Function: =org-babel-template-initiate-session= <<org-babel-template-initiate-session>>
* TODO Function: =org-babel-expand-body:template= <<org-babel-expand-body:template>>
Called by [[org-babel-execute:template][=org-babel-execute:template=]].

* TODO Function: =org-babel-prep-session:template= <<org-babel-prep-session:template>>
* TODO Function: =org-babel-template-var-to-template= <<org-babel-template-var-to-template>>
This is cruft.  It's only defined in =ob-template.el=. All it does is this:

#+begin_src emacs-lisp :eval never
(defun org-babel-template-var-to-template (var)
  "Convert an elisp var into a string of template source code
specifying a var of the same value."
  (format "%S" var))
#+end_src

Surely this doesn't deserve a whole function.

* TODO Function: =org-babel-template-table-or-string= <<org-babel-template-table-or-string>>
This is cruft. It is defined in =ob-template.el= but not used
anywhere, in that file or elsewhere.

* Function: =org-babel-process-params= <<org-babel-process-params>>
  "Expand variables in PARAMS and add summary parameters."

* TODO Variable: =org-babel-tangle-lang-exts= <<org-babel-tangle-lang-exts>>
Gets called by =ob-tangle.el= in the
=org-babel-effective-tangled-filename= function.
* TODO Variable: =org-babel-default-header-args:template= <<org-babel-default-header-args:template>>
* TODO Function: =org-babel-header-args:template= <<org-babel-header-args:template>>
* Function: =org-babel-load-session:template= <<org-babel-load-session:template>>
Called by the =ob-core.el= function =org-babel-load-in-session=. That
function handles header args, mainly no-web expansion. It then starts
the comint process and opens the associated process buffer.  The
=org-babel-load-in-session= function is, basically, called by the
"org-metaup-hook" which is attached to M-up by default.  Otherwise,
this function is never called. Also, AFAICT, this behavior is not
documented anywhere.

For ob-shell.el, this function is implemented, but only for the
"shell" language. For example, press M-up when in the following
block. The body will be inserted in a new process buffer "my-shell".

: #+begin_src shell :session my-shell
: echo "hello, world!"
: #+end_src
:
: #+begin_src sh :session my-sh
: echo "hello, world!"
: #+end_src
:
: #+begin_src bash :session my-bash
: echo "hello, world!"
: #+end_src

* Function: =org-babel-variable-assignments:template= <<org-babel-variable-assignments:template>>
Used during tangle (ob-tangle.el) and eval.

It is used as an argument for [[https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/lisp/ob-core.el#n810][=org-babel-expand-body:generic=]].

It is used in =org-babel-prep-session:template=

#+begin_quote
"Return list of template statements assigning the block's variables."
#+end_quote

* TODO Function: org-babel-comint-wait-for-output <<org-babel-comint-wait-for-output>>
* Undocumented behavior
Don't be surprised if there is undocumented behavior defined within an
ob file.

The following were undocumented features of =ob-shell.el=:

  - :stdin passes org reference (#+name) as stdin
  - :cmdline with :shebang runs body with given command-line arg(s)
  - :cmdline without :shebang runs body with shell-file-name and command-line arg(s)
  - org-babel-load-session:shell (see [[org-babel-load-session:template][org-babel-load-session:template]])

* Terminology
To quote [[https://orgmode.org/manual/Structure-of-Code-Blocks.html][(org) Structure of Code Blocks]],

#+begin_example org
A source code block conforms to this structure:

    #+NAME: <name>
    #+BEGIN_SRC <language> <switches> <header arguments>
    <body>
    #+END_SRC
#+end_example

** TODO info
** params
'params' is an alist derived from the src block header.  Element keys
 are referred to as "header args". These are defined in
 =org-babel-common-header-args-w-values=, but are not exclusive to
 that list. Any ":header-arg value" pair defined in the header will
 appear in the list.

** switch
Switches control evaluation, export and tangling of code blocks.

For example, the following block uses the "-n" switch to toggle line
numbers in the HTML output:

#+begin_example org
#+begin_src elisp -n
(message "hello, world!")
#+end_src
#+end_example

#+attr_html: :class pre-top pre-bottom
#+begin_src elisp -n
(message "hello, world!")
#+end_src

** header argument
Header arguments are keywords which control block evaluation or
expansion.  For instance, the =:session <session-name>= argument runs
a block in a separate shell. Another example is the =:var NAME=VALUE=
argument which passes data between blocks such that =NAME= gets
defined as =VALUE= within the block environment.

Header arguments can be more general, however. To quote [[https://git.sr.ht/~bzg/worg/tree/master/item/org-contrib/babel/ob-template.el][ob-template.el]],

#+begin_quote
"you are free to define any new header arguments which you feel may be
useful -- all header arguments specified by the user will be available
in the PARAMS variable."
#+end_quote

Arguments can be system-wide, using =org-babel-default-header-args= or
language specific with =org-babel-default-header-args:<LANG>=.

For common header arguments, see
=org-babel-common-header-args-w-values=.

** expansion
"Expand" and "expansion" are overloaded terms. Generally, they mean to
replace something within a form.

When org-babel "expands" the body of a source block, it resolves any
noweb references (noweb is the functionality that allows different
source blocks to reference one another).

To expand a variable, in the context of Org babel, means to assign a
header variable as a variable within the language runtime.

For example, the table named =tbl= gets "expanded" and assigned to
=data= within the Python runtime.

#+begin_example
#+name: tbl
| 1 | 2 |
| 3 | 4 |

#+BEGIN_SRC python :var data=tbl :results value
return data
#+END_SRC

#+RESULTS:
| 1 | 2 |
| 3 | 4 |
#+end_example

* The =:var= parameter
The =:var= parameter has the form

#+begin_example
(:var name . value)
#+end_example

where "value", the cdr, should be a cons cell whose car is the name of
the variable "as a symbol" and whose value is the value of the
variable.

That is,

#+begin_example
(:var name . (name-symbol . "it's value"))
#+end_example

* References
- https://lists.gnu.org/archive/html/emacs-orgmode/2015-09/msg00487.html

* Footnotes

[fn:1] https://orgmode.org/

[fn:2] https://orgmode.org/worg/worg-about.html

[fn:3] https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/lisp

[fn:4] Technically, Emacs doesn't have namespaces. Instead, a prefix
is given to symbols so that various packages don't conflict.