Raku evaluation for org-babel

Raku evaluation for org-babel

[Tim Van den Langenbergh]

[-- Attachment #1: Type: text/plain, Size: 8482 bytes --]

Hello,

I have spitballed together a module that adds Raku (formerly known as Perl 6) evaluation to Org-Babel.

According to the org-babel docs I should contact this mailing list for help to get it ready for inclusion on elpa. This is my first module for Emacs, so I'm not familiar with proper process, I believe I have to add the .el file to this e-mail, so I shall do so.

If there is anything I have to do before I can proceed, I would be grateful for any advise offered.

Vale,

-Tim Van den Langenbergh

--- ob-raku.el

;; Copyright (C) Tim Van den Langenbergh

;; Author: Tim Van den Langenbergh <tmt_vdl@gmx.com>
;; Keywords: literate programming, reproducible research
;; Homepage: https://github.com/tmtvl/ob-raku
;; Version: 0.05
;; News: 0.05 --- Added initial support for parentheses and commas in strings in lists without breaking the lists on return.
;;       0.04 --- Added square brackets to list splitting, so as to split embedded arrays as well as lists.
;;       0.03 --- Removed the double execution, simplified the formatting of the Raku output, fixed hline support.
;;       0.02 --- Added support for tables, removed unneeded require statements, error when trying to use a session.
;;       0.01 --- Initial release. Accept inputs, support for output and value results.

;;; License:

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation; either version 3, or (at your option)
;; any later version.
;;
;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.
;;
;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs; see the file COPYING.  If not, write to the
;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
;; Boston, MA 02110-1301, USA.

;;; Commentary:

;; Bindings for org-babel support for Raku (née Perl6).

;;; Requirements:

;; Requires a working Raku interpreter to be installed.
;; (Optionally) requires Perl6 mode to be installed.

;;; Code:

(require 'ob)
(require 'ob-eval)
(require 'ob-ref)

(add-to-list 'org-babel-tangle-lang-exts '("raku" . "raku"))

(defvar org-babel-default-header-args:raku '())

(defvar org-babel-raku-command "raku"
  "Command to run Raku.")

(defun org-babel-expand-body:raku (body params &optional processed-params)
  "Expand BODY according to the header arguments specified in PARAMS.
Use the PROCESSED-PARAMS if defined."
  (let ((vars
	 (delq nil
	       (mapcar
		(lambda (pair)
		  (when (eq (car pair) :var) (cdr pair)))
		(or processed-params (org-babel-process-params params))))))
    (concat
     (mapconcat
      (lambda (pair)
	(format
	 "my %s%s = %s;"
	 (if (listp (cdr pair))
	     "@"
	   "$")
	 (car pair)
	 (org-babel-raku-var-to-raku (cdr pair))))
      vars
      "\n")
     "\n" body "\n")))

(defun org-babel-execute:raku (body params)
  "Execute the BODY of Raku code processing it according to PARAMS."
  (let* ((processed-params (org-babel-process-params params))
	 (session (cdr (assoc :session params)))
	 (result-type (cdr (assoc :result-type params)))
	 (result-params (cdr (assoc :result-params params)))
	 (full-body (org-babel-expand-body:raku body params processed-params)))
    (org-babel-reassemble-table
     (org-babel-raku-evaluate full-body session result-type)
     (org-babel-pick-name
      (cdr (assoc :colname-names params)) (cdr (assoc :colnames params)))
     (org-babel-pick-name
      (cdr (assoc :rowname-names params)) (cdr (assoc :rownames params))))))

(defun org-babel-prep-session:raku (session params)
  "Prepare SESSION according to the header arguments specified in PARAMS."
  (error "Sessions are not supported for Raku"))

(defun org-babel-raku-var-to-raku (var)
  "Convert an elisp value VAR to a Raku definition of the same value."
  (if (listp var)
      (concat
       "("
       (mapconcat
	#'org-babel-raku-var-to-raku
	var
	", ")
       ")")
    (if (equal var 'hline)
	"\"HLINE\""
      (format "%S" var))))

(defun org-babel-raku-escape-nested-list-delimiters (list)
  "Escapes any commas or parentheses found in strings contained in the given LIST."
  (let ((in-string nil))
    (mapconcat
	   (lambda (string)
	     (cond
	      ((string= string "\"")
	       (setq in-string (not in-string))
	       string)
	      ((and
		in-string
		(or
		 (string= string "(")
		 (string= string ")")
		 (string= string "[")
		 (string= string "]")
		 (string= string ",")))
	       (concat "\\" string))
	      (t string)))
	   (split-string list "" t)
	   "")))

(defun org-babel-raku-unescape-parens-and-commas (string)
  "Unescapes parentheses and commas in STRING."
  ;;(replace-regexp-in-string "\\\\\([][(),]\)" "\1" string) ;; This doesn't work.
  (let ((index (string-match "\\\\[][(),]" string)))
    (if index
	(concat
	 (substring string 0 index)
	 (org-babel-raku-unescape-parens-and-commas
	  (substring string (+ index 1))))
      string)))

(defun org-babel-raku-split-list (list)
  "Split LIST on a comma or parentheses, ignoring those in a string."
  (mapcar
   (lambda (pairstring)
     (mapcar
      (lambda (string)
	(org-babel-raku-unescape-parens-and-commas string))
      (split-string pairstring "[^\\], " t)))
   (split-string (org-babel-raku-escape-nested-list-delimiters (substring list 2 -2))
		 "[^\\][][()]"
		 t)))

(defun org-babel-raku-sanitize-table (table)
  "Recursively sanitize the values in the given TABLE."
  (if (listp table)
      (let ((sanitized-table (mapcar 'org-babel-raku-sanitize-table table)))
	(if (and (stringp (car sanitized-table))
		 (string= (car sanitized-table) "HLINE"))
	    'hline
	  sanitized-table))
    (org-babel-script-escape table)))

(defun org-babel-raku-table-or-string (results)
  "If RESULTS look like a table, then convert them into an elisp table.
Otherwise return RESULTS as a string."
  (cond
   ((or (string-prefix-p "$[" results)
	(string-prefix-p "$(" results))
    (org-babel-raku-sanitize-table
     (org-babel-raku-split-list results)))
   ((string-prefix-p "{" results)
    (org-babel-raku-sanitize-table
     (mapcar
      (lambda (pairstring)
	(split-string pairstring " => " t))
      (split-string
       (substring results 1 -2)
       ", "
       t))))
   (t (org-babel-script-escape results))))

(defun org-babel-raku-initiate-session (&optional session)
  "If there is not a current inferior-process-buffer in SESSION then create.
Return the initialized SESSION."
  (unless (string= session "none")
    (if session (error "Sessions are not supported for Raku"))))

(defun org-babel-raku-evaluate (body &optional session result-type)
  "Evaluate the BODY with Raku.
If SESSION is not provided, evaluate in an external process.
If RESULT-TYPE is not provided, assume \"value\"."
  (org-babel-raku-table-or-string (if (and session (not (string= session "none")))
      (org-babel-raku-evaluate-session session body result-type)
    (org-babel-raku-evaluate-external body result-type))))

(defconst org-babel-raku-wrapper
  "sub _MAIN {
%s
}

sub _FORMATTER ($result) {
return $result.gist if $result.WHAT ~~ Hash;
$result.raku
}

\"%s\".IO.spurt(\"{ _FORMATTER(_MAIN()) }\\n\");"
  "Wrapper for grabbing the final value from Raku code.")

(defun org-babel-raku-evaluate-external (body &optional result-type)
  "Evaluate the BODY with an external Raku process.
If RESULT-TYPE is not provided, assume \"value\"."
  (if (and result-type (string= result-type "output"))
      (org-babel-eval org-babel-raku-command body)
    (let ((temp-file (org-babel-temp-file "raku-" ".raku")))
      (org-babel-eval
       org-babel-raku-command
       (format
	org-babel-raku-wrapper
	body
	(org-babel-process-file-name temp-file 'noquote)))
      (org-babel-eval-read-file temp-file))))

(defun org-babel-raku-evaluate-session (session body &optional result-type)
  "Evaluate the BODY with the Raku process running in SESSION.
If RESULT-TYPE is not provided, assume \"value\"."
  (error "Sessions are not supported for Raku"))

(provide 'ob-raku)
;;; ob-raku.el ends here

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Raku evaluation for org-babel

[Bruno Félix Rezende Ribeiro]

[-- Attachment #1: Type: text/plain, Size: 345 bytes --]

Hello Tim,

Tim Van den Langenbergh <tmt_vdl@gmx.com> writes:

> I have spitballed together a module that adds Raku (formerly known as
> Perl 6) evaluation to Org-Babel. [...] get it ready for inclusion on
> elpa.

Was this followed upon?

-- 
Bruno Félix Rezende Ribeiro (oitofelix) [0x28D618AF]
<http://oitofelix.freeshell.org/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]

Re: Raku evaluation for org-babel

[Tim Van den Langenbergh]

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

On Thursday, 9 April 2020 05:14:13 CEST you wrote:
> Hello Tim,
>
> Tim Van den Langenbergh <tmt_vdl@gmx.com> writes:
>
> > I have spitballed together a module that adds Raku (formerly known as
> > Perl 6) evaluation to Org-Babel. [...] get it ready for inclusion on
> > elpa.
>
> Was this followed upon?
>
>

Not yet, though I have read the ELPA README and plan on getting the package ready according to the provided guidelines.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Raku evaluation for org-babel

[Stefan Monnier]

> I have spitballed together a module that adds Raku (formerly known as
> Perl 6) evaluation to Org-Babel.
> According to the org-babel docs I should contact this mailing list for
> help to get it ready for inclusion on elpa.

I'm not very familiar with Org-Babel, so I was wondering if it's best to
distribute such packages as standalone packages or include them in
Org-Babel, but if the org-babel doc pointed you here, I guess that's the
place they prefer, which is fine by me.

> This is my first module for Emacs, so I'm not familiar with proper
> process, I believe I have to add the .el file to this e-mail, so
> I shall do so.

Actually, if you have public Git branch somewhere, it's better, this way
we can preserve the (pre)history.

> If there is anything I have to do before I can proceed, I would be
> grateful for any advise offered.

See some comments below but the main issue is that it seems you haven't
signed the needed copyright paperwork yet.  I'll send you the forms
off-list for that.

> ;; Author: Tim Van den Langenbergh <tmt_vdl@gmx.com>
> ;; Keywords: literate programming, reproducible research
> ;; Homepage: https://github.com/tmtvl/ob-raku
> ;; Version: 0.05

This is equal to 0.5 (GNU ELPA's versions are lists of numbers and 05 is
the same number as 5).

It'd be good to add a `Package-Requires: ((emacs "NN"))` to clarify
with which versions of Emacs it's supposed to work.

> ;; News: 0.05 --- Added initial support for parentheses and commas in strings in lists without breaking the lists on return.
> ;;       0.04 --- Added square brackets to list splitting, so as to split embedded arrays as well as lists.
> ;;       0.03 --- Removed the double execution, simplified the formatting of the Raku output, fixed hline support.
> ;;       0.02 --- Added support for tables, removed unneeded require statements, error when trying to use a session.
> ;;       0.01 --- Initial release. Accept inputs, support for output and value results.

We usually prefer to put the news in a separate ";;;; News" section.
Also we recommend to avoid using more than 80 columns.

> ;; You should have received a copy of the GNU General Public License
> ;; along with GNU Emacs; see the file COPYING.  If not, write to the
> ;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
> ;; Boston, MA 02110-1301, USA.

This has been updated to:

    ;; You should have received a copy of the GNU General Public License
    ;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.

> (add-to-list 'org-babel-tangle-lang-exts '("raku" . "raku"))

IIUC this is the part that tells org-babel about the existence of support
for raku, but it's only executed if `ob-raku` is loaded.

It'd be nice to make it so that the user doesn't need to explicitly
require `ob-raku`, i.e. advertise the existence of raku support to
org-babel before the file is loaded.  For that you'll want to add
some ;;;###autoload cookies at a few appropriate places.

Sadly, the above `add-to-list` can't be autoloaded as-is since it will
will signal error if executed before `org-babel-tangle-lang-exts` is defined.
I don't know how it's supposed to be done for org-babel, so you may need
to ask the org-babel about that.

>     (mapconcat
> 	   (lambda (string)
> 	     (cond
> 	      ((string= string "\"")
> 	       (setq in-string (not in-string))
> 	       string)
> 	      ((and
> 		in-string
> 		(or
> 		 (string= string "(")
> 		 (string= string ")")
> 		 (string= string "[")
> 		 (string= string "]")
> 		 (string= string ",")))

Last few lines => (member string '("(" ")" "[" "]" ","))

> 	       (concat "\\" string))
> 	      (t string)))
> 	   (split-string list "" t)
> 	   "")))

Turning your string into a list of single-char strings is terribly
wasteful.  Every char in your original string will end up occupying
something like:
- 2 words for the cons cell.
- 4 words for the actual Lisp_String object.
- 2 words for the actual string bytes (1 word of header, 1 byte for the
  char, 1 terminating NUL byte, plus alignment padding).
So 8 words per char, which on a 64bit system means 64 bytes per char.

It also makes the subsequent tests more expensive since
`(string= string "(")` takes significantly more effort than (eq char ?\().

>   ;;(replace-regexp-in-string "\\\\\([][(),]\)" "\1" string) ;; This doesn't work.
                                     ^^       ^^   ^^^
                                     \\(     \\)   "\\1"?

\( and \) in a string are not special, so they're treated just like
( and ) and the regexp engine never sees your backslash!

> 	    (and (stringp (car sanitized-table))
> 		 (string= (car sanitized-table) "HLINE"))

Aka (equal (car sanitized-table) "HLINE")


        Stefan

Re: Raku evaluation for org-babel

[Stefan Monnier]

> See some comments below but the main issue is that it seems you haven't
> signed the needed copyright paperwork yet.  I'll send you the forms
> off-list for that.

Here it is.  Please fill the form and then send it as instructed to the
FSF so they can send you the relevant paperwork to sign.
Thanks,


        Stefan


Please email the following information to assign@gnu.org, and we
will send you the assignment form for your past and future changes.

Please use your full legal name (in ASCII characters) as the subject
line of the message.
----------------------------------------------------------------------
REQUEST: SEND FORM FOR PAST AND FUTURE CHANGES

[What is the name of the program or package you're contributing to?]
Emacs

[Did you copy any files or text written by someone else in these changes?
Even if that material is free software, we need to know about it.]


[Do you have an employer who might have a basis to claim to own
your changes?  Do you attend a school which might make such a claim?]


[For the copyright registration, what country are you a citizen of?]


[What year were you born?]


[Please write your email address here.]


[Please write your postal address here.]





[Which files have you changed so far, and which new files have you written
so far?]

Re: Raku evaluation for org-babel

[Tim Van den Langenbergh]

[-- Attachment #1: Type: text/plain, Size: 119 bytes --]

Thank you for the pointers, Stefan. I shall get the paperwork done and work on getting your suggested changes in.

-Tim

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]