Problem with org-babel and geiser

Problem with org-babel and geiser

[Bob Heffernan]

Dear all,

I am having trouble with evaluating scheme source-blocks in org-mode.
Specifically, in a minimal example such as this:

#+title: Test
#+PROPERTY: header-args :session *scheme*

#+begin_src scheme
  (+ 1 1)
#+end_src

When I evaluate the src block, the results come back empty.

The *geiser-messages* buffer has this:
WARNING: File /home/bob/.chez-geiser does not exist, so it's not added to CLI args

INFO: REQUEST: <1>: (begin (import (geiser)) (write `((result ) (output . ""))) (newline))

INFO: RETORT: ((result) (output . ""))

INFO: <1>: processed

ERROR: <2>: continuation failed "(geiser:eval '#f '(begin ;; -*- geiser-scheme-implementation: chez -*-
(+ 1 1)
))"
	(error Selecting deleted buffer)

In an effort to make sure it's not something funny in my config, I reduced my init.el to the following:

(require 'package)
(add-to-list 'package-archives
  '("nongnu" . "https://elpa.nongnu.org/nongnu/"))
(package-initialize)

(require 'geiser)
(require 'geiser-chez)
(setq geiser-chez-binary "chez")
(org-babel-do-load-languages
 'org-babel-load-languages
 '((scheme . t)))

but the problem persists.

I happen to be using chez scheme, but the same issue arises with other implementations (I tried guile, chicken and racket).  On the other hand, evaluating an elisp block works fine.

This might be a geiser problem, rather than an org problem, but geiser appears to be working fine otherwise.

I'm running Emacs 27.2 on Arch Linux with Org 9.4.4.

Any help would be appreciated.

Regards,
Bob Heffernan

Re: Problem with org-babel and geiser

[Ihor Radchenko]

Bob Heffernan <bob.heffernan@gmail.com> writes:

> I am having trouble with evaluating scheme source-blocks in org-mode.
> Specifically, in a minimal example such as this:
>
> #+title: Test
> #+PROPERTY: header-args :session *scheme*
>
> #+begin_src scheme
>   (+ 1 1)
> #+end_src
>
> When I evaluate the src block, the results come back empty.

Does the patch in
https://orgmode.org/list/3462869.oopU1TnKvU@felipe-thinkpad-x200 help?

Best,
Ihor

Re: Problem with org-babel and geiser

[Bob Heffernan]

On 22-02-04 15:00, Ihor Radchenko wrote:
> Does the patch in
> https://orgmode.org/list/3462869.oopU1TnKvU@felipe-thinkpad-x200 help?

Ihor,

Thank you for your response.
The patch you linked does seem to be intended to fix this issue.

However, I have no experience with applying a patch (to Emacs or anything else).
On my system, ob-scheme.el.gz can be found at
/usr/share/emacs/27.2/lisp/org
can I uncompress the file, patch that, then compress again?

Or, would I need to download the Emacs source, apply the patch and then recompile?

Thank you!

Bob Heffernan

Re: Problem with org-babel and geiser

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 1184 bytes --]

Bob Heffernan <bob.heffernan@gmail.com> writes:

> Thank you for your response.
> The patch you linked does seem to be intended to fix this issue.
>
> However, I have no experience with applying a patch (to Emacs or anything else).
> On my system, ob-scheme.el.gz can be found at
> /usr/share/emacs/27.2/lisp/org
> can I uncompress the file, patch that, then compress again?
>
> Or, would I need to download the Emacs source, apply the patch and then recompile?

It's not a good idea to modify Org packaged together with Emacs. For
testing, you can just use the attached version of ob-scheme.el. Load
your Emacs and Org mode, open the attached ob-scheme.el, and run M-x
eval-buffer. That should be sufficient to test the proposed patch on
your side.

If the patch solves the problem you are experiencing, let us know. I
will then apply the proposed patch upstream.

Note that even if I apply the patch, it will only be available on the
latest version of Org mode (Org 9.5.2). It is _not_ what is shipped
together with your Emacs. You will need to upgrade your Org mode using
M-x package-install. See
https://orgmode.org/manual/Installation.html#Installation for details.

Best,
Ihor


[-- Attachment #2: ob-scheme.el --]
[-- Type: text/plain, Size: 9417 bytes --]

;;; ob-scheme.el --- Babel Functions for Scheme      -*- lexical-binding: t; -*-

;; Copyright (C) 2010-2022 Free Software Foundation, Inc.

;; Authors: Eric Schulte
;;	    Michael Gauland
;; Keywords: literate programming, reproducible research, scheme
;; Homepage: https://orgmode.org

;; This file is part of GNU Emacs.

;; GNU Emacs is free software: you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; GNU Emacs is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:

;; Now working with SBCL for both session and external evaluation.
;;
;; This certainly isn't optimally robust, but it seems to be working
;; for the basic use cases.

;;; Requirements:

;; - a working scheme implementation
;;   (e.g. guile https://www.gnu.org/software/guile/guile.html)
;;
;; - for session based evaluation geiser is required, which is available from
;;   ELPA.

;;; Code:
(require 'ob)
(require 'geiser nil t)
(require 'geiser-impl nil t)
(defvar geiser-repl--repl)             ; Defined in geiser-repl.el
(defvar geiser-impl--implementation)   ; Defined in geiser-impl.el
(defvar geiser-scheme-implementation)  ; Defined in geiser-impl.el
(defvar geiser-default-implementation) ; Defined in geiser-impl.el
(defvar geiser-active-implementations) ; Defined in geiser-impl.el
(defvar geiser-debug-show-debug-p)     ; Defined in geiser-debug.el
(defvar geiser-debug-jump-to-debug-p)  ; Defined in geiser-debug.el
(defvar geiser-repl-use-other-window)  ; Defined in geiser-repl.el
(defvar geiser-repl-window-allow-split)	; Defined in geiser-repl.el

(declare-function run-geiser "ext:geiser-repl" (impl))
(declare-function geiser-mode "ext:geiser-mode" ())
(declare-function geiser-eval-region "ext:geiser-mode"
                  (start end &optional and-go raw nomsg))
(declare-function geiser-repl-exit "ext:geiser-repl" (&optional arg))
(declare-function geiser-eval--retort-output "ext:geiser-eval" (ret))
(declare-function geiser-eval--retort-result-str "ext:geiser-eval" (ret prefix))

(defcustom org-babel-scheme-null-to 'hline
  "Replace `null' and empty lists in scheme tables with this before returning."
  :group 'org-babel
  :version "26.1"
  :package-version '(Org . "9.1")
  :type 'symbol)

(defvar org-babel-default-header-args:scheme '()
  "Default header arguments for scheme code blocks.")

(defun org-babel-expand-body:scheme (body params)
  "Expand BODY according to PARAMS, return the expanded body."
  (let ((vars (org-babel--get-vars params))
	(prepends (cdr (assq :prologue params)))
	(postpends (cdr (assq :epilogue params))))
    (concat (and prepends (concat prepends "\n"))
	    (if (null vars) body
	      (format "(let (%s)\n%s\n)"
		      (mapconcat
		       (lambda (var)
			 (format "%S" (print `(,(car var) ',(cdr var)))))
		       vars
		       "\n      ")
		      body))
	    (and postpends (concat "\n" postpends)))))


(defvar org-babel-scheme-repl-map (make-hash-table :test #'equal)
  "Map of scheme sessions to session names.")

(defun org-babel-scheme-cleanse-repl-map ()
  "Remove dead buffers from the REPL map."
  (maphash
   (lambda (x y) (unless (buffer-name y) (remhash x org-babel-scheme-repl-map)))
   org-babel-scheme-repl-map))

(defun org-babel-scheme-get-session-buffer (session-name)
  "Look up the scheme buffer for a session; return nil if it doesn't exist."
  (org-babel-scheme-cleanse-repl-map) ; Prune dead sessions
  (gethash session-name org-babel-scheme-repl-map))

(defun org-babel-scheme-set-session-buffer (session-name buffer)
  "Record the scheme buffer used for a given session."
  (puthash session-name buffer org-babel-scheme-repl-map))

(defun org-babel-scheme-get-buffer-impl (buffer)
  "Return the scheme implementation geiser associates with the buffer."
  (with-current-buffer (set-buffer buffer)
    geiser-impl--implementation))

(defun org-babel-scheme-get-repl (impl name)
  "Switch to a scheme REPL, creating it if it doesn't exist."
  (let ((buffer (org-babel-scheme-get-session-buffer name)))
    (or buffer
	(progn
	  (run-geiser impl)
	  (when name
	    (rename-buffer name t)
	    (org-babel-scheme-set-session-buffer name (current-buffer)))
	  (current-buffer)))))

(defun org-babel-scheme-make-session-name (buffer name impl)
  "Generate a name for the session buffer.

For a named session, the buffer name will be the session name.

If the session is unnamed (nil), generate a name.

If the session is `none', use nil for the session name, and
org-babel-scheme-execute-with-geiser will use a temporary session."
  (cond ((not name) (concat buffer " " (symbol-name impl) " REPL"))
	((string= name "none") nil)
	(name)))

(defmacro org-babel-scheme-capture-current-message (&rest body)
  "Capture current message in both interactive and noninteractive mode."
  `(if noninteractive
       (let ((original-message (symbol-function 'message))
             (current-message nil))
         (unwind-protect
             (progn
               (defun message (&rest args)
                 (setq current-message (apply original-message args)))
               ,@body
               current-message)
           (fset 'message original-message)))
     (progn
       ,@body
       (current-message))))

(defun org-babel-scheme-execute-with-geiser (code output impl repl)
  "Execute code in specified REPL.
If the REPL doesn't exist, create it using the given scheme
implementation.

Returns the output of executing the code if the OUTPUT parameter
is true; otherwise returns the last value."
  (let ((result nil))
    (with-temp-buffer
      (insert (format ";; -*- geiser-scheme-implementation: %s -*-" impl))
      (newline)
      (insert code)
      (geiser-mode)
      (let ((geiser-repl-window-allow-split nil)
	    (geiser-repl-use-other-window nil))
	(let ((repl-buffer (save-current-buffer
			     (org-babel-scheme-get-repl impl repl))))
	  (when (not (eq impl (org-babel-scheme-get-buffer-impl
			       (current-buffer))))
	    (message "Implementation mismatch: %s (%s) %s (%s)" impl (symbolp impl)
		     (org-babel-scheme-get-buffer-impl (current-buffer))
		     (symbolp (org-babel-scheme-get-buffer-impl
			       (current-buffer)))))
	  (setq geiser-repl--repl repl-buffer)
	  (setq geiser-impl--implementation nil)
	  (let ((geiser-debug-jump-to-debug-p nil)
		(geiser-debug-show-debug-p nil))
	    (let ((ret (funcall
			;; use `geiser-eval-region/wait' only when available
			;; in newer versions of `geiser'
			(if (fboundp 'geiser-eval-region/wait)
			    'geiser-eval-region/wait
			  'geiser-eval-region)
			(point-min)
			(point-max))))
	      (setq result (if output
			       (or (geiser-eval--retort-output ret)
				   "Geiser Interpreter produced no output")
			     (geiser-eval--retort-result-str ret "")))))
	  (when (not repl)
	    (save-current-buffer (set-buffer repl-buffer)
				 (geiser-repl-exit))
	    (set-process-query-on-exit-flag (get-buffer-process repl-buffer) nil)
	    (kill-buffer repl-buffer)))))
    result))

(defun org-babel-scheme--table-or-string (results)
  "Convert RESULTS into an appropriate elisp value.
If the results look like a list or tuple, then convert them into an
Emacs-lisp table, otherwise return the results as a string."
  (let ((res (org-babel-script-escape results)))
    (cond ((listp res)
           (mapcar (lambda (el)
		     (if (or (null el) (eq el 'null))
			 org-babel-scheme-null-to
		       el))
                   res))
	  (t res))))

(defun org-babel-execute:scheme (body params)
  "Execute a block of Scheme code with org-babel.
This function is called by `org-babel-execute-src-block'."
  (let* ((source-buffer (current-buffer))
	 (source-buffer-name (replace-regexp-in-string ;; zap surrounding *
			      "^ ?\\*\\([^*]+\\)\\*" "\\1"
			      (buffer-name source-buffer))))
    (save-excursion
      (let* ((result-type (cdr (assq :result-type params)))
	     (impl (or (when (cdr (assq :scheme params))
			 (intern (cdr (assq :scheme params))))
		       geiser-scheme-implementation
		       geiser-default-implementation
		       (car geiser-active-implementations)))
	     (session (org-babel-scheme-make-session-name
		       source-buffer-name (cdr (assq :session params)) impl))
	     (full-body (org-babel-expand-body:scheme body params))
	     (result-params (cdr (assq :result-params params)))
	     (result
	      (org-babel-scheme-execute-with-geiser
	       full-body		       ; code
	       (string= result-type "output")  ; output?
	       impl			       ; implementation
	       (and (not (string= session "none")) session)))) ; session
	(let ((table
	       (org-babel-reassemble-table
		result
		(org-babel-pick-name (cdr (assq :colname-names params))
				     (cdr (assq :colnames params)))
		(org-babel-pick-name (cdr (assq :rowname-names params))
				     (cdr (assq :rownames params))))))
	  (org-babel-result-cond result-params
	    result
	    (org-babel-scheme--table-or-string table)))))))

(provide 'ob-scheme)

;;; ob-scheme.el ends here

Re: Problem with org-babel and geiser

[Bob Heffernan]

On 22-02-05 19:39, Ihor Radchenko wrote:
> It's not a good idea to modify Org packaged together with Emacs. For
> testing, you can just use the attached version of ob-scheme.el. Load
> your Emacs and Org mode, open the attached ob-scheme.el, and run M-x
> eval-buffer. That should be sufficient to test the proposed patch on
> your side.
>
> If the patch solves the problem you are experiencing, let us know. I
> will then apply the proposed patch upstream.
>
> Note that even if I apply the patch, it will only be available on the
> latest version of Org mode (Org 9.5.2). It is _not_ what is shipped
> together with your Emacs. You will need to upgrade your Org mode using
> M-x package-install. See
> https://orgmode.org/manual/Installation.html#Installation for details.

Ihor,

I did as you described and the patched version of ob-scheme.el *does* seem to solve the problem.

Thanks & best regards,
Bob

Re: Problem with org-babel and geiser

[Rudolf Adamkovič]

Bob Heffernan <bob.heffernan@gmail.com> writes:

> I did as you described and the patched version of ob-scheme.el *does*
> seem to solve the problem.

Today, I needed to use Scheme with Org (9.5.2-24-g668205), and I found
that the #+RESULTS: always come back empty.  Oh, well!  I use Guile and
have everything configured correctly.  Has anyone merged the patch
discussed in this thread into Org?

Thank you.

Rudy
-- 
"Thinking is a momentary dismissal of irrelevancies."
-- Richard Buckminster Fuller, 1969

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: Problem with org-babel and geiser

[Neil Jerram]

Hi Rudy,

On Thu, 17 Mar 2022 at 17:07, Rudolf Adamkovič <salutis@me.com> wrote:
>
> Bob Heffernan <bob.heffernan@gmail.com> writes:
>
> > I did as you described and the patched version of ob-scheme.el *does*
> > seem to solve the problem.
>
> Today, I needed to use Scheme with Org (9.5.2-24-g668205), and I found
> that the #+RESULTS: always come back empty.  Oh, well!  I use Guile and
> have everything configured correctly.

What do you see if you add ":results output" to the "#+begin_src"
line, and then C-c C-c again?

> Has anyone merged the patch
> discussed in this thread into Org?

Did the patch help in your case?

Best wishes,
    Neil

Re: Problem with org-babel and geiser

[Rudolf Adamkovič]

Neil Jerram <neiljerram@gmail.com> writes:

Hi Neil!

> What do you see if you add ":results output" to the "#+begin_src"
> line, and then C-c C-c again?

#+BEGIN_SRC scheme :results output verbatim
(+ 1 1)
#+END_SRC

#+RESULTS:
: Geiser Interpreter produced no output

> Did the patch help in your case?

I had to update the file name inside of the patch:

- old: lisp/org/ob-scheme.el
- new: lisp/ob-scheme.el

After that, the patch applied cleanly on top of 'main'.

So, I restarted Emacs and tried:

#+BEGIN_SRC scheme
(+ 1 1)
#+END_SRC

#+RESULTS:
: 2

Further, I also tried to execute '(display "ABC")' while capturing
standard output using ':results output verbatim'.  It worked.

Lastly, I tried to use a persistent ':session' to avoid waiting for the
Geiser/Guile when executing code blocks.  It worked as well.

TL;DR The patch works.

P.S.

We should add 'testing/lisp/test-ob-scheme.el' or it will break again
sooner or later.  Can we make the Org test suite depend on Geiser and
Guile?  Given that Org uses Geiser out of the box, I think we should.
What do you think?

Rudy
-- 
"The whole science is nothing more than a refinement of everyday
thinking."
-- Albert Einstein, 1879-1955

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: Problem with org-babel and geiser

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

> P.S.
>
> We should add 'testing/lisp/test-ob-scheme.el' or it will break again
> sooner or later.  Can we make the Org test suite depend on Geiser and
> Guile?  Given that Org uses Geiser out of the box, I think we should.
> What do you think?

Sounds reasonable. Patches are welcome!

Best,
Ihor

Re: Re: Problem with org-babel and geiser

[Zelphir Kaltstahl]

Hello Rudy!

On 3/18/22 17:00, emacs-orgmode-request@gnu.org wrote:
> Today, I needed to use Scheme with Org (9.5.2-24-g668205), and I found
> that the #+RESULTS: always come back empty.  Oh, well!  I use Guile and
> have everything configured correctly.  Has anyone merged the patch
> discussed in this thread into Org?
>
> Thank you.
>
> Rudy

I've had lots of problems with getting it to work as well and have not solved 
all of them. At some point something changed, I think, which made my setup not 
work correctly any longer. You might be interested in the following issue on gitlab:

https://gitlab.com/emacs-geiser/guile/-/issues/13

I've not followed up further yet.

Best regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl

Re: Re: Problem with org-babel and geiser

[Rudolf Adamkovič]

Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:

Hello Zelphir!

> At some point something changed, I think, […]

In Geiser or in Org?  I use Geiser for work heavily and it works like
magic.  I ask because if the problem resides in Geiser, we could report
a new bug there.

Rudy
-- 
"One can begin to reason only when a clear picture has been formed in
the imagination."
-- Walter Warwick Sawyer, Mathematician's Delight, 1943

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: Problem with org-babel and geiser

[Zelphir Kaltstahl]

[-- Attachment #1: Type: text/plain, Size: 1603 bytes --]

Hello Rudy!

On 3/22/22 21:51, Rudolf Adamkovič wrote:
> Zelphir Kaltstahl<zelphirkaltstahl@posteo.de>  writes:
>
> Hello Zelphir!
>
>> At some point something changed, I think, […]
> In Geiser or in Org?  I use Geiser for work heavily and it works like
> magic.  I ask because if the problem resides in Geiser, we could report
> a new bug there.
>
> Rudy

Geiser seems to work, if I use /bin/sh with some arguments instead of GNU Bash. I am not sure what is going on with that though, because I did not have to do that in the past. In org-mode it still does not work. I will have to get into the topic again at some point, as I am unsatisfied without having GNU Guile source blocks in Org-mode working.

I've successfully used those before in multiple personal or learning projects (like inhttps://notabug.org/ZelphirKaltstahl/the-little-schemer/src/master/chapter-09-y-combinator/code.org). I spent so much time on fixing this problem, that I neglected my actual project that I wanted to continue. After 2 or 3 weekends, I left the issue at the status in that gitlab issue. I completely blended that one out for now and continued by copy pasting into Geiser REPL or using C-c C-r to run regions of code. That works at least. But it is less satisfying than doing literate programming in org-mode.

If anyone has concrete steps to run, to figure out the problem, I might find the energy to get into the matter again. For now I have put it up to "later" / "in the future" / "when some update comes" or something like that.

Best regards,
Zelphir

-- 
repositories:https://notabug.org/ZelphirKaltstahl

[-- Attachment #2: Type: text/html, Size: 2481 bytes --]

Re: Problem with org-babel and geiser

[Rudolf Adamkovič]

Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:

Hi Zelphir,

> Geiser seems to work, if I use /bin/sh with some arguments instead of
> GNU Bash.

I have no idea it means to "use /bin/sh with some arguments instead of
GNU Bash" in the context of Org Babel, Geiser, and Guile.  Then, please
see my other mail, the reply to Neil, where I test the patch posted
upthread and find that it indeed works well!

Rudy
-- 
"Programming reliably -- must be an activity of an undeniably
mathematical nature […] You see, mathematics is about thinking, and
doing mathematics is always trying to think as well as possible."
-- Edsger W. Dijkstra, 1981

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia