all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* [PATCH] ob-lua: Support all types and multiple values in results
@ 2024-04-23 16:44 Rudolf Adamkovič
  2024-04-23 17:05 ` Rudolf Adamkovič
  2024-04-24 13:05 ` Ihor Radchenko
  0 siblings, 2 replies; 18+ messages in thread
From: Rudolf Adamkovič @ 2024-04-23 16:44 UTC (permalink / raw)
  To: emacs-orgmode; +Cc: Rudolf Adamkovič

* etc/ORG-NEWS
(New and changed options): Describe the new option
'org-babel-lua-multiple-values-separator'.
(New features): Describe the main change, as per the title of this
commit message.
* lisp/ob-lua.el
(org-babel-lua-multiple-values-separator): Enable the user to
customize the string that separates the individual values in
multi-valued returns.
(org-babel-lua-wrapper-method): Support all Lua types and multi-valued
returns.  Further, do not pretty-print tables with one or more
extraneous newline characters.
(org-babel-lua-pp-wrapper-method): Remove in favor of the new, more
general 'org-babel-lua-wrapper-method'.
(org-babel-lua-evaluate-external-process): Adapt for the new
'org-babel-lua-wrapper-method'.
* testing/lisp/test-ob-lua.el
(test-ob-lua/colnames-yes-header-argument-pp):
(test-ob-lua/colnames-nil-header-argument):
(test-ob-lua/colnames-no-header-argument): Stop expecting extraneous
newlines, now that the pretty printer does not output them.
(test-ob-lua/types): Test nil, boolean, number, string, and table
results.
(test-ob-lua/multiple-values): Test multi-valued results.
---
 etc/ORG-NEWS                | 17 +++++++++
 lisp/ob-lua.el              | 73 ++++++++++++++++++++++---------------
 testing/lisp/test-ob-lua.el | 48 ++++++++++++++++++++++--
 3 files changed, 105 insertions(+), 33 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index fc2ff9e00..696f46e53 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -637,6 +637,11 @@ link when storing any type of external link type in an Org file, not
 just =id:= links.
 
 ** New and changed options
+*** ~org-babel-lua-multiple-values-separator~
+
+The string that separates the values of multi-valued results returned
+from Lua code blocks.
+
 *** =.avif= images are now recognized in ~org-html-inline-image-rules~
 
 In =ox-html=, =.avif= image links are now inlined by default.
@@ -1012,6 +1017,18 @@ The option can be customized either by
 2. by setting the file local keyword =LATEX_FOOTNOTE_COMMAND=
 
 ** New features
+*** =ob-lua=: Support all types and multiple values in results
+
+Lua code blocks can now return values of any type and can also return
+multiple values.  Previously, values of certain types were incorrectly
+converted to the empty string =""=, which broke HTML export for inline
+code blocks, and multiple values were incorrectly concatenated, where
+~return 1, 2, 3~ was evaluated as =123=.
+
+Multiple values are comma-separated by default, so that they work well
+with inline code blocks.  To change the string used as the separator,
+customize ~org-babel-lua-multiple-values-separator~.
+
 *** ~org-paste-subtree~ now handles =C-u= and =C-u C-u= prefix arguments specially
 
 With =C-u= prefix argument, force inserting a sibling heading below.
diff --git a/lisp/ob-lua.el b/lisp/ob-lua.el
index b241fccdc..92a1b3344 100644
--- a/lisp/ob-lua.el
+++ b/lisp/ob-lua.el
@@ -81,6 +81,13 @@ This will typically be `lua-mode'."
   :package-version '(Org . "8.3")
   :type 'symbol)
 
+(defcustom org-babel-lua-multiple-values-separator ", "
+  "Separate multiple values with this string."
+  :group 'org-babel
+  :version "30.0"
+  :package-version '(Org . "9.7")
+  :type 'string)
+
 (defun org-babel-execute:lua (body params)
   "Execute Lua BODY according to PARAMS.
 This function is called by `org-babel-execute-src-block'."
@@ -251,41 +258,47 @@ function main()
 %s
 end
 
-fd=io.open(\"%s\", \"w\")
-fd:write( main() )
-fd:close()")
-(defvar org-babel-lua-pp-wrapper-method
-  "
--- table to string
-function t2s(t, indent)
+function dump(it, indent)
    if indent == nil then
-      indent = \"\"
+      indent = ''
    end
-   if type(t) == \"table\" then
-      ts = \"\"
-      for k,v in pairs(t) do
-         if type(v) == \"table\" then
-            ts = ts .. indent .. t2s(k,indent .. \"  \") .. \" = \\n\" ..
-               t2s(v, indent .. \"  \")
-         else
-            ts = ts .. indent .. t2s(k,indent .. \"  \") .. \" = \" ..
-               t2s(v, indent .. \"  \") .. \"\\n\"
+   if type(it) == 'table' and %s then
+      local count = 0
+      for _ in pairs(it) do
+         count = count + 1
+      end
+      local result = ''
+      if #indent ~= 0 then
+         result = result .. '\\n'
+      end
+      for key, value in pairs(it) do
+         result = result
+            .. indent
+            .. dump(key)
+            .. ' = '
+            .. dump(value, indent .. '  ')
+         count = count - 1
+         if count ~= 0 then
+            result = result .. '\\n'
          end
       end
-      return ts
+      return result
    else
-      return tostring(t)
+      return tostring(it)
    end
 end
 
-
-function main()
-%s
+function combine(...)
+  local result = {}
+  for index = 1, select('#', ...) do
+    result[index] = dump(select(index, ...))
+  end
+  return table.concat(result, '%s')
 end
 
-fd=io.open(\"%s\", \"w\")
-fd:write(t2s(main()))
-fd:close()")
+output = io.open('%s', 'w')
+output:write(combine(main()))
+output:close()")
 
 (defun org-babel-lua-evaluate
     (session body &optional result-type result-params preamble)
@@ -319,15 +332,17 @@ PREAMBLE string is appended to BODY."
 		      (concat
 		       preamble (and preamble "\n")
 		       (format
-			(if (member "pp" result-params)
-			    org-babel-lua-pp-wrapper-method
-			  org-babel-lua-wrapper-method)
+                        org-babel-lua-wrapper-method
 			(mapconcat
 			 (lambda (line) (format "\t%s" line))
 			 (split-string
 			  (org-remove-indentation
 			   (org-trim body))
-			  "[\r\n]") "\n")
+			  "[\r\n]")
+                         "\n")
+                        (if (member "pp" result-params)
+                            "true" "false")
+                        org-babel-lua-multiple-values-separator
 			(org-babel-process-file-name tmp-file 'noquote))))
 		     (org-babel-eval-read-file tmp-file))))))
     (org-babel-result-cond result-params
diff --git a/testing/lisp/test-ob-lua.el b/testing/lisp/test-ob-lua.el
index f30e65bb3..0a60c68ca 100644
--- a/testing/lisp/test-ob-lua.el
+++ b/testing/lisp/test-ob-lua.el
@@ -77,9 +77,9 @@ return x[1]
 
 
 (ert-deftest test-ob-lua/colnames-yes-header-argument-pp ()
-  "Test table passing with `colnames' header and pp option."
+  "Test table passing with `colnames' header and `pp' option."
   (should
-   (equal "a = 12\nb = 13\n"
+   (equal "a = 12\nb = 13"
 	  (org-test-with-temp-text
 	      "#+name: eg
 | col | val |
@@ -99,7 +99,7 @@ return x
 (ert-deftest test-ob-lua/colnames-nil-header-argument ()
   "Test table with `colnames' set to `nil'."
   (should
-   (equal "1 = a\n2 = b\n"
+   (equal "1 = a\n2 = b"
 	  (org-test-with-temp-text
 	      "#+name: eg
 | col |
@@ -119,7 +119,7 @@ return x
 (ert-deftest test-ob-lua/colnames-no-header-argument ()
   "Test table passing without `colnames'."
   (should
-   (equal "1 = col\n2 = a\n3 = b\n"
+   (equal "1 = col\n2 = a\n3 = b"
 	  (org-test-with-temp-text
 	      "#+name: eg
 | col |
@@ -136,6 +136,46 @@ return x
 	    (org-babel-next-src-block)
 	    (org-babel-execute-src-block)))))
 
+(ert-deftest test-ob-lua/types ()
+  "Test returning different types."
+  (should
+   (equal "nil"
+          (org-test-with-temp-text "src_lua{return nil}"
+            (org-babel-execute-src-block))))
+  (should
+   (equal "true"
+          (org-test-with-temp-text "src_lua{return true}"
+            (org-babel-execute-src-block))))
+  (should
+   (equal "false"
+          (org-test-with-temp-text "src_lua{return false}"
+            (org-babel-execute-src-block))))
+  (should
+   (equal 1
+          (org-test-with-temp-text "src_lua{return 1}"
+            (org-babel-execute-src-block))))
+  (should
+   (equal "hello world"
+          (org-test-with-temp-text "src_lua{return 'hello world'}"
+            (org-babel-execute-src-block))))
+  (should
+   (equal 0
+          (string-match "table: 0x[0-9A-F]+"
+                        (org-test-with-temp-text "src_lua{return {}}"
+                          (org-babel-execute-src-block))))))
+
+(ert-deftest test-ob-lua/multiple-values ()
+  "Test returning multiple values."
+  (should
+   (equal "1, 2, 3"
+          (org-test-with-temp-text "src_lua{return 1, 2, 3}"
+            (org-babel-execute-src-block))))
+  (should
+   (equal "1|2|3"
+          (let ((org-babel-lua-multiple-values-separator "|"))
+            (org-test-with-temp-text "src_lua{return 1, 2, 3}"
+              (org-babel-execute-src-block))))))
+
 (provide 'test-ob-lua)
 
 ;;; test-ob-lua.el ends here
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-23 16:44 [PATCH] ob-lua: Support all types and multiple values in results Rudolf Adamkovič
@ 2024-04-23 17:05 ` Rudolf Adamkovič
  2024-04-24 14:20   ` Ihor Radchenko
  2024-04-24 13:05 ` Ihor Radchenko
  1 sibling, 1 reply; 18+ messages in thread
From: Rudolf Adamkovič @ 2024-04-23 17:05 UTC (permalink / raw)
  To: emacs-orgmode

Rudolf Adamkovič <rudolf@adamkovic.org> writes:

> +Lua code blocks can now return values of any type and can also return
> +multiple values.  Previously, values of certain types were incorrectly
> +converted to the empty string =""=, which broke HTML export for inline
> +code blocks, and multiple values were incorrectly concatenated, where
> +~return 1, 2, 3~ was evaluated as =123=.

Also,

  #+BEGIN_SRC lua
  return {1, 2, 3}
  #+END_SRC

previously crashed, as did

  #+BEGIN_SRC lua
  function divide()
     error("oops")
     return divident / divisor
  end
  return pcall(oops)
  #+END_SRC

All that works now, with no more crashes.

P.S. #1

We still have an old bug where

  src_lua{return string.match("A {B} C", "%b{}")}

is misjudged to be a table:

  org-babel-insert-result: Inline error: list result cannot be used

P.S. #2

I did not update any session-related code.

Currently,

  #+BEGIN_SRC lua :session x
  print 1
  #+END_SRC

gives

  ... Sessions currently not supported, work in progress

This is also documented in the header

  ;; Requirements:
  ;; for session support, lua-mode is needed.
  ;; [...]
  ;; However, sessions are not yet working.

This half-finished session support should be removed, IMHO.

If someone needs it and are willing to finish it, they can still dig it
up it in the history.  That is what VC is for, after all.

As of now, all that session-related dead code only pollutes the file,
making it harder to change, as it drifts out of sync...

Rudy
-- 
"All you have to do is write one true sentence.  Write the truest
sentence that you know."  --- Ernest Miller Hemingway (1899-1961)

Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-23 16:44 [PATCH] ob-lua: Support all types and multiple values in results Rudolf Adamkovič
  2024-04-23 17:05 ` Rudolf Adamkovič
@ 2024-04-24 13:05 ` Ihor Radchenko
  2024-04-24 15:01   ` Rudolf Adamkovič
  1 sibling, 1 reply; 18+ messages in thread
From: Ihor Radchenko @ 2024-04-24 13:05 UTC (permalink / raw)
  To: Rudolf Adamkovič; +Cc: emacs-orgmode

Rudolf Adamkovič <rudolf@adamkovic.org> writes:

> * etc/ORG-NEWS
> (New and changed options): Describe the new option
> 'org-babel-lua-multiple-values-separator'.
> (New features): Describe the main change, as per the title of this
> commit message.

Thanks!
Applied, onto main, after removing redundant :version keyword from
defcustom.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=252cc0be0

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-23 17:05 ` Rudolf Adamkovič
@ 2024-04-24 14:20   ` Ihor Radchenko
  0 siblings, 0 replies; 18+ messages in thread
From: Ihor Radchenko @ 2024-04-24 14:20 UTC (permalink / raw)
  To: Rudolf Adamkovič; +Cc: emacs-orgmode

Rudolf Adamkovič <rudolf@adamkovic.org> writes:

> P.S. #1
>
> We still have an old bug where
>
>   src_lua{return string.match("A {B} C", "%b{}")}
>
> is misjudged to be a table:
>
>   org-babel-insert-result: Inline error: list result cannot be used

May you create a test for this with expected failure?

> P.S. #2
>
> I did not update any session-related code.
>
> Currently,
>
>   #+BEGIN_SRC lua :session x
>   print 1
>   #+END_SRC
>
> gives
>
>   ... Sessions currently not supported, work in progress
>
> This is also documented in the header
>
>   ;; Requirements:
>   ;; for session support, lua-mode is needed.
>   ;; [...]
>   ;; However, sessions are not yet working.
>
> This half-finished session support should be removed, IMHO.

+1

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-24 13:05 ` Ihor Radchenko
@ 2024-04-24 15:01   ` Rudolf Adamkovič
  2024-04-26 13:40     ` Ihor Radchenko
  0 siblings, 1 reply; 18+ messages in thread
From: Rudolf Adamkovič @ 2024-04-24 15:01 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: emacs-orgmode

Ihor Radchenko <yantar92@posteo.net> writes:

> May you create a test for this with expected failure?

Sure!  Here is one:

  (should
   (equal "{B}"
          (org-test-with-temp-text
              "src_lua{return string.match('A {B} C', '%b{}')}"
            (org-babel-execute-src-block))))

The return value from 'string.match' is the substring "{B}", which Babel
misjudges to be a list and signals the error:

  (user-error "Inline error: list result cannot be used")

> Applied, onto main, after [...]

Thank you, Ihor!

Rudy
-- 
"I do not fear death.  I had been dead for billions and billions of
years before I was born, and had not suffered the slightest
inconvenience from it."  --- Mark Twain, paraphrased

Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-24 15:01   ` Rudolf Adamkovič
@ 2024-04-26 13:40     ` Ihor Radchenko
  2024-04-27 10:57       ` Max Nikulin
  2024-04-27 16:33       ` Rudolf Adamkovič
  0 siblings, 2 replies; 18+ messages in thread
From: Ihor Radchenko @ 2024-04-26 13:40 UTC (permalink / raw)
  To: Rudolf Adamkovič; +Cc: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 657 bytes --]

Rudolf Adamkovič <rudolf@adamkovic.org> writes:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> May you create a test for this with expected failure?
>
> Sure!  Here is one:
>
>   (should
>    (equal "{B}"
>           (org-test-with-temp-text
>               "src_lua{return string.match('A {B} C', '%b{}')}"
>             (org-babel-execute-src-block))))
>
> The return value from 'string.match' is the substring "{B}", which Babel
> misjudges to be a list and signals the error:
>
>   (user-error "Inline error: list result cannot be used")

Maybe something like the attached.
(I am a casual lua user, so better double check)


[-- Attachment #2: 0001-ob-lua-Fix-edge-case-when-output-is-string.patch --]
[-- Type: text/x-patch, Size: 1840 bytes --]

From bfe0fa3d78cc13b84dec33c230755c9f4693a685 Mon Sep 17 00:00:00 2001
Message-ID: <bfe0fa3d78cc13b84dec33c230755c9f4693a685.1714138810.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Fri, 26 Apr 2024 16:39:07 +0300
Subject: [PATCH] ob-lua: Fix edge case when output is {string}
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/ob-lua.el (org-babel-lua-wrapper-method): Make sure that string
result is enclosed in "...", so that Org mode does not try to guess
the output type heuristically.
* testing/lisp/test-ob-lua.el (test-ob-lua/types): New test.

Co-authored-by: Rudolf Adamkovič <rudolf@adamkovic.org>
---
 lisp/ob-lua.el              | 2 ++
 testing/lisp/test-ob-lua.el | 6 ++++++
 2 files changed, 8 insertions(+)

diff --git a/lisp/ob-lua.el b/lisp/ob-lua.el
index 041abfabc..afe88065d 100644
--- a/lisp/ob-lua.el
+++ b/lisp/ob-lua.el
@@ -282,6 +282,8 @@ (defvar org-babel-lua-wrapper-method
          end
       end
       return result
+   elseif type(it) == 'string' then
+      return '\"' .. it .. '\"'
    else
       return tostring(it)
    end
diff --git a/testing/lisp/test-ob-lua.el b/testing/lisp/test-ob-lua.el
index 0a60c68ca..398ab761d 100644
--- a/testing/lisp/test-ob-lua.el
+++ b/testing/lisp/test-ob-lua.el
@@ -158,6 +158,12 @@ (ert-deftest test-ob-lua/types ()
    (equal "hello world"
           (org-test-with-temp-text "src_lua{return 'hello world'}"
             (org-babel-execute-src-block))))
+  ;; Edge case: string matching list syntax.
+  (should
+   (equal "{B}"
+	  (org-test-with-temp-text
+	      "src_lua{return string.match('A {B} C', '%b{}')}"
+	    (org-babel-execute-src-block))))
   (should
    (equal 0
           (string-match "table: 0x[0-9A-F]+"
-- 
2.44.0


[-- Attachment #3: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-26 13:40     ` Ihor Radchenko
@ 2024-04-27 10:57       ` Max Nikulin
  2024-04-27 16:33       ` Rudolf Adamkovič
  1 sibling, 0 replies; 18+ messages in thread
From: Max Nikulin @ 2024-04-27 10:57 UTC (permalink / raw)
  To: emacs-orgmode

On 26/04/2024 20:40, Ihor Radchenko wrote:
> +++ b/lisp/ob-lua.el
> @@ -282,6 +282,8 @@ (defvar org-babel-lua-wrapper-method
>            end
>         end
>         return result
> +   elseif type(it) == 'string' then
> +      return '\"' .. it .. '\"'

If you are adding quotes around then quote characters in the value 
should be escaped.



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-26 13:40     ` Ihor Radchenko
  2024-04-27 10:57       ` Max Nikulin
@ 2024-04-27 16:33       ` Rudolf Adamkovič
  2024-04-28 12:36         ` Ihor Radchenko
  1 sibling, 1 reply; 18+ messages in thread
From: Rudolf Adamkovič @ 2024-04-27 16:33 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: emacs-orgmode

Ihor Radchenko <yantar92@posteo.net> writes:

> Maybe something like the attached.

Nice:

  src_elisp{"foo"} {{{results(=foo=)}}}
  src_python{return "foo"} {{{results(=foo=)}}}
  src_lua{return "foo"} {{{results(=foo=)}}}

That said, I have just noticed:

  # expected: a, b, c
  src_lua{return "a", "b", "c"} {{{results(="a"  "b"  "c"=)}}}

  # expected: 1, 2
  src_lua{return 1, 2} {{{results(=1\, 2=)}}}

Oops!

We need *better tests*, otherwise all this is kind of useless.

I will hack on this some more...

Rudy
-- 
"Thinking is a momentary dismissal of irrelevancies."
--- Richard Buckminster Fuller, 1969

Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-27 16:33       ` Rudolf Adamkovič
@ 2024-04-28 12:36         ` Ihor Radchenko
  2024-04-29 15:57           ` Rudolf Adamkovič
  0 siblings, 1 reply; 18+ messages in thread
From: Ihor Radchenko @ 2024-04-28 12:36 UTC (permalink / raw)
  To: Rudolf Adamkovič; +Cc: emacs-orgmode

Rudolf Adamkovič <rudolf@adamkovic.org> writes:

> ...
> We need *better tests*, otherwise all this is kind of useless.
>
> I will hack on this some more...

Will you be willing to provide some?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-28 12:36         ` Ihor Radchenko
@ 2024-04-29 15:57           ` Rudolf Adamkovič
  2024-04-29 20:26             ` Rudolf Adamkovič
  0 siblings, 1 reply; 18+ messages in thread
From: Rudolf Adamkovič @ 2024-04-29 15:57 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: emacs-orgmode

Ihor Radchenko <yantar92@posteo.net> writes:

>> We need *better tests*, otherwise all this is kind of useless.
>>
>> I will hack on this some more...
>
> Will you be willing to provide some?

Definitely!  I am on it.

Rudy
-- 
"Genius is 1% inspiration and 99% perspiration."
--- Thomas Alva Edison, 1932

Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-29 15:57           ` Rudolf Adamkovič
@ 2024-04-29 20:26             ` Rudolf Adamkovič
  2024-04-30 10:24               ` Ihor Radchenko
  2024-05-01 11:05               ` Max Nikulin
  0 siblings, 2 replies; 18+ messages in thread
From: Rudolf Adamkovič @ 2024-04-29 20:26 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 134 bytes --]

Rudolf Adamkovič <rudolf@adamkovic.org> writes:

> Definitely!  I am on it.

All right, how about the attached patch?

Rudy


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-lua-Quote-list-like-strings-sort-tables-and-parse.patch --]
[-- Type: text/x-patch, Size: 14384 bytes --]

From 40270bc62f951cfd20916c17e2dfc52863d6b8e4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Rudolf=20Adamkovi=C4=8D?= <rudolf@adamkovic.org>
Date: Mon, 29 Apr 2024 21:42:04 +0200
Subject: [PATCH] ob-lua: Quote list-like strings, sort tables, and parse bare
 returns

* lisp/ob-lua.el (org-babel-lua-multiple-values-separator): Change the
default value from ", " to "|" to avoid '\\,' appearing in strings.
(org-babel-lua-wrapper-method): Improve 3 aspects: (1) Quote strings
with list-like content to prevent Org Babel from incorrectly guessing
their type, (2) sort pretty-printed non-sequential tables to make them
reproducible, and (3) handle implicit nils produced by 'return' used
with no arguments.
* testing/lisp/test-ob-lua.el (test-ob-lua/result/nil):
(test-ob-lua/result/nil/explicit):
(test-ob-lua/result/boolean):
(test-ob-lua/results/number/integer):
(test-ob-lua/results/number/integer/negative):
(test-ob-lua/results/number/integer/multiple):
(test-ob-lua/results/number/real):
(test-ob-lua/results/number/real/multiple):
(test-ob-lua/results/number/infinity):
(test-ob-lua/results/string/single-quotes):
(test-ob-lua/results/string/double-quotes):
(test-ob-lua/results/string/multiple):
(test-ob-lua/results/string/list-like):
(test-ob-lua/results/string/list-like/brackets):
(test-ob-lua/results/string/list-like/curlies):
(test-ob-lua/result/table):
(test-ob-lua/result/table/pretty-print):
(test-ob-lua/result/table/pretty-print/sorted):
(test-ob-lua/results/value-separator): New tests.
---
 lisp/ob-lua.el              |  41 ++++--
 testing/lisp/test-ob-lua.el | 264 ++++++++++++++++++++++++++++++++----
 2 files changed, 266 insertions(+), 39 deletions(-)

diff --git a/lisp/ob-lua.el b/lisp/ob-lua.el
index 041abfabc..5a0fdb18b 100644
--- a/lisp/ob-lua.el
+++ b/lisp/ob-lua.el
@@ -81,7 +81,7 @@ This will typically be `lua-mode'."
   :package-version '(Org . "8.3")
   :type 'symbol)
 
-(defcustom org-babel-lua-multiple-values-separator ", "
+(defcustom org-babel-lua-multiple-values-separator "|"
   "Separate multiple values with this string."
   :group 'org-babel
   :package-version '(Org . "9.7")
@@ -261,26 +261,33 @@ function dump(it, indent)
    if indent == nil then
       indent = ''
    end
+
    if type(it) == 'table' and %s then
-      local count = 0
-      for _ in pairs(it) do
-         count = count + 1
-      end
       local result = ''
+
       if #indent ~= 0 then
          result = result .. '\\n'
       end
-      for key, value in pairs(it) do
+
+      local keys = {}
+      for key in pairs(it) do
+        table.insert(keys, key)
+      end
+
+      table.sort(keys)
+
+      for index, key in pairs(keys) do
+         local value = it[key]
          result = result
             .. indent
             .. dump(key)
             .. ' = '
             .. dump(value, indent .. '  ')
-         count = count - 1
-         if count ~= 0 then
+         if index ~= #keys then
             result = result .. '\\n'
          end
       end
+
       return result
    else
       return tostring(it)
@@ -288,11 +295,27 @@ function dump(it, indent)
 end
 
 function combine(...)
+  local quotes = '\"'
   local result = {}
+
   for index = 1, select('#', ...) do
     result[index] = dump(select(index, ...))
   end
-  return table.concat(result, '%s')
+
+  if #result == 0 then
+    return dump(nil)
+  end
+
+  if #result == 1 then
+    local value = result[1]
+    if string.find(value, '[%%(%%[{]') == 1 then
+      return quotes .. value .. quotes
+    else
+      return value
+    end
+  end
+
+  return quotes .. table.concat(result, '%s') .. quotes
 end
 
 output = io.open('%s', 'w')
diff --git a/testing/lisp/test-ob-lua.el b/testing/lisp/test-ob-lua.el
index 0a60c68ca..775a5cf14 100644
--- a/testing/lisp/test-ob-lua.el
+++ b/testing/lisp/test-ob-lua.el
@@ -136,45 +136,249 @@ return x
 	    (org-babel-next-src-block)
 	    (org-babel-execute-src-block)))))
 
-(ert-deftest test-ob-lua/types ()
-  "Test returning different types."
+(ert-deftest test-ob-lua/result/nil ()
+  "Test returning nothing."
   (should
-   (equal "nil"
-          (org-test-with-temp-text "src_lua{return nil}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return} {{{results(=nil=)}}}"
+    (org-test-with-temp-text "src_lua{return}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/result/nil/explicit ()
+  "Test returning nothing explicitly."
+  (should
+   (equal
+    "src_lua{return nil} {{{results(=nil=)}}}"
+    (org-test-with-temp-text "src_lua{return nil}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/result/boolean ()
+  "Test returning the boolean values true and false."
+  (should
+   (equal
+    "src_lua{return true} {{{results(=true=)}}}"
+    (org-test-with-temp-text "src_lua{return true}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max)))))
   (should
-   (equal "true"
-          (org-test-with-temp-text "src_lua{return true}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return false} {{{results(=false=)}}}"
+    (org-test-with-temp-text "src_lua{return false}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/number/integer ()
+  "Test returning integers."
   (should
-   (equal "false"
-          (org-test-with-temp-text "src_lua{return false}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return 1} {{{results(=1=)}}}"
+    (org-test-with-temp-text "src_lua{return 1}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/number/integer/negative ()
+  "Test returning negative integers."
   (should
-   (equal 1
-          (org-test-with-temp-text "src_lua{return 1}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return -1} {{{results(=-1=)}}}"
+    (org-test-with-temp-text "src_lua{return -1}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/number/integer/multiple ()
+  "Test returning multiple integers at once."
   (should
-   (equal "hello world"
-          (org-test-with-temp-text "src_lua{return 'hello world'}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return 1, 2, 3} {{{results(=1|2|3=)}}}"
+    (org-test-with-temp-text "src_lua{return 1, 2, 3}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/number/real ()
+  "Test returning real numbers."
   (should
-   (equal 0
-          (string-match "table: 0x[0-9A-F]+"
-                        (org-test-with-temp-text "src_lua{return {}}"
-                          (org-babel-execute-src-block))))))
+   (equal
+    "src_lua{return 1.5} {{{results(=1.5=)}}}"
+    (org-test-with-temp-text "src_lua{return 1.5}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
 
-(ert-deftest test-ob-lua/multiple-values ()
-  "Test returning multiple values."
+(ert-deftest test-ob-lua/results/number/real/multiple ()
+  "Test returning multiple real numbers at once."
   (should
-   (equal "1, 2, 3"
-          (org-test-with-temp-text "src_lua{return 1, 2, 3}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return 1.5, 2.5, 3.5} {{{results(=1.5|2.5|3.5=)}}}"
+    (org-test-with-temp-text "src_lua{return 1.5, 2.5, 3.5}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/number/infinity ()
+  "Test returning the infinity."
+  (should
+   (equal
+    "src_lua{return 1 / 0} {{{results(=inf=)}}}"
+    (org-test-with-temp-text "src_lua{return 1 / 0}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/single-quotes ()
+  "Test returning strings in single quotes."
+  (should
+   (equal
+    "src_lua{return 'hello world'} {{{results(=hello world=)}}}"
+    (org-test-with-temp-text "src_lua{return 'hello world'}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/double-quotes ()
+  "Test returning strings in double quotes."
+  (should
+   (equal
+    "src_lua{return \"hello world\"} {{{results(=hello world=)}}}"
+    (org-test-with-temp-text "src_lua{return \"hello world\"}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/multiple ()
+  "Test returning multiple strings at once."
+  (should
+   (equal
+    "src_lua{return 'a', 'b'} {{{results(=a|b=)}}}"
+    (org-test-with-temp-text "src_lua{return 'a', 'b'}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/list-like ()
+  "Test returning strings that look like \"(...)\" lists."
+  (should
+   (equal
+    (concat "src_lua{return string.match('A (B) C', '%b()')}"
+            " {{{results(=(B)=)}}}")
+    (org-test-with-temp-text
+        "src_lua{return string.match('A (B) C', '%b()')}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/list-like/brackets ()
+  "Test returning strings that look like \"[...]\" lists."
+  (should
+   (equal
+    (concat "src_lua{return string.match('A [B] C', '%b[]')}"
+            " {{{results(=[B]=)}}}")
+    (org-test-with-temp-text
+        "src_lua{return string.match('A [B] C', '%b[]')}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/list-like/curlies ()
+  "Test returning strings that look like \"{...}\" lists."
+  (should
+   (equal
+    (concat "src_lua{return string.match('A {B} C', '%b{}')}"
+            " {{{results(={B}=)}}}")
+    (org-test-with-temp-text
+	"src_lua{return string.match('A {B} C', '%b{}')}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/result/table ()
+  "Test returning table references."
+  (should
+   (equal
+    0
+    (string-match
+     "src_lua{return {}} {{{results(=table: 0x[0-9A-F]+=)}}}"
+     (org-test-with-temp-text "src_lua{return {}}"
+       (org-babel-execute-src-block)
+       (buffer-substring-no-properties (point-min)
+                                       (point-max)))))))
+
+(ert-deftest test-ob-lua/result/table/pretty-print ()
+  "Test returning and pretty-printing sequential tables."
+  (should
+   (equal (string-join
+           '("#+BEGIN_SRC lua :results pp"
+             "return {10, {20, 30, {40, 50}, 60}, 70}"
+             "#+END_SRC"
+             ""
+             "#+RESULTS:"
+             ": 1 = 10"
+             ": 2 = "                   ; FIXME Trailing space.
+             ":   1 = 20"
+             ":   2 = 30"
+             ":   3 = "                 ; FIXME Trailing space.
+             ":     1 = 40"
+             ":     2 = 50"
+             ":   4 = 60"
+             ": 3 = 70"
+             "")
+           "\n")
+          (org-test-with-temp-text
+              (string-join
+               '("#+BEGIN_SRC lua :results pp"
+                 "return {10, {20, 30, {40, 50}, 60}, 70}<point>"
+                 "#+END_SRC")
+               "\n")
+	    (org-babel-execute-src-block)
+            (buffer-substring-no-properties (point-min)
+                                            (point-max))))))
+
+(ert-deftest test-ob-lua/result/table/pretty-print/sorted ()
+  "Test returning and pretty-printing non-sequential tables."
+  (should
+   (equal (string-join
+           '("#+BEGIN_SRC lua :results pp"
+             "return {b = 20, c = 30, a = 10}"
+             "#+END_SRC"
+             ""
+             "#+RESULTS:"
+             ;; NOTE The keys are sorted alphabetically.
+             ": a = 10"
+             ": b = 20"
+             ": c = 30"
+             "")
+           "\n")
+          (org-test-with-temp-text
+              (string-join
+               '("#+BEGIN_SRC lua :results pp"
+                 "return {b = 20, c = 30, a = 10}"
+                 "#+END_SRC")
+               "\n")
+	    (org-babel-execute-src-block)
+            (buffer-substring-no-properties (point-min)
+                                            (point-max))))))
+
+(ert-deftest test-ob-lua/results/value-separator ()
+  "Test customizing the separator of multiple values."
+  ;; TODO Once Org Babel supports returning lists from inline blocks,
+  ;; instead of trapping with the user error: "Inline error: list
+  ;; result cannot be used", use those for multiple values.
   (should
-   (equal "1|2|3"
-          (let ((org-babel-lua-multiple-values-separator "|"))
-            (org-test-with-temp-text "src_lua{return 1, 2, 3}"
-              (org-babel-execute-src-block))))))
+   (equal
+    "src_lua{return 1, 2, 3} {{{results(=1\t2\t3=)}}}"
+    (org-test-with-temp-text "src_lua{return 1, 2, 3}"
+      (let ((org-babel-lua-multiple-values-separator "\t"))
+        (org-babel-execute-src-block))
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
 
 (provide 'test-ob-lua)
 
-- 
2.39.3 (Apple Git-146)


[-- Attachment #3: Type: text/plain, Size: 277 bytes --]

-- 
"It is no paradox to say that in our most theoretical moods we may be
nearest to our most practical applications."
--- Alfred North Whitehead, 1861-1947

Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-29 20:26             ` Rudolf Adamkovič
@ 2024-04-30 10:24               ` Ihor Radchenko
  2024-05-02 15:57                 ` Rudolf Adamkovič
  2024-05-02 15:58                 ` Rudolf Adamkovič
  2024-05-01 11:05               ` Max Nikulin
  1 sibling, 2 replies; 18+ messages in thread
From: Ihor Radchenko @ 2024-04-30 10:24 UTC (permalink / raw)
  To: Rudolf Adamkovič; +Cc: emacs-orgmode

Rudolf Adamkovič <rudolf@adamkovic.org> writes:

> Rudolf Adamkovič <rudolf@adamkovic.org> writes:
>
>> Definitely!  I am on it.

Thanks!

> All right, how about the attached patch?
> ...
> * lisp/ob-lua.el (org-babel-lua-multiple-values-separator): Change the
> default value from ", " to "|" to avoid '\\,' appearing in strings.

May you please elaborate why this breaking change is going to lead to
significant improvement? How is using "," worse than using "|"? Either
way, strings containing the separator will become problematic and need
to be escaped. Moreover, using "|" may lead to interpreting the output
as Org tables, which may be surprising to users.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-29 20:26             ` Rudolf Adamkovič
  2024-04-30 10:24               ` Ihor Radchenko
@ 2024-05-01 11:05               ` Max Nikulin
  2024-05-02 16:00                 ` Rudolf Adamkovič
  1 sibling, 1 reply; 18+ messages in thread
From: Max Nikulin @ 2024-05-01 11:05 UTC (permalink / raw)
  To: Rudolf Adamkovič; +Cc: emacs-orgmode

On 30/04/2024 03:26, Rudolf Adamkovič wrote:
> +    local value = result[1]
> +    if string.find(value, '[%%(%%[{]') == 1 then
> +      return quotes .. value .. quotes

Am I wrong that quotes in value may cause issues? I expect some way of 
escaping " characters.


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-30 10:24               ` Ihor Radchenko
@ 2024-05-02 15:57                 ` Rudolf Adamkovič
  2024-05-02 15:58                 ` Rudolf Adamkovič
  1 sibling, 0 replies; 18+ messages in thread
From: Rudolf Adamkovič @ 2024-05-02 15:57 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: emacs-orgmode

Ihor Radchenko <yantar92@posteo.net> writes:

> May you please elaborate why this breaking change is going to lead to
> significant improvement? How is using "," worse than using "|"? Either
> way, strings containing the separator will become problematic and need
> to be escaped. Moreover, using "|" may lead to interpreting the output
> as Org tables, which may be surprising to users.

Thank you for the review!

The change of the separator was a lazy move on my part.  I appologize.
I changed the separator because I could not figure out how to make Org
behave sanely.  The problem is not specific to Lua:

  src_elisp{"3, 2, 1, and go!"} {{{results(=3\, 2\, 1\, and go!=)}}}

Why Org is escaping commas here?  In other words, should my tests expect
\-escapes in front of commas, or is that a bug?

I am attaching a V2 of the patch that:

- does not modify the separator anymore (resulting in 3 failing tests)
- adds one more test (which is just a sanity check, you can ignore it)
- fixes Lua code indentation (as indented by the Lua major mode)

Thank you for your guidance.

Rudy
-- 
"Be especially critical of any statement following the word
'obviously.'"
--- Anna Pell Wheeler, 1883-1966

Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-04-30 10:24               ` Ihor Radchenko
  2024-05-02 15:57                 ` Rudolf Adamkovič
@ 2024-05-02 15:58                 ` Rudolf Adamkovič
  2024-05-02 16:02                   ` Rudolf Adamkovič
  1 sibling, 1 reply; 18+ messages in thread
From: Rudolf Adamkovič @ 2024-05-02 15:58 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: emacs-orgmode

Ihor Radchenko <yantar92@posteo.net> writes:

> May you please elaborate why this breaking change is going to lead to
> significant improvement? How is using "," worse than using "|"? Either
> way, strings containing the separator will become problematic and need
> to be escaped. Moreover, using "|" may lead to interpreting the output
> as Org tables, which may be surprising to users.

Thank you for the review!

The change of the separator was a lazy move on my part.  I appologize.
I changed the separator because I could not figure out how to make Org
behave sanely.  The problem is not specific to Lua:

  src_elisp{"3, 2, 1, and go!"} {{{results(=3\, 2\, 1\, and go!=)}}}

Why Org is escaping commas here?  In other words, should my tests expect
\-escapes in front of commas, or is that a bug?

I am attaching a V2 of the patch that:

- does not modify the separator anymore (resulting in 3 failing tests)
- adds one more test (which is just a sanity check, you can ignore it)
- fixes Lua code indentation (as indented by the Lua major mode)

Thank you for your guidance.

Rudy
-- 
"Be especially critical of any statement following the word
'obviously.'"
--- Anna Pell Wheeler, 1883-1966

Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-05-01 11:05               ` Max Nikulin
@ 2024-05-02 16:00                 ` Rudolf Adamkovič
  2024-05-02 16:41                   ` Ihor Radchenko
  0 siblings, 1 reply; 18+ messages in thread
From: Rudolf Adamkovič @ 2024-05-02 16:00 UTC (permalink / raw)
  To: Max Nikulin; +Cc: emacs-orgmode

Max Nikulin <manikulin@gmail.com> writes:

> On 30/04/2024 03:26, Rudolf Adamkovič wrote:
>> +    local value = result[1]
>> +    if string.find(value, '[%%(%%[{]') == 1 then
>> +      return quotes .. value .. quotes
>
> Am I wrong that quotes in value may cause issues? I expect some way of
> escaping " characters.

What would the reproduction steps look like?

Rudy
-- 
"It is far better to have a question that can't be answered than an answer that
can't be questioned."
--- Carl Sagan

Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-05-02 15:58                 ` Rudolf Adamkovič
@ 2024-05-02 16:02                   ` Rudolf Adamkovič
  0 siblings, 0 replies; 18+ messages in thread
From: Rudolf Adamkovič @ 2024-05-02 16:02 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 157 bytes --]

Rudolf Adamkovič <rudolf@adamkovic.org> writes:

> I am attaching a V2 of the patch that:

Oops, I forgot to attach the patch.  Here it is. :)

Rudy

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-lua-Quote-list-like-strings-sort-tables-and-parse.patch --]
[-- Type: text/x-patch, Size: 14832 bytes --]

From b4d61bf511df74094fbcdf13c8c0c2b57b1e4a22 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Rudolf=20Adamkovi=C4=8D?= <rudolf@adamkovic.org>
Date: Mon, 29 Apr 2024 21:42:04 +0200
Subject: [PATCH] ob-lua: Quote list-like strings, sort tables, and parse bare
 returns

* lisp/ob-lua.el (org-babel-lua-wrapper-method): 3 changes: (1) Quote
strings with list-like content to prevent Org Babel from incorrectly
guessing their type, (2) sort pretty-printed non-sequential tables to
make them reproducible, and (3) handle implicit nils produced by
'return' used with no arguments.
* testing/lisp/test-ob-lua.el (test-ob-lua/result/nil):
(test-ob-lua/result/nil/explicit):
(test-ob-lua/result/boolean):
(test-ob-lua/results/number/integer):
(test-ob-lua/results/number/integer/negative):
(test-ob-lua/results/number/integer/multiple):
(test-ob-lua/results/number/real):
(test-ob-lua/results/number/real/multiple):
(test-ob-lua/results/number/infinity):
(test-ob-lua/results/string/single-quotes):
(test-ob-lua/results/string/double-quotes):
(test-ob-lua/results/string/multiple):
(test-ob-lua/results/string/list-like):
(test-ob-lua/results/string/list-like/brackets):
(test-ob-lua/results/string/list-like/curlies):
(test-ob-lua/results/string/list-like/standard-output):
(test-ob-lua/result/table):
(test-ob-lua/result/table/pretty-print):
(test-ob-lua/result/table/pretty-print/sorted):
(test-ob-lua/results/value-separator): New tests.
---
 lisp/ob-lua.el              |  49 +++++--
 testing/lisp/test-ob-lua.el | 278 ++++++++++++++++++++++++++++++++----
 2 files changed, 284 insertions(+), 43 deletions(-)

diff --git a/lisp/ob-lua.el b/lisp/ob-lua.el
index 041abfabc..19950f2cb 100644
--- a/lisp/ob-lua.el
+++ b/lisp/ob-lua.el
@@ -254,33 +254,40 @@ then create.  Return the initialized session."
 (defvar org-babel-lua-wrapper-method
   "
 function main()
-%s
+   %s
 end
 
 function dump(it, indent)
    if indent == nil then
       indent = ''
    end
+
    if type(it) == 'table' and %s then
-      local count = 0
-      for _ in pairs(it) do
-         count = count + 1
-      end
       local result = ''
+
       if #indent ~= 0 then
          result = result .. '\\n'
       end
-      for key, value in pairs(it) do
+
+      local keys = {}
+      for key in pairs(it) do
+         table.insert(keys, key)
+      end
+
+      table.sort(keys)
+
+      for index, key in pairs(keys) do
+         local value = it[key]
          result = result
             .. indent
             .. dump(key)
             .. ' = '
             .. dump(value, indent .. '  ')
-         count = count - 1
-         if count ~= 0 then
+         if index ~= #keys then
             result = result .. '\\n'
          end
       end
+
       return result
    else
       return tostring(it)
@@ -288,11 +295,27 @@ function dump(it, indent)
 end
 
 function combine(...)
-  local result = {}
-  for index = 1, select('#', ...) do
-    result[index] = dump(select(index, ...))
-  end
-  return table.concat(result, '%s')
+   local quotes = '\"'
+   local result = {}
+
+   for index = 1, select('#', ...) do
+      result[index] = dump(select(index, ...))
+   end
+
+   if #result == 0 then
+      return dump(nil)
+   end
+
+   if #result == 1 then
+      local value = result[1]
+      if string.find(value, '[%%(%%[{]') == 1 then
+         return quotes .. value .. quotes
+      else
+         return value
+      end
+   end
+
+   return quotes .. table.concat(result, '%s') .. quotes
 end
 
 output = io.open('%s', 'w')
diff --git a/testing/lisp/test-ob-lua.el b/testing/lisp/test-ob-lua.el
index 0a60c68ca..a0a3b178c 100644
--- a/testing/lisp/test-ob-lua.el
+++ b/testing/lisp/test-ob-lua.el
@@ -136,45 +136,263 @@ return x
 	    (org-babel-next-src-block)
 	    (org-babel-execute-src-block)))))
 
-(ert-deftest test-ob-lua/types ()
-  "Test returning different types."
+(ert-deftest test-ob-lua/result/nil ()
+  "Test returning nothing."
   (should
-   (equal "nil"
-          (org-test-with-temp-text "src_lua{return nil}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return} {{{results(=nil=)}}}"
+    (org-test-with-temp-text "src_lua{return}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/result/nil/explicit ()
+  "Test returning nothing explicitly."
+  (should
+   (equal
+    "src_lua{return nil} {{{results(=nil=)}}}"
+    (org-test-with-temp-text "src_lua{return nil}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/result/boolean ()
+  "Test returning the boolean values true and false."
+  (should
+   (equal
+    "src_lua{return true} {{{results(=true=)}}}"
+    (org-test-with-temp-text "src_lua{return true}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max)))))
+  (should
+   (equal
+    "src_lua{return false} {{{results(=false=)}}}"
+    (org-test-with-temp-text "src_lua{return false}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/number/integer ()
+  "Test returning integers."
+  (should
+   (equal
+    "src_lua{return 1} {{{results(=1=)}}}"
+    (org-test-with-temp-text "src_lua{return 1}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/number/integer/negative ()
+  "Test returning negative integers."
   (should
-   (equal "true"
-          (org-test-with-temp-text "src_lua{return true}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return -1} {{{results(=-1=)}}}"
+    (org-test-with-temp-text "src_lua{return -1}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/number/integer/multiple ()
+  "Test returning multiple integers at once."
+  (should
+   (equal
+    "src_lua{return 1, 2, 3} {{{results(=1, 2, 3=)}}}"
+    (org-test-with-temp-text "src_lua{return 1, 2, 3}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/number/real ()
+  "Test returning real numbers."
+  (should
+   (equal
+    "src_lua{return 1.5} {{{results(=1.5=)}}}"
+    (org-test-with-temp-text "src_lua{return 1.5}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/number/real/multiple ()
+  "Test returning multiple real numbers at once."
   (should
-   (equal "false"
-          (org-test-with-temp-text "src_lua{return false}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return 1.5, 2.5, 3.5} {{{results(=1.5, 2.5, 3.5=)}}}"
+    (org-test-with-temp-text "src_lua{return 1.5, 2.5, 3.5}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/number/infinity ()
+  "Test returning the infinity."
   (should
-   (equal 1
-          (org-test-with-temp-text "src_lua{return 1}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return 1 / 0} {{{results(=inf=)}}}"
+    (org-test-with-temp-text "src_lua{return 1 / 0}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/single-quotes ()
+  "Test returning strings in single quotes."
   (should
-   (equal "hello world"
-          (org-test-with-temp-text "src_lua{return 'hello world'}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return 'hello world'} {{{results(=hello world=)}}}"
+    (org-test-with-temp-text "src_lua{return 'hello world'}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/double-quotes ()
+  "Test returning strings in double quotes."
   (should
-   (equal 0
-          (string-match "table: 0x[0-9A-F]+"
-                        (org-test-with-temp-text "src_lua{return {}}"
-                          (org-babel-execute-src-block))))))
+   (equal
+    "src_lua{return \"hello world\"} {{{results(=hello world=)}}}"
+    (org-test-with-temp-text "src_lua{return \"hello world\"}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
 
-(ert-deftest test-ob-lua/multiple-values ()
-  "Test returning multiple values."
+(ert-deftest test-ob-lua/results/string/multiple ()
+  "Test returning multiple strings at once."
   (should
-   (equal "1, 2, 3"
-          (org-test-with-temp-text "src_lua{return 1, 2, 3}"
-            (org-babel-execute-src-block))))
+   (equal
+    "src_lua{return 'a', 'b'} {{{results(=a, b=)}}}"
+    (org-test-with-temp-text "src_lua{return 'a', 'b'}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/list-like ()
+  "Test returning strings that look like \"(...)\" lists."
+  (should
+   (equal
+    (concat "src_lua{return string.match('A (B) C', '%b()')}"
+            " {{{results(=(B)=)}}}")
+    (org-test-with-temp-text
+        "src_lua{return string.match('A (B) C', '%b()')}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/list-like/brackets ()
+  "Test returning strings that look like \"[...]\" lists."
+  (should
+   (equal
+    (concat "src_lua{return string.match('A [B] C', '%b[]')}"
+            " {{{results(=[B]=)}}}")
+    (org-test-with-temp-text
+        "src_lua{return string.match('A [B] C', '%b[]')}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/list-like/curlies ()
+  "Test returning strings that look like \"{...}\" lists."
+  (should
+   (equal
+    (concat "src_lua{return string.match('A {B} C', '%b{}')}"
+            " {{{results(={B}=)}}}")
+    (org-test-with-temp-text
+	"src_lua{return string.match('A {B} C', '%b{}')}"
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/results/string/list-like/standard-output ()
+  "Test returning strings that look like lists to standard output."
+  (should
+   (equal
+    (concat "src_lua[:results output]"
+            "{print(string.match('A (B) C', '%b()'))}"
+            " {{{results(=(B)=)}}}")
+    (org-test-with-temp-text
+        (concat "src_lua[:results output]"
+                "{print(string.match('A (B) C', '%b()'))}")
+      (org-babel-execute-src-block)
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
+
+(ert-deftest test-ob-lua/result/table ()
+  "Test returning table references."
+  (should
+   (equal
+    0
+    (string-match
+     "src_lua{return {}} {{{results(=table: 0x[0-9A-F]+=)}}}"
+     (org-test-with-temp-text "src_lua{return {}}"
+       (org-babel-execute-src-block)
+       (buffer-substring-no-properties (point-min)
+                                       (point-max)))))))
+
+(ert-deftest test-ob-lua/result/table/pretty-print ()
+  "Test returning and pretty-printing sequential tables."
+  (should
+   (equal (string-join
+           '("#+BEGIN_SRC lua :results pp"
+             "return {10, {20, 30, {40, 50}, 60}, 70}"
+             "#+END_SRC"
+             ""
+             "#+RESULTS:"
+             ": 1 = 10"
+             ": 2 = "                   ; FIXME Trailing space.
+             ":   1 = 20"
+             ":   2 = 30"
+             ":   3 = "                 ; FIXME Trailing space.
+             ":     1 = 40"
+             ":     2 = 50"
+             ":   4 = 60"
+             ": 3 = 70"
+             "")
+           "\n")
+          (org-test-with-temp-text
+              (string-join
+               '("#+BEGIN_SRC lua :results pp"
+                 "return {10, {20, 30, {40, 50}, 60}, 70}<point>"
+                 "#+END_SRC")
+               "\n")
+	    (org-babel-execute-src-block)
+            (buffer-substring-no-properties (point-min)
+                                            (point-max))))))
+
+(ert-deftest test-ob-lua/result/table/pretty-print/sorted ()
+  "Test returning and pretty-printing non-sequential tables."
+  (should
+   (equal (string-join
+           '("#+BEGIN_SRC lua :results pp"
+             "return {b = 20, c = 30, a = 10}"
+             "#+END_SRC"
+             ""
+             "#+RESULTS:"
+             ;; NOTE The keys are sorted alphabetically.
+             ": a = 10"
+             ": b = 20"
+             ": c = 30"
+             "")
+           "\n")
+          (org-test-with-temp-text
+              (string-join
+               '("#+BEGIN_SRC lua :results pp"
+                 "return {b = 20, c = 30, a = 10}"
+                 "#+END_SRC")
+               "\n")
+	    (org-babel-execute-src-block)
+            (buffer-substring-no-properties (point-min)
+                                            (point-max))))))
+
+(ert-deftest test-ob-lua/results/value-separator ()
+  "Test customizing the separator of multiple values."
+  ;; TODO Once Org Babel supports returning lists from inline blocks,
+  ;; instead of trapping with the user error: "Inline error: list
+  ;; result cannot be used", use those for multiple values.
   (should
-   (equal "1|2|3"
-          (let ((org-babel-lua-multiple-values-separator "|"))
-            (org-test-with-temp-text "src_lua{return 1, 2, 3}"
-              (org-babel-execute-src-block))))))
+   (equal
+    "src_lua{return 1, 2, 3} {{{results(=1\t2\t3=)}}}"
+    (org-test-with-temp-text "src_lua{return 1, 2, 3}"
+      (let ((org-babel-lua-multiple-values-separator "\t"))
+        (org-babel-execute-src-block))
+      (buffer-substring-no-properties (point-min)
+                                      (point-max))))))
 
 (provide 'test-ob-lua)
 
-- 
2.39.3 (Apple Git-146)


[-- Attachment #3: Type: text/plain, Size: 259 bytes --]

-- 
“Those who cannot remember the past are condemned to repeat it.”
--- George Santayana, Life of Reason: Reason in Common Sense, 1905

Rudolf Adamkovič <rudolf@adamkovic.org> [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: [PATCH] ob-lua: Support all types and multiple values in results
  2024-05-02 16:00                 ` Rudolf Adamkovič
@ 2024-05-02 16:41                   ` Ihor Radchenko
  0 siblings, 0 replies; 18+ messages in thread
From: Ihor Radchenko @ 2024-05-02 16:41 UTC (permalink / raw)
  To: Rudolf Adamkovič; +Cc: Max Nikulin, emacs-orgmode

Rudolf Adamkovič <rudolf@adamkovic.org> writes:

> Max Nikulin <manikulin@gmail.com> writes:
>
>> On 30/04/2024 03:26, Rudolf Adamkovič wrote:
>>> +    local value = result[1]
>>> +    if string.find(value, '[%%(%%[{]') == 1 then
>>> +      return quotes .. value .. quotes
>>
>> Am I wrong that quotes in value may cause issues? I expect some way of
>> escaping " characters.
>
> What would the reproduction steps look like?

I guess

#+begin_src lua
return '"hello " world"'
#+end_src

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>


^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2024-05-02 16:41 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-23 16:44 [PATCH] ob-lua: Support all types and multiple values in results Rudolf Adamkovič
2024-04-23 17:05 ` Rudolf Adamkovič
2024-04-24 14:20   ` Ihor Radchenko
2024-04-24 13:05 ` Ihor Radchenko
2024-04-24 15:01   ` Rudolf Adamkovič
2024-04-26 13:40     ` Ihor Radchenko
2024-04-27 10:57       ` Max Nikulin
2024-04-27 16:33       ` Rudolf Adamkovič
2024-04-28 12:36         ` Ihor Radchenko
2024-04-29 15:57           ` Rudolf Adamkovič
2024-04-29 20:26             ` Rudolf Adamkovič
2024-04-30 10:24               ` Ihor Radchenko
2024-05-02 15:57                 ` Rudolf Adamkovič
2024-05-02 15:58                 ` Rudolf Adamkovič
2024-05-02 16:02                   ` Rudolf Adamkovič
2024-05-01 11:05               ` Max Nikulin
2024-05-02 16:00                 ` Rudolf Adamkovič
2024-05-02 16:41                   ` Ihor Radchenko

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.