all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: John C <john.ciolfi.32@gmail.com>
To: Ihor Radchenko <yantar92@posteo.net>
Cc: emacs-orgmode@gnu.org
Subject: Re: ob-octave: improve MATLAB support
Date: Thu, 2 Jan 2025 18:56:48 -0500	[thread overview]
Message-ID: <CACb3vdSc2ypq5hdd3Kgb3K+wek-0LDwUqUPugzUr2tt5yyg23Q@mail.gmail.com> (raw)
In-Reply-To: <CACb3vdQLF8D6buAv0CQZoYmwajwkAhCsCcwW+BVdv6v=amgzYA@mail.gmail.com>


[-- Attachment #1.1: Type: text/plain, Size: 11465 bytes --]

Attaching the updated org-matlab.patch

On Thu, Jan 2, 2025 at 6:55 PM John C <john.ciolfi.32@gmail.com> wrote:

> Hi
>
> Thanks for the feedback. See updated org-matlab.patch. Note
> code evaluation will now error out when using an out-of-date matlab-mode
> (rather than generating a warning) because when using an older matlab-mode,
> code block evaluation doesn't work. We need to wait for the matlab-shell to
> be ready. Without that capability found in newer matlab-mode, the errors
> are very hard to understand. I also added tests. Using steps outlined in
> the ob-matlab-test.el comment to setup for MATLAB testing, I get on Linux:
>
> matlab-mode, version 6.3
> Detected MATLAB R2024b (24.2)  -- Loading history file
> matlab-shell: starting server with name server-2055614
>    passed   127/1250  ob-matlab/results-file-graphics (9.712776 sec)
>    passed   128/1250  ob-matlab/results-file-graphics-with-space (0.332004
> sec)
>    passed   129/1250  ob-matlab/results-output (0.013681 sec)
>    passed   130/1250  ob-matlab/results-output-latex (0.459980 sec)
>    passed   131/1250  ob-matlab/results-output-preserve-whitespace
> (0.281001 sec)
>    passed   132/1250  ob-matlab/results-output-reuse-a (0.039130 sec)
>    passed   133/1250  ob-matlab/results-output-reuse-b (0.043762 sec)
>    passed   134/1250  ob-matlab/results-output-reuse-clear (0.301597 sec)
>    passed   135/1250  ob-matlab/results-verbatim (0.172567 sec)
>
> You mentioned that we should expect org users to understand sessions,
> however the doc on sessions is a bit light. Searching
> https://orgmode.org/org.html for "session", we see it used two different
> ways "Emacs session" and babel code block evaluations sessions described in
> "Using sessions". What is there is okay, but doesn't really help one
> understand when they should/shouldn't use babel code block evaluation
> sessions. What would be nice is if "Using sessions" were titled "Babel code
> block evaluation sessions" containing the existing content minus the "Only
> languages that provide interactive evaluation ..." paragraph because this
> is vague. In addition, each language should have a way of describing what
> :session means to them, perhaps by extracting this info from ob-LANGUAGE.el
> or maybe ob-LANGUAGE.org, or maybe something else that inserts the language
> specific into the org manual.
>
> With good MATLAB integration in org-mode, we'll see scientists and
> engineers who have limited programming background leverage org-mode for
> their scientific papers. Not having to worry about more advanced concepts
> like sessions is nice, so thanks for letting us default the :session for
> MATLAB to make it work as one would expect.
>
> Here's the commit info:
>
> ob-matlab.el: improve MATLAB support
>
> * lisp/ob-matlab.el (header): Update URL for MATLAB
>
> * lisp/ob-octave.el (org-babel-octave-evaluate): Fixed MATLAB support
>   - Deprecate variables related to MATLAB Emacs Link and removed the code.
>     Emacs Link capability was removed from MATLAB release R2009a, 15 years
> ago.
>   - Fixed the following type of org block evaluation:
>     1) #+begin_src matlab :results verbatim
>     2) #+begin_src matlab :results output
>     3) #+begin_src matlab :results output latex
>     4) #+begin_src matlab :results file graphics
>     which aid in writing scientific papers.
>   - Minor point, the correct spelling of MATLAB when referencing the
> product is
>     all upper case.
>
> * lisp/ob-comint.el: enhanced org-babel-comint-with-output
>   - The META argument of org-babel-comint-with-output now supports an
> optional
>     STRIP-REGEXPS which can be used to remove content from the returned
> output.
>
> * lisp/org.el
>   - Add MATLAB as one of the options for org-babel-load-languages
>
> * mk/default.mk
>   - Add support for testing matlab code blocks.
>
> * testing/examples/ob-matlab-test.org, testing/lisp/test-ob-matlab.el
>   - Test for matlab code block.
>
> * testing/org-test.el
>   - Added org-test-get-code-block which is for use by
> testing/lisp/test*.el files
>     to extract code blocks from testing/examples/*.org files for on-the-fly
>     testing using org-test-with-temp-text.
>
> * etc/ORG-NEWS (New functions and changes in function arguments):
>   Added entry "ob-octave: improved MATLAB support"
>
> Thanks
> John
>
>
> On Sun, Dec 29, 2024 at 2:41 AM Ihor Radchenko <yantar92@posteo.net>
> wrote:
>
>> John C <john.ciolfi.32@gmail.com> writes:
>>
>> > See attached org-matlab.patch which addresses all feedback. Here's the
>> > commit info.
>>
>> Thanks!
>> See my comments inline.
>>
>> > +*** ob-matlab: fixed MATLAB support
>> > +
>> > +Fixed MATLAB babel code blocks processing. MATLAB code blocks,
>> ~#+begin_src matlab~, with ~:results
>> > +verbatim~, ~:results output~, ~:results output latex~, or ~:results
>> file graphics~ now work.  Fixes
>> > +include (1) waiting for matlab-shell to start before evaluating MATLAB
>> code, (2) correctly showing
>> > +the results using writematrix, (3) removing the code block lines from
>> the result, (4) correctly
>> > +handling graphics results by invoking print correctly. To use MATLAB
>> with org, you need
>> > +https://github.com/MathWorks/Emacs-MATLAB-Mode.
>>
>> There is no need to provide so many details.
>> Just leave the most important things:
>>
>> 1. MATLAB is no longer broken
>> 2. Emacs-MATLAB-Mode is required now
>>
>> > +*** ob-matlab: MATLAB behavior change
>> > +
>> > +MATLAB code blocks now reuse the ~MATLAB*~ buffer created by ~M-x
>> matlab-shell~, whereas the
>> > +prior version started a new shell for each evaluation.  The benefit of
>> this is that
>> > +evaluations are very fast after the first evaluation and that state is
>> maintained between
>> > +evaluations, which you can clear using the MATLAB ~clear~ command.
>> Another benefit of this
>> > +behavior is that it is consistent with how MATLAB works.
>>
>> No need to explain in so much details, I think.
>> Just say that MATLAB uses session by default and them mention that users
>> may customize `org-babel-default-header-args:matlab' to disable session.
>>
>> > +(defun org-babel-comint--strip-regexps (result strip-regexps)
>> > +  "STRIP-REGEXPS from RESULT list of strings."
>> > +  (dolist (strip-regexp strip-regexps)
>> > +    (let ((new-result '()))
>> > +      (dolist (line result)
>> > +        (setq line (replace-regexp-in-string strip-regexp "" line))
>> > +        (when (not (string= line ""))
>> > +          (setq new-result (append new-result `(,line)))))
>>
>> It is more efficient to use `push' + `nreverse' instead of `append'.
>>
>> > -(defvar org-babel-default-header-args:matlab '())
>> > +;; With `org-babel-default-header-args:matlab' set to
>> > +;;  '((:session . "*MATLAB*")))
>> > +;; ...
>> > +;; If you want a new session each time you evaluate a MATLAB code
>> block,
>> > +;;   (setq 'org-babel-default-header-args:matlab '())
>> > +;; However, this will make evaluations slower and is not consistent
>> with how
>> > +;; MATLAB works.  MATLAB is designed for many evaluations.
>> > +(defvar org-babel-default-header-args:matlab '((:session .
>> "*MATLAB*")))
>>
>> You don't need that long comment in the source code.
>> If you think that explaining the details about session is necessary (it
>> may or may not be, but we should assume that Org users are familiar with
>> the notion of sessions in code blocks), please do it in the
>> documentation, not in the code.
>>
>> More generally, your motivation is not specific to matlab. Yet, we
>> default to no session in most babel backends. So, it is not a question
>> of session being faster or slower, but a question of consistency.
>>
>> That said, some babel backends do default to session, so I do not oppose
>> this change too much.
>>
>> > +(make-obsolete-variable 'org-babel-matlab-with-emacs-link
>> > +                        "MATLAB removed EmacsLink in R2009a." "2009")
>> > +
>> > +(make-obsolete-variable 'org-babel-matlab-emacs-link-wrapper-method
>> > +                        "MATLAB removed EmacsLink in R2009a." "2009")
>>
>> Please use Org version in WHEN argument of `make-obsolete-variable'.
>> The WHEN should be "9.8".
>>
>> > +(defun org-babel-matlab-shell ()
>> > +  "Start and/or wait for MATLAB shell."
>> > +  (require 'matlab-shell) ;; make `matlab-shell-busy-checker' available
>> > +  (cond
>> > +   ((fboundp 'matlab-shell-busy-checker)
>> > +    ;; Start the shell if needed.  `matlab-shell' will reuse existing
>> if already running.
>> > +    (matlab-shell)
>> > +    ;; If we just started the matlab-shell, wait for the prompt.  If
>> we do not
>> > +    ;; wait, then the startup messages will show up in the evaluation
>> results.
>> > +    (matlab-shell-busy-checker 'wait-for-prompt))
>> > +   (t
>> > +    (message (concat "You version of matlab-mode is old.\n"
>> > +                     "Please update, see
>> https://github.com/mathworks/Emacs-MATLAB-Mode\n"
>> > +                     "Updating will eliminate unexpected output in
>> your results\n"))
>> > +    (sit-for 3)
>>
>> Instead of `message' + `sit-fit', you can simply use `display-warning'.
>> It will give users more control.
>>
>> > +(defun org-babel-body-for-output (body matlabp)
>> > +  "If MATLABP, fixup BODY for MATLAB output result-type."
>> > +  (when matlabp
>> > +    ;; When we send multi-line input to `matlab-shell', we'll see the
>> "body"
>> > +    ;; code lines echoed in the output which is not what one would
>> expect.  To
>> > +    ;; remove these unwanted lines, we append a comment "%-<org-eval>"
>> to each
>> > +    ;; line in the body MATLAB code.  After we collect the results from
>> > +    ;; evaluation, we leverage the "%-<org-eval>" to remove the
>> unwanted lines.
>> > +    ;; Example of desired behavior:
>> > ...
>>
>> I think that an important point here is that MATLAB does not echo the
>> whole BODY all at once and instead mixes it with the output. Which is
>> why we need to do something non-standard to filter out the body in
>> matlab specifically.
>>
>> > +    (setq body (replace-regexp-in-string "\n" " %-<org-eval>-\n" body))
>> > +    (when (not (string-match "\n\\'" body))
>> > +      (setq body (concat body " %-<org-eval>-"))))
>> > +  body)
>>
>> Please put this %-<org-eval> into an internal constant and then reuse
>> it when building the regexp to filter.
>>
>> > +               (when matlabp
>> > +                 '(;; MATLAB echo's all input lines, so use the
>> %-<org-eval> comments to strip
>> > +                   ;; them from the output
>> > +                   "^[^\n]*%-<org-eval>-\n"
>> > +                   ;; Remove starting blank line caused by stripping
>> %-<org-eval>
>> > +                   "\\`[[:space:]\r\n]+"
>> > +                   ;; Strip <ERRORTXT> and </ERRORTXT> matlab-shell
>> error indicators
>> > +                   "</?ERRORTXT>\n")))
>>
>> Same here. Please put these regexps into a constant.
>>
>> --
>> Ihor Radchenko // yantar92,
>> Org mode maintainer,
>> Learn more about Org mode at <https://orgmode.org/>.
>> Support Org development at <https://liberapay.com/org-mode>,
>> or support my work at <https://liberapay.com/yantar92>
>>
>

[-- Attachment #1.2: Type: text/html, Size: 13865 bytes --]

[-- Attachment #2: org-matlab.patch --]
[-- Type: text/x-patch, Size: 29430 bytes --]

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 5d421172f..4f9523cf3 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -162,6 +162,17 @@ bibliography format requires them to be written in title-case.
 
 # This also includes changes in function behavior from Elisp perspective.
 
+*** ob-matlab: fixed MATLAB support
+
+Fixed MATLAB babel code blocks processing. MATLAB code blocks, ~#+begin_src matlab~, with ~:results
+verbatim~, ~:results output~, ~:results output latex~, or ~:results file graphics~ now work.  To use
+MATLAB with org, you need https://github.com/MathWorks/Emacs-MATLAB-Mode.
+
+*** ob-matlab: MATLAB behavior change
+
+MATLAB code blocks now reuse the ~MATLAB*~ buffer created by ~M-x matlab-shell~ by default. To
+change this behavior, customize ~org-babel-default-header-args:matlab~.
+
 *** ob-sqlite: Added ability to open a database in readonly mode
 
 Added option :readonly to ob-sqlite.
diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el
index b88ac445a..2246badc2 100644
--- a/lisp/ob-comint.el
+++ b/lisp/ob-comint.el
@@ -101,15 +101,28 @@ PROMPT-REGEXP defaults to `comint-prompt-regexp'."
        (setq string (substring string (match-end 0))))
   string)
 
+(defun org-babel-comint--strip-regexps (result strip-regexps)
+  "STRIP-REGEXPS from RESULT list of strings."
+  (dolist (strip-regexp strip-regexps)
+    (let ((new-result '()))
+      (dolist (str result)
+        (setq str (replace-regexp-in-string strip-regexp "" str))
+        (when (not (string= str ""))
+          (push str new-result)))
+      (setq result (nreverse new-result))))
+  result)
+
 (defmacro org-babel-comint-with-output (meta &rest body)
   "Evaluate BODY in BUFFER and return process output.
 Will wait until EOE-INDICATOR appears in the output, then return
 all process output.  If REMOVE-ECHO and FULL-BODY are present and
-non-nil, then strip echo'd body from the returned output.  META
-should be a list containing the following where the last two
-elements are optional.
+non-nil, then strip echoed body from the returned output.  If
+optional STRIP-REGEXPS, a list of regular expressions, is
+present, then all matches will be removed from the returned
+output.  META should be a list containing the following where the
+last three elements are optional.
 
- (BUFFER EOE-INDICATOR REMOVE-ECHO FULL-BODY)
+ (BUFFER EOE-INDICATOR REMOVE-ECHO FULL-BODY STRIP-REGEXPS)
 
 This macro ensures that the filter is removed in case of an error
 or user `keyboard-quit' during execution of body."
@@ -117,7 +130,8 @@ or user `keyboard-quit' during execution of body."
   (let ((buffer (nth 0 meta))
 	(eoe-indicator (nth 1 meta))
 	(remove-echo (nth 2 meta))
-	(full-body (nth 3 meta)))
+	(full-body (nth 3 meta))
+        (strip-regexps (nth 4 meta)))
     `(org-babel-comint-in-buffer ,buffer
        (let* ((string-buffer "")
 	      (comint-output-filter-functions
@@ -165,8 +179,12 @@ or user `keyboard-quit' during execution of body."
          (and ,remove-echo ,full-body
               (setq string-buffer (org-babel-comint--echo-filter string-buffer ,full-body)))
 
-         ;; Filter out prompts.
-         (org-babel-comint--prompt-filter string-buffer)))))
+         ;; Filter out prompts from returned output result.
+         (let ((result (org-babel-comint--prompt-filter string-buffer)))
+           ;; Remove all matches of STRIP-REGEXPS in returned output result.
+           (when ,strip-regexps
+             (setq result (org-babel-comint--strip-regexps result ,strip-regexps)))
+           result)))))
 
 (defun org-babel-comint-input-command (buffer cmd)
   "Pass CMD to BUFFER.
diff --git a/lisp/ob-matlab.el b/lisp/ob-matlab.el
index de8deadbe..083dcdec3 100644
--- a/lisp/ob-matlab.el
+++ b/lisp/ob-matlab.el
@@ -28,11 +28,10 @@
 
 ;;; Requirements:
 
-;; Matlab
-
-;; matlab.el required for interactive emacs sessions and matlab-mode
-;; major mode for source code editing buffer
-;; https://matlab-emacs.sourceforge.net/
+;; 1) MATLAB
+;; 2) https://github.com/mathworks/Emacs-MATLAB-Mode
+;;    For matlab-shell to run MATLAB within Emacs and matlab-mode
+;;    major mode for source code editing buffer
 
 ;;; Code:
 
diff --git a/lisp/ob-octave.el b/lisp/ob-octave.el
index 005990f20..5e622df9a 100644
--- a/lisp/ob-octave.el
+++ b/lisp/ob-octave.el
@@ -1,8 +1,8 @@
-;;; ob-octave.el --- Babel Functions for Octave and Matlab -*- lexical-binding: t; -*-
+;;; ob-octave.el --- Babel Functions for Octave and MATLAB -*- lexical-binding: t; -*-
 
 ;; Copyright (C) 2010-2024 Free Software Foundation, Inc.
 
-;; Author: Dan Davison
+;; Author: Dan Davison (Octave), John Ciolfi (MATLAB)
 ;; Keywords: literate programming, reproducible research
 ;; URL: https://orgmode.org
 
@@ -30,6 +30,8 @@
 
 ;;; Code:
 
+(require 'cl-seq)
+
 (require 'org-macs)
 (org-assert-version)
 
@@ -39,7 +41,11 @@
 (declare-function matlab-shell "ext:matlab-mode")
 (declare-function matlab-shell-run-region "ext:matlab-mode")
 
-(defvar org-babel-default-header-args:matlab '())
+(defvar org-babel-default-header-args:matlab '((:session . "*MATLAB*"))
+  "Reuse the matlab-shell buffer for code block evaluations.
+Add the MATLAB clear command to your code block to clear the
+MATLAB workspace.")
+
 (defvar org-babel-default-header-args:octave '())
 
 (defvar org-babel-matlab-shell-command "matlab -nosplash"
@@ -47,18 +53,42 @@
 (defvar org-babel-octave-shell-command "octave -q"
   "Shell command to run octave as an external process.")
 
-(defvar org-babel-matlab-with-emacs-link nil
-  "If non-nil use matlab-shell-run-region for session evaluation.
-This will use EmacsLink if (matlab-with-emacs-link) evaluates
-to a non-nil value.")
-
-(defvar org-babel-matlab-emacs-link-wrapper-method
-  "%s
-if ischar(ans), fid = fopen('%s', 'w'); fprintf(fid, '%%s\\n', ans); fclose(fid);
-else, save -ascii %s ans
-end
-delete('%s')
+(make-obsolete-variable 'org-babel-matlab-with-emacs-link
+                        "MATLAB removed EmacsLink in R2009a." "9.8")
+
+(make-obsolete-variable 'org-babel-matlab-emacs-link-wrapper-method
+                        "MATLAB removed EmacsLink in R2009a." "9.8")
+
+(defvar org-babel-matlab-print "print(\"-dpng\", %S);\nans=%S;"
+  ;; MATLAB command-function duality requires that the file name be specified
+  ;; without quotes. Using: print -dpng "file.png", would produce a file with
+  ;; the quotes in the file name on disk. Therefore, use the functional form
+  ;; to handle files with spaces, print("-dpng", "file.png").
+  ;; Example:
+  ;;    #+begin_src matlab :file "sine wave.png" :results file graphics
+  ;;      t = [0 : 0.1 : 2*pi];
+  ;;      y = sin(t);
+  ;;      plot(t, y);
+  ;;      set(gcf, 'PaperUnits', 'inches', 'PaperPosition', [0 0 4 3]) % Set the size to 4" x 3"
+  ;;    #+end_src
+  ;;
+  ;;    #+RESULTS:
+  ;;    [[file:sine wave.png]]
+  "MATLAB format specifier to print current figure to a file.")
+
+(defvar org-babel-octave-print "print -dpng %S\nans=%S"
+  "Octave format specifier to print current figure to a file.")
+
+(defvar org-babel-matlab-wrapper-method
+  (concat "\
+cd('%s');
+%s
+if ~exist('ans', 'var') ans = ''; end; \
+writematrix(ans, '%s', 'Delimiter', 'tab');
 ")
+  "Format specifier used when evaluating MATLAB code blocks.
+Arguments are the `default-directory', the MATLAB code, and a result file.txt.")
+
 (defvar org-babel-octave-wrapper-method
   "%s
 if ischar(ans), fid = fopen('%s', 'w'); fdisp(fid, ans); fclose(fid);
@@ -92,7 +122,10 @@ When MATLABP is non-nil, execute Matlab.  Otherwise, execute Octave."
 				 (list
 				  "set (0, \"defaultfigurevisible\", \"off\");"
 				  full-body
-				  (format "print -dpng %S\nans=%S" gfx-file gfx-file))
+				  (format (if matlabp
+                                              org-babel-matlab-print
+                                            org-babel-octave-print)
+                                          gfx-file gfx-file))
 				 "\n")
 		    full-body)
 		  result-type matlabp)))
@@ -153,6 +186,19 @@ If there is not a current inferior-process-buffer in SESSION then
 create.  Return the initialized session.  PARAMS are src block parameters."
   (org-babel-octave-initiate-session session params 'matlab))
 
+(defun org-babel-matlab-shell ()
+  "Start and/or wait for MATLAB shell."
+  (require 'matlab-shell) ;; make `matlab-shell-busy-checker' available
+  (if (fboundp 'matlab-shell-busy-checker)
+      (progn
+        ;; Start the shell if needed.  `matlab-shell' will be reused if it is already running.
+        (matlab-shell)
+        ;; If we just started the matlab-shell, wait for the prompt.  If we do not
+        ;; wait, then the startup messages will show up in the evaluation results.
+        (matlab-shell-busy-checker 'wait-for-prompt))
+    (user-error "A newer version of matlab-mode is required, see \
+https://github.com/mathworks/Emacs-MATLAB-Mode\n")))
+
 (defun org-babel-octave-initiate-session (&optional session _params matlabp)
   "Create an octave inferior process buffer.
 If there is not a current inferior-process-buffer in SESSION then
@@ -165,9 +211,15 @@ Octave session, unless MATLABP is non-nil."
   (unless (string= session "none")
     (let ((session (or session
 		       (if matlabp "*Inferior Matlab*" "*Inferior Octave*"))))
-      (if (org-babel-comint-buffer-livep session) session
+      (if (org-babel-comint-buffer-livep session)
+          (progn
+            (when (and matlabp (fboundp 'matlab-shell-busy-checker))
+              ;; Can't evaluate if the matlab-shell is currently running code
+              (matlab-shell-busy-checker 'error-if-busy))
+            session)
 	(save-window-excursion
-	  (if matlabp (unless org-babel-matlab-with-emacs-link (matlab-shell))
+	  (if matlabp
+              (org-babel-matlab-shell)
 	    (run-octave))
 	  (rename-buffer (if (bufferp session) (buffer-name session)
 			   (if (stringp session) session (buffer-name))))
@@ -183,79 +235,125 @@ value of the last statement in BODY, as elisp."
       (org-babel-octave-evaluate-session session body result-type matlabp)
     (org-babel-octave-evaluate-external-process body result-type matlabp)))
 
+(defun org-babel-octave-wrapper-tmp-file (matlabp)
+  "Return a local tmp file with name adjusted for MATLABP."
+  (if matlabp
+      ;; writematrix requires a file ending with '.txt'
+      (org-babel-temp-file "matlab-" ".txt")
+    (org-babel-temp-file "octave-")))
+
+(defun org-babel-octave-get-code-to-eval (body tmp-file matlabp)
+  "Format BODY of the code block for evaluation saving results to TMP-FILE.
+If MATLABP, format for MATLAB, else format for Octave."
+    (if matlabp
+        (format org-babel-matlab-wrapper-method default-directory body tmp-file)
+      (format org-babel-octave-wrapper-method body tmp-file tmp-file)))
+
 (defun org-babel-octave-evaluate-external-process (body result-type matlabp)
-  "Evaluate BODY in an external Octave or Matalab process.
+  "Evaluate BODY in an external Octave or MATLAB process.
 Process the result as RESULT-TYPE.  Use Octave, unless MATLABP is non-nil."
   (let ((cmd (if matlabp
 		 org-babel-matlab-shell-command
 	       org-babel-octave-shell-command)))
     (pcase result-type
       (`output (org-babel-eval cmd body))
-      (`value (let ((tmp-file (org-babel-temp-file "octave-")))
+      (`value (let ((tmp-file (org-babel-process-file-name
+                               (org-babel-octave-wrapper-tmp-file matlabp)
+                               'noquote)))
 	        (org-babel-eval
 		 cmd
-		 (format org-babel-octave-wrapper-method body
-			 (org-babel-process-file-name tmp-file 'noquote)
-			 (org-babel-process-file-name tmp-file 'noquote)))
+                 (org-babel-octave-get-code-to-eval body tmp-file matlabp))
 	        (org-babel-octave-import-elisp-from-file tmp-file))))))
 
+(defvar org-babel-octave--matlab-line-indicator " %-<org-eval>\n"
+  "Comment appended to each code line being evaluated.")
+
+(defvar org-babel-octave--matlab-error-indicator-re "</?ERRORTXT>\r?\n?"
+  "MATLAB shell error indicators.")
+
+(defun org-babel-octave-body-for-output (body matlabp)
+  "If MATLABP, fix up BODY for MATLAB output result-type."
+  (when matlabp
+    ;; MATLAB does not echo the whole BODY all at once and instead mixes input
+    ;; with the output.  Therefore, when we send multi-line input to
+    ;; `matlab-shell', we'll see the "body" code lines echoed in the output
+    ;; which is not what one would expect.  To remove these unwanted lines,
+    ;; we append `org-babel-octave--matlab-line-indicator' to each line in the BODY.
+    ;; We leverage this indicator to remove the unwanted lines.
+    ;; Example of desired behavior:
+    ;;     #+begin_src matlab :results output
+    ;;       disp('The results are:')
+    ;;       a = [1, 2; 3, 4]
+    ;;       b = a * 2
+    ;;     #+end_src
+    ;;
+    ;;     #+RESULTS:
+    ;;     #+begin_example
+    ;;     The results are:
+    ;;
+    ;;     a =
+    ;;
+    ;;          1     2
+    ;;          3     4
+    ;;
+    ;;     b =
+    ;;
+    ;;          2     4
+    ;;          6     8
+    ;;     #+end_example
+    (setq body (replace-regexp-in-string "\r" "" body)) ;; CRLF => LF
+    (setq body (concat (string-trim-right body) "\n"))  ;; Ensure a single final newline
+    (setq body (replace-regexp-in-string "\n" org-babel-octave--matlab-line-indicator body)))
+  body)
+
 (defun org-babel-octave-evaluate-session
     (session body result-type &optional matlabp)
   "Evaluate BODY in SESSION."
-  (let* ((tmp-file (org-babel-temp-file (if matlabp "matlab-" "octave-")))
-	 (wait-file (org-babel-temp-file "matlab-emacs-link-wait-signal-"))
+  (let* ((tmp-file (org-babel-octave-wrapper-tmp-file matlabp))
 	 (full-body
 	  (pcase result-type
 	    (`output
 	     (mapconcat
 	      #'org-babel-chomp
-	      (list body org-babel-octave-eoe-indicator) "\n"))
+	      (list (org-babel-octave-body-for-output body matlabp)
+                    org-babel-octave-eoe-indicator)
+              "\n"))
 	    (`value
-	     (if (and matlabp org-babel-matlab-with-emacs-link)
-		 (concat
-		  (format org-babel-matlab-emacs-link-wrapper-method
-			  body
-			  (org-babel-process-file-name tmp-file 'noquote)
-			  (org-babel-process-file-name tmp-file 'noquote) wait-file) "\n")
-	       (mapconcat
-		#'org-babel-chomp
-		(list (format org-babel-octave-wrapper-method
-			      body
-			      (org-babel-process-file-name tmp-file 'noquote)
-			      (org-babel-process-file-name tmp-file 'noquote))
-		      org-babel-octave-eoe-indicator) "\n")))))
-	 (raw (if (and matlabp org-babel-matlab-with-emacs-link)
-		  (save-window-excursion
-		    (with-temp-buffer
-		      (insert full-body)
-		      (write-region "" 'ignored wait-file nil nil nil 'excl)
-		      (matlab-shell-run-region (point-min) (point-max))
-		      (message "Waiting for Matlab Emacs Link")
-		      (while (file-exists-p wait-file) (sit-for 0.01))
-		      "")) ;; matlab-shell-run-region doesn't seem to
-		;; make *matlab* buffer contents easily
-		;; available, so :results output currently
-		;; won't work
-		(org-babel-comint-with-output
-		    (session
-		     (if matlabp
-			 org-babel-octave-eoe-indicator
-		       org-babel-octave-eoe-output)
-		     t full-body)
-		  (insert full-body) (comint-send-input nil t))))
-	 results)
+	     (mapconcat
+	      #'org-babel-chomp
+	      (list (org-babel-octave-get-code-to-eval body tmp-file matlabp)
+		    org-babel-octave-eoe-indicator)
+              "\n"))))
+	 (raw-results
+	  (org-babel-comint-with-output
+	      (session
+	       (if matlabp
+		   org-babel-octave-eoe-indicator
+		 org-babel-octave-eoe-output)
+	       t full-body ;; Remove echo'd full-body from result
+               (when matlabp
+                 `(;; MATLAB echo's input interleaved w/output, so strip inputs
+                   ,(concat "^.*" org-babel-octave--matlab-line-indicator)
+                   ;; Strip matlab-shell error indicators
+                   ,org-babel-octave--matlab-error-indicator-re)))
+            (insert full-body) (comint-send-input nil t))))
     (pcase result-type
       (`value
        (org-babel-octave-import-elisp-from-file tmp-file))
       (`output
-       (setq results
-	     (if matlabp
-		 (cdr (reverse (delete "" (mapcar #'org-strip-quotes
-					          (mapcar #'org-trim raw)))))
-	       (cdr (member org-babel-octave-eoe-output
-			    (reverse (mapcar #'org-strip-quotes
-					     (mapcar #'org-trim raw)))))))
-       (mapconcat #'identity (reverse results) "\n")))))
+       (if matlabp
+           (let* ((stripped (delete "" (mapcar #'org-strip-quotes raw-results)))
+                  ;; Trim extra newline, keeping MATLAB's newlines
+                  (trimmed (mapcar (lambda (str)
+                                     (replace-regexp-in-string "\n\n\\'" "\n" str))
+                                   stripped))
+                  (reversed (cdr (reverse trimmed))))
+             (concat (string-trim (mapconcat #'identity (reverse reversed)) "[\r\n]+")
+                     "\n"))
+         (let ((reversed (cdr (member org-babel-octave-eoe-output
+		                      (reverse (mapcar #'org-strip-quotes
+					               (mapcar #'org-trim raw-results)))))))
+	   (mapconcat #'identity (reverse reversed))))))))
 
 (defun org-babel-octave-import-elisp-from-file (file-name)
   "Import data from FILE-NAME.
diff --git a/lisp/org.el b/lisp/org.el
index 819a82eb9..1e46d238e 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -332,9 +332,10 @@ requirement."
 		 (const :tag "Lisp" lisp)
                  (const :tag "Lua" lua)
 		 (const :tag "Makefile" makefile)
+                 (const :tag "MATLAB" matlab)
 		 (const :tag "Maxima" maxima)
                  (const :tag "OCaml" ocaml)
-		 (const :tag "Octave and MatLab" octave)
+		 (const :tag "Octave" octave)
 		 (const :tag "Org" org)
 		 (const :tag "Perl" perl)
                  (const :tag "Processing" processing)
diff --git a/mk/default.mk b/mk/default.mk
index c5ea1ba01..2635d3a98 100644
--- a/mk/default.mk
+++ b/mk/default.mk
@@ -53,7 +53,7 @@ BTEST_POST  =
               # -L <path-to>/ert      # needed for Emacs23, Emacs24 has ert built in
               # -L <path-to>/ess      # needed for running R tests
               # -L <path-to>/htmlize  # need at least version 1.34 for source code formatting
-BTEST_OB_LANGUAGES = awk C fortran maxima lilypond octave perl python java sqlite eshell calc
+BTEST_OB_LANGUAGES = awk C fortran matlab maxima lilypond octave perl python java sqlite eshell calc
               # R                     # requires ESS to be installed and configured
               # ruby                  # requires inf-ruby to be installed and configured
 # extra packages to require for testing
diff --git a/testing/examples/ob-matlab-test.org b/testing/examples/ob-matlab-test.org
new file mode 100644
index 000000000..63500f223
--- /dev/null
+++ b/testing/examples/ob-matlab-test.org
@@ -0,0 +1,114 @@
+#+Title: A collection of examples for ob-matlab tests
+#+OPTIONS: ^:nil
+
+* Test MATLAB results output
+:PROPERTIES:
+:ID:       99332277-5e0e-4834-a3ad-ebcaddb855ab
+:END:
+
+#+begin_src matlab :results output
+  disp('The results are:')
+  a = [1, 2; 3, 4]
+  b = a * 2
+#+end_src
+
+#+RESULTS:
+#+begin_example
+The results are:
+
+a =
+
+     1     2
+     3     4
+
+b =
+
+     2     4
+     6     8
+#+end_example
+
+Following validates whitespace is preserved
+
+#+begin_src matlab :results output
+  disp('  +---+')
+  disp('  |one|')
+  disp('  +---+')
+  disp(newline)
+  disp(['    ', num2str(1)]);
+  disp(newline)
+  disp('  +---+')
+  disp('  |two|')
+  disp('  +---+')
+  disp(newline)
+  disp(['    ', num2str(2)]);
+  disp(newline)
+#+end_src
+
+* Test reuse of MATLAB buffer with results output
+:PROPERTIES:
+:ID:       a68db9ff-efdf-42b9-85fc-174015cd1f77
+:END:
+
+#+begin_src matlab :results output
+  a = 123
+#+end_src
+
+#+RESULTS:
+: a =
+: 
+:    123
+
+#+begin_src matlab :exports both :results output
+  b = a + 1000
+#+end_src
+
+#+RESULTS:
+: b =
+: 
+:         1123
+
+Following should give: Unrecognized function or variable 'b'.
+
+#+begin_src matlab :results output
+  clear
+  c = b * 2
+#+end_src
+
+#+RESULTS:
+: Unrecognized function or variable 'b'.
+
+* Test results verbatim
+:PROPERTIES:
+:ID:       278047b6-4b87-4852-9050-e3e99fcaabb8
+:END:
+
+#+begin_src matlab :results verbatim
+  a = 2 + 3;
+  ans = magic(a);
+#+end_src
+
+* Test results output latex
+:PROPERTIES:
+:ID:       7a8190be-d674-4944-864e-6fdaa7362585
+:END:
+
+#+begin_src matlab :results output latex
+  m = [4*pi, 3*pi; 2*pi, pi];
+  result = latex(sym(m));
+  disp(result)
+#+end_src
+
+* Test results file graphics
+:PROPERTIES:
+:ID:       5bee8841-a898-4135-b44b-f1bd5465ceed
+:END:
+
+#+begin_src matlab :results file graphics :file NAME.png
+  t = [0 : 0.1 : 2*pi];
+  y = sin(t);
+  plot(t, y);
+  set(gcf, 'PaperUnits', 'inches', 'PaperPosition', [0 0 4 3]) % Set the size to 4" x 3"
+#+end_src
+
+
+# LocalWords:  ebcaddb efdf fc fcaabb fdaa ceed sinewave
diff --git a/testing/lisp/test-ob-matlab.el b/testing/lisp/test-ob-matlab.el
new file mode 100644
index 000000000..476e5b1e4
--- /dev/null
+++ b/testing/lisp/test-ob-matlab.el
@@ -0,0 +1,191 @@
+;;; test-ob-matlab.el --- tests for ob-matlab.el  -*- lexical-binding: t; -*-
+
+;;; Commentary:
+
+;; Copyright 2024 Free Software Foundation
+;; Authors: John Ciolfi
+
+;; This file is not part of GNU Emacs.
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;; ----------
+;; Code blocks reside in: ../examples/ob-matlab-test.org
+;; To run test-ob-matlab.el using latest matlab-mode and org-mode:
+;; 1. Download and build latest org-mode, https://orgmode.org/worg/org-contribute.html
+;;    Example:
+;;      cd ~/github
+;;      git clone https://git.sv.gnu.org/git/emacs/org-mode.git
+;;      make -C org-mode compile -j32 && make prefix=~/github/org-mode-install install
+;; 2. Download and build latest matlab-mode, https://github.com/MathWorks/Emacs-MATLAB-Mode
+;;    Example:
+;;      cd ~/github
+;;      git clone https://github.com/mathworks/Emacs-MATLAB-Mode.git
+;;      make -C Emacs-MATLAB-Mode -j32
+;; 3. Test uses matlab-shell which requires that "matlab" is on the PATH:
+;;    Example:
+;;      cd ~/github/org-mode
+;;      env PATH=/path/to/MATLAB-INSTALL/bin:$PATH \
+;;          make -j32 test \
+;;          BTEST_POST="-L ~/github/Emacs-MATLAB-Mode -l ~/Emacs-MATLAB-Mode/matlab-autoload.el"
+
+;;; Code:
+
+(require 'ob-core)
+
+(org-test-for-executable "matlab")
+
+(unless (fboundp 'matlab-shell)
+  (signal 'missing-test-dependency '("Support for MATLAB code blocks")))
+
+;;-------------------------------;;
+;; Basic ":results output" tests ;;
+;;-------------------------------;;
+(ert-deftest ob-matlab/results-output ()
+  "Test matlab :results output code block."
+  (let ((expected "\
+The results are:
+
+a =
+
+     1     2
+     3     4
+
+b =
+
+     2     4
+     6     8
+"))
+    (org-test-at-id "99332277-5e0e-4834-a3ad-ebcaddb855ab"
+                    (org-babel-next-src-block)
+                    (should (equal expected (org-babel-execute-src-block))))))
+
+(ert-deftest ob-matlab/results-output-preserve-whitespace ()
+  "Test matlab :results output code block preserves whitespace."
+  (let ((expected "\
+  +---+
+  |one|
+  +---+
+
+    1
+
+  +---+
+  |two|
+  +---+
+
+    2
+"))
+    (org-test-at-id "99332277-5e0e-4834-a3ad-ebcaddb855ab"
+                    (org-babel-next-src-block 2)
+                    (should (equal expected (org-babel-execute-src-block))))))
+
+;;------------------------------------------------;;
+;; MATLAB workspace reuse tests (:results output) ;;
+;;------------------------------------------------;;
+
+(ert-deftest ob-matlab/results-output-reuse-a ()
+  "Test matlab :results output defining variable, a."
+  (let ((expected "\
+a =
+
+   123
+"))
+    (org-test-at-id "a68db9ff-efdf-42b9-85fc-174015cd1f77"
+                    (org-babel-next-src-block)
+                    (should (equal expected (org-babel-execute-src-block))))))
+
+(ert-deftest ob-matlab/results-output-reuse-b ()
+  "Test matlab :results output reusing variable, a, to compute variable, b."
+  (let ((expected "\
+b =
+
+        1123
+"))
+    (org-test-at-id "a68db9ff-efdf-42b9-85fc-174015cd1f77"
+                    (org-babel-next-src-block 2)
+                    (should (equal expected (org-babel-execute-src-block))))))
+
+(ert-deftest ob-matlab/results-output-reuse-clear ()
+  "Test matlab :results output with clear resulting in error.
+Also validates that we strip the </?ERRORTXT> matlab-shell indicators."
+  (let ((expected "\
+Unrecognized function or variable 'b'.
+"))
+    (org-test-at-id "a68db9ff-efdf-42b9-85fc-174015cd1f77"
+                    (org-babel-next-src-block 3)
+                    (should (equal expected (org-babel-execute-src-block))))))
+
+;;---------------------------;;
+;; ":results verbatim" tests ;;
+;;---------------------------;;
+(ert-deftest ob-matlab/results-verbatim ()
+  "Test matlab :results verbatim."
+  (let ((expected '((17 24 1 8 15)
+                    (23 5 7 14 16)
+                    (4 6 13 20 22)
+                    (10 12 19 21 3)
+                    (11 18 25 2 9))))
+    (org-test-at-id "278047b6-4b87-4852-9050-e3e99fcaabb8"
+                    (org-babel-next-src-block)
+                    (should (equal expected (org-babel-execute-src-block))))))
+
+;;-------------------------------;;
+;; ":results output latex" tests ;;
+;;-------------------------------;;
+
+(ert-deftest ob-matlab/results-output-latex ()
+  "Test matlab :results output latex."
+  (let ((expected "\
+\\left(\\begin{array}{cc} 4\\,\\pi  & 3\\,\\pi \\\\ 2\\,\\pi  & \\pi  \\end{array}\\right)
+"))
+    (org-test-at-id "7a8190be-d674-4944-864e-6fdaa7362585"
+                    (org-babel-next-src-block)
+                    (let ((got (org-babel-execute-src-block)))
+                      (should (equal expected got))))))
+
+
+;;--------------------------------;;
+;; ":results file graphics" tests ;;
+;;--------------------------------;;
+
+(ert-deftest ob-matlab/results-file-graphics ()
+  "Test matlab :results file graphics."
+  (let ((code-block (org-test-get-code-block "5bee8841-a898-4135-b44b-f1bd5465ceed"))
+        (temp-file-png (make-temp-file "test-ob-matlab-" nil ".png")))
+    (setq code-block (replace-regexp-in-string "NAME\\.png" temp-file-png code-block))
+    (unwind-protect
+        (org-test-with-temp-text
+         code-block
+         (org-babel-execute-src-block)
+         (should (search-forward (format "[[file:%s]]" temp-file-png) nil nil))
+         (should (file-readable-p temp-file-png)))
+      (delete-file temp-file-png))))
+
+(ert-deftest ob-matlab/results-file-graphics-with-space ()
+  "Test matlab :results file graphics using a file name with a space."
+  (let ((code-block (org-test-get-code-block "5bee8841-a898-4135-b44b-f1bd5465ceed"))
+        (temp-file-png (make-temp-file "test ob-matlab-" nil ".png")))
+    (setq code-block (replace-regexp-in-string "NAME\\.png" temp-file-png code-block))
+    (unwind-protect
+        (org-test-with-temp-text
+         code-block
+         (org-babel-execute-src-block)
+         (should (search-forward (format "[[file:%s]]" temp-file-png) nil nil))
+         (should (file-readable-p temp-file-png)))
+      (delete-file temp-file-png))))
+
+(provide 'test-ob-matlab)
+;;; test-ob-matlab.el ends here (emacs-lisp-checkdoc)
+
+;; LocalWords:  fboundp ebcaddb efdf fc ERRORTXT fcaabb fdaa ceed setq env BTEST
diff --git a/testing/org-test.el b/testing/org-test.el
index 52d38c3fd..93dfa46fc 100644
--- a/testing/org-test.el
+++ b/testing/org-test.el
@@ -133,6 +133,27 @@ currently executed.")
        (unless (or visited-p (not to-be-removed))
 	 (kill-buffer to-be-removed)))))
 
+(defun org-test-get-code-block (id &optional count)
+  "Get the COUNT, defaulting to first, code block after ID."
+  (let* ((id-location (org-id-find id))
+         (id-file (car id-location))
+         (visited-p (get-file-buffer id-file))
+         code-block
+         to-be-removed)
+    (unwind-protect
+        (save-window-excursion
+          (save-match-data
+            (org-id-goto id)
+            (org-babel-next-src-block count)
+            (beginning-of-line)
+            (setq code-block (buffer-substring-no-properties
+                              (point)
+                              (re-search-forward "^[ \t]*#\\+end_src[ \t\r]*\n")))
+            (setq to-be-removed (current-buffer))))
+      (unless (or visited-p (not to-be-removed))
+        (kill-buffer to-be-removed)))
+    code-block))
+
 (defmacro org-test-in-example-file (file &rest body)
   "Execute body in the Org example file."
   (declare (indent 1) (debug t))

  reply	other threads:[~2025-01-03  0:33 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-11-08 19:30 ob-octave: improve MATLAB support John C
2024-11-09  9:32 ` Ihor Radchenko
2024-11-13 19:10   ` John C
2024-11-15 13:29     ` John C
2024-11-23 15:57     ` Ihor Radchenko
     [not found]       ` <87o712lv3t.fsf@localhost>
2024-12-29  3:04         ` John C
2024-12-29  7:42           ` Ihor Radchenko
2025-01-02 23:55             ` John C
2025-01-02 23:56               ` John C [this message]
2025-01-04 13:02                 ` Ihor Radchenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CACb3vdSc2ypq5hdd3Kgb3K+wek-0LDwUqUPugzUr2tt5yyg23Q@mail.gmail.com \
    --to=john.ciolfi.32@gmail.com \
    --cc=emacs-orgmode@gnu.org \
    --cc=yantar92@posteo.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.