From: John C <john.ciolfi.32@gmail.com>
To: Ihor Radchenko <yantar92@posteo.net>
Cc: emacs-orgmode@gnu.org
Subject: Re: ob-octave: improve MATLAB support
Date: Thu, 2 Jan 2025 18:56:48 -0500 [thread overview]
Message-ID: <CACb3vdSc2ypq5hdd3Kgb3K+wek-0LDwUqUPugzUr2tt5yyg23Q@mail.gmail.com> (raw)
In-Reply-To: <CACb3vdQLF8D6buAv0CQZoYmwajwkAhCsCcwW+BVdv6v=amgzYA@mail.gmail.com>
[-- Attachment #1.1: Type: text/plain, Size: 11465 bytes --]
Attaching the updated org-matlab.patch
On Thu, Jan 2, 2025 at 6:55 PM John C <john.ciolfi.32@gmail.com> wrote:
> Hi
>
> Thanks for the feedback. See updated org-matlab.patch. Note
> code evaluation will now error out when using an out-of-date matlab-mode
> (rather than generating a warning) because when using an older matlab-mode,
> code block evaluation doesn't work. We need to wait for the matlab-shell to
> be ready. Without that capability found in newer matlab-mode, the errors
> are very hard to understand. I also added tests. Using steps outlined in
> the ob-matlab-test.el comment to setup for MATLAB testing, I get on Linux:
>
> matlab-mode, version 6.3
> Detected MATLAB R2024b (24.2) -- Loading history file
> matlab-shell: starting server with name server-2055614
> passed 127/1250 ob-matlab/results-file-graphics (9.712776 sec)
> passed 128/1250 ob-matlab/results-file-graphics-with-space (0.332004
> sec)
> passed 129/1250 ob-matlab/results-output (0.013681 sec)
> passed 130/1250 ob-matlab/results-output-latex (0.459980 sec)
> passed 131/1250 ob-matlab/results-output-preserve-whitespace
> (0.281001 sec)
> passed 132/1250 ob-matlab/results-output-reuse-a (0.039130 sec)
> passed 133/1250 ob-matlab/results-output-reuse-b (0.043762 sec)
> passed 134/1250 ob-matlab/results-output-reuse-clear (0.301597 sec)
> passed 135/1250 ob-matlab/results-verbatim (0.172567 sec)
>
> You mentioned that we should expect org users to understand sessions,
> however the doc on sessions is a bit light. Searching
> https://orgmode.org/org.html for "session", we see it used two different
> ways "Emacs session" and babel code block evaluations sessions described in
> "Using sessions". What is there is okay, but doesn't really help one
> understand when they should/shouldn't use babel code block evaluation
> sessions. What would be nice is if "Using sessions" were titled "Babel code
> block evaluation sessions" containing the existing content minus the "Only
> languages that provide interactive evaluation ..." paragraph because this
> is vague. In addition, each language should have a way of describing what
> :session means to them, perhaps by extracting this info from ob-LANGUAGE.el
> or maybe ob-LANGUAGE.org, or maybe something else that inserts the language
> specific into the org manual.
>
> With good MATLAB integration in org-mode, we'll see scientists and
> engineers who have limited programming background leverage org-mode for
> their scientific papers. Not having to worry about more advanced concepts
> like sessions is nice, so thanks for letting us default the :session for
> MATLAB to make it work as one would expect.
>
> Here's the commit info:
>
> ob-matlab.el: improve MATLAB support
>
> * lisp/ob-matlab.el (header): Update URL for MATLAB
>
> * lisp/ob-octave.el (org-babel-octave-evaluate): Fixed MATLAB support
> - Deprecate variables related to MATLAB Emacs Link and removed the code.
> Emacs Link capability was removed from MATLAB release R2009a, 15 years
> ago.
> - Fixed the following type of org block evaluation:
> 1) #+begin_src matlab :results verbatim
> 2) #+begin_src matlab :results output
> 3) #+begin_src matlab :results output latex
> 4) #+begin_src matlab :results file graphics
> which aid in writing scientific papers.
> - Minor point, the correct spelling of MATLAB when referencing the
> product is
> all upper case.
>
> * lisp/ob-comint.el: enhanced org-babel-comint-with-output
> - The META argument of org-babel-comint-with-output now supports an
> optional
> STRIP-REGEXPS which can be used to remove content from the returned
> output.
>
> * lisp/org.el
> - Add MATLAB as one of the options for org-babel-load-languages
>
> * mk/default.mk
> - Add support for testing matlab code blocks.
>
> * testing/examples/ob-matlab-test.org, testing/lisp/test-ob-matlab.el
> - Test for matlab code block.
>
> * testing/org-test.el
> - Added org-test-get-code-block which is for use by
> testing/lisp/test*.el files
> to extract code blocks from testing/examples/*.org files for on-the-fly
> testing using org-test-with-temp-text.
>
> * etc/ORG-NEWS (New functions and changes in function arguments):
> Added entry "ob-octave: improved MATLAB support"
>
> Thanks
> John
>
>
> On Sun, Dec 29, 2024 at 2:41 AM Ihor Radchenko <yantar92@posteo.net>
> wrote:
>
>> John C <john.ciolfi.32@gmail.com> writes:
>>
>> > See attached org-matlab.patch which addresses all feedback. Here's the
>> > commit info.
>>
>> Thanks!
>> See my comments inline.
>>
>> > +*** ob-matlab: fixed MATLAB support
>> > +
>> > +Fixed MATLAB babel code blocks processing. MATLAB code blocks,
>> ~#+begin_src matlab~, with ~:results
>> > +verbatim~, ~:results output~, ~:results output latex~, or ~:results
>> file graphics~ now work. Fixes
>> > +include (1) waiting for matlab-shell to start before evaluating MATLAB
>> code, (2) correctly showing
>> > +the results using writematrix, (3) removing the code block lines from
>> the result, (4) correctly
>> > +handling graphics results by invoking print correctly. To use MATLAB
>> with org, you need
>> > +https://github.com/MathWorks/Emacs-MATLAB-Mode.
>>
>> There is no need to provide so many details.
>> Just leave the most important things:
>>
>> 1. MATLAB is no longer broken
>> 2. Emacs-MATLAB-Mode is required now
>>
>> > +*** ob-matlab: MATLAB behavior change
>> > +
>> > +MATLAB code blocks now reuse the ~MATLAB*~ buffer created by ~M-x
>> matlab-shell~, whereas the
>> > +prior version started a new shell for each evaluation. The benefit of
>> this is that
>> > +evaluations are very fast after the first evaluation and that state is
>> maintained between
>> > +evaluations, which you can clear using the MATLAB ~clear~ command.
>> Another benefit of this
>> > +behavior is that it is consistent with how MATLAB works.
>>
>> No need to explain in so much details, I think.
>> Just say that MATLAB uses session by default and them mention that users
>> may customize `org-babel-default-header-args:matlab' to disable session.
>>
>> > +(defun org-babel-comint--strip-regexps (result strip-regexps)
>> > + "STRIP-REGEXPS from RESULT list of strings."
>> > + (dolist (strip-regexp strip-regexps)
>> > + (let ((new-result '()))
>> > + (dolist (line result)
>> > + (setq line (replace-regexp-in-string strip-regexp "" line))
>> > + (when (not (string= line ""))
>> > + (setq new-result (append new-result `(,line)))))
>>
>> It is more efficient to use `push' + `nreverse' instead of `append'.
>>
>> > -(defvar org-babel-default-header-args:matlab '())
>> > +;; With `org-babel-default-header-args:matlab' set to
>> > +;; '((:session . "*MATLAB*")))
>> > +;; ...
>> > +;; If you want a new session each time you evaluate a MATLAB code
>> block,
>> > +;; (setq 'org-babel-default-header-args:matlab '())
>> > +;; However, this will make evaluations slower and is not consistent
>> with how
>> > +;; MATLAB works. MATLAB is designed for many evaluations.
>> > +(defvar org-babel-default-header-args:matlab '((:session .
>> "*MATLAB*")))
>>
>> You don't need that long comment in the source code.
>> If you think that explaining the details about session is necessary (it
>> may or may not be, but we should assume that Org users are familiar with
>> the notion of sessions in code blocks), please do it in the
>> documentation, not in the code.
>>
>> More generally, your motivation is not specific to matlab. Yet, we
>> default to no session in most babel backends. So, it is not a question
>> of session being faster or slower, but a question of consistency.
>>
>> That said, some babel backends do default to session, so I do not oppose
>> this change too much.
>>
>> > +(make-obsolete-variable 'org-babel-matlab-with-emacs-link
>> > + "MATLAB removed EmacsLink in R2009a." "2009")
>> > +
>> > +(make-obsolete-variable 'org-babel-matlab-emacs-link-wrapper-method
>> > + "MATLAB removed EmacsLink in R2009a." "2009")
>>
>> Please use Org version in WHEN argument of `make-obsolete-variable'.
>> The WHEN should be "9.8".
>>
>> > +(defun org-babel-matlab-shell ()
>> > + "Start and/or wait for MATLAB shell."
>> > + (require 'matlab-shell) ;; make `matlab-shell-busy-checker' available
>> > + (cond
>> > + ((fboundp 'matlab-shell-busy-checker)
>> > + ;; Start the shell if needed. `matlab-shell' will reuse existing
>> if already running.
>> > + (matlab-shell)
>> > + ;; If we just started the matlab-shell, wait for the prompt. If
>> we do not
>> > + ;; wait, then the startup messages will show up in the evaluation
>> results.
>> > + (matlab-shell-busy-checker 'wait-for-prompt))
>> > + (t
>> > + (message (concat "You version of matlab-mode is old.\n"
>> > + "Please update, see
>> https://github.com/mathworks/Emacs-MATLAB-Mode\n"
>> > + "Updating will eliminate unexpected output in
>> your results\n"))
>> > + (sit-for 3)
>>
>> Instead of `message' + `sit-fit', you can simply use `display-warning'.
>> It will give users more control.
>>
>> > +(defun org-babel-body-for-output (body matlabp)
>> > + "If MATLABP, fixup BODY for MATLAB output result-type."
>> > + (when matlabp
>> > + ;; When we send multi-line input to `matlab-shell', we'll see the
>> "body"
>> > + ;; code lines echoed in the output which is not what one would
>> expect. To
>> > + ;; remove these unwanted lines, we append a comment "%-<org-eval>"
>> to each
>> > + ;; line in the body MATLAB code. After we collect the results from
>> > + ;; evaluation, we leverage the "%-<org-eval>" to remove the
>> unwanted lines.
>> > + ;; Example of desired behavior:
>> > ...
>>
>> I think that an important point here is that MATLAB does not echo the
>> whole BODY all at once and instead mixes it with the output. Which is
>> why we need to do something non-standard to filter out the body in
>> matlab specifically.
>>
>> > + (setq body (replace-regexp-in-string "\n" " %-<org-eval>-\n" body))
>> > + (when (not (string-match "\n\\'" body))
>> > + (setq body (concat body " %-<org-eval>-"))))
>> > + body)
>>
>> Please put this %-<org-eval> into an internal constant and then reuse
>> it when building the regexp to filter.
>>
>> > + (when matlabp
>> > + '(;; MATLAB echo's all input lines, so use the
>> %-<org-eval> comments to strip
>> > + ;; them from the output
>> > + "^[^\n]*%-<org-eval>-\n"
>> > + ;; Remove starting blank line caused by stripping
>> %-<org-eval>
>> > + "\\`[[:space:]\r\n]+"
>> > + ;; Strip <ERRORTXT> and </ERRORTXT> matlab-shell
>> error indicators
>> > + "</?ERRORTXT>\n")))
>>
>> Same here. Please put these regexps into a constant.
>>
>> --
>> Ihor Radchenko // yantar92,
>> Org mode maintainer,
>> Learn more about Org mode at <https://orgmode.org/>.
>> Support Org development at <https://liberapay.com/org-mode>,
>> or support my work at <https://liberapay.com/yantar92>
>>
>
[-- Attachment #1.2: Type: text/html, Size: 13865 bytes --]
[-- Attachment #2: org-matlab.patch --]
[-- Type: text/x-patch, Size: 29430 bytes --]
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 5d421172f..4f9523cf3 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -162,6 +162,17 @@ bibliography format requires them to be written in title-case.
# This also includes changes in function behavior from Elisp perspective.
+*** ob-matlab: fixed MATLAB support
+
+Fixed MATLAB babel code blocks processing. MATLAB code blocks, ~#+begin_src matlab~, with ~:results
+verbatim~, ~:results output~, ~:results output latex~, or ~:results file graphics~ now work. To use
+MATLAB with org, you need https://github.com/MathWorks/Emacs-MATLAB-Mode.
+
+*** ob-matlab: MATLAB behavior change
+
+MATLAB code blocks now reuse the ~MATLAB*~ buffer created by ~M-x matlab-shell~ by default. To
+change this behavior, customize ~org-babel-default-header-args:matlab~.
+
*** ob-sqlite: Added ability to open a database in readonly mode
Added option :readonly to ob-sqlite.
diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el
index b88ac445a..2246badc2 100644
--- a/lisp/ob-comint.el
+++ b/lisp/ob-comint.el
@@ -101,15 +101,28 @@ PROMPT-REGEXP defaults to `comint-prompt-regexp'."
(setq string (substring string (match-end 0))))
string)
+(defun org-babel-comint--strip-regexps (result strip-regexps)
+ "STRIP-REGEXPS from RESULT list of strings."
+ (dolist (strip-regexp strip-regexps)
+ (let ((new-result '()))
+ (dolist (str result)
+ (setq str (replace-regexp-in-string strip-regexp "" str))
+ (when (not (string= str ""))
+ (push str new-result)))
+ (setq result (nreverse new-result))))
+ result)
+
(defmacro org-babel-comint-with-output (meta &rest body)
"Evaluate BODY in BUFFER and return process output.
Will wait until EOE-INDICATOR appears in the output, then return
all process output. If REMOVE-ECHO and FULL-BODY are present and
-non-nil, then strip echo'd body from the returned output. META
-should be a list containing the following where the last two
-elements are optional.
+non-nil, then strip echoed body from the returned output. If
+optional STRIP-REGEXPS, a list of regular expressions, is
+present, then all matches will be removed from the returned
+output. META should be a list containing the following where the
+last three elements are optional.
- (BUFFER EOE-INDICATOR REMOVE-ECHO FULL-BODY)
+ (BUFFER EOE-INDICATOR REMOVE-ECHO FULL-BODY STRIP-REGEXPS)
This macro ensures that the filter is removed in case of an error
or user `keyboard-quit' during execution of body."
@@ -117,7 +130,8 @@ or user `keyboard-quit' during execution of body."
(let ((buffer (nth 0 meta))
(eoe-indicator (nth 1 meta))
(remove-echo (nth 2 meta))
- (full-body (nth 3 meta)))
+ (full-body (nth 3 meta))
+ (strip-regexps (nth 4 meta)))
`(org-babel-comint-in-buffer ,buffer
(let* ((string-buffer "")
(comint-output-filter-functions
@@ -165,8 +179,12 @@ or user `keyboard-quit' during execution of body."
(and ,remove-echo ,full-body
(setq string-buffer (org-babel-comint--echo-filter string-buffer ,full-body)))
- ;; Filter out prompts.
- (org-babel-comint--prompt-filter string-buffer)))))
+ ;; Filter out prompts from returned output result.
+ (let ((result (org-babel-comint--prompt-filter string-buffer)))
+ ;; Remove all matches of STRIP-REGEXPS in returned output result.
+ (when ,strip-regexps
+ (setq result (org-babel-comint--strip-regexps result ,strip-regexps)))
+ result)))))
(defun org-babel-comint-input-command (buffer cmd)
"Pass CMD to BUFFER.
diff --git a/lisp/ob-matlab.el b/lisp/ob-matlab.el
index de8deadbe..083dcdec3 100644
--- a/lisp/ob-matlab.el
+++ b/lisp/ob-matlab.el
@@ -28,11 +28,10 @@
;;; Requirements:
-;; Matlab
-
-;; matlab.el required for interactive emacs sessions and matlab-mode
-;; major mode for source code editing buffer
-;; https://matlab-emacs.sourceforge.net/
+;; 1) MATLAB
+;; 2) https://github.com/mathworks/Emacs-MATLAB-Mode
+;; For matlab-shell to run MATLAB within Emacs and matlab-mode
+;; major mode for source code editing buffer
;;; Code:
diff --git a/lisp/ob-octave.el b/lisp/ob-octave.el
index 005990f20..5e622df9a 100644
--- a/lisp/ob-octave.el
+++ b/lisp/ob-octave.el
@@ -1,8 +1,8 @@
-;;; ob-octave.el --- Babel Functions for Octave and Matlab -*- lexical-binding: t; -*-
+;;; ob-octave.el --- Babel Functions for Octave and MATLAB -*- lexical-binding: t; -*-
;; Copyright (C) 2010-2024 Free Software Foundation, Inc.
-;; Author: Dan Davison
+;; Author: Dan Davison (Octave), John Ciolfi (MATLAB)
;; Keywords: literate programming, reproducible research
;; URL: https://orgmode.org
@@ -30,6 +30,8 @@
;;; Code:
+(require 'cl-seq)
+
(require 'org-macs)
(org-assert-version)
@@ -39,7 +41,11 @@
(declare-function matlab-shell "ext:matlab-mode")
(declare-function matlab-shell-run-region "ext:matlab-mode")
-(defvar org-babel-default-header-args:matlab '())
+(defvar org-babel-default-header-args:matlab '((:session . "*MATLAB*"))
+ "Reuse the matlab-shell buffer for code block evaluations.
+Add the MATLAB clear command to your code block to clear the
+MATLAB workspace.")
+
(defvar org-babel-default-header-args:octave '())
(defvar org-babel-matlab-shell-command "matlab -nosplash"
@@ -47,18 +53,42 @@
(defvar org-babel-octave-shell-command "octave -q"
"Shell command to run octave as an external process.")
-(defvar org-babel-matlab-with-emacs-link nil
- "If non-nil use matlab-shell-run-region for session evaluation.
-This will use EmacsLink if (matlab-with-emacs-link) evaluates
-to a non-nil value.")
-
-(defvar org-babel-matlab-emacs-link-wrapper-method
- "%s
-if ischar(ans), fid = fopen('%s', 'w'); fprintf(fid, '%%s\\n', ans); fclose(fid);
-else, save -ascii %s ans
-end
-delete('%s')
+(make-obsolete-variable 'org-babel-matlab-with-emacs-link
+ "MATLAB removed EmacsLink in R2009a." "9.8")
+
+(make-obsolete-variable 'org-babel-matlab-emacs-link-wrapper-method
+ "MATLAB removed EmacsLink in R2009a." "9.8")
+
+(defvar org-babel-matlab-print "print(\"-dpng\", %S);\nans=%S;"
+ ;; MATLAB command-function duality requires that the file name be specified
+ ;; without quotes. Using: print -dpng "file.png", would produce a file with
+ ;; the quotes in the file name on disk. Therefore, use the functional form
+ ;; to handle files with spaces, print("-dpng", "file.png").
+ ;; Example:
+ ;; #+begin_src matlab :file "sine wave.png" :results file graphics
+ ;; t = [0 : 0.1 : 2*pi];
+ ;; y = sin(t);
+ ;; plot(t, y);
+ ;; set(gcf, 'PaperUnits', 'inches', 'PaperPosition', [0 0 4 3]) % Set the size to 4" x 3"
+ ;; #+end_src
+ ;;
+ ;; #+RESULTS:
+ ;; [[file:sine wave.png]]
+ "MATLAB format specifier to print current figure to a file.")
+
+(defvar org-babel-octave-print "print -dpng %S\nans=%S"
+ "Octave format specifier to print current figure to a file.")
+
+(defvar org-babel-matlab-wrapper-method
+ (concat "\
+cd('%s');
+%s
+if ~exist('ans', 'var') ans = ''; end; \
+writematrix(ans, '%s', 'Delimiter', 'tab');
")
+ "Format specifier used when evaluating MATLAB code blocks.
+Arguments are the `default-directory', the MATLAB code, and a result file.txt.")
+
(defvar org-babel-octave-wrapper-method
"%s
if ischar(ans), fid = fopen('%s', 'w'); fdisp(fid, ans); fclose(fid);
@@ -92,7 +122,10 @@ When MATLABP is non-nil, execute Matlab. Otherwise, execute Octave."
(list
"set (0, \"defaultfigurevisible\", \"off\");"
full-body
- (format "print -dpng %S\nans=%S" gfx-file gfx-file))
+ (format (if matlabp
+ org-babel-matlab-print
+ org-babel-octave-print)
+ gfx-file gfx-file))
"\n")
full-body)
result-type matlabp)))
@@ -153,6 +186,19 @@ If there is not a current inferior-process-buffer in SESSION then
create. Return the initialized session. PARAMS are src block parameters."
(org-babel-octave-initiate-session session params 'matlab))
+(defun org-babel-matlab-shell ()
+ "Start and/or wait for MATLAB shell."
+ (require 'matlab-shell) ;; make `matlab-shell-busy-checker' available
+ (if (fboundp 'matlab-shell-busy-checker)
+ (progn
+ ;; Start the shell if needed. `matlab-shell' will be reused if it is already running.
+ (matlab-shell)
+ ;; If we just started the matlab-shell, wait for the prompt. If we do not
+ ;; wait, then the startup messages will show up in the evaluation results.
+ (matlab-shell-busy-checker 'wait-for-prompt))
+ (user-error "A newer version of matlab-mode is required, see \
+https://github.com/mathworks/Emacs-MATLAB-Mode\n")))
+
(defun org-babel-octave-initiate-session (&optional session _params matlabp)
"Create an octave inferior process buffer.
If there is not a current inferior-process-buffer in SESSION then
@@ -165,9 +211,15 @@ Octave session, unless MATLABP is non-nil."
(unless (string= session "none")
(let ((session (or session
(if matlabp "*Inferior Matlab*" "*Inferior Octave*"))))
- (if (org-babel-comint-buffer-livep session) session
+ (if (org-babel-comint-buffer-livep session)
+ (progn
+ (when (and matlabp (fboundp 'matlab-shell-busy-checker))
+ ;; Can't evaluate if the matlab-shell is currently running code
+ (matlab-shell-busy-checker 'error-if-busy))
+ session)
(save-window-excursion
- (if matlabp (unless org-babel-matlab-with-emacs-link (matlab-shell))
+ (if matlabp
+ (org-babel-matlab-shell)
(run-octave))
(rename-buffer (if (bufferp session) (buffer-name session)
(if (stringp session) session (buffer-name))))
@@ -183,79 +235,125 @@ value of the last statement in BODY, as elisp."
(org-babel-octave-evaluate-session session body result-type matlabp)
(org-babel-octave-evaluate-external-process body result-type matlabp)))
+(defun org-babel-octave-wrapper-tmp-file (matlabp)
+ "Return a local tmp file with name adjusted for MATLABP."
+ (if matlabp
+ ;; writematrix requires a file ending with '.txt'
+ (org-babel-temp-file "matlab-" ".txt")
+ (org-babel-temp-file "octave-")))
+
+(defun org-babel-octave-get-code-to-eval (body tmp-file matlabp)
+ "Format BODY of the code block for evaluation saving results to TMP-FILE.
+If MATLABP, format for MATLAB, else format for Octave."
+ (if matlabp
+ (format org-babel-matlab-wrapper-method default-directory body tmp-file)
+ (format org-babel-octave-wrapper-method body tmp-file tmp-file)))
+
(defun org-babel-octave-evaluate-external-process (body result-type matlabp)
- "Evaluate BODY in an external Octave or Matalab process.
+ "Evaluate BODY in an external Octave or MATLAB process.
Process the result as RESULT-TYPE. Use Octave, unless MATLABP is non-nil."
(let ((cmd (if matlabp
org-babel-matlab-shell-command
org-babel-octave-shell-command)))
(pcase result-type
(`output (org-babel-eval cmd body))
- (`value (let ((tmp-file (org-babel-temp-file "octave-")))
+ (`value (let ((tmp-file (org-babel-process-file-name
+ (org-babel-octave-wrapper-tmp-file matlabp)
+ 'noquote)))
(org-babel-eval
cmd
- (format org-babel-octave-wrapper-method body
- (org-babel-process-file-name tmp-file 'noquote)
- (org-babel-process-file-name tmp-file 'noquote)))
+ (org-babel-octave-get-code-to-eval body tmp-file matlabp))
(org-babel-octave-import-elisp-from-file tmp-file))))))
+(defvar org-babel-octave--matlab-line-indicator " %-<org-eval>\n"
+ "Comment appended to each code line being evaluated.")
+
+(defvar org-babel-octave--matlab-error-indicator-re "</?ERRORTXT>\r?\n?"
+ "MATLAB shell error indicators.")
+
+(defun org-babel-octave-body-for-output (body matlabp)
+ "If MATLABP, fix up BODY for MATLAB output result-type."
+ (when matlabp
+ ;; MATLAB does not echo the whole BODY all at once and instead mixes input
+ ;; with the output. Therefore, when we send multi-line input to
+ ;; `matlab-shell', we'll see the "body" code lines echoed in the output
+ ;; which is not what one would expect. To remove these unwanted lines,
+ ;; we append `org-babel-octave--matlab-line-indicator' to each line in the BODY.
+ ;; We leverage this indicator to remove the unwanted lines.
+ ;; Example of desired behavior:
+ ;; #+begin_src matlab :results output
+ ;; disp('The results are:')
+ ;; a = [1, 2; 3, 4]
+ ;; b = a * 2
+ ;; #+end_src
+ ;;
+ ;; #+RESULTS:
+ ;; #+begin_example
+ ;; The results are:
+ ;;
+ ;; a =
+ ;;
+ ;; 1 2
+ ;; 3 4
+ ;;
+ ;; b =
+ ;;
+ ;; 2 4
+ ;; 6 8
+ ;; #+end_example
+ (setq body (replace-regexp-in-string "\r" "" body)) ;; CRLF => LF
+ (setq body (concat (string-trim-right body) "\n")) ;; Ensure a single final newline
+ (setq body (replace-regexp-in-string "\n" org-babel-octave--matlab-line-indicator body)))
+ body)
+
(defun org-babel-octave-evaluate-session
(session body result-type &optional matlabp)
"Evaluate BODY in SESSION."
- (let* ((tmp-file (org-babel-temp-file (if matlabp "matlab-" "octave-")))
- (wait-file (org-babel-temp-file "matlab-emacs-link-wait-signal-"))
+ (let* ((tmp-file (org-babel-octave-wrapper-tmp-file matlabp))
(full-body
(pcase result-type
(`output
(mapconcat
#'org-babel-chomp
- (list body org-babel-octave-eoe-indicator) "\n"))
+ (list (org-babel-octave-body-for-output body matlabp)
+ org-babel-octave-eoe-indicator)
+ "\n"))
(`value
- (if (and matlabp org-babel-matlab-with-emacs-link)
- (concat
- (format org-babel-matlab-emacs-link-wrapper-method
- body
- (org-babel-process-file-name tmp-file 'noquote)
- (org-babel-process-file-name tmp-file 'noquote) wait-file) "\n")
- (mapconcat
- #'org-babel-chomp
- (list (format org-babel-octave-wrapper-method
- body
- (org-babel-process-file-name tmp-file 'noquote)
- (org-babel-process-file-name tmp-file 'noquote))
- org-babel-octave-eoe-indicator) "\n")))))
- (raw (if (and matlabp org-babel-matlab-with-emacs-link)
- (save-window-excursion
- (with-temp-buffer
- (insert full-body)
- (write-region "" 'ignored wait-file nil nil nil 'excl)
- (matlab-shell-run-region (point-min) (point-max))
- (message "Waiting for Matlab Emacs Link")
- (while (file-exists-p wait-file) (sit-for 0.01))
- "")) ;; matlab-shell-run-region doesn't seem to
- ;; make *matlab* buffer contents easily
- ;; available, so :results output currently
- ;; won't work
- (org-babel-comint-with-output
- (session
- (if matlabp
- org-babel-octave-eoe-indicator
- org-babel-octave-eoe-output)
- t full-body)
- (insert full-body) (comint-send-input nil t))))
- results)
+ (mapconcat
+ #'org-babel-chomp
+ (list (org-babel-octave-get-code-to-eval body tmp-file matlabp)
+ org-babel-octave-eoe-indicator)
+ "\n"))))
+ (raw-results
+ (org-babel-comint-with-output
+ (session
+ (if matlabp
+ org-babel-octave-eoe-indicator
+ org-babel-octave-eoe-output)
+ t full-body ;; Remove echo'd full-body from result
+ (when matlabp
+ `(;; MATLAB echo's input interleaved w/output, so strip inputs
+ ,(concat "^.*" org-babel-octave--matlab-line-indicator)
+ ;; Strip matlab-shell error indicators
+ ,org-babel-octave--matlab-error-indicator-re)))
+ (insert full-body) (comint-send-input nil t))))
(pcase result-type
(`value
(org-babel-octave-import-elisp-from-file tmp-file))
(`output
- (setq results
- (if matlabp
- (cdr (reverse (delete "" (mapcar #'org-strip-quotes
- (mapcar #'org-trim raw)))))
- (cdr (member org-babel-octave-eoe-output
- (reverse (mapcar #'org-strip-quotes
- (mapcar #'org-trim raw)))))))
- (mapconcat #'identity (reverse results) "\n")))))
+ (if matlabp
+ (let* ((stripped (delete "" (mapcar #'org-strip-quotes raw-results)))
+ ;; Trim extra newline, keeping MATLAB's newlines
+ (trimmed (mapcar (lambda (str)
+ (replace-regexp-in-string "\n\n\\'" "\n" str))
+ stripped))
+ (reversed (cdr (reverse trimmed))))
+ (concat (string-trim (mapconcat #'identity (reverse reversed)) "[\r\n]+")
+ "\n"))
+ (let ((reversed (cdr (member org-babel-octave-eoe-output
+ (reverse (mapcar #'org-strip-quotes
+ (mapcar #'org-trim raw-results)))))))
+ (mapconcat #'identity (reverse reversed))))))))
(defun org-babel-octave-import-elisp-from-file (file-name)
"Import data from FILE-NAME.
diff --git a/lisp/org.el b/lisp/org.el
index 819a82eb9..1e46d238e 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -332,9 +332,10 @@ requirement."
(const :tag "Lisp" lisp)
(const :tag "Lua" lua)
(const :tag "Makefile" makefile)
+ (const :tag "MATLAB" matlab)
(const :tag "Maxima" maxima)
(const :tag "OCaml" ocaml)
- (const :tag "Octave and MatLab" octave)
+ (const :tag "Octave" octave)
(const :tag "Org" org)
(const :tag "Perl" perl)
(const :tag "Processing" processing)
diff --git a/mk/default.mk b/mk/default.mk
index c5ea1ba01..2635d3a98 100644
--- a/mk/default.mk
+++ b/mk/default.mk
@@ -53,7 +53,7 @@ BTEST_POST =
# -L <path-to>/ert # needed for Emacs23, Emacs24 has ert built in
# -L <path-to>/ess # needed for running R tests
# -L <path-to>/htmlize # need at least version 1.34 for source code formatting
-BTEST_OB_LANGUAGES = awk C fortran maxima lilypond octave perl python java sqlite eshell calc
+BTEST_OB_LANGUAGES = awk C fortran matlab maxima lilypond octave perl python java sqlite eshell calc
# R # requires ESS to be installed and configured
# ruby # requires inf-ruby to be installed and configured
# extra packages to require for testing
diff --git a/testing/examples/ob-matlab-test.org b/testing/examples/ob-matlab-test.org
new file mode 100644
index 000000000..63500f223
--- /dev/null
+++ b/testing/examples/ob-matlab-test.org
@@ -0,0 +1,114 @@
+#+Title: A collection of examples for ob-matlab tests
+#+OPTIONS: ^:nil
+
+* Test MATLAB results output
+:PROPERTIES:
+:ID: 99332277-5e0e-4834-a3ad-ebcaddb855ab
+:END:
+
+#+begin_src matlab :results output
+ disp('The results are:')
+ a = [1, 2; 3, 4]
+ b = a * 2
+#+end_src
+
+#+RESULTS:
+#+begin_example
+The results are:
+
+a =
+
+ 1 2
+ 3 4
+
+b =
+
+ 2 4
+ 6 8
+#+end_example
+
+Following validates whitespace is preserved
+
+#+begin_src matlab :results output
+ disp(' +---+')
+ disp(' |one|')
+ disp(' +---+')
+ disp(newline)
+ disp([' ', num2str(1)]);
+ disp(newline)
+ disp(' +---+')
+ disp(' |two|')
+ disp(' +---+')
+ disp(newline)
+ disp([' ', num2str(2)]);
+ disp(newline)
+#+end_src
+
+* Test reuse of MATLAB buffer with results output
+:PROPERTIES:
+:ID: a68db9ff-efdf-42b9-85fc-174015cd1f77
+:END:
+
+#+begin_src matlab :results output
+ a = 123
+#+end_src
+
+#+RESULTS:
+: a =
+:
+: 123
+
+#+begin_src matlab :exports both :results output
+ b = a + 1000
+#+end_src
+
+#+RESULTS:
+: b =
+:
+: 1123
+
+Following should give: Unrecognized function or variable 'b'.
+
+#+begin_src matlab :results output
+ clear
+ c = b * 2
+#+end_src
+
+#+RESULTS:
+: Unrecognized function or variable 'b'.
+
+* Test results verbatim
+:PROPERTIES:
+:ID: 278047b6-4b87-4852-9050-e3e99fcaabb8
+:END:
+
+#+begin_src matlab :results verbatim
+ a = 2 + 3;
+ ans = magic(a);
+#+end_src
+
+* Test results output latex
+:PROPERTIES:
+:ID: 7a8190be-d674-4944-864e-6fdaa7362585
+:END:
+
+#+begin_src matlab :results output latex
+ m = [4*pi, 3*pi; 2*pi, pi];
+ result = latex(sym(m));
+ disp(result)
+#+end_src
+
+* Test results file graphics
+:PROPERTIES:
+:ID: 5bee8841-a898-4135-b44b-f1bd5465ceed
+:END:
+
+#+begin_src matlab :results file graphics :file NAME.png
+ t = [0 : 0.1 : 2*pi];
+ y = sin(t);
+ plot(t, y);
+ set(gcf, 'PaperUnits', 'inches', 'PaperPosition', [0 0 4 3]) % Set the size to 4" x 3"
+#+end_src
+
+
+# LocalWords: ebcaddb efdf fc fcaabb fdaa ceed sinewave
diff --git a/testing/lisp/test-ob-matlab.el b/testing/lisp/test-ob-matlab.el
new file mode 100644
index 000000000..476e5b1e4
--- /dev/null
+++ b/testing/lisp/test-ob-matlab.el
@@ -0,0 +1,191 @@
+;;; test-ob-matlab.el --- tests for ob-matlab.el -*- lexical-binding: t; -*-
+
+;;; Commentary:
+
+;; Copyright 2024 Free Software Foundation
+;; Authors: John Ciolfi
+
+;; This file is not part of GNU Emacs.
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+;; ----------
+;; Code blocks reside in: ../examples/ob-matlab-test.org
+;; To run test-ob-matlab.el using latest matlab-mode and org-mode:
+;; 1. Download and build latest org-mode, https://orgmode.org/worg/org-contribute.html
+;; Example:
+;; cd ~/github
+;; git clone https://git.sv.gnu.org/git/emacs/org-mode.git
+;; make -C org-mode compile -j32 && make prefix=~/github/org-mode-install install
+;; 2. Download and build latest matlab-mode, https://github.com/MathWorks/Emacs-MATLAB-Mode
+;; Example:
+;; cd ~/github
+;; git clone https://github.com/mathworks/Emacs-MATLAB-Mode.git
+;; make -C Emacs-MATLAB-Mode -j32
+;; 3. Test uses matlab-shell which requires that "matlab" is on the PATH:
+;; Example:
+;; cd ~/github/org-mode
+;; env PATH=/path/to/MATLAB-INSTALL/bin:$PATH \
+;; make -j32 test \
+;; BTEST_POST="-L ~/github/Emacs-MATLAB-Mode -l ~/Emacs-MATLAB-Mode/matlab-autoload.el"
+
+;;; Code:
+
+(require 'ob-core)
+
+(org-test-for-executable "matlab")
+
+(unless (fboundp 'matlab-shell)
+ (signal 'missing-test-dependency '("Support for MATLAB code blocks")))
+
+;;-------------------------------;;
+;; Basic ":results output" tests ;;
+;;-------------------------------;;
+(ert-deftest ob-matlab/results-output ()
+ "Test matlab :results output code block."
+ (let ((expected "\
+The results are:
+
+a =
+
+ 1 2
+ 3 4
+
+b =
+
+ 2 4
+ 6 8
+"))
+ (org-test-at-id "99332277-5e0e-4834-a3ad-ebcaddb855ab"
+ (org-babel-next-src-block)
+ (should (equal expected (org-babel-execute-src-block))))))
+
+(ert-deftest ob-matlab/results-output-preserve-whitespace ()
+ "Test matlab :results output code block preserves whitespace."
+ (let ((expected "\
+ +---+
+ |one|
+ +---+
+
+ 1
+
+ +---+
+ |two|
+ +---+
+
+ 2
+"))
+ (org-test-at-id "99332277-5e0e-4834-a3ad-ebcaddb855ab"
+ (org-babel-next-src-block 2)
+ (should (equal expected (org-babel-execute-src-block))))))
+
+;;------------------------------------------------;;
+;; MATLAB workspace reuse tests (:results output) ;;
+;;------------------------------------------------;;
+
+(ert-deftest ob-matlab/results-output-reuse-a ()
+ "Test matlab :results output defining variable, a."
+ (let ((expected "\
+a =
+
+ 123
+"))
+ (org-test-at-id "a68db9ff-efdf-42b9-85fc-174015cd1f77"
+ (org-babel-next-src-block)
+ (should (equal expected (org-babel-execute-src-block))))))
+
+(ert-deftest ob-matlab/results-output-reuse-b ()
+ "Test matlab :results output reusing variable, a, to compute variable, b."
+ (let ((expected "\
+b =
+
+ 1123
+"))
+ (org-test-at-id "a68db9ff-efdf-42b9-85fc-174015cd1f77"
+ (org-babel-next-src-block 2)
+ (should (equal expected (org-babel-execute-src-block))))))
+
+(ert-deftest ob-matlab/results-output-reuse-clear ()
+ "Test matlab :results output with clear resulting in error.
+Also validates that we strip the </?ERRORTXT> matlab-shell indicators."
+ (let ((expected "\
+Unrecognized function or variable 'b'.
+"))
+ (org-test-at-id "a68db9ff-efdf-42b9-85fc-174015cd1f77"
+ (org-babel-next-src-block 3)
+ (should (equal expected (org-babel-execute-src-block))))))
+
+;;---------------------------;;
+;; ":results verbatim" tests ;;
+;;---------------------------;;
+(ert-deftest ob-matlab/results-verbatim ()
+ "Test matlab :results verbatim."
+ (let ((expected '((17 24 1 8 15)
+ (23 5 7 14 16)
+ (4 6 13 20 22)
+ (10 12 19 21 3)
+ (11 18 25 2 9))))
+ (org-test-at-id "278047b6-4b87-4852-9050-e3e99fcaabb8"
+ (org-babel-next-src-block)
+ (should (equal expected (org-babel-execute-src-block))))))
+
+;;-------------------------------;;
+;; ":results output latex" tests ;;
+;;-------------------------------;;
+
+(ert-deftest ob-matlab/results-output-latex ()
+ "Test matlab :results output latex."
+ (let ((expected "\
+\\left(\\begin{array}{cc} 4\\,\\pi & 3\\,\\pi \\\\ 2\\,\\pi & \\pi \\end{array}\\right)
+"))
+ (org-test-at-id "7a8190be-d674-4944-864e-6fdaa7362585"
+ (org-babel-next-src-block)
+ (let ((got (org-babel-execute-src-block)))
+ (should (equal expected got))))))
+
+
+;;--------------------------------;;
+;; ":results file graphics" tests ;;
+;;--------------------------------;;
+
+(ert-deftest ob-matlab/results-file-graphics ()
+ "Test matlab :results file graphics."
+ (let ((code-block (org-test-get-code-block "5bee8841-a898-4135-b44b-f1bd5465ceed"))
+ (temp-file-png (make-temp-file "test-ob-matlab-" nil ".png")))
+ (setq code-block (replace-regexp-in-string "NAME\\.png" temp-file-png code-block))
+ (unwind-protect
+ (org-test-with-temp-text
+ code-block
+ (org-babel-execute-src-block)
+ (should (search-forward (format "[[file:%s]]" temp-file-png) nil nil))
+ (should (file-readable-p temp-file-png)))
+ (delete-file temp-file-png))))
+
+(ert-deftest ob-matlab/results-file-graphics-with-space ()
+ "Test matlab :results file graphics using a file name with a space."
+ (let ((code-block (org-test-get-code-block "5bee8841-a898-4135-b44b-f1bd5465ceed"))
+ (temp-file-png (make-temp-file "test ob-matlab-" nil ".png")))
+ (setq code-block (replace-regexp-in-string "NAME\\.png" temp-file-png code-block))
+ (unwind-protect
+ (org-test-with-temp-text
+ code-block
+ (org-babel-execute-src-block)
+ (should (search-forward (format "[[file:%s]]" temp-file-png) nil nil))
+ (should (file-readable-p temp-file-png)))
+ (delete-file temp-file-png))))
+
+(provide 'test-ob-matlab)
+;;; test-ob-matlab.el ends here (emacs-lisp-checkdoc)
+
+;; LocalWords: fboundp ebcaddb efdf fc ERRORTXT fcaabb fdaa ceed setq env BTEST
diff --git a/testing/org-test.el b/testing/org-test.el
index 52d38c3fd..93dfa46fc 100644
--- a/testing/org-test.el
+++ b/testing/org-test.el
@@ -133,6 +133,27 @@ currently executed.")
(unless (or visited-p (not to-be-removed))
(kill-buffer to-be-removed)))))
+(defun org-test-get-code-block (id &optional count)
+ "Get the COUNT, defaulting to first, code block after ID."
+ (let* ((id-location (org-id-find id))
+ (id-file (car id-location))
+ (visited-p (get-file-buffer id-file))
+ code-block
+ to-be-removed)
+ (unwind-protect
+ (save-window-excursion
+ (save-match-data
+ (org-id-goto id)
+ (org-babel-next-src-block count)
+ (beginning-of-line)
+ (setq code-block (buffer-substring-no-properties
+ (point)
+ (re-search-forward "^[ \t]*#\\+end_src[ \t\r]*\n")))
+ (setq to-be-removed (current-buffer))))
+ (unless (or visited-p (not to-be-removed))
+ (kill-buffer to-be-removed)))
+ code-block))
+
(defmacro org-test-in-example-file (file &rest body)
"Execute body in the Org example file."
(declare (indent 1) (debug t))
next prev parent reply other threads:[~2025-01-03 0:33 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-08 19:30 ob-octave: improve MATLAB support John C
2024-11-09 9:32 ` Ihor Radchenko
2024-11-13 19:10 ` John C
2024-11-15 13:29 ` John C
2024-11-23 15:57 ` Ihor Radchenko
[not found] ` <87o712lv3t.fsf@localhost>
2024-12-29 3:04 ` John C
2024-12-29 7:42 ` Ihor Radchenko
2025-01-02 23:55 ` John C
2025-01-02 23:56 ` John C [this message]
2025-01-04 13:02 ` Ihor Radchenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CACb3vdSc2ypq5hdd3Kgb3K+wek-0LDwUqUPugzUr2tt5yyg23Q@mail.gmail.com \
--to=john.ciolfi.32@gmail.com \
--cc=emacs-orgmode@gnu.org \
--cc=yantar92@posteo.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).