SageMath and Emacs

SageMath and Emacs

[Rahguzar]

Dear Emacs developers,

I want to ask opinions about including support for SageMath in Emacs or
GNU ELPA.

SageMath is a niche software but I think the niche is a very important
one. It is also the best computer algebra system which is also free
software and hence important to advance free software in a field
dominated proprietary software. (This dominance is especially sad since
proprietary software is a very shaky foundation for scientific
research.)

The SageMath project project provides Emacs modes both for source files
and repl. They code can be found at
https://github.com/sagemath/sage-shell-mode

It is a large code base but looking at it a year or so ago, I realized
that most of the code was implementing functionality that is by now
included in the python.el in Emacs. SageMath is a thin wrapper around Python
and as a result basic support for it can be provided by setting some
variables provided by python.el appropriately.

I wrote such a wrapper around python.el which is hosted at
https://codeberg.org/rahguzar/sage-mode

There is more code there than needed because I wanted to change some
behaviors of python.el but now I think those deviations should be
removed. As a result I think basic functionality can be provided by
addition of 200-300 lines in python.el (or a separate file requiring
it). If there is an interest in providing out of box support for
SageMath I can prepare such a patch.

I have been meaning to send this email for a while but never came around
to it. I am sending this now because I saw Eli's message about a pretest
for Emacs 30 being imminent. It might be already too late for Emacs 30
but if the pretest is a week or two away there might be time.

If inclusion in Emacs core is not considered a good option, I will like
to contribute support to ELPA. I would like to hear opinions of the
people maintaining the current SageMath support and opened an issue at
https://github.com/sagemath/sage-shell-mode/issues/80 but I haven't
heard from them (it has only been a few days).

Thanks,
Rahguzar

Re: SageMath and Emacs

[Eli Zaretskii]

> From: Rahguzar <rahguzar@zohomail.eu>
> Date: Thu, 09 May 2024 12:35:37 +0200
> 
> If inclusion in Emacs core is not considered a good option, I will like
> to contribute support to ELPA.

Since you say SageMath is niche software, it sounds like ELPA is a
better home for it than the core?

Re: SageMath and Emacs

[Rahguzar]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

> Since you say SageMath is niche software, it sounds like ELPA is a
> better home for it than the core?

It is niche but I think (definitely hope) that its use will become much
more common in academia especially in cases where proprietary
alternatives are now widespread.

ELPA might still be a better home but I think the patch will be small
enough that an additional packages might not be warranted. Hence, my
interest in exploring if it can be added to core.

In line with Philip's suggestion in the other message I can prepare a
patch to help with the decision as to where the support should go.
However if you greatly prefer the ELPA route I will wait a little bit so
that people maintaining the Emacs integration on SageMath project can
respond and then ask for inclusion in GNU ELPA.

[ELPA] Re: SageMath and Emacs

[Philip Kaludercic]

Rahguzar <rahguzar@zohomail.eu> writes:

> Dear Emacs developers,
>
> I want to ask opinions about including support for SageMath in Emacs or
> GNU ELPA.
>
> SageMath is a niche software but I think the niche is a very important
> one. It is also the best computer algebra system which is also free
> software and hence important to advance free software in a field
> dominated proprietary software. (This dominance is especially sad since
> proprietary software is a very shaky foundation for scientific
> research.)
>
> The SageMath project project provides Emacs modes both for source files
> and repl. They code can be found at
> https://github.com/sagemath/sage-shell-mode
>
> It is a large code base but looking at it a year or so ago, I realized
> that most of the code was implementing functionality that is by now
> included in the python.el in Emacs. SageMath is a thin wrapper around Python
> and as a result basic support for it can be provided by setting some
> variables provided by python.el appropriately.
>
> I wrote such a wrapper around python.el which is hosted at
> https://codeberg.org/rahguzar/sage-mode
>
> There is more code there than needed because I wanted to change some
> behaviors of python.el but now I think those deviations should be
> removed. As a result I think basic functionality can be provided by
> addition of 200-300 lines in python.el (or a separate file requiring
> it). If there is an interest in providing out of box support for
> SageMath I can prepare such a patch.
>
> I have been meaning to send this email for a while but never came around
> to it. I am sending this now because I saw Eli's message about a pretest
> for Emacs 30 being imminent. It might be already too late for Emacs 30
> but if the pretest is a week or two away there might be time.
>
> If inclusion in Emacs core is not considered a good option, I will like
> to contribute support to ELPA. I would like to hear opinions of the
> people maintaining the current SageMath support and opened an issue at
> https://github.com/sagemath/sage-shell-mode/issues/80 but I haven't
> heard from them (it has only been a few days).

It certainly can be added to ELPA, but I guess it would be easier to
discuss the matter if you could provide a patch for emacs.git, if that
is not too much work.

> Thanks,
> Rahguzar
>
>

-- 
	Philip Kaludercic on peregrine

Re: [ELPA] Re: SageMath and Emacs

[Philip Kaludercic]

Philip Kaludercic <philipk@posteo.net> writes:

> Rahguzar <rahguzar@zohomail.eu> writes:
>
>> Dear Emacs developers,
>>
>> I want to ask opinions about including support for SageMath in Emacs or
>> GNU ELPA.
>>
>> SageMath is a niche software but I think the niche is a very important
>> one. It is also the best computer algebra system which is also free
>> software and hence important to advance free software in a field
>> dominated proprietary software. (This dominance is especially sad since
>> proprietary software is a very shaky foundation for scientific
>> research.)
>>
>> The SageMath project project provides Emacs modes both for source files
>> and repl. They code can be found at
>> https://github.com/sagemath/sage-shell-mode
>>
>> It is a large code base but looking at it a year or so ago, I realized
>> that most of the code was implementing functionality that is by now
>> included in the python.el in Emacs. SageMath is a thin wrapper around Python
>> and as a result basic support for it can be provided by setting some
>> variables provided by python.el appropriately.
>>
>> I wrote such a wrapper around python.el which is hosted at
>> https://codeberg.org/rahguzar/sage-mode
>>
>> There is more code there than needed because I wanted to change some
>> behaviors of python.el but now I think those deviations should be
>> removed. As a result I think basic functionality can be provided by
>> addition of 200-300 lines in python.el (or a separate file requiring
>> it). If there is an interest in providing out of box support for
>> SageMath I can prepare such a patch.
>>
>> I have been meaning to send this email for a while but never came around
>> to it. I am sending this now because I saw Eli's message about a pretest
>> for Emacs 30 being imminent. It might be already too late for Emacs 30
>> but if the pretest is a week or two away there might be time.
>>
>> If inclusion in Emacs core is not considered a good option, I will like
>> to contribute support to ELPA. I would like to hear opinions of the
>> people maintaining the current SageMath support and opened an issue at
>> https://github.com/sagemath/sage-shell-mode/issues/80 but I haven't
>> heard from them (it has only been a few days).
>
> It certainly can be added to ELPA, but I guess it would be easier to
> discuss the matter if you could provide a patch for emacs.git, if that
> is not too much work.

Ping?

>> Thanks,
>> Rahguzar
>>
>>

-- 
	Philip Kaludercic on peregrine

Re: [ELPA] Re: SageMath and Emacs

[Rahguzar]

[-- Attachment #1: Type: text/plain, Size: 541 bytes --]

Hi Philip,

Philip Kaludercic <philipk@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
> Ping?
>

Sorry, I got busy with some stuff. I have attached two patches with this
email. The first one provides basic support for SageMath source files
and repls. The second one adds a command to look up documentation of
sage objects. I don't think it uses anything sage specific so it might
make sense to have some such facility for python generally. I tried but
couldn't find something in python.el for it.

Thanks,
Rahguzar


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Support-for-SageMath.patch --]
[-- Type: text/x-patch, Size: 7303 bytes --]

From 8ff7f1d19efc6362b8d090236e7174bd17aedfa1 Mon Sep 17 00:00:00 2001
From: Rahguzar <rahguzar@zohomail.eu>
Date: Mon, 20 May 2024 16:07:22 +0200
Subject: [PATCH 1/2] Support for SageMath

* lisp/progmodes/sage.el (python):
(sage): New customization group.
(sage-shell-interpreter, sage-shell-interpreter-args):
New custom options.
(sage-shell-eval-setup-code): New variable.
(sage-shell-mode, sage-mode): New major modes.
(sage-mode-map): Keymap for sage-mode.
(run-sage): New function for starting repl.
---
 lisp/progmodes/sage.el | 162 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 162 insertions(+)
 create mode 100644 lisp/progmodes/sage.el

diff --git a/lisp/progmodes/sage.el b/lisp/progmodes/sage.el
new file mode 100644
index 00000000000..cba3421cbaa
--- /dev/null
+++ b/lisp/progmodes/sage.el
@@ -0,0 +1,162 @@
+;;; sage.el --- SageMath support for Emacs -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2024 Free Software Foundation, Inc.
+
+;; Author: Rahguzar <rahguzar@zohomail.eu>
+;; Maintainer: Rahguzar <rahguzar@zohomail.eu>
+;; Keywords: languages
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; Major modes for editing Sage files and interacting with sage process.  These
+;; are thin wrappers around `python-mode' and `inferior-python-mode'.  To
+;; change the behavior of these modes, the facilities provided by `python.el'
+;; can be used in conjunction with `sage-mode-hook' and `sage-shell-mode-hook'.
+
+;;; Code:
+(require 'python)
+
+;;;; Variables
+(defgroup sage nil
+  "Emacs interface to sagemath."
+  :group 'languages)
+
+(defcustom sage-shell-interpreter (executable-find "sage")
+  "Interpreter for sage code.
+Sage version should be at least 9.3."
+  :type 'string)
+
+(defcustom sage-shell-interpreter-args '("--simple-prompt")
+  "Arguments for the sage interpreter.
+Note: \"--simple-promot\" is required for ipython to work."
+  :type '(repeat string))
+
+(defvar sage-shell-eval-setup-code
+  "\
+import warnings
+
+def __PYTHON_EL_eval(source, filename):
+    import ast, sys
+    from sage.repl.preparse import preparse_file
+    if isinstance(source, str):
+        source = preparse_file(source)
+    else:
+        source = preparse_file(source.decode())
+    if sys.version_info[0] == 2:
+        from __builtin__ import compile, eval, globals
+    else:
+        from builtins import compile, eval, globals
+    try:
+        p, e = ast.parse(source, filename), None
+    except SyntaxError:
+        t, v, tb = sys.exc_info()
+        sys.excepthook(t, v, tb.tb_next)
+        return
+    if p.body and isinstance(p.body[-1], ast.Expr):
+        e = p.body.pop()
+    try:
+        g = globals()
+        exec(compile(p, filename, 'exec'), g, g)
+        if e:
+            return eval(compile(ast.Expression(e.value), filename, 'eval'), g, g)
+    except Exception:
+        t, v, tb = sys.exc_info()
+        sys.excepthook(t, v, tb.tb_next)
+"
+  "Code for sending code to a sage process.
+This is essentially same as `python-shell-eval-setup-code' except the addition
+of a preparse step that converts sage code to python code.")
+
+;;;; Sage shell mode
+(define-derived-mode sage-shell-mode inferior-python-mode
+  "Sage Repl" "Execute Sage commands interactively."
+  (setq-local python-shell-eval-setup-code sage-shell-eval-setup-code))
+
+;;;###autoload
+(defun run-sage (&optional dedicated show)
+  "Run an inferior sage process.
+
+The command to run the process is calculated using `sage-shell-interpreter'
+and `sage-shell-interpreter-args'.
+
+When called interactively with `prefix-arg', it allows
+the user to choose whether the interpreter should be DEDICATED to the current
+buffer or project.  When numeric prefix arg is other than 0 or 4 do not SHOW.
+
+For a given buffer and same values of DEDICATED, if a process is
+already running for it, it will do nothing.  This means that if
+the current buffer is using a global process, the user is still
+able to switch it to use a dedicated one.
+
+Type \\[describe-mode] in the process buffer for a list of commands."
+  (interactive
+   (if current-prefix-arg
+       (list
+        (alist-get (car (read-multiple-choice "Make dedicated process?"
+                                              '((?b "to buffer")
+                                                (?p "to project")
+                                                (?n "no"))))
+                   '((?b . buffer) (?p . project)))
+        (= (prefix-numeric-value current-prefix-arg) 4))
+     (list python-shell-dedicated t)))
+  (let* ((proc (python-shell-get-process-name dedicated))
+         (buf (get-buffer-create (format "*%s*" proc))))
+    (unless (get-buffer-process buf)
+      (with-current-buffer buf
+        (let ((inhibit-read-only t))
+          (erase-buffer))
+        (apply #'make-comint-in-buffer proc buf sage-shell-interpreter nil
+               sage-shell-interpreter-args)
+        ;; Ideally we should able to bind `python-shell-interpreter' and
+        ;; `python-shell-interpreter-args' and call `python-run'. However, that
+        ;; runs into problems since `inferior-python-mode' uses
+        ;; `python-shell--interpreter' and `python-shell--interpreter-args'
+        ;; which can only be let bound from within `python.el'. This makes it
+        ;; impossible to call any derived mode of `inferior-python-mode'
+        ;; without dealing with these internal variables.
+        (dlet ((python-shell--interpreter "sage")
+               (python-shell--interpreter-args "--simple-prompt"))
+          (let ((python-shell-prompt-detect-enabled nil)
+                (python-shell-interpreter-interactive-arg "")
+                (python-shell-prompt-regexp "sage: "))
+            (sage-shell-mode))
+          (setq-local python-shell-prompt-detect-enabled nil
+                      python-shell-interpreter-interactive-arg ""
+                      python-shell-prompt-regexp "sage: "))))
+    (if show (display-buffer buf))
+    proc))
+
+;;;; Sage Mode
+(defvar-keymap sage-mode-map
+  "<remap> <run-python>" #'run-sage)
+
+;;;###autoload
+(define-derived-mode sage-mode python-mode "Sage"
+  :group 'sage
+  (setq-local python-shell-buffer-name "Sage"
+              python-shell-interpreter sage-shell-interpreter
+              python-shell-interpreter-args (combine-and-quote-strings
+                                             sage-shell-interpreter-args)
+              python-shell-eval-setup-code sage-shell-eval-setup-code))
+
+;;;###autoload
+(cl-pushnew `(,(rx ".sage" eos) . sage-mode) auto-mode-alist :test #'equal)
+
+;;;; Provide
+(provide 'sage)
+;;; sage.el ends here
-- 
2.45.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-Function-to-look-up-documentation-for-sage-objects.patch --]
[-- Type: text/x-patch, Size: 4409 bytes --]

From 78b125eb925f5a2428aaecc540680cae64f37b67 Mon Sep 17 00:00:00 2001
From: Rahguzar <rahguzar@zohomail.eu>
Date: Mon, 20 May 2024 17:12:17 +0200
Subject: [PATCH 2/2] Function to look up documentation for sage objects

* lisp/progmodes/sage.el
(sage-lookup-doc): New function.
(sage-shell-mode): Setup for sage-lookup-doc.
(sage-mode-map): Add keybinding for sage-lookup-doc.
(sage--redirect-send): Internal function.
---
 lisp/progmodes/sage.el | 65 ++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 63 insertions(+), 2 deletions(-)

diff --git a/lisp/progmodes/sage.el b/lisp/progmodes/sage.el
index cba3421cbaa..730725924a3 100644
--- a/lisp/progmodes/sage.el
+++ b/lisp/progmodes/sage.el
@@ -82,10 +82,40 @@ sage-shell-eval-setup-code
 This is essentially same as `python-shell-eval-setup-code' except the addition
 of a preparse step that converts sage code to python code.")
 
+(defvar sage-shell-doc-setup-code
+  "\
+from IPython.terminal.prompts import Prompts, Token
+class __SAGEPrompt(type(get_ipython().prompts)):
+     redirect_happening = False
+     initial_prompts = get_ipython().prompts
+     def in_prompt_tokens(self, cli=None):
+         if self.redirect_happening:
+             self.redirect_happening = False
+             return [(Token.Prompt,'***** COMINT REDIRECT ENDED *****')]
+         else:
+             return self.initial_prompts.in_prompt_tokens()
+get_ipython().prompts = __SAGEPrompt(get_ipython())
+
+def __SAGE_get_doc(str):
+    In.pop()
+    get_ipython().prompts.redirect_happening = True
+    get_ipython().run_line_magic('pinfo', str)
+"
+  "Code to setup fetching documentation from a Sage process.
+It is needed because the documentation somtimes contains bare prompts which
+trip up the comint redirection machinery.  This works around that.")
+
 ;;;; Sage shell mode
+(defun sage-shell-setup-doc ()
+  "Send `sage-shell-doc-setup-code' to Sage process."
+  (python-shell-send-string-no-output sage-shell-doc-setup-code))
+
 (define-derived-mode sage-shell-mode inferior-python-mode
   "Sage Repl" "Execute Sage commands interactively."
-  (setq-local python-shell-eval-setup-code sage-shell-eval-setup-code))
+  (setq-local python-shell-eval-setup-code sage-shell-eval-setup-code
+              comint-redirect-completed t)
+  (add-hook 'python-shell-first-prompt-hook #'sage-shell-setup-doc)
+  (add-hook 'comint-redirect-filter-functions #'ansi-color-apply nil t))
 
 ;;;###autoload
 (defun run-sage (&optional dedicated show)
@@ -143,7 +173,8 @@ run-sage
 
 ;;;; Sage Mode
 (defvar-keymap sage-mode-map
-  "<remap> <run-python>" #'run-sage)
+  "<remap> <run-python>" #'run-sage
+  "<remap> <python-eldoc-at-point>" #'sage-lookup-doc)
 
 ;;;###autoload
 (define-derived-mode sage-mode python-mode "Sage"
@@ -157,6 +188,36 @@ sage-mode
 ;;;###autoload
 (cl-pushnew `(,(rx ".sage" eos) . sage-mode) auto-mode-alist :test #'equal)
 
+;;;; Documentation
+(defun sage--redirect-send (command output-buffer process echo no-display)
+  "Version of `comint-redirect-send-command-to-process'.
+Which see for COMMAND, OUTPUT-BUFFER, PROCESS, ECHO and NO-DISPLAY.
+The COMMAND must output ***** COMINT REDIRECT ENDED ***** as the last line."
+  (with-current-buffer (process-buffer process)
+    (if (and (python-util-comint-end-of-output-p)
+             comint-redirect-completed)
+        (let ((comint-redirect-perform-sanity-check nil)
+              (comint-prompt-regexp (rx bol "***** COMINT REDIRECT ENDED *****")))
+          (comint-redirect-send-command-to-process
+           command output-buffer process echo no-display))
+      (error "Sage process is busy"))))
+
+(defun sage-lookup-doc (symbol &optional proc display)
+  "Look up documentation for the SYMBOL.
+It is the symbol at point when called interactively.  PROC is the sage process.
+If DISPLAY is non-nil, buffer is displayed."
+  (interactive (list (python-info-current-symbol) (python-shell-get-process) t))
+  (let ((buf (get-buffer-create "*Sage Documentation*"))
+        (proc (or proc (python-shell-get-process))))
+    (prog1 buf
+      (with-current-buffer buf
+        (erase-buffer)
+        (setq mode-line-format nil)
+        (font-lock-mode)
+        (visual-line-mode))
+      (sage--redirect-send (format "__SAGE_get_doc('%s')" symbol)
+                           buf proc nil (not display)))))
+
 ;;;; Provide
 (provide 'sage)
 ;;; sage.el ends here
-- 
2.45.1

Re: [ELPA] Re: SageMath and Emacs

[Augusto Stoffel]

Hi Rahguzar,

On Mon, 20 May 2024 at 17:47, Rahguzar wrote:

> From 78b125eb925f5a2428aaecc540680cae64f37b67 Mon Sep 17 00:00:00 2001
> From: Rahguzar <rahguzar@zohomail.eu>
> Date: Mon, 20 May 2024 17:12:17 +0200
> Subject: [PATCH 2/2] Function to look up documentation for sage objects

not trying to advertise a package here, but implementing this kind of
thing might be easier and cleaner with dREPL
(https://elpa.gnu.org/packages/drepl.html).  There's already an IPython
version, I guess Sage can be adapter from it.

Re: [ELPA] Re: SageMath and Emacs

[Rahguzar]

Hi Augusto,

Augusto Stoffel <arstoffel@gmail.com> writes:

> Hi Rahguzar,
>
> On Mon, 20 May 2024 at 17:47, Rahguzar wrote:
>
> not trying to advertise a package here, but implementing this kind of
> thing might be easier and cleaner with dREPL
> (https://elpa.gnu.org/packages/drepl.html).  There's already an IPython
> version, I guess Sage can be adapter from it.

dREPL looks very interesting but my first attempt will be to stay as
close as possible to facilities in python.el.

I do agree that comint can be improved, particularly around redirection,
so I will keep dREPL in mind for future.

Thanks,
Rahguzar