ELPA submission: drepl (REPL protocol)

ELPA submission: drepl (REPL protocol)

[Augusto Stoffel]

Hi again,

I would like to submit the following package to ELPA:

  https://github.com/astoff/drepl

It provides alternative shells for the Python and Lua languages.  More
importantly, however, it defines a framework and a protocol to create
new REPLs which hopefully helps doing some things which are tricky in
Comint (especially completion and multi-line editing).

This is kind of an experiment at this point, but I would like to make it
public and see what people think.

Re: ELPA submission: drepl (REPL protocol)

[Stefan Kangas]

Augusto Stoffel <arstoffel@gmail.com> writes:

> I would like to submit the following package to ELPA:
>
>   https://github.com/astoff/drepl

To clarify, do you mean GNU ELPA here?

Re: ELPA submission: drepl (REPL protocol)

[Augusto Stoffel]

On Mon, 30 Oct 2023 at 15:33, Stefan Kangas wrote:

> Augusto Stoffel <arstoffel@gmail.com> writes:
>
>> I would like to submit the following package to ELPA:
>>
>>   https://github.com/astoff/drepl
>
> To clarify, do you mean GNU ELPA here?

Also yes.

By the way, this package has several files, including non-Elisp ones.  I
don't know what ELPA picks by default, but the glob pattern drepl* would
be what I need.

Re: ELPA submission: drepl (REPL protocol)

[Stefan Kangas]

Augusto Stoffel <arstoffel@gmail.com> writes:

> By the way, this package has several files, including non-Elisp ones.  I
> don't know what ELPA picks by default, but the glob pattern drepl* would
> be what I need.

Please add any files that shouldn't be installed to a new file .elpaignore
and they will be ignored.  All the rest will be installed.

Re: ELPA submission: drepl (REPL protocol)

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 762 bytes --]

Augusto Stoffel <arstoffel@gmail.com> writes:

> Hi again,
>
> I would like to submit the following package to ELPA:
>
>   https://github.com/astoff/drepl
>
> It provides alternative shells for the Python and Lua languages.  More
> importantly, however, it defines a framework and a protocol to create
> new REPLs which hopefully helps doing some things which are tricky in
> Comint (especially completion and multi-line editing).
>
> This is kind of an experiment at this point, but I would like to make it
> public and see what people think.

The idea seems interesting, I'll have to try it out at some point
(though I don't really use Python or Lua much, so I hope you plan to add
more languages in the future).

Code-wise, I just have a few minor comments:


[-- Attachment #2: Type: text/plain, Size: 1738 bytes --]

diff --git a/drepl.el b/drepl.el
index 1c6bd71..1a11b47 100644
--- a/drepl.el
+++ b/drepl.el
@@ -4,6 +4,8 @@
 
 ;; Author: Augusto Stoffel <arstoffel@gmail.com>
 ;; Keywords: languages, processes
+;; Version: 1.0.0
+;; Package-Requires: ((emacs "29.1"))
 
 ;; This program is free software; you can redistribute it and/or modify
 ;; it under the terms of the GNU General Public License as published by
@@ -49,7 +51,7 @@
 (defface drepl-prompt-incomplete '((t :inherit (comint-highlight-prompt default)))
   "Face for continuation prompts when input is incomplete but valid.")
 
-(defface drepl-prompt-invalid '((t :inherit (error default)))
+(defface drepl-prompt-invalid '((t :inherit (error default))) ;why both?
   "Face for continuation prompts when input is invalid.")
 
 (defcustom drepl-directory #'drepl--project-directory
@@ -68,6 +70,8 @@ buffers, this is the dREPL buffer or nil.")
 
 ;;; Basic definitions
 
+;; Are classes needed here or could you use `cl-defstruct' with
+;; `:include'?
 (defclass drepl-base ()
   ((buffer :initarg :buffer :reader drepl--buffer)
    (status :initform nil :accessor drepl--status)
@@ -96,6 +100,8 @@ The message is formed by calling `format' with FMTSTRING and ARGS."
 
 ;;; Communication protocol
 
+;; 1. Mention /what/ is not available.
+;; 2. why not fall back to json.el?
 (defalias 'drepl--json-decode
   (if (json-available-p)
       (lambda (s)
@@ -413,7 +419,7 @@ if needed."
 (cl-defgeneric drepl--command (repl)
   "The command to start the REPL interpreter as a list of strings."
   (ignore repl)
-  (error "This needs an implementation"))
+  (error "This needs an implementation")) ;Mention what "this" is
 
 (cl-defgeneric drepl--init (repl)
   "Initialization code for REPL.

[-- Attachment #3: Type: text/plain, Size: 23 bytes --]


-- 
Philip Kaludercic

Re: ELPA submission: drepl (REPL protocol)

[Augusto Stoffel]

On Tue, 31 Oct 2023 at 09:08, Philip Kaludercic wrote:

> The idea seems interesting, I'll have to try it out at some point
> (though I don't really use Python or Lua much, so I hope you plan to add
> more languages in the future).

Sure, let me know if you have any ideas.  A good candidate, as I
mention in the readme, would be a language that has a good embeddable
REPL library that you hack into (as opposed to a program where the
REP-loop is more or less hardcoded).

> Code-wise, I just have a few minor comments:

> -(defface drepl-prompt-invalid '((t :inherit (error default)))
> +(defface drepl-prompt-invalid '((t :inherit (error default))) ;why both?
>    "Face for continuation prompts when input is invalid.")

This is really needed to avoid comint-highlight-input from interfering
in continuation prompts.

> +;; Are classes needed here or could you use `cl-defstruct' with
> +;; `:include'?
>  (defclass drepl-base ()
>    ((buffer :initarg :buffer :reader drepl--buffer)
>     (status :initform nil :accessor drepl--status)

I would use a more lightweight alternative to EIEIO if available, but
cl-defstructs don't form a hierarchy for method dispatch purposes.  (Not
that I'm using this feature _right now_, but morally it's the right
thing.)

>  (cl-defgeneric drepl--command (repl)
>    "The command to start the REPL interpreter as a list of strings."
>    (ignore repl)
> -  (error "This needs an implementation"))
> +  (error "This needs an implementation")) ;Mention what "this" is

_This_ is the method.  This is one of the two things that cannot have a
default implementation and should be implemented by every subclass :-).
Is the message confusing?

Re: ELPA submission: drepl (REPL protocol)

[Philip Kaludercic]

Augusto Stoffel <arstoffel@gmail.com> writes:

> On Tue, 31 Oct 2023 at 09:08, Philip Kaludercic wrote:
>
>> The idea seems interesting, I'll have to try it out at some point
>> (though I don't really use Python or Lua much, so I hope you plan to add
>> more languages in the future).

> Sure, let me know if you have any ideas.  A good candidate, as I
> mention in the readme, would be a language that has a good embeddable
> REPL library that you hack into (as opposed to a program where the
> REP-loop is more or less hardcoded).

What does a REPL library need, that most languages couldn't implement
themselves if they have an eval function?

>>  (cl-defgeneric drepl--command (repl)
>>    "The command to start the REPL interpreter as a list of strings."
>>    (ignore repl)
>> -  (error "This needs an implementation"))
>> +  (error "This needs an implementation")) ;Mention what "this" is
>
> _This_ is the method.  This is one of the two things that cannot have a
> default implementation and should be implemented by every subclass :-).
> Is the message confusing?

I am just imagining an error message appearing in the mini buffer saying
"This needs an implementation", without an indication where the error is
coming from.

Re: ELPA submission: drepl (REPL protocol)

[Augusto Stoffel]

On Wed,  1 Nov 2023 at 18:38, Philip Kaludercic wrote:

> Augusto Stoffel <arstoffel@gmail.com> writes:
>
>> On Tue, 31 Oct 2023 at 09:08, Philip Kaludercic wrote:
>>
>>> The idea seems interesting, I'll have to try it out at some point
>>> (though I don't really use Python or Lua much, so I hope you plan to add
>>> more languages in the future).
>
>> Sure, let me know if you have any ideas.  A good candidate, as I
>> mention in the readme, would be a language that has a good embeddable
>> REPL library that you hack into (as opposed to a program where the
>> REP-loop is more or less hardcoded).
>
> What does a REPL library need, that most languages couldn't implement
> themselves if they have an eval function?

Yes, this is all you need to define a Comint.  But if you want
e.g. completions, then a _function_ that computes the completions of
given a prefix is helpful.

Contrast this with bash: it has a completion system, but, AFAIK, that
can be accessed only through readline.  It's not impossible, in Comint,
to send a TAB character, block until you detect a prompt, and try to
parse the text printed in the meanwhile (purportedly the completion
candidates).  But experience seems to show this is very hard to get
right.

The idea here is to avoid this kind of chicanery (and pursue a different
kind of chicanery).  If I succeeded in my design, then any embeddable
REPL library, including niceties such as a function that completes a
list of completions from a given piece of text, should be fairly easy to
plug into.

>>>  (cl-defgeneric drepl--command (repl)
>>>    "The command to start the REPL interpreter as a list of strings."
>>>    (ignore repl)
>>> -  (error "This needs an implementation"))
>>> +  (error "This needs an implementation")) ;Mention what "this" is
>>
>> _This_ is the method.  This is one of the two things that cannot have a
>> default implementation and should be implemented by every subclass :-).
>> Is the message confusing?
>
> I am just imagining an error message appearing in the mini buffer saying
> "This needs an implementation", without an indication where the error is
> coming from.

Yep, it's a message for the REPL implementor, not the user... I'll
rephrase a bit.

Re: ELPA submission: drepl (REPL protocol)

[João Távora]

On Wed, Nov 1, 2023 at 6:38 PM Philip Kaludercic <philipk@posteo.net> wrote:
>
> Augusto Stoffel <arstoffel@gmail.com> writes:
>
> > On Tue, 31 Oct 2023 at 09:08, Philip Kaludercic wrote:
> >
> >> The idea seems interesting, I'll have to try it out at some point
> >> (though I don't really use Python or Lua much, so I hope you plan to add
> >> more languages in the future).
>
> > Sure, let me know if you have any ideas.  A good candidate, as I
> > mention in the readme, would be a language that has a good embeddable
> > REPL library that you hack into (as opposed to a program where the
> > REP-loop is more or less hardcoded).
>
> What does a REPL library need, that most languages couldn't implement
> themselves if they have an eval function?
>
> >>  (cl-defgeneric drepl--command (repl)
> >>    "The command to start the REPL interpreter as a list of strings."
> >>    (ignore repl)
> >> -  (error "This needs an implementation"))
> >> +  (error "This needs an implementation")) ;Mention what "this" is
> >
> > _This_ is the method.  This is one of the two things that cannot have a
> > default implementation and should be implemented by every subclass :-).
> > Is the message confusing?
>
> I am just imagining an error message appearing in the mini buffer saying
> "This needs an implementation", without an indication where the error is
> coming from.

Two notes here:

* If this generic is something that anyone else besides drepl.el
  is meant to add methods to, then it shouldn't be a '--' symbol.

* Normally, instead of "error", CL programmers just leave out such
  a default implementation.  The standard "No applicable method" error
  already signals that some generic function was invoked with arguments
  for which  there wasn't an applicable method, and the error message
  contains   information about those arguments.

João

Re: ELPA submission: drepl (REPL protocol)

[Augusto Stoffel]

On Thu,  2 Nov 2023 at 14:26, João Távora wrote:

> Two notes here:
>
> * If this generic is something that anyone else besides drepl.el
>   is meant to add methods to, then it shouldn't be a '--' symbol.

Well, those symbols are somewhere in between "public" and "private".  As
a user you shouldn't fiddle with them (unless you know what you're
doing).  If anyone wanted to implement a subclass elsewhere, they
certainly would have to add methods to some '--' symbols.  But then the
burden of keeping up with changes in drepl.el is on them and not me.

I suppose that's what people understand by double-dashed symbols...?

> * Normally, instead of "error", CL programmers just leave out such
>   a default implementation.  The standard "No applicable method" error
>   already signals that some generic function was invoked with arguments
>   for which  there wasn't an applicable method, and the error message
>   contains   information about those arguments.

Ah, sure, that's what I should do.

Re: ELPA submission: drepl (REPL protocol)

[João Távora]

On Thu, Nov 2, 2023 at 4:08 PM Augusto Stoffel <arstoffel@gmail.com> wrote:
>
> On Thu,  2 Nov 2023 at 14:26, João Távora wrote:
>
> > Two notes here:
> >
> > * If this generic is something that anyone else besides drepl.el
> >   is meant to add methods to, then it shouldn't be a '--' symbol.
>
> Well, those symbols are somewhere in between "public" and "private".  As
> a user you shouldn't fiddle with them (unless you know what you're
> doing).  If anyone wanted to implement a subclass elsewhere, they
> certainly would have to add methods to some '--' symbols.  But then the
> burden of keeping up with changes in drepl.el is on them and not me.
>
> I suppose that's what people understand by double-dashed symbols...?

Often, there are two types of interfaces to a library: the user interface
and the programmer's interface, aka the API.  The symbols for both of
them should be external meaning changes to their semantics could result,
respectively, in breaking a user's setup or breaking the program.using
the API.

Also it's fine to offer both in the same Elisp package, of course.

Anyway, that's what '--' means.  It means: this may change without
warning.  If you publish an API based on '--' symbols, then it's not
much of an API IMHO, at least I wouldn't want to write a program
against it.

João

Re: ELPA submission: drepl (REPL protocol)

[Augusto Stoffel]

On Tue, 31 Oct 2023 at 09:08, Philip Kaludercic wrote:

> The idea seems interesting, I'll have to try it out at some point
> (though I don't really use Python or Lua much, so I hope you plan to add
> more languages in the future).

Sure, let me know if you have any ideas.  A good candidate, as I
mention in the readme, would be a language that has a good embeddable
REPL library that you hack into (as opposed to a program where the
REP-loop is more or less hardcoded).

> Code-wise, I just have a few minor comments:

> -(defface drepl-prompt-invalid '((t :inherit (error default)))
> +(defface drepl-prompt-invalid '((t :inherit (error default))) ;why both?
>    "Face for continuation prompts when input is invalid.")

This is really needed to avoid comint-highlight-input from interfering
in continuation prompts.

> +;; Are classes needed here or could you use `cl-defstruct' with
> +;; `:include'?
>  (defclass drepl-base ()
>    ((buffer :initarg :buffer :reader drepl--buffer)
>     (status :initform nil :accessor drepl--status)

I would use a more lightweight alternative to EIEIO if available, but
cl-defstructs don't form a hierarchy for method dispatch purposes.  (Not
that I'm using this feature _right now_, but morally it's the right
thing.)

>  (cl-defgeneric drepl--command (repl)
>    "The command to start the REPL interpreter as a list of strings."
>    (ignore repl)
> -  (error "This needs an implementation"))
> +  (error "This needs an implementation")) ;Mention what "this" is

_This_ is the method.  This is one of the two things that cannot have a
default implementation and should be implemented by every subclass :-).
Is the message confusing?

Re: ELPA submission: drepl (REPL protocol)

[Philip Kaludercic]

Philip Kaludercic <philipk@posteo.net> writes:

> Augusto Stoffel <arstoffel@gmail.com> writes:
>
>> Hi again,
>>
>> I would like to submit the following package to ELPA:
>>
>>   https://github.com/astoff/drepl
>>
>> It provides alternative shells for the Python and Lua languages.  More
>> importantly, however, it defines a framework and a protocol to create
>> new REPLs which hopefully helps doing some things which are tricky in
>> Comint (especially completion and multi-line editing).
>>
>> This is kind of an experiment at this point, but I would like to make it
>> public and see what people think.
>
> The idea seems interesting, I'll have to try it out at some point
> (though I don't really use Python or Lua much, so I hope you plan to add
> more languages in the future).
>
> Code-wise, I just have a few minor comments:
>
> diff --git a/drepl.el b/drepl.el
> index 1c6bd71..1a11b47 100644
> --- a/drepl.el
> +++ b/drepl.el
> @@ -4,6 +4,8 @@
>  
>  ;; Author: Augusto Stoffel <arstoffel@gmail.com>
>  ;; Keywords: languages, processes
> +;; Version: 1.0.0
> +;; Package-Requires: ((emacs "29.1"))

I have added the package, but it won't appear until you apply these
changes and tag a new release.

Re: ELPA submission: drepl (REPL protocol)

[Augusto Stoffel]

On Tue,  7 Nov 2023 at 08:20, Philip Kaludercic wrote:

> I have added the package, but it won't appear until you apply these
> changes and tag a new release.

Thanks!  I will do that soonish.