Need help interoperating between comint and CLI app's multi-line mode

Need help interoperating between comint and CLI app's multi-line mode

[pareto optimal via Users list for the GNU Emacs text editor]

Hello all, been stuck on this one for quite some hours :)

I'm trying to make an inferior comint based mode for memgpt.

In a terminal outside of emacs, when I use `memgpt run`, it has a special multi-line mode you can enter with `//`.

It looks like:

```
> Enter your message: //
> Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
>
  - say hi
  - count to 10
  - say sayonara (I press Alt+enter)
💭 User wants to say hi, count to 10, and say sayonara. Handling the commands one by one.
🤖 Hello! How are you doing today? I'm all ears. Oh, and one, two, three...
```

The problem is in emacs if you just do something like this to run it with comint:

```
(progn
  (make-comint "memgpt" "memgpt" nil "run" "--strip-ui")
  (pop-to-buffer "*memgpt*"))
```

Then you go into multi-line mode, you'll get an error about `M-RET` not being bound. That's surmountable since I can bind `M-RET` to something.

```
(process-send-string "*memgpt*" "\e\r") ;; send escape codes for ESC enter to underlying process
```

The problem is that process-send-string sends at the beginning of the line.

I can prove that. Given comint buffer below and `|` denoting the cursor:

```
> Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
  - say hi
  - count to 10
  - say sayonara|
```

Send:

```
(process-send-string "*memgpt*" "****")
```

and you get:

```
> Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
****  - say hi
  - count to 10
  - say sayonara
```

So when I send `(process-send-string "*memgpt*" "\e\r")` I predictably get this result:

```
Empty input received. Try again!
> Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
  - say hi
  - count to 10
  - say sayonara
```

Well a little confusing since it puts the response above your input string.

Then I went to the docs and found `comint-accumulate`. This is a not too horrible workaround and does work, but the output is a little noisy and strange if you don't know exactly whats happening:

```
> Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
  - say hi
  - count to 10
  - say sayonara

>
    - say hi
      - count to 10
        - say sayonara
        
        
        > Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
>
    - say hi
      - count to 10
        - say sayonara
        
        
        
💭 INSERT silly wrong LLM output
```

Any guidance or advice would really help so I can have a comint memgpt that isn't much more terrible to use than the normal version.

Thanks!

Re: Need help interoperating between comint and CLI app's multi-line mode

[pareto optimal via Users list for the GNU Emacs text editor]

I feel like this is definitely not the best way, but it works and may help others understand my problem:

```
(defvar multi-line-mode nil "Flag to track if we're in multi-line mode.")
(defvar multi-line-input-list nil "List to store multi-line inputs.")

(defun send-multi-line-command ()
  (interactive)
  (if (not multi-line-mode)
      (progn
        (setq multi-line-input-list '())
        (setq multi-line-mode t)
        (message "Multi-line mode: ON"))
    (let ((input (string-join multi-line-input-list "\n")))
      (process-send-string "*memgpt*" (concat input "\n"))
      ;; Here we send the equivalent of 'Alt+Enter'
      (process-send-string "*memgpt*" "\e\r")
      (setq multi-line-mode nil)
      (setq multi-line-input-list nil)
      (message "Multi-line mode: OFF"))))

(defun capture-multi-line-input ()
  (interactive)
  (when multi-line-mode
    (end-of-buffer)
    (let ((input (read-string "Multi-line input: ")))
      (add-to-list 'multi-line-input-list input t))))

(global-set-key (kbd "C-c C-m") 'capture-multi-line-input)
(global-set-key (kbd "M-RET") 'send-multi-line-command)
```

On Dec 10, 2023 at 5:21 AM, pareto optimal <pareto.optimal@mailfence.com> wrote:
Hello all, been stuck on this one for quite some hours :)

I'm trying to make an inferior comint based mode for memgpt.

In a terminal outside of emacs, when I use `memgpt run`, it has a special multi-line mode you can enter with `//`.

It looks like:

```
> Enter your message: //
> Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
>
  - say hi
  - count to 10
  - say sayonara (I press Alt+enter)
💭 User wants to say hi, count to 10, and say sayonara. Handling the commands one by one.
🤖 Hello! How are you doing today? I'm all ears. Oh, and one, two, three...
```

The problem is in emacs if you just do something like this to run it with comint:

```
(progn
  (make-comint "memgpt" "memgpt" nil "run" "--strip-ui")
  (pop-to-buffer "*memgpt*"))
```

Then you go into multi-line mode, you'll get an error about `M-RET` not being bound. That's surmountable since I can bind `M-RET` to something.

```
(process-send-string "*memgpt*" "\e\r") ;; send escape codes for ESC enter to underlying process
```

The problem is that process-send-string sends at the beginning of the line.

I can prove that. Given comint buffer below and `|` denoting the cursor:

```
> Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
  - say hi
  - count to 10
  - say sayonara|
```

Send:

```
(process-send-string "*memgpt*" "****")
```

and you get:

```
> Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
****  - say hi
  - count to 10
  - say sayonara
```

So when I send `(process-send-string "*memgpt*" "\e\r")` I predictably get this result:

```
Empty input received. Try again!
> Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
  - say hi
  - count to 10
  - say sayonara
```

Well a little confusing since it puts the response above your input string.

Then I went to the docs and found `comint-accumulate`. This is a not too horrible workaround and does work, but the output is a little noisy and strange if you don't know exactly whats happening:

```
> Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
  - say hi
  - count to 10
  - say sayonara

>
    - say hi
      - count to 10
        - say sayonara
        
        
        > Enter your message:  (Finish with 'Alt+Enter' or 'Esc then Enter')
>
    - say hi
      - count to 10
        - say sayonara
        
        
        
💭 INSERT silly wrong LLM output
```

Any guidance or advice would really help so I can have a comint memgpt that isn't much more terrible to use than the normal version.

Thanks!