How to pass messages between emacs and a Python program? (goal: trying to use emacs as a UI)

How to pass messages between emacs and a Python program? (goal: trying to use emacs as a UI)

[Brian Merchant]

Hi all,

Say I have opened up a file in emacs, and I type in something like:

`x \and \top`

Then, a Python program reads that file, and:

* replaces all `\and` to `∧`
* replaces all `\top` to `T`
* it does some thinking on the expression `x \and \top` and decides that
that is the same as `x`, so will append an `= x`

The final result of all the changes made by the program to the file will
leave it like so:

`x ∧ T = x`

Okay, so much for what I want. For getting there though:

I don't want the Python program to be constantly polling the file for
changes (using a `while` loop), and I probably don't want emacs to be
constantly polling the file for updates (which I know how to do using the
`auto-revert` command).

Maybe I press some key combination, and then that sends a message to a
Python script that its time to read the file and make updates and then the
Python script would message emacs and ask it to update what it is
displaying in its buffer.

Could this be done?

A bit of background on my goal: I'd like to make a proof editor/"IDE", and
while I have the programming know-how (at least in Python and C++, not in
emacs Lisp) to code the background analyzers/proof-engines, I am horrible
at putting pixels on a screen. I have experimented a bit with putting
pixels on a screen, and writing my own simple editor, but suffice it to say
that I have come to appreciate the immense amount of effort it takes to
build something as smooth as emacs. I think it would be much better for me
to take advantage of emacs' capabilities in terms of showing stuff on a
screen, and manipulating/editing that stuff. I worry though that there
might not be any easy way to do this...

Anyway, would love to hear your thoughts and guidance!

Kind regards,
Brian

Re: How to pass messages between emacs and a Python program? (goal: trying to use emacs as a UI)

[Michael Albinus]

Brian Merchant <bhmerchant@gmail.com> writes:

> Hi all,

Hi Brian,

> Say I have opened up a file in emacs, and I type in something like:
>
> `x \and \top`
>
> Then, a Python program reads that file, and:
>
> * replaces all `\and` to `∧`
> * replaces all `\top` to `T`
> * it does some thinking on the expression `x \and \top` and decides that
> that is the same as `x`, so will append an `= x`
>
> The final result of all the changes made by the program to the file will
> leave it like so:
>
> `x ∧ T = x`
>
> Okay, so much for what I want. For getting there though:
>
> I don't want the Python program to be constantly polling the file for
> changes (using a `while` loop), and I probably don't want emacs to be
> constantly polling the file for updates (which I know how to do using the
> `auto-revert` command).
>
> Maybe I press some key combination, and then that sends a message to a
> Python script that its time to read the file and make updates and then the
> Python script would message emacs and ask it to update what it is
> displaying in its buffer.
>
> Could this be done?

I have no idea about python and its features in this respect. From the
Emacs point of view, I see two possibilities:

- Use file notifications. Emacs supports file notifications for several
  operating systems, like GNU/Linux, *BSD, OS X, Cygnus, MS
  Windows. That is, if a file is changed in the file system, an event is
  raised by the kernel, and Emacs is able to catch this event and to
  run a handler (a Lisp function) bound to this event. If python could
  do similar, you have it.

- Use D-Bus messages. This is a communication channel for applications
  of different type to talk to each other. Emacs has D-Bus support, and
  python has it also (a short web search gave me
  <https://dbus.freedesktop.org/doc/dbus-python/doc/tutorial.html>. This
  is restricted to operating systems with D-Bus support, mainly
  GNU/Linux and Cygwin. Both Emacs and your python program could send a
  D-Bus message to the partner, indicating that a file has changed.

> Kind regards,
> Brian

Best regards, Michael.

Re: How to pass messages between emacs and a Python program? (goal: trying to use emacs as a UI)

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 756 bytes --]

() Brian Merchant <bhmerchant@gmail.com>
() Thu, 25 Aug 2016 15:18:19 -0700

   Could this be done?

If the program can read stdin and write stdout, you could make
it into a REPL.  Probably the easiest would be to adapt Inferior
Python mode (try ‘M-x run-python RET’ and then ‘C-h m’ in the
resulting buffer, for details).

I think that would be easier than a "live update"-style design,
although certainly that is possible as well (w/ a SMOP).

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (type via)
   (case type
     (technical (eq 'mailing-list via))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: How to pass messages between emacs and a Python program? (goal: trying to use emacs as a UI)

[Alexis Roda]

El 26/08/16 a les 00:18, Brian Merchant ha escrit:
> I don't want the Python program to be constantly polling the file for
> changes (using a `while` loop), and I probably don't want emacs to be
> constantly polling the file for updates (which I know how to do using the
> `auto-revert` command).

Hi,
not sure what you mean by "polling", for me it means:

while True:
     if file_changed():
         do_something()
     sleep(some_time)

pynofify (https://github.com/seb-m/pyinotify/wiki) can help with that. 
It's event driven, it monitors a set of files and reacts to changes on 
them. Unfortunately, AFAIK, it's not portable, it only works on linux.

Anyway, depending on your use case, this may not be the right approach.

> Maybe I press some key combination, and then that sends a message to a
> Python script that its time to read the file and make updates and then the
> Python script would message emacs and ask it to update what it is
> displaying in its buffer.
>
> Could this be done?

Yes, but you'll need to learn some emacs lisp.

The simplest solution I can think of that does not require a lot of 
emacs lisp is pymacs (https://github.com/pinard/Pymacs). In short, it 
starts a python interpreter in the background and exposes a set of 
python functions to emacs so that you can use them as if they where 
native emacs functions. From memory:

mymodule.py:

def do_something(path):
     # do something on file 'path'
     return changed  # return True or False

emacs:

(require 'pymacs)
(pymacs-load "mymodule.py" "mm")

 From now on you can call functions defined in mymodule.py prefixing 
them with "mm-":

(mm-do-something "/path/to/buffer")

Pymacs will take care of "encoding" the function call, pass it to 
python, get back the return value and "decode" it.

If required you can define a command that operates on the current buffer:

(defun do-something-on-current-buffer ()
   "call do_something on current buffer"
   (interactive)
   ;; TODO: check if buffer has changes not saved to disk and
   ;; prompt the user to save them.
   (if (mm-do-something (buffer-file-name))
     ;; if content changed reload buffer contents
     (revert-buffer t t)))

and bind it to a key:

(global-set-key (kbd "<f12>") 'do-something-on-current-buffer)

The pros are that you do all your coding in python and write a little 
bit of elisp glue code, the cons are that it's seem unmaintained and I'm 
not sure if it will work with python 3.


Packages like elpy or jedi start an RPC server. The idea is pretty much 
the same as pymacs, take care of forwarding function calls to python and 
getting back the result, but using a different communication 
channel/protocol.



HTH

Re: How to pass messages between emacs and a Python program? (goal: trying to use emacs as a UI)

[Pascal J. Bourguignon]

Brian Merchant <bhmerchant@gmail.com> writes:

> Hi all,
>
> Say I have opened up a file in emacs, and I type in something like:
>
> `x \and \top`
>
> Then, a Python program reads that file, and:
>
> * replaces all `\and` to `∧`
> * replaces all `\top` to `T`
> * it does some thinking on the expression `x \and \top` and decides that
> that is the same as `x`, so will append an `= x`
>
> The final result of all the changes made by the program to the file will
> leave it like so:
>
> `x ∧ T = x`
>
> Okay, so much for what I want. For getting there though:
>
> I don't want the Python program to be constantly polling the file for
> changes (using a `while` loop), and I probably don't want emacs to be
> constantly polling the file for updates (which I know how to do using the
> `auto-revert` command).
>
> Maybe I press some key combination, and then that sends a message to a
> Python script that its time to read the file and make updates and then the
> Python script would message emacs and ask it to update what it is
> displaying in its buffer.
>
> Could this be done?

Yes.


> A bit of background on my goal: I'd like to make a proof editor/"IDE", and
> while I have the programming know-how (at least in Python and C++, not in
> emacs Lisp) to code the background analyzers/proof-engines, I am horrible
> at putting pixels on a screen. I have experimented a bit with putting
> pixels on a screen, and writing my own simple editor, but suffice it to say
> that I have come to appreciate the immense amount of effort it takes to
> build something as smooth as emacs. I think it would be much better for me
> to take advantage of emacs' capabilities in terms of showing stuff on a
> screen, and manipulating/editing that stuff. I worry though that there
> might not be any easy way to do this...
>
> Anyway, would love to hear your thoughts and guidance!


1- it would be better to do it in emacs lisp, that's why emacs has emacs
   lisp!

2- there are systems to bridge lisp and python, but AFAIK, they're all
   implemented for Common Lisp.

   See for example:

      * CLAUDE - The Common Lisp Library Audience Expansion Toolkit
        http://www.european-lisp-symposium.org/editions/2014/levine.zip
        http://nicklevine.org/els2014/levine.mp3
        http://medias.ircam.fr/x31d466

      * cl-python https://common-lisp.net/project/clpython/

   So some porting work would be needed to use a similar solution with
   emacs.

   For example, cl-python which is written in Common Lisp, could run on
   emacs with emacs-cl, a CL implementation written in emacs lisp.
   Unfortunately, this emacs-cl has bit-rot since it doesn't take into
   account lexical closure introduced in emacs 24: it would need some
   work to upgrade it to emacs 24+, and some more to complete the
   conformity and coverage of the standard.  Then it would be trivial to
   use all the CL tools in emacs, such as cl-python.

3- there is slime/swank.
   https://github.com/fgallina/swank-python
   (I have no idea how complete and functional this is).
   While not directly meant for this, slime/swank can be used for RPC
   between emacs lisp and the inferior process.
   
   There are also direct emacs-python RPC such as:
   https://github.com/tkf/python-epc

4- the situation would be much easier if emacs was written in Common
   Lisp instead of C.  For this reason I've started to write a C
   compiler in Common Lisp targetting Common Lisp, with the objective to
   compile/translate the C code of emacs lisp into Common Lisp.  Then
   the solutions available in points 2 and 3 would be trivially
   available.  Unfortunately, my tax inspector requires me to work for
   money, so I couldn't complete this compiler yet. (I only have the C
   pre-processor done for now).
   
   Some people also envisionned rewritting this emacs C code manually,
   but this is as big a task, if not bigger, than to write a C compiler,
   so they didn't make any much more progress than me.

5- if python is that good, why isn't there an emacs clone written in
   python using python as scripting language instead of lisp?  It would
   be a trivial solution to your problem.
   
   

So, to make it Q&D, perhaps the easiest for you will be to use
something like https://github.com/tkf/python-epc
to call your python processing code from emacs when convenient.


-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: How to pass messages between emacs and a Python program? (goal: trying to use emacs as a UI)

[Tomas Nordin]

Brian Merchant <bhmerchant@gmail.com> writes:

> I don't want the Python program to be constantly polling the file for
> changes (using a `while` loop), and I probably don't want emacs to be
> constantly polling the file for updates (which I know how to do using the
> `auto-revert` command).

Make the python code callable from command line.

>
> Maybe I press some key combination, and then that sends a message to a
> Python script that its time to read the file and make updates and then the
> Python script would message emacs and ask it to update what it is
> displaying in its buffer.

Then have a key combination calling your python parser. There is
built-in support in elisp for synchronous process calls for example. You
can receive return value and process output into emacs. If the python
code is prepared for it (import fileinput?), the (unsaved) buffer could
be sent as stdin to your program.