Integrating Midi into Emacs

[David Kastrup <dak@gnu.org>]

Hi,

I've been meddling with CCL programs and stuff for integrating Midi
input received via make-serial-process into Emacs.  I've encountered
several puzzling design decisions and drawbacks.

There are several crucial ones, however.  At its core, pressing keys on
a musical keyboard will result in the insertion of note names into the
current buffer.  This basically is similar to an input method.

Here are several respects causing a bad fit:

a) user expectation will be to have music keyboard keys insert into the
selected window.  make-serial-process can not really have the process be
tied into one editing buffer since those editing buffers run processes
of their own (viewer, compilation with LilyPond etc) and
get-buffer-process is used for checking for conflicts.  Also, one cannot
reattach the process to different buffers when buffers are getting
switched.  So basically make-serial-process happens for a buffer of its
own, and this buffer distributes the received information through its
process filter.  This process filter then does something akin to

(defun notename-filter (process string)
  (with-current-buffer (window-buffer)
    (if	(= (aref string 0) ?\ )
	(if (and (= (window-point) (process-mark process))
		 (eq (current-buffer) (marker-buffer (process-mark process))))
	    (insert string)
	  (insert (substring string 1)))
      (insert string))
    (move-marker (process-mark process) (window-point) (current-buffer))))

Apart from the complications arising from trying to put spaces between
successive entries only if no editing happened in between, the thing to
note is that this works on (window-buffer), the currently selected
buffer.  It is not clear how many save-excursions may intervene, and
when a process filter actually gets triggered.  So point and mark in the
buffer may or may not terminally advance.  One really would want to feed
this through the regular input queue instead of going directly into some
buffer.

Which gets us to point
b) Ultimately, input should be time-stamped.  It's reasonably easy to do
this using the process-filter, tacking on appropriate text properties,
but the timestamp may be slightly stale when Emacs is busy as process
filters are only run at appropriately idle times.

c) the inserted text actually depends on buffer-local variables, for
example the current notename language (English writes cs for c-sharp,
Dutch writes cis, Italian writes dod), and the current key (the Midi
event for c-sharp is indistinguishable from d flat, and the decision is
made based on the current key in the current piece).  Switching CCL
programs in the decoder is tricky as they do not lend themselves to
closures with internal states.  Also, one would need one closure per
language/key pairing currently active.

This kind of flexible back-and-forth mapping is actually better
accomplished by swapping around keymaps rather than encodings.

What this leads up to is that a better approach would be to have Midi
events in the Emacs event queue.  They would want timestamps like mouse
events, they would want some identifiable nice names/identifications for
binding them to actions.  Since they are primarily going to be used as
an input method with one hand on the computer keyboard and one hand on
the Midi keyboard, it might even make sense to allow for keyboard
modifiers like shift/alt/control in order to signify particular modified
actions.

So basically it would seem nice to extend the Emacs input events.  That,
in itself, feels like something that Emacs should support out of the box
without recompilation.  If it doesn't, it might be worth thinking about
what might be necessary to make it do so.  There is also a chance that
an implementation along that lines could be made to work with XEmacs
without requiring compilation.

However, to really get timestamps with the accuracy of the arriving
input, it would make sense to use the binary ALSA API for communicating
with Midi input devices.

One advantage of tieing Midi input in at a basic level might also be
that one can use it to convey modifiers like Hyper and Super that avid
Emacs typists might want to enter using foot pedals.  The number of USB
computer keyboards providing for Emacs-friendly foot pedals is
minuscule.  Midi-capable footpedals, however, are not exactly a dime a
dozen but they are quite ubiquitous.

So being able to exploit Midi signals with a timing accuracy making it
possible to match them to computer keyboard input would make it much
easier to acquire workable modifier pedals even for regular non-musician
uses.

Thoughts?

-- 
David Kastrup