Re: Integrating Midi into Emacs

[joakim@verona.se]

David Kastrup <dak@gnu.org> writes:

> Hi,
>
> I've been meddling with CCL programs and stuff for integrating Midi
> input received via make-serial-process into Emacs.  I've encountered
> several puzzling design decisions and drawbacks.
>
> There are several crucial ones, however.  At its core, pressing keys on
> a musical keyboard will result in the insertion of note names into the
> current buffer.  This basically is similar to an input method.
>
> Here are several respects causing a bad fit:
>
> a) user expectation will be to have music keyboard keys insert into the
> selected window.  make-serial-process can not really have the process be
> tied into one editing buffer since those editing buffers run processes
> of their own (viewer, compilation with LilyPond etc) and
> get-buffer-process is used for checking for conflicts.  Also, one cannot
> reattach the process to different buffers when buffers are getting
> switched.  So basically make-serial-process happens for a buffer of its
> own, and this buffer distributes the received information through its
> process filter.  This process filter then does something akin to
>
> (defun notename-filter (process string)
>   (with-current-buffer (window-buffer)
>     (if	(= (aref string 0) ?\ )
> 	(if (and (= (window-point) (process-mark process))
> 		 (eq (current-buffer) (marker-buffer (process-mark process))))
> 	    (insert string)
> 	  (insert (substring string 1)))
>       (insert string))
>     (move-marker (process-mark process) (window-point) (current-buffer))))
>
> Apart from the complications arising from trying to put spaces between
> successive entries only if no editing happened in between, the thing to
> note is that this works on (window-buffer), the currently selected
> buffer.  It is not clear how many save-excursions may intervene, and
> when a process filter actually gets triggered.  So point and mark in the
> buffer may or may not terminally advance.  One really would want to feed
> this through the regular input queue instead of going directly into some
> buffer.
>
> Which gets us to point
> b) Ultimately, input should be time-stamped.  It's reasonably easy to do
> this using the process-filter, tacking on appropriate text properties,
> but the timestamp may be slightly stale when Emacs is busy as process
> filters are only run at appropriately idle times.
>
> c) the inserted text actually depends on buffer-local variables, for
> example the current notename language (English writes cs for c-sharp,
> Dutch writes cis, Italian writes dod), and the current key (the Midi
> event for c-sharp is indistinguishable from d flat, and the decision is
> made based on the current key in the current piece).  Switching CCL
> programs in the decoder is tricky as they do not lend themselves to
> closures with internal states.  Also, one would need one closure per
> language/key pairing currently active.
>
> This kind of flexible back-and-forth mapping is actually better
> accomplished by swapping around keymaps rather than encodings.
>
> What this leads up to is that a better approach would be to have Midi
> events in the Emacs event queue.  They would want timestamps like mouse
> events, they would want some identifiable nice names/identifications for
> binding them to actions.  Since they are primarily going to be used as
> an input method with one hand on the computer keyboard and one hand on
> the Midi keyboard, it might even make sense to allow for keyboard
> modifiers like shift/alt/control in order to signify particular modified
> actions.
>
> So basically it would seem nice to extend the Emacs input events.  That,
> in itself, feels like something that Emacs should support out of the box
> without recompilation.  If it doesn't, it might be worth thinking about
> what might be necessary to make it do so.  There is also a chance that
> an implementation along that lines could be made to work with XEmacs
> without requiring compilation.
>
> However, to really get timestamps with the accuracy of the arriving
> input, it would make sense to use the binary ALSA API for communicating
> with Midi input devices.
>
> One advantage of tieing Midi input in at a basic level might also be
> that one can use it to convey modifiers like Hyper and Super that avid
> Emacs typists might want to enter using foot pedals.  The number of USB
> computer keyboards providing for Emacs-friendly foot pedals is
> minuscule.  Midi-capable footpedals, however, are not exactly a dime a
> dozen but they are quite ubiquitous.
>
> So being able to exploit Midi signals with a timing accuracy making it
> possible to match them to computer keyboard input would make it much
> easier to acquire workable modifier pedals even for regular non-musician
> uses.
>
> Thoughts?

Since you said "Thoughts", I have some.

- I'm building a combined computer/midi keyboard, and it would
  be really nice to use it with emacs with your idea. (If I ever manage
  to finish the project)

- I'm sure you considered OSC, but heres a link anyway:
  http://delysid.org/emacs/osc.el

My idea with OSC was to embedd gtk sliders in a buffer, and drive a
softsynth made with Overtone with these sliders using OSC.

Anyway, good luck with your project!


-- 
Joakim Verona