Integrating Midi into Emacs

Integrating Midi into Emacs

[David Kastrup]

Hi,

I've been meddling with CCL programs and stuff for integrating Midi
input received via make-serial-process into Emacs.  I've encountered
several puzzling design decisions and drawbacks.

There are several crucial ones, however.  At its core, pressing keys on
a musical keyboard will result in the insertion of note names into the
current buffer.  This basically is similar to an input method.

Here are several respects causing a bad fit:

a) user expectation will be to have music keyboard keys insert into the
selected window.  make-serial-process can not really have the process be
tied into one editing buffer since those editing buffers run processes
of their own (viewer, compilation with LilyPond etc) and
get-buffer-process is used for checking for conflicts.  Also, one cannot
reattach the process to different buffers when buffers are getting
switched.  So basically make-serial-process happens for a buffer of its
own, and this buffer distributes the received information through its
process filter.  This process filter then does something akin to

(defun notename-filter (process string)
  (with-current-buffer (window-buffer)
    (if	(= (aref string 0) ?\ )
	(if (and (= (window-point) (process-mark process))
		 (eq (current-buffer) (marker-buffer (process-mark process))))
	    (insert string)
	  (insert (substring string 1)))
      (insert string))
    (move-marker (process-mark process) (window-point) (current-buffer))))

Apart from the complications arising from trying to put spaces between
successive entries only if no editing happened in between, the thing to
note is that this works on (window-buffer), the currently selected
buffer.  It is not clear how many save-excursions may intervene, and
when a process filter actually gets triggered.  So point and mark in the
buffer may or may not terminally advance.  One really would want to feed
this through the regular input queue instead of going directly into some
buffer.

Which gets us to point
b) Ultimately, input should be time-stamped.  It's reasonably easy to do
this using the process-filter, tacking on appropriate text properties,
but the timestamp may be slightly stale when Emacs is busy as process
filters are only run at appropriately idle times.

c) the inserted text actually depends on buffer-local variables, for
example the current notename language (English writes cs for c-sharp,
Dutch writes cis, Italian writes dod), and the current key (the Midi
event for c-sharp is indistinguishable from d flat, and the decision is
made based on the current key in the current piece).  Switching CCL
programs in the decoder is tricky as they do not lend themselves to
closures with internal states.  Also, one would need one closure per
language/key pairing currently active.

This kind of flexible back-and-forth mapping is actually better
accomplished by swapping around keymaps rather than encodings.

What this leads up to is that a better approach would be to have Midi
events in the Emacs event queue.  They would want timestamps like mouse
events, they would want some identifiable nice names/identifications for
binding them to actions.  Since they are primarily going to be used as
an input method with one hand on the computer keyboard and one hand on
the Midi keyboard, it might even make sense to allow for keyboard
modifiers like shift/alt/control in order to signify particular modified
actions.

So basically it would seem nice to extend the Emacs input events.  That,
in itself, feels like something that Emacs should support out of the box
without recompilation.  If it doesn't, it might be worth thinking about
what might be necessary to make it do so.  There is also a chance that
an implementation along that lines could be made to work with XEmacs
without requiring compilation.

However, to really get timestamps with the accuracy of the arriving
input, it would make sense to use the binary ALSA API for communicating
with Midi input devices.

One advantage of tieing Midi input in at a basic level might also be
that one can use it to convey modifiers like Hyper and Super that avid
Emacs typists might want to enter using foot pedals.  The number of USB
computer keyboards providing for Emacs-friendly foot pedals is
minuscule.  Midi-capable footpedals, however, are not exactly a dime a
dozen but they are quite ubiquitous.

So being able to exploit Midi signals with a timing accuracy making it
possible to match them to computer keyboard input would make it much
easier to acquire workable modifier pedals even for regular non-musician
uses.

Thoughts?

-- 
David Kastrup

Re: Integrating Midi into Emacs

[davi]

David Kastrup wrote:
> ... workable modifier pedals even for regular non-musician uses.
> 
> Thoughts?

+1 to very cool innovations.

Re: Integrating Midi into Emacs

[Christopher Allan Webber]

David Kastrup writes:

> One advantage of tieing Midi input in at a basic level might also be
> that one can use it to convey modifiers like Hyper and Super that avid
> Emacs typists might want to enter using foot pedals.  The number of USB
> computer keyboards providing for Emacs-friendly foot pedals is
> minuscule.  Midi-capable footpedals, however, are not exactly a dime a
> dozen but they are quite ubiquitous.
>
> So being able to exploit Midi signals with a timing accuracy making it
> possible to match them to computer keyboard input would make it much
> easier to acquire workable modifier pedals even for regular non-musician
> uses.
>
> Thoughts?

This is an interesting idea.  I use foot pedals myself for coding to
mitigate my RSI.  It's hard to recommend to people though since the ones
I have from Kinesis are a little pricey (but not as pricey as having RSI
issues).

BTW, I've been meaning to document if for some time but haven't, but if
anyone here is using foot pedals for typing, I highly recommend turning
the foot pedals *backwards* so you press it with your heel, and prop
books up in front of the pedals... this means that pressing ctrl & alt
requires only small movements.  I found that pressing ctrl and alt with
my foot forward-facing was irritating my foot very quickly.

 - cwebb

Re: Integrating Midi into Emacs

[David Kastrup]

Christopher Allan Webber <cwebber@dustycloud.org> writes:

> David Kastrup writes:
>
>> One advantage of tieing Midi input in at a basic level might also be
>> that one can use it to convey modifiers like Hyper and Super that avid
>> Emacs typists might want to enter using foot pedals.  The number of USB
>> computer keyboards providing for Emacs-friendly foot pedals is
>> minuscule.  Midi-capable footpedals, however, are not exactly a dime a
>> dozen but they are quite ubiquitous.
>>
>> So being able to exploit Midi signals with a timing accuracy making it
>> possible to match them to computer keyboard input would make it much
>> easier to acquire workable modifier pedals even for regular non-musician
>> uses.
>>
>> Thoughts?
>
> This is an interesting idea.  I use foot pedals myself for coding to
> mitigate my RSI.  It's hard to recommend to people though since the ones
> I have from Kinesis are a little pricey (but not as pricey as having RSI
> issues).
>
> BTW, I've been meaning to document if for some time but haven't, but if
> anyone here is using foot pedals for typing, I highly recommend turning
> the foot pedals *backwards* so you press it with your heel, and prop
> books up in front of the pedals... this means that pressing ctrl & alt
> requires only small movements.  I found that pressing ctrl and alt with
> my foot forward-facing was irritating my foot very quickly.

The basic "sustain pedal" contains just a switch (typically, you can
select whether it's on or off when pushing the pedal down, and a number
of keyboards don't actually care but just consider the polarity
encountered on power-on as off) on a 6.3mm jack.  So that does not yet
give you a Midi signal unless you have some keyboard and/or
"midification kit" connected.

But a number of organ pedalboards produce Midi as their main output, and
there are programmable footswitch controllers as well.  However, the
programmable footswitch controllers are more intended for changing
registers and keys and patterns, so they are intended more for
occasional kick action rather than continuous use.

So you are likely better off with the mechanics of organ pedalboards
(though you'll not need most of the keys) or sustain pedals.

On the reverse route, it might be interesting to use something akin to
Emacs keyboard methods to produce Midi: if you take a look at a
chromatic button keyboard (say,
<URL:http://buttonbox.com/images/acu0035-01b.jpg>, the side with the big
buttons) you'll find that the chromatic button keyboard (anything after
the third button row is redundant and most players will get along fine
without it) is remarkably similar to a computer keyboard.  There are
"virtual Midi" keyboards available but they are based on piano keyboards
(and thus make worse use of the computer keyboard) and most particularly
they run in another window.

When one switches back and forth between Midi input in Emacs and normal
input _and_ one wants to use the computer keyboard for music entry
rather than a separate keyboard, having to mess with input focus,
windows etc etc is going to be a nuisance.  An input method just
producing Midi would be preferable.

How-e-ver: to actually get key press and key release events, one would
have to work with a more raw form of X11 keyboard events than what is
usually fed into Emacs input methods.

At any rate, there is quite a bit of potential for integrating Emacs
better into a musician's workflow.

-- 
David Kastrup

Re: Integrating Midi into Emacs

[joakim]

David Kastrup <dak@gnu.org> writes:

> Hi,
>
> I've been meddling with CCL programs and stuff for integrating Midi
> input received via make-serial-process into Emacs.  I've encountered
> several puzzling design decisions and drawbacks.
>
> There are several crucial ones, however.  At its core, pressing keys on
> a musical keyboard will result in the insertion of note names into the
> current buffer.  This basically is similar to an input method.
>
> Here are several respects causing a bad fit:
>
> a) user expectation will be to have music keyboard keys insert into the
> selected window.  make-serial-process can not really have the process be
> tied into one editing buffer since those editing buffers run processes
> of their own (viewer, compilation with LilyPond etc) and
> get-buffer-process is used for checking for conflicts.  Also, one cannot
> reattach the process to different buffers when buffers are getting
> switched.  So basically make-serial-process happens for a buffer of its
> own, and this buffer distributes the received information through its
> process filter.  This process filter then does something akin to
>
> (defun notename-filter (process string)
>   (with-current-buffer (window-buffer)
>     (if	(= (aref string 0) ?\ )
> 	(if (and (= (window-point) (process-mark process))
> 		 (eq (current-buffer) (marker-buffer (process-mark process))))
> 	    (insert string)
> 	  (insert (substring string 1)))
>       (insert string))
>     (move-marker (process-mark process) (window-point) (current-buffer))))
>
> Apart from the complications arising from trying to put spaces between
> successive entries only if no editing happened in between, the thing to
> note is that this works on (window-buffer), the currently selected
> buffer.  It is not clear how many save-excursions may intervene, and
> when a process filter actually gets triggered.  So point and mark in the
> buffer may or may not terminally advance.  One really would want to feed
> this through the regular input queue instead of going directly into some
> buffer.
>
> Which gets us to point
> b) Ultimately, input should be time-stamped.  It's reasonably easy to do
> this using the process-filter, tacking on appropriate text properties,
> but the timestamp may be slightly stale when Emacs is busy as process
> filters are only run at appropriately idle times.
>
> c) the inserted text actually depends on buffer-local variables, for
> example the current notename language (English writes cs for c-sharp,
> Dutch writes cis, Italian writes dod), and the current key (the Midi
> event for c-sharp is indistinguishable from d flat, and the decision is
> made based on the current key in the current piece).  Switching CCL
> programs in the decoder is tricky as they do not lend themselves to
> closures with internal states.  Also, one would need one closure per
> language/key pairing currently active.
>
> This kind of flexible back-and-forth mapping is actually better
> accomplished by swapping around keymaps rather than encodings.
>
> What this leads up to is that a better approach would be to have Midi
> events in the Emacs event queue.  They would want timestamps like mouse
> events, they would want some identifiable nice names/identifications for
> binding them to actions.  Since they are primarily going to be used as
> an input method with one hand on the computer keyboard and one hand on
> the Midi keyboard, it might even make sense to allow for keyboard
> modifiers like shift/alt/control in order to signify particular modified
> actions.
>
> So basically it would seem nice to extend the Emacs input events.  That,
> in itself, feels like something that Emacs should support out of the box
> without recompilation.  If it doesn't, it might be worth thinking about
> what might be necessary to make it do so.  There is also a chance that
> an implementation along that lines could be made to work with XEmacs
> without requiring compilation.
>
> However, to really get timestamps with the accuracy of the arriving
> input, it would make sense to use the binary ALSA API for communicating
> with Midi input devices.
>
> One advantage of tieing Midi input in at a basic level might also be
> that one can use it to convey modifiers like Hyper and Super that avid
> Emacs typists might want to enter using foot pedals.  The number of USB
> computer keyboards providing for Emacs-friendly foot pedals is
> minuscule.  Midi-capable footpedals, however, are not exactly a dime a
> dozen but they are quite ubiquitous.
>
> So being able to exploit Midi signals with a timing accuracy making it
> possible to match them to computer keyboard input would make it much
> easier to acquire workable modifier pedals even for regular non-musician
> uses.
>
> Thoughts?

Since you said "Thoughts", I have some.

- I'm building a combined computer/midi keyboard, and it would
  be really nice to use it with emacs with your idea. (If I ever manage
  to finish the project)

- I'm sure you considered OSC, but heres a link anyway:
  http://delysid.org/emacs/osc.el

My idea with OSC was to embedd gtk sliders in a buffer, and drive a
softsynth made with Overtone with these sliders using OSC.

Anyway, good luck with your project!


-- 
Joakim Verona

Re: Integrating Midi into Emacs

[David Kastrup]

[-- Attachment #1: Type: text/plain, Size: 2220 bytes --]

joakim@verona.se writes:

> David Kastrup <dak@gnu.org> writes:
>
>> I've been meddling with CCL programs and stuff for integrating Midi
>> input received via make-serial-process into Emacs.  I've encountered
>> several puzzling design decisions and drawbacks.
>>
>> There are several crucial ones, however.  At its core, pressing keys
>> on a musical keyboard will result in the insertion of note names into
>> the current buffer.  This basically is similar to an input method.

>> c) the inserted text actually depends on buffer-local variables, for
>> example the current notename language (English writes cs for c-sharp,
>> Dutch writes cis, Italian writes dod), and the current key (the Midi
>> event for c-sharp is indistinguishable from d flat, and the decision is
>> made based on the current key in the current piece).  Switching CCL
>> programs in the decoder is tricky as they do not lend themselves to
>> closures with internal states.  Also, one would need one closure per
>> language/key pairing currently active.
>>
>> This kind of flexible back-and-forth mapping is actually better
>> accomplished by swapping around keymaps rather than encodings.
>>
>> What this leads up to is that a better approach would be to have Midi
>> events in the Emacs event queue.
>>
>> Thoughts?
>
> Since you said "Thoughts", I have some.
>
> - I'm building a combined computer/midi keyboard, and it would
>   be really nice to use it with emacs with your idea. (If I ever manage
>   to finish the project)
>
> - I'm sure you considered OSC, but heres a link anyway:
>   http://delysid.org/emacs/osc.el

Never heard of it.  I'll take a look eventually.

> My idea with OSC was to embedd gtk sliders in a buffer, and drive a
> softsynth made with Overtone with these sliders using OSC.
>
> Anyway, good luck with your project!

Here is the current pitch I am working with without recompiling the
binary, so obviously not taking modifier keys or timings at the moment.
Also, no strategy yet for encoding key-release events.

You just call M-x midikbd-open RET, specify a raw midi device (if
necessary, provided by the snd-virmidi module), and then you can choose
yourself how you are going to bind the resulting key events.


[-- Attachment #2: midi-kbd.el --]
[-- Type: application/emacs-lisp, Size: 5160 bytes --]

[-- Attachment #3: Type: text/plain, Size: 294 bytes --]


Of course this is ongoing work, but what I got here is a nice
self-contained module doing exactly one job, that of delivering events.
And obviously, if one were to write native Midi support, that's exactly
the scope of what this native Midi support would be supposed to do.

-- 
David Kastrup

Re: Integrating Midi into Emacs

[David Kastrup]

joakim@verona.se writes:

> - I'm sure you considered OSC, but heres a link anyway:
>   http://delysid.org/emacs/osc.el
>
> My idea with OSC was to embedd gtk sliders in a buffer, and drive a
> softsynth made with Overtone with these sliders using OSC.
>
> Anyway, good luck with your project!

I actually don't have the foggiest what I am supposed to do with OSC and
osc.el.  At any rate, opening a link into some generic sound control
network server is not at the same level of complexity as operating some
Midi device.  So I'm going to pursue my Midi entry project
independently.  Whether or not one can use OSC as an alternative source
of the respective events: no idea.

-- 
David Kastrup

Re: Integrating Midi into Emacs

[joakim]

David Kastrup <dak@gnu.org> writes:

> joakim@verona.se writes:
>
>> - I'm sure you considered OSC, but heres a link anyway:
>>   http://delysid.org/emacs/osc.el>
>> My idea with OSC was to embedd gtk sliders in a buffer, and drive a
>> softsynth made with Overtone with these sliders using OSC.
>>
>> Anyway, good luck with your project!
>
> I actually don't have the foggiest what I am supposed to do with OSC and
> osc.el.  At any rate, opening a link into some generic sound control
> network server is not at the same level of complexity as operating some
> Midi device.  So I'm going to pursue my Midi entry project
> independently.  Whether or not one can use OSC as an alternative source
> of the respective events: no idea.

OSC is indeed on a level above plain Midi, so they are not mutually
exclusive. I mostly thought it was related, thats all.


-- 
Joakim Verona

Re: Integrating Midi into Emacs

[joakim]

David Kastrup <dak@gnu.org> writes:

> joakim@verona.se writes:
>
>> David Kastrup <dak@gnu.org> writes:
>>
>>> I've been meddling with CCL programs and stuff for integrating Midi
>>> input received via make-serial-process into Emacs.  I've encountered
>>> several puzzling design decisions and drawbacks.
>>>
>>> There are several crucial ones, however.  At its core, pressing keys
>>> on a musical keyboard will result in the insertion of note names into
>>> the current buffer.  This basically is similar to an input method.
>
>>> c) the inserted text actually depends on buffer-local variables, for
>>> example the current notename language (English writes cs for c-sharp,
>>> Dutch writes cis, Italian writes dod), and the current key (the Midi
>>> event for c-sharp is indistinguishable from d flat, and the decision is
>>> made based on the current key in the current piece).  Switching CCL
>>> programs in the decoder is tricky as they do not lend themselves to
>>> closures with internal states.  Also, one would need one closure per
>>> language/key pairing currently active.
>>>
>>> This kind of flexible back-and-forth mapping is actually better
>>> accomplished by swapping around keymaps rather than encodings.
>>>
>>> What this leads up to is that a better approach would be to have Midi
>>> events in the Emacs event queue.
>>>
>>> Thoughts?
>>
>> Since you said "Thoughts", I have some.
>>
>> - I'm building a combined computer/midi keyboard, and it would
>>   be really nice to use it with emacs with your idea. (If I ever manage
>>   to finish the project)
>>
>> - I'm sure you considered OSC, but heres a link anyway:
>>   http://delysid.org/emacs/osc.el
>
> Never heard of it.  I'll take a look eventually.
>
>> My idea with OSC was to embedd gtk sliders in a buffer, and drive a
>> softsynth made with Overtone with these sliders using OSC.
>>
>> Anyway, good luck with your project!
>
> Here is the current pitch I am working with without recompiling the
> binary, so obviously not taking modifier keys or timings at the moment.
> Also, no strategy yet for encoding key-release events.
>
> You just call M-x midikbd-open RET, specify a raw midi device (if
> necessary, provided by the snd-virmidi module), and then you can choose
> yourself how you are going to bind the resulting key events.

Very interesting, I will try it with my Midi equipment!

>
> ;;; midi-kbd.el --- Create keyboard events from Midi input  -*- lexical-binding: t; -*-
>
> ;; Copyright (C) 2015  David Kastrup
>
> ;; Author: David Kastrup <dak@gnu.org>
> ;; Keywords: convenience, hardware, multimedia
>
> ;; This program is free software; you can redistribute it and/or modify
> ;; it under the terms of the GNU General Public License as published by
> ;; the Free Software Foundation, either version 3 of the License, or
> ;; (at your option) any later version.
>
> ;; This program is distributed in the hope that it will be useful,
> ;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> ;; GNU General Public License for more details.
>
> ;; You should have received a copy of the GNU General Public License
> ;; along with this program.  If not, see <http://www.gnu.org/licenses/>.
>
> ;;; Commentary:
>
> ;; This generates events from an ALSA Midi device.  The names of the
> ;; events are modeled after the nomenclature of Emacs Calc's notenames.
> ;; Channel switching events are <Channel1> ... <Channel16>
> ;; Note Events are <C_-1> <Csharp_-1> ... <G_9>
> ;; Since Midi does not encode enharmonics, there are no *flat_* key
> ;; names: it is the job of the key bindings to establish tonality
> ;; beyond the chromatic pitch.
> ;;
> ;; It would make sense to provide this kind of functionality directly
> ;; from within Emacs, making use of libportmidi.  That way, it could
> ;; get supported by operating systems not providing a raw Midi port
> ;; accessible as a character device.  This would also improve the
> ;; possibilities to integrate timing and modifiers into events like it
> ;; is done with mouse events.
>
> ;;; Code:
>
> ;; Should this be a char-table?
> (defvar midikbd-keys
>   [C_-1 Csharp_-1 D_-1 Dsharp_-1 E_-1 F_-1
> 	Fsharp_-1 G_-1 Gsharp_-1 A_-1 Asharp_-1 B_-1
>    C_0 Csharp_0 D_0 Dsharp_0 E_0 F_0
> 	Fsharp_0 G_0 Gsharp_0 A_0 Asharp_0 B_0
>    C_1 Csharp_1 D_1 Dsharp_1 E_1 F_1
> 	Fsharp_1 G_1 Gsharp_1 A_1 Asharp_1 B_1
>    C_2 Csharp_2 D_2 Dsharp_2 E_2 F_2
> 	Fsharp_2 G_2 Gsharp_2 A_2 Asharp_2 B_2
>    C_3 Csharp_3 D_3 Dsharp_3 E_3 F_3
> 	Fsharp_3 G_3 Gsharp_3 A_3 Asharp_3 B_3
>    C_4 Csharp_4 D_4 Dsharp_4 E_4 F_4
> 	Fsharp_4 G_4 Gsharp_4 A_4 Asharp_4 B_4
>    C_5 Csharp_5 D_5 Dsharp_5 E_5 F_5
> 	Fsharp_5 G_5 Gsharp_5 A_5 Asharp_5 B_5
>    C_6 Csharp_6 D_6 Dsharp_6 E_6 F_6
> 	Fsharp_6 G_6 Gsharp_6 A_6 Asharp_6 B_6
>    C_7 Csharp_7 D_7 Dsharp_7 E_7 F_7
> 	Fsharp_7 G_7 Gsharp_7 A_7 Asharp_7 B_7
>    C_8 Csharp_8 D_8 Dsharp_8 E_8 F_8
> 	Fsharp_8 G_8 Gsharp_8 A_8 Asharp_8 B_8
>    C_9 Csharp_9 D_9 Dsharp_9 E_9 F_9
> 	Fsharp_9 G_9
>    Channel1 Channel2 Channel3 Channel4 Channel5 Channel6 Channel7 Channel8
>    Channel9 Channel10 Channel11 Channel12 Channel13 Channel14 Channel15 Channel16]
>    "Map CCL program codes to symbols used as keycodes.
>
> CCL program `midikbd-decoder' produces 0..127 for the pitches and
> 128..143 for the channel switches.  `midikbd-keys' maps this into
> symbols used as keycodes.")
>
> ;; CCL programs used in coding systems apparently don't save registers
> ;; across suspension so we don't use a coding system.  Instead our CCL
> ;; program is run using ccl-execute-on-string in the filter routine.
> ;; That allows us to interpret all _completed_ Midi commands without
> ;; getting confused, and it also gives us a well-defined internal
> ;; state (namely one for every call of midikbd-filter-create).
>
> (define-ccl-program midikbd-decoder
>   '(1
>     ((r1 = 16)
>      (loop
>       (read r0)
>       (if ((r0 & 240) == 144)
> 	  (read-if (r2 < 128)		; pitch
> 		   (read-if (r3 > 0)	; velocity
> 					; Some Midi devices use velocity 0
> 					; for switching notes off, so skip
> 					; velocity 0 notes
> 			    ((r0 &= 15)
> 			     (if (r0 != r1)
> 				 ((r1 = r0)
> 				  (write (r0 + 128))))
> 			     (write-repeat r2)))))
>       (repeat)))))
>
> (defun midikbd-filter-create ()
>   "Create one Midi process filter keeping state across calls."
>   (let ((state (make-vector 9 nil)))
>     (lambda (_process string)
>       (setq unread-command-events
> 	    (nconc unread-command-events
> 		   (mapcar (lambda (x) (aref midikbd-keys x))
> 			   (ccl-execute-on-string 'midikbd-decoder
> 						  state string t t)))))))
>
> ;;;###autoload
> (defun midikbd-open (file)
>   "Open the raw Midi device FILE as a source for Midi input.
> This should be an ALSA device like \"/dev/snd/midiC1D0\".  If your
> Midi producing device is a software Midi device, you might need to
> call
>
>     sudo modprobe snd-virmidi
>
> in order to have some virtual ALSA ports available as such raw Midi
> devices."
>   (interactive (list (read-file-name "Midi device: "
> 				     "/dev/snd/" "midiC1D0" t nil
> 				     #'file-readable-p)))
>   (let* ((file (expand-file-name file "/dev/snd/"))
> 	 (buffer (get-buffer-create (concat " *Midi process " file " *")))
> 	 (oldproc (get-buffer-process buffer)))
>     (if (processp oldproc) (delete-process oldproc))
>     (make-serial-process :port file
> 			 :speed nil
> 			 :buffer buffer
> 			 :coding 'raw-text
> 			 :filter (midikbd-filter-create)
> 			 :sentinel #'ignore
> 			 :noquery t)))
>
> (provide 'midi-kbd)
> ;;; midi-kbd.el ends here
>
>
> Of course this is ongoing work, but what I got here is a nice
> self-contained module doing exactly one job, that of delivering events.
> And obviously, if one were to write native Midi support, that's exactly
> the scope of what this native Midi support would be supposed to do.

-- 
Joakim Verona

Re: Integrating Midi into Emacs

[Niels Giesen]

[-- Attachment #1: Type: text/plain, Size: 9511 bytes --]

Hi David,

This looks really cool to me. I tried to use your code, but cannot for the
life of me find out how to bind functions to those events. Could you give a
simple example?

It would be great if indeed all (details of) midi events could be made
available to Emacs.

By the way, it might be nice for Emacs to be able to act as a midi device
itself too, both for input and output in Alsa, and to be able to hook Emacs
up to other programs inside qjackctl or st..

Please keep us updated on this exciting work.

On Sat, Jan 10, 2015 at 9:32 AM, <joakim@verona.se> wrote:

> David Kastrup <dak@gnu.org> writes:
>
> > joakim@verona.se writes:
> >
> >> David Kastrup <dak@gnu.org> writes:
> >>
> >>> I've been meddling with CCL programs and stuff for integrating Midi
> >>> input received via make-serial-process into Emacs.  I've encountered
> >>> several puzzling design decisions and drawbacks.
> >>>
> >>> There are several crucial ones, however.  At its core, pressing keys
> >>> on a musical keyboard will result in the insertion of note names into
> >>> the current buffer.  This basically is similar to an input method.
> >
> >>> c) the inserted text actually depends on buffer-local variables, for
> >>> example the current notename language (English writes cs for c-sharp,
> >>> Dutch writes cis, Italian writes dod), and the current key (the Midi
> >>> event for c-sharp is indistinguishable from d flat, and the decision is
> >>> made based on the current key in the current piece).  Switching CCL
> >>> programs in the decoder is tricky as they do not lend themselves to
> >>> closures with internal states.  Also, one would need one closure per
> >>> language/key pairing currently active.
> >>>
> >>> This kind of flexible back-and-forth mapping is actually better
> >>> accomplished by swapping around keymaps rather than encodings.
> >>>
> >>> What this leads up to is that a better approach would be to have Midi
> >>> events in the Emacs event queue.
> >>>
> >>> Thoughts?
> >>
> >> Since you said "Thoughts", I have some.
> >>
> >> - I'm building a combined computer/midi keyboard, and it would
> >>   be really nice to use it with emacs with your idea. (If I ever manage
> >>   to finish the project)
> >>
> >> - I'm sure you considered OSC, but heres a link anyway:
> >>   http://delysid.org/emacs/osc.el
> >
> > Never heard of it.  I'll take a look eventually.
> >
> >> My idea with OSC was to embedd gtk sliders in a buffer, and drive a
> >> softsynth made with Overtone with these sliders using OSC.
> >>
> >> Anyway, good luck with your project!
> >
> > Here is the current pitch I am working with without recompiling the
> > binary, so obviously not taking modifier keys or timings at the moment.
> > Also, no strategy yet for encoding key-release events.
> >
> > You just call M-x midikbd-open RET, specify a raw midi device (if
> > necessary, provided by the snd-virmidi module), and then you can choose
> > yourself how you are going to bind the resulting key events.
>
> Very interesting, I will try it with my Midi equipment!
>
> >
> > ;;; midi-kbd.el --- Create keyboard events from Midi input  -*-
> lexical-binding: t; -*-
> >
> > ;; Copyright (C) 2015  David Kastrup
> >
> > ;; Author: David Kastrup <dak@gnu.org>
> > ;; Keywords: convenience, hardware, multimedia
> >
> > ;; This program is free software; you can redistribute it and/or modify
> > ;; it under the terms of the GNU General Public License as published by
> > ;; the Free Software Foundation, either version 3 of the License, or
> > ;; (at your option) any later version.
> >
> > ;; This program is distributed in the hope that it will be useful,
> > ;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> > ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > ;; GNU General Public License for more details.
> >
> > ;; You should have received a copy of the GNU General Public License
> > ;; along with this program.  If not, see <http://www.gnu.org/licenses/>.
> >
> > ;;; Commentary:
> >
> > ;; This generates events from an ALSA Midi device.  The names of the
> > ;; events are modeled after the nomenclature of Emacs Calc's notenames.
> > ;; Channel switching events are <Channel1> ... <Channel16>
> > ;; Note Events are <C_-1> <Csharp_-1> ... <G_9>
> > ;; Since Midi does not encode enharmonics, there are no *flat_* key
> > ;; names: it is the job of the key bindings to establish tonality
> > ;; beyond the chromatic pitch.
> > ;;
> > ;; It would make sense to provide this kind of functionality directly
> > ;; from within Emacs, making use of libportmidi.  That way, it could
> > ;; get supported by operating systems not providing a raw Midi port
> > ;; accessible as a character device.  This would also improve the
> > ;; possibilities to integrate timing and modifiers into events like it
> > ;; is done with mouse events.
> >
> > ;;; Code:
> >
> > ;; Should this be a char-table?
> > (defvar midikbd-keys
> >   [C_-1 Csharp_-1 D_-1 Dsharp_-1 E_-1 F_-1
> >       Fsharp_-1 G_-1 Gsharp_-1 A_-1 Asharp_-1 B_-1
> >    C_0 Csharp_0 D_0 Dsharp_0 E_0 F_0
> >       Fsharp_0 G_0 Gsharp_0 A_0 Asharp_0 B_0
> >    C_1 Csharp_1 D_1 Dsharp_1 E_1 F_1
> >       Fsharp_1 G_1 Gsharp_1 A_1 Asharp_1 B_1
> >    C_2 Csharp_2 D_2 Dsharp_2 E_2 F_2
> >       Fsharp_2 G_2 Gsharp_2 A_2 Asharp_2 B_2
> >    C_3 Csharp_3 D_3 Dsharp_3 E_3 F_3
> >       Fsharp_3 G_3 Gsharp_3 A_3 Asharp_3 B_3
> >    C_4 Csharp_4 D_4 Dsharp_4 E_4 F_4
> >       Fsharp_4 G_4 Gsharp_4 A_4 Asharp_4 B_4
> >    C_5 Csharp_5 D_5 Dsharp_5 E_5 F_5
> >       Fsharp_5 G_5 Gsharp_5 A_5 Asharp_5 B_5
> >    C_6 Csharp_6 D_6 Dsharp_6 E_6 F_6
> >       Fsharp_6 G_6 Gsharp_6 A_6 Asharp_6 B_6
> >    C_7 Csharp_7 D_7 Dsharp_7 E_7 F_7
> >       Fsharp_7 G_7 Gsharp_7 A_7 Asharp_7 B_7
> >    C_8 Csharp_8 D_8 Dsharp_8 E_8 F_8
> >       Fsharp_8 G_8 Gsharp_8 A_8 Asharp_8 B_8
> >    C_9 Csharp_9 D_9 Dsharp_9 E_9 F_9
> >       Fsharp_9 G_9
> >    Channel1 Channel2 Channel3 Channel4 Channel5 Channel6 Channel7
> Channel8
> >    Channel9 Channel10 Channel11 Channel12 Channel13 Channel14 Channel15
> Channel16]
> >    "Map CCL program codes to symbols used as keycodes.
> >
> > CCL program `midikbd-decoder' produces 0..127 for the pitches and
> > 128..143 for the channel switches.  `midikbd-keys' maps this into
> > symbols used as keycodes.")
> >
> > ;; CCL programs used in coding systems apparently don't save registers
> > ;; across suspension so we don't use a coding system.  Instead our CCL
> > ;; program is run using ccl-execute-on-string in the filter routine.
> > ;; That allows us to interpret all _completed_ Midi commands without
> > ;; getting confused, and it also gives us a well-defined internal
> > ;; state (namely one for every call of midikbd-filter-create).
> >
> > (define-ccl-program midikbd-decoder
> >   '(1
> >     ((r1 = 16)
> >      (loop
> >       (read r0)
> >       (if ((r0 & 240) == 144)
> >         (read-if (r2 < 128)           ; pitch
> >                  (read-if (r3 > 0)    ; velocity
> >                                       ; Some Midi devices use velocity 0
> >                                       ; for switching notes off, so skip
> >                                       ; velocity 0 notes
> >                           ((r0 &= 15)
> >                            (if (r0 != r1)
> >                                ((r1 = r0)
> >                                 (write (r0 + 128))))
> >                            (write-repeat r2)))))
> >       (repeat)))))
> >
> > (defun midikbd-filter-create ()
> >   "Create one Midi process filter keeping state across calls."
> >   (let ((state (make-vector 9 nil)))
> >     (lambda (_process string)
> >       (setq unread-command-events
> >           (nconc unread-command-events
> >                  (mapcar (lambda (x) (aref midikbd-keys x))
> >                          (ccl-execute-on-string 'midikbd-decoder
> >                                                 state string t t)))))))
> >
> > ;;;###autoload
> > (defun midikbd-open (file)
> >   "Open the raw Midi device FILE as a source for Midi input.
> > This should be an ALSA device like \"/dev/snd/midiC1D0\".  If your
> > Midi producing device is a software Midi device, you might need to
> > call
> >
> >     sudo modprobe snd-virmidi
> >
> > in order to have some virtual ALSA ports available as such raw Midi
> > devices."
> >   (interactive (list (read-file-name "Midi device: "
> >                                    "/dev/snd/" "midiC1D0" t nil
> >                                    #'file-readable-p)))
> >   (let* ((file (expand-file-name file "/dev/snd/"))
> >        (buffer (get-buffer-create (concat " *Midi process " file " *")))
> >        (oldproc (get-buffer-process buffer)))
> >     (if (processp oldproc) (delete-process oldproc))
> >     (make-serial-process :port file
> >                        :speed nil
> >                        :buffer buffer
> >                        :coding 'raw-text
> >                        :filter (midikbd-filter-create)
> >                        :sentinel #'ignore
> >                        :noquery t)))
> >
> > (provide 'midi-kbd)
> > ;;; midi-kbd.el ends here
> >
> >
> > Of course this is ongoing work, but what I got here is a nice
> > self-contained module doing exactly one job, that of delivering events.
> > And obviously, if one were to write native Midi support, that's exactly
> > the scope of what this native Midi support would be supposed to do.
>
> --
> Joakim Verona
>
>


-- 
http://pft.github.com

[-- Attachment #2: Type: text/html, Size: 12430 bytes --]

Re: Integrating Midi into Emacs

[David Kastrup]

Niels Giesen <niels.giesen@gmail.com> writes:

> Hi David,
>
> This looks really cool to me. I tried to use your code, but cannot for the
> life of me find out how to bind functions to those events. Could you give a
> simple example?

Well, this is work in progress.  I've been tripped up by quirks of the
raw Midi spec: to make things more challenging, it allows inserting
foreign command bytes ("realtime events") in the middle of other
commands.  And it allows leaving off command bytes altogether ("running
status byte").  When accessing Midi devices through libraries or the
Midi API, those peculiarities are taken care off, but not so with the
raw Midi stream.

I'm appending the current files I am working with.  They are _not_ a
finished application yet.  The midikbd module produces events.  The
decoding framework should now be working reliably, but currently only
the events I am interested in working with get produced.  And the
lily-midi module integrates this into LilyPond-mode, assuming that it
has already been loaded (again: work in progress, so load order,
autoloads etc are all not in place).  The notenames are hardwired to
dutch, channel changes are not currently used, timing is not recorded
and so on.  The last thing I did was implementing a recording key
signature so that pitches are produced with accidentals chosen in a
useful relation to the key signature.

The long pitch would be to use libmidiport in Emacs proper, and produce
both key up and down events (with at least the up events ignored by
default) and full timing info, similar to mouse events.  Most of this
(apart from the default binding behavior) can probably be presimulated
by extending midi-kbd.el, but the precision of the timing would likely
suffer.

-- 
David Kastrup

Re: Integrating Midi into Emacs

[David Kastrup]

[-- Attachment #1: Type: text/plain, Size: 1896 bytes --]

David Kastrup <dak@gnu.org> writes:

> Niels Giesen <niels.giesen@gmail.com> writes:
>
>> Hi David,
>>
>> This looks really cool to me. I tried to use your code, but cannot for the
>> life of me find out how to bind functions to those events. Could you give a
>> simple example?
>
> Well, this is work in progress.  I've been tripped up by quirks of the
> raw Midi spec: to make things more challenging, it allows inserting
> foreign command bytes ("realtime events") in the middle of other
> commands.  And it allows leaving off command bytes altogether ("running
> status byte").  When accessing Midi devices through libraries or the
> Midi API, those peculiarities are taken care off, but not so with the
> raw Midi stream.
>
> I'm appending the current files I am working with.  They are _not_ a
> finished application yet.  The midikbd module produces events.  The
> decoding framework should now be working reliably, but currently only
> the events I am interested in working with get produced.  And the
> lily-midi module integrates this into LilyPond-mode, assuming that it
> has already been loaded (again: work in progress, so load order,
> autoloads etc are all not in place).  The notenames are hardwired to
> dutch, channel changes are not currently used, timing is not recorded
> and so on.  The last thing I did was implementing a recording key
> signature so that pitches are produced with accidentals chosen in a
> useful relation to the key signature.
>
> The long pitch would be to use libmidiport in Emacs proper, and produce
> both key up and down events (with at least the up events ignored by
> default) and full timing info, similar to mouse events.  Most of this
> (apart from the default binding behavior) can probably be presimulated
> by extending midi-kbd.el, but the precision of the timing would likely
> suffer.

Oh fiddlesticks.  Should have appended the files.


[-- Attachment #2: midi-kbd.el --]
[-- Type: application/emacs-lisp, Size: 6444 bytes --]

[-- Attachment #3: lily-midi.el --]
[-- Type: application/emacs-lisp, Size: 4509 bytes --]

[-- Attachment #4: Type: text/plain, Size: 20 bytes --]



-- 
David Kastrup