Sound in Emacs

Sound in Emacs

[Lars Magne Ingebrigtsen]

Emacs has rudimentary support for playing sound -- .wav and .au files,
if I read the code correctly.

Has anybody considered adding support for modern sound formats via a
library like SDL?  

http://icculus.org/SDL_sound/

It seems like it has support for all the common formats -- .flac and
.ogg and .mp3 and .etc.

There's probably no extremely compelling reasons to add built-in support
for a real sound player in Emacs, but it might be kinda neat.  If you're
in a dired buffer, you can hit RET to see images, but sound files aren't
as available.  Wouldn't it be nice if you hit RET on an .mp3 file, and
Emacs pops up a waveform buffer and starts playing the file?  And you
can skip around in the song/podcast...

Has anybody done any work in that direction?  And if not, would Emacs
(after 24.1, or course) be open to adding such functionality if somebody
(ahem) were to find the time to implement it?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Sound in Emacs

[Julien Danjou]

[-- Attachment #1: Type: text/plain, Size: 795 bytes --]

On Mon, Oct 03 2011, Lars Magne Ingebrigtsen wrote:

> There's probably no extremely compelling reasons to add built-in support
> for a real sound player in Emacs, but it might be kinda neat.  If you're
> in a dired buffer, you can hit RET to see images, but sound files aren't
> as available.  Wouldn't it be nice if you hit RET on an .mp3 file, and
> Emacs pops up a waveform buffer and starts playing the file?  And you
> can skip around in the song/podcast...
>
> Has anybody done any work in that direction?  And if not, would Emacs
> (after 24.1, or course) be open to adding such functionality if somebody
> (ahem) were to find the time to implement it?

I suggest to look at libcanberra or even GStreamer if you want really to
work on that. Not SDL.

-- 
Julien Danjou

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Sound in Emacs

[joakim]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Emacs has rudimentary support for playing sound -- .wav and .au files,
> if I read the code correctly.
>
> Has anybody considered adding support for modern sound formats via a
> library like SDL?  
>
> http://icculus.org/SDL_sound/
>
> It seems like it has support for all the common formats -- .flac and
> .ogg and .mp3 and .etc.
>
> There's probably no extremely compelling reasons to add built-in support
> for a real sound player in Emacs, but it might be kinda neat.  If you're
> in a dired buffer, you can hit RET to see images, but sound files aren't
> as available.  Wouldn't it be nice if you hit RET on an .mp3 file, and
> Emacs pops up a waveform buffer and starts playing the file?  And you
> can skip around in the song/podcast...
>
> Has anybody done any work in that direction?  And if not, would Emacs
> (after 24.1, or course) be open to adding such functionality if somebody
> (ahem) were to find the time to implement it?

I'm interested in using Emacs for controlling various media. The Emacs
Xwidget branch was for instance originally conceived to implement
sliders in Emacs to control sound servers over OSC.

There are pretty big Emacs user groups that use Emacs a lot for this,
using the CSound and the Supercollider tools for instance. Snd is an
audio editor that is influenced by Emacs.

And we also of course have EMMS. The thing, though, is that it is pretty
seamless to just use an external binary to play the sound, unlike having
to start an external binary to watch an image. So my vote would be on
improving EMMS.

If you still feel a compelling need to implement audio directly in Emacs
that's cool also. Then I'd vote for implementing the Emacs buffer model
on audio files and use some nice simple portable audio api such as
Portaudio. It should be more than enough for Emacs and is, ahem, portable.








-- 
Joakim Verona

Re: Sound in Emacs

[Jan D.]

joakim@verona.se skrev 2011-10-03 23:04:

> And we also of course have EMMS. The thing, though, is that it is pretty
> seamless to just use an external binary to play the sound, unlike having
> to start an external binary to watch an image. So my vote would be on
> improving EMMS.
>

If you just are out to get a more fancy beep, starting a new external 
binary for every beep may not be so seamless.

	Jan D.

Re: Sound in Emacs

[Tim Cross]

On Tue, Oct 4, 2011 at 6:01 PM, Jan D. <jan.h.d@swipnet.se> wrote:
> joakim@verona.se skrev 2011-10-03 23:04:
>
>> And we also of course have EMMS. The thing, though, is that it is pretty
>> seamless to just use an external binary to play the sound, unlike having
>> to start an external binary to watch an image. So my vote would be on
>> improving EMMS.
>>
>
> If you just are out to get a more fancy beep, starting a new external binary
> for every beep may not be so seamless.
>
>        Jan D.
>
>
>
>

As someone who makes use of sound from within emacs quite extensively
(emacspeak user), I can confirm that using exgternal binaries to play
sound works quite well. However, depending on what you need, it does
have some limitations. For example, if it is just for a 'new beep',
you can get lag or other issues when lots of beeps are requested at
the same time.

I think the main issue with extending support for sound within emacs
will centre around the amount of variation in sound architectures used
by the various platforms emacs runs on. While there are libraries like
portaudio, these can introduce their own problems. I've run into
significant problems with such compatibility libraries and have
frequently found they can adversely impact on
performance/respnsiveness.

To answer the question whether to enhacne/extend sound support, we
probably need to define what we mean - do we mean better 'beeps' or do
we mean the ability to play mp3/ogg files form dired similar to how we
view images or do we mean something else.

One thing which I have thought about is whether there would be any
benefit from a dbus type interface to external binaries to render
sound files.

Tim



-- 
Tim Cross
Phone: 0428 212 217

Re: Sound in Emacs

[Lars Ingebrigtsen]

Tim Cross <theophilusx@gmail.com> writes:

> To answer the question whether to enhacne/extend sound support, we
> probably need to define what we mean - do we mean better 'beeps' or do
> we mean the ability to play mp3/ogg files form dired similar to how we
> view images or do we mean something else.

I'm not particularly interested in the `beep' case, but more interested
in the dired case.

But I'm not sure there's really a compelling case for incorporating such
support in Emacs.  I mean, all music I play go through Emacs, but use
external programs for playing, and that works kinda OK.  But if there
were good decoding and outputting libraries available (and it's getting
sort of unclear if there really are), then playing music though Emacs
would Just Work instead of being something you'd have to work at to get.

But it's like ImageMagick support.  Does displaying images inside Emacs
give you a better experience?  I think it does.  And I have a feeling
the same would be the case with sound support.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Sound in Emacs

[chad]

[-- Attachment #1: Type: text/plain, Size: 1327 bytes --]


On Oct 4, 2011, at 5:23 PM, Lars Ingebrigtsen wrote:
> 
> But it's like ImageMagick support.  Does displaying images inside Emacs
> give you a better experience?  I think it does.  And I have a feeling
> the same would be the case with sound support.

I suspect that it's the `browsing' versus `working with' difference; I
find image viewing in Emacs to be very useful for looking through
things that contain images, but I've never really missed the ability
to crop, rotate, zoom, etc -- although those would be neat, I haven't
really wanted them.

There is a big UX difference between using emacs to manage a music
collection, adding ui sounds to gnus, and using emacs to figure out
that sound/892794.ogg is `ding' while sound/234676.ogg is `quack'.  I
don't know that EMMS benefits much from internal players, or that a
software replacement for the dectalk is interesting.  I imagine that
many emacs hackers would hate sounds in gnus (look at how many disable
the bell), but I suspect many of the newer users would like it (I've
actually turned them back on in my non-Emacs email environment,
because I like to hear when mail has actually been sent). I imagine
most people either don't care or would like to have emacs' help
browsing sound files in dired.

I hope that this musing hasn't been entirely useless. :-)
*Chad




[-- Attachment #2: Type: text/html, Size: 1871 bytes --]

Re: Sound in Emacs

[Stephen J. Turnbull]

chad writes:

 > There is a big UX difference between using emacs to manage a music
 > collection, adding ui sounds to gnus, and using emacs to figure out
 > that sound/892794.ogg is `ding' while sound/234676.ogg is `quack'.

And none of those really require internal support AFAICS.  But that's
not what I would want Emacs support for.

The thing is, I'm a console-oriented kinda guy, and I find the lack of
imagination, not to mention documentation, of the people who design
GUIs really off-putting and constraining.  Take Audacity (which is a
fine application in most respects).  I use it to record lectures and
things like that, but what I would *really* like is a way to insert
markers from the keyboard -- the idea being to mark at each change of
slide.

Similarly for WYSIWYG draw programs.  Most of my drawing needs are
pretty simple: flow charts and things like that.  And the sizes etc
are usually easily computed, or pretty standard.

The applications do allow creation of macros and things like that, but
it requires a lot more effort than I want to put in to learn the
specifics.  Whipping up UIs (modes and keymaps) for that kind of thing
is something Emacs Lisp is really good for.

Re: Sound in Emacs

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

() Lars Ingebrigtsen <larsi@gnus.org>
() Wed, 05 Oct 2011 02:23:40 +0200

   then playing music though Emacs would Just Work instead
   of being something you'd have to work at to get.

Are you sure?  I get the impression sound reproduction is
more timing sensitive than, say, image display.

   But it's like ImageMagick support.  Does displaying
   images inside Emacs give you a better experience?  I
   think it does.  And I have a feeling the same would be
   the case with sound support.

Hmmm.

FWIW, here is a heavily modified TRAT, originally by Vikas
Gorur.  There are other playlist manglers of course...


[-- Attachment #2: trat.el --]
[-- Type: application/emacs-lisp, Size: 10052 bytes --]

Re: Sound in Emacs

[Óscar Fuentes]

"Jan D." <jan.h.d@swipnet.se> writes:

> joakim@verona.se skrev 2011-10-03 23:04:
>
>> And we also of course have EMMS. The thing, though, is that it is pretty
>> seamless to just use an external binary to play the sound, unlike having
>> to start an external binary to watch an image. So my vote would be on
>> improving EMMS.
>>
>
> If you just are out to get a more fancy beep, starting a new external
> binary for every beep may not be so seamless.

As Emacs' `(beep)' does not work on Kubuntu 10.10, my .emacs contains

  (setq ring-bell-function
	(lambda ()
	  (call-process "beep" nil 0 nil "-f" "400" "-l" "100")))

Although the `beep' executable is as lightweight as it can possibly be,
the results are far from perfect. For instance

(beep)
(sit-for 0.1)
(beep)

usually makes two audible beeps, but often only one beep is
perceived. On the machines where Emacs' (beep) works, there is no such
problem.

Re: Sound in Emacs

[andersvi]

    L> If you're in a dired buffer, you can hit RET to see images, but
    L> sound files aren't as available.  Wouldn't it be nice if you hit
    L> RET on an .mp3 file, and Emacs pops up a waveform buffer and
    L> starts playing the file?  And you can skip around in the
    L> song/podcast...

Possibly the closest you get atm is setting up Snd
(https://ccrma.stanford.edu/software/snd/) as an inferior scheme
process, and control it from emacs via existing major or minor-modes, or
build further to integrate more tightly into eg. dired.  Everything in
Snd is controllable and accessible from a running scheme.

Linking in libsndfile, which is rather x-plattform i beleive, would
provide builtin access for reading (and writing) the various sound-file
formats around.

A project id like to get going is setting up a database type app for
sound - something similar to various photo-managers - where info from a
sound gets stored and managed - diskfile, region, format, tags and
metadata of all sorts - - and made available through a general
interface.  Send a query to find certain regions from some files, and
ask a running sound-editor to show them.  Edit them, update metadata
info, headers etc. on a set of matches (ie. add a tag or similar) etc.

Ideally an interchange format (SDIF?) would make exported data usable by
all sorts of clients, import into your DAW, export a region into a
soundeditor and get the edited version back w. updated metadata...

Emacs would be an ideal environment to access all this.

-anders

Re: Sound in Emacs

[joakim]

andersvi@notam02.no writes:

>     L> If you're in a dired buffer, you can hit RET to see images, but
>     L> sound files aren't as available.  Wouldn't it be nice if you hit
>     L> RET on an .mp3 file, and Emacs pops up a waveform buffer and
>     L> starts playing the file?  And you can skip around in the
>     L> song/podcast...
>
> Possibly the closest you get atm is setting up Snd
> (https://ccrma.stanford.edu/software/snd/) as an inferior scheme
> process, and control it from emacs via existing major or minor-modes, or
> build further to integrate more tightly into eg. dired.  Everything in
> Snd is controllable and accessible from a running scheme.
>
> Linking in libsndfile, which is rather x-plattform i beleive, would
> provide builtin access for reading (and writing) the various sound-file
> formats around.
>
> A project id like to get going is setting up a database type app for
> sound - something similar to various photo-managers - where info from a
> sound gets stored and managed - diskfile, region, format, tags and
> metadata of all sorts - - and made available through a general
> interface.  Send a query to find certain regions from some files, and
> ask a running sound-editor to show them.  Edit them, update metadata
> info, headers etc. on a set of matches (ie. add a tag or similar) etc.
>
> Ideally an interchange format (SDIF?) would make exported data usable by
> all sorts of clients, import into your DAW, export a region into a
> soundeditor and get the edited version back w. updated metadata...
>
> Emacs would be an ideal environment to access all this.

I'm very interested in this sort of thing also. I suppose it's off topic
for this thread though. Anyway, briefly, my conclusion is the same as
yours. All filetypes can be meta tagged the same way technically. then
you need a common storage(I've been experimenting with rdf indexing on
top of xattrs) and indeed Emacs is ideal for large scale tagging.

>
> -anders
>

-- 
Joakim Verona

Re: Sound in Emacs

[andersvi]

    j> All filetypes can be meta tagged the same way technically. then
    j> you need a common storage(I've been experimenting with rdf
    j> indexing on top of xattrs) and indeed Emacs is ideal for large
    j> scale tagging.

I dont know of any existing things doing this, which is a pity, it would
make for a very effective tool for managing all kinds of large
collections of sound.  There are some metadata-editors for sound around,
but afaik only 'standalone' apps, really not offering much more in
functionality than file-browsers.

One obvious problem in a db-approach is keeping collected info updated
after say moving files around or editing them.

This could be managed by including historic information in the actual
files (version, name, location...) for a 'detective'-script to be able
to keep track of new names, locations or other changes, and keep a db
updated accordingly.  Maybe xattrs already solves this?

Re: Sound in Emacs

[joakim]

andersvi@notam02.no writes:

>     j> All filetypes can be meta tagged the same way technically. then
>     j> you need a common storage(I've been experimenting with rdf
>     j> indexing on top of xattrs) and indeed Emacs is ideal for large
>     j> scale tagging.
>
> I dont know of any existing things doing this, which is a pity, it would
> make for a very effective tool for managing all kinds of large
> collections of sound.  There are some metadata-editors for sound around,
> but afaik only 'standalone' apps, really not offering much more in
> functionality than file-browsers.
>
> One obvious problem in a db-approach is keeping collected info updated
> after say moving files around or editing them.
>
> This could be managed by including historic information in the actual
> files (version, name, location...) for a 'detective'-script to be able
> to keep track of new names, locations or other changes, and keep a db
> updated accordingly.  Maybe xattrs already solves this?

xattrs are like image exif tags, or mp3 id3 tags, so they move along
with the file. Sadly there doesn't seem to be a set of standard xattrs
one can use, the facility as such is just a name value storage in a
recourse fork of the file. Indexing needs to be done separately and
there are a bunch of daemons that do that.

Anyway, my idea was to implement an xattrs editor in Emacs, the other
issues are outside of the scope for Emacs. If you are interested in this
line of thought we could set up a shared code repository and take it
from there.


-- 
Joakim Verona

RE: Sound in Emacs

[Drew Adams]

>>> A project id like to get going is setting up a database
>>> type app for sound - something similar to various
>>> photo-managers - where info from a sound gets stored
>>> and managed - diskfile, region, format, tags and
>>> metadata of all sorts - - and made available through a
>>> general interface.  Send a query to find certain regions
>>> from some files, and ask a running sound-editor to show
>>> them.  Edit them, update metadata info, headers etc. on
>>> a set of matches (ie. add a tag or similar) etc.
>j>
>j> All filetypes can be meta tagged the same way technically. then
>j> you need a common storage(I've been experimenting with rdf
>j> indexing on top of xattrs) and indeed Emacs is ideal for large
>j> scale tagging.
> 
> I dont know of any existing things doing this, which is a 
> pity, it would make for a very effective tool for managing
> all kinds of large collections of sound.  There are some
> metadata-editors for sound around, but afaik only 'standalone'
> apps, really not offering much more in functionality than
> file-browsers.
> 
> One obvious problem in a db-approach is keeping collected info updated
> after say moving files around or editing them.
> 
> This could be managed by including historic information in the actual
> files (version, name, location...) for a 'detective'-script to be able
> to keep track of new names, locations or other changes, and keep a db
> updated accordingly.

FWIW/FYI-

No doubt it does not correspond to everything you both are envisioning, but
Bookmark+ lets you easily register (aka "bookmark") any type of file, associate
metatdata with it, tag it in various ways and access it using tag set
operations, attach any additional code to its activation (aka "jumping to it"),
group it in various ways with other files (e.g. a playlist), and so on.  

(Not sure what you meant by "regions", but you can also bookmark specific
regions of text files.  And bookmarks generally handle "moving around",
renaming, and editing.)

Possibly some of what you envision could be built using what is already there -
dunno.  Mentioning Bookmark+ because much of what you describe sounds similar.

http://www.emacswiki.org/emacs/BookmarkPlus

Alternatively, you can perhaps leverage Org mode's features involving metadata
and tagging - dunno.  And then there is Stefan's Music Player Daemon (MPD) -
dunno about that either, but it sounds like it employs an actual "database"
(unlike Bookmark+ (and Org, I imagine), which holds the metadata in plain text
files).

Re: Sound in Emacs

[Nix]

On 6 Oct 2011, Drew Adams outgrape:
>                       And then there is Stefan's Music Player Daemon (MPD) -
> dunno about that either, but it sounds like it employs an actual "database"
> (unlike Bookmark+ (and Org, I imagine), which holds the metadata in plain text
> files).

MPD's 'database' is just a (UTF-8-encoded) text file (as is its
saved-state file in the same directory).

-- 
NULL && (void)

RE: Sound in Emacs

[Drew Adams]

> MPD's 'database' is just a (UTF-8-encoded) text file (as is its
> saved-state file in the same directory).

I see.  So in that, at least, it is similar to Bookmark+ (and, I imagine, Org).

Re: Sound in Emacs

[Stefan Monnier]

>> And then there is Stefan's Music Player Daemon (MPD) -
>> dunno about that either, but it sounds like it employs an actual "database"
>> (unlike Bookmark+ (and Org, I imagine), which holds the metadata in plain text
>> files).
> MPD's 'database' is just a (UTF-8-encoded) text file (as is its
> saved-state file in the same directory).

Yes and no: it's saved as a plain text file, but it's loaded upon
startup and the indices are kept in some kind of balanced trees.


        Stefan

Re: Sound in Emacs

[Nix]

On 6 Oct 2011, Stefan Monnier said:

>>> And then there is Stefan's Music Player Daemon (MPD) -
>>> dunno about that either, but it sounds like it employs an actual "database"
>>> (unlike Bookmark+ (and Org, I imagine), which holds the metadata in plain text
>>> files).
>> MPD's 'database' is just a (UTF-8-encoded) text file (as is its
>> saved-state file in the same directory).
>
> Yes and no: it's saved as a plain text file, but it's loaded upon
> startup and the indices are kept in some kind of balanced trees.

Yeah, so as with other dbs you can't edit it freely (e.g. not without
shutting mpd down first).

-- 
NULL && (void)

Re: Sound in Emacs

[andersvi]

>>>>> "D" == Drew Adams <drew.adams@oracle.com> writes:
    D> No doubt it does not correspond to everything you both are
    D> envisioning, but Bookmark+ lets you easily register (aka
    D> "bookmark") any type of file, associate metatdata with it, tag it
    D> in various ways and access it using tag set operations, attach
    D> any additional code to its activation (aka "jumping to it"),
    D> group it in various ways with other files (e.g. a playlist), and
    D> so on.

Looks great.  Indeed, if Lars' initial suggestion of linking in native
'sound'-support to emacs Bookmark+ (or similar) would come quite far.

    D> (Not sure what you meant by "regions", but you can also bookmark
    D> specific regions of text files.

This is perhaps more for people working with sound in analytical or
creative contexts, ie. composers or researchers etc.  Most sounds in
these working-contexts are typically contained in recorded (or
processed) files, any one file containing several 'regions' -
start-time, duration, channel-index... - one would want to look up
separately without necessary splicing each region out as one soundfile.

    D> Alternatively, you can perhaps leverage Org mode's features
    D> involving metadata and tagging - dunno.

I think someone made a snd-interface for org-babel.  These are nice
places to start.

    D>     And then there is Stefan's Music Player Daemon (MPD) - dunno
    D> about that either, but it sounds like it employs an actual
    D> "database" (unlike Bookmark+ (and Org, I imagine), which holds
    D> the metadata in plain text files).

Storage isnt a problem as such.  Any efficient db system, say mysql,
would work great.  Typically sound-collections (in the contexts
mentioned above) grow large, and often are kept on separate
mount-points, and special sound-disks.  Looking up a db-server, perhaps
as an option, would be a bonus.