Re: ELisp futures and continuations/coroutines

[SAKURAI Masashi <m.sakurai@kiwanami.net>]

At Tue, 24 May 2011 21:07:50 -0500,
Ted Zlatanov wrote:
>  :
> Also please see the discussion in this thread about url-future.el.  If
> you can consider augmenting that defstruct instead of using the one in
> deferred.el, it would be nice so we're all speaking the same data
> language.  I asked the same of Thien-Thi Nguyen for fsm.el.  But it's
> not a requirement for inclusion in the GNU ELPA and you don't have to do
> it.

I have read all discussion messages in this thread (about more
url-utils and future in elisp) carefully at last. 
Those discussions are so interesting. 
I wrote my comment as follows. Sorry for long comment.


About the "future" type for the just url-fetch function, I think it is
good enough to transfer values asynchronously to the waiting function.
However, I think it is somewhat primitive to construct general
asynchronous programs.

I have met so many difficulties during developing some Emacs
applications[1] which has rich UIs and communication with external
resources, such as handling many callbacks and waiting for them to
finish. For example, getting images by wget -> making thumbnails ->
displaying the images in the buffer, during then, the application
notifies the user of progress of tasks and may display results or
error messages.

OTOH, in JavaScript, there are async supports in many major libs:
jQuery.Deferred, Mochikit.Async, dojo.Deferred and so on. We can
easily write async codes right now. Now, even .NET4[2] provides an
async lib like JSDeferred. So, I'm sure elisp should have async
support too.

While I'm not a researcher for the concurrent programing,
investigating such other async libraries, I found that asynchronous
programing needs at least following two functions:

1) sequential connecting
2) waiting for async tasks done (all of them or the earliest one).

One can define many other aspects of async programing.
Ex. composition of tasks, error handling, canceling, propagating
values, adding tasks after executed, restarting tasks, notifying
progress and so on[3].

Recently, I found an utility program url-queue.el (Fetching web pages
in parallel) was added to lisp/url in trunk. In the single-thread
model like the Emacs, the task queue requires the callback
interface. So, I think some library which supports concurrent
programing model will be needed in near future, like deferred.el 
and concurrent.el.

Furthermore, url-queue.el seems to implement the semaphore which is
one of concurrent patterns. I made the semaphore in concurrent.el
because I needed. I have also seen the similar code in
emacs-w3m[4]. The code of w3m-proc.el manages number of sub-processes
and abstraction of async and sync calling.  So, I think those
concurrent patterns have been needed already.

Though concurrent.el has some patterns those were implemented for my
applications, of course, it doesn't cover all patterns. According to
other languages and some books, STM, Agent, Actor, Reentrant Lock and
Read-Write Lock are argued as concurrent programing.


Last, if my experience of development of deferred.el and concurrent.el
would help the Emacs's advance, I would be happy. I don't mind if
the libraries will be added to GNU ELPA or Emacs, even the maintainers
write a subset code from scratch.

Foot notes:
[1] My deferred.el applications
  * A network multiplayer game which I mentioned the previous post.
  https://github.com/kiwanami/emacs-3d-demo
  (movie) http://www.youtube.com/watch?v=YS5CUj9oUhs
  * In-line image manager with cacoo.com, the on-line diagramming tool
  https://github.com/kiwanami/emacs-cacoo
  (movie) http://www.youtube.com/watch?v=pwZvT5HFQjY
  * PDF file manager
  https://github.com/kiwanami/emacs-anything-books
  (movie) http://www.youtube.com/watch?v=jfEHmgCIc08
  * Inertial scrolling (see the movie of cacoo.el)
  https://github.com/kiwanami/emacs-inertial-scroll

[2] .NET
  * Async Tasks - Simplify Asynchronous Programming with Tasks
  http://msdn.microsoft.com/en-us/magazine/ff959203.aspx
  Moreover, Task and Rx enables to collect async resources by LINQ
  * Reactive Extensions
  http://msdn.microsoft.com/en-us/data/gg577609

[3] Aspects of async libs
  Sorry for only Japanese article I found.
  * (in Japanese)
  http://digitechlog.com/2010/03/12/7-asynchronous-processing-open-source-actionscript-library.html 
  * (machine translation)
  http://translate.google.co.jp/translate?hl=ja&sl=ja&tl=en&u=http://digitechlog.com/2010/03/12/7-asynchronous-processing-open-source-actionscript-library.html
  Following article may be helpful for designing async lib.
  * Virtual Panel: How to Survive Asynchronous Programming in JavaScript (InfoQ)
  http://www.infoq.com/articles/surviving-asynchronous-programming-in-javascript

[4] Emacs-w3m
  * web site: http://emacs-w3m.namazu.org/index-en.html
  * w3m-proc.el: http://cvs.namazu.org/emacs-w3m/w3m-proc.el?view=markup

--
SAKURAI, Masashi (family, given)