all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* async.el: A simple asynchronous framework for Emacs
@ 2012-06-19  2:51 John Wiegley
  2012-06-19  5:11 ` Thien-Thi Nguyen
                   ` (3 more replies)
  0 siblings, 4 replies; 24+ messages in thread
From: John Wiegley @ 2012-06-19  2:51 UTC (permalink / raw)
  To: emacs-devel

Hello,

I think we've all wanted threading and asynchronicity in Emacs for some time
now (Gnus, anyone?), and there have been many attempts to provide it.  I
propose a module for inclusion in Emacs, async.el, which offers a very ease to
use model for asynchronicity, without the need for any threading (and its
attendant complexities).  It should work on every platform that supports
asynchronous processes using `start-process'.

The whole interface is two functions: `async-start' and `async-get' (of which
the latter is optional).  Here is the basic form of use:

    (async-start (lambda () ...)
                 'function-to-call-when-done)

This will execute the lambda (which must *not* be byte-compiled -- in other
words, don't use `function' or #') in a child Emacs asynchronously.  When it's
done, the return value is passed to `function-to-call-when-done' as an
argument.  (If you don't care about the return value, pass the `ignore' symbol
as the second argument).

If you pass no second argument, a future is returned.  You can later call
`async-get' on this future to obtain the value, blocking if necessary.

That's it.  All you need to do asynchronous computation within Emacs.

Since it's likely that you'll want the child Emacs to heavy lifting based on
the parent Emacs' configuration, you can use `async-inject-environment' to
pass variable definitions across the process boundary:

    (async-start (lambda ()
                   (require 'some-module)
                   (async-inject-environment "\\`some-module-")
                   ...))

The variable definitions from the module "some-module" will be passed into the
child.

Using these facilities, here's all it takes to send e-mail asynchronously with
smtpmail.el:

    (defun async-smtpmail-send-it ()
      (async-start
       `(lambda ()
          (require 'smtpmail)
          (with-temp-buffer
            (insert ,(buffer-substring-no-properties (point-min) (point-max)))
            ;; Pass in the variable environment for smtpmail
            ,(async-inject-environment "\\`\\(smtpmail\\|\\(user-\\)?mail\\)-")
            (smtpmail-send-it)))
       'ignore))

I've also written dired-async.el, which performs copies, moves and deletes
asynchronously.  It works great with Tramp.

The files are hosted on GitHub presently:

    https://github.com/jwiegley/emacs-async

One thing I would love to do is to work with the authors of other modules --
such as one of my all-time favorites, Gnus -- to see how a facility like this
can help improve user experience.

Comments welcome,
  John

p.s. This e-mail sent asynchronously with smtpmail.el :)



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-19  2:51 async.el: A simple asynchronous framework for Emacs John Wiegley
@ 2012-06-19  5:11 ` Thien-Thi Nguyen
  2012-06-19  5:36   ` John Wiegley
  2012-06-19 13:29 ` Stefan Monnier
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 24+ messages in thread
From: Thien-Thi Nguyen @ 2012-06-19  5:11 UTC (permalink / raw)
  To: emacs-devel

() John Wiegley <jwiegley@gmail.com>
() Mon, 18 Jun 2012 21:51:31 -0500

   Since it's likely that you'll want the child Emacs to heavy
   lifting based on the parent Emacs' configuration, you can use
   `async-inject-environment' to pass variable definitions across
   the process boundary:

       (async-start (lambda ()
                      (require 'some-module)
                      (async-inject-environment "\\`some-module-")
                      ...))

   The variable definitions from the module "some-module" will be
   passed into the child.

The variable values must be of somewhat limited type, right?



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-19  5:11 ` Thien-Thi Nguyen
@ 2012-06-19  5:36   ` John Wiegley
  0 siblings, 0 replies; 24+ messages in thread
From: John Wiegley @ 2012-06-19  5:36 UTC (permalink / raw)
  To: emacs-devel

>>>>> Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> The variable values must be of somewhat limited type, right?

They can be anything for which:

    (assert (equal x (read (prin1-to-string x))))

This excludes unprintable values, most notably buffers, byte-compiled lambdas,
windows, frames, etc.

John



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-19  2:51 async.el: A simple asynchronous framework for Emacs John Wiegley
  2012-06-19  5:11 ` Thien-Thi Nguyen
@ 2012-06-19 13:29 ` Stefan Monnier
  2012-06-19 20:44   ` John Wiegley
  2012-06-19 21:38 ` Christopher Allan Webber
  2012-06-21  8:31 ` Thierry Volpiatto
  3 siblings, 1 reply; 24+ messages in thread
From: Stefan Monnier @ 2012-06-19 13:29 UTC (permalink / raw)
  To: emacs-devel

> The whole interface is two functions: `async-start' and `async-get' (of which
> the latter is optional).  Here is the basic form of use:

Looks very interesting, thank you.

> This will execute the lambda (which must *not* be byte-compiled -- in other
> words, don't use `function' or #') in a child Emacs asynchronously.

Can you explain why it can't be byte-compiled?
Also, I'd guess that your package could be improved if Emacs provided
a `fork' primitive, right?


        Stefan



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-19 13:29 ` Stefan Monnier
@ 2012-06-19 20:44   ` John Wiegley
  2012-06-19 21:46     ` Stefan Monnier
  0 siblings, 1 reply; 24+ messages in thread
From: John Wiegley @ 2012-06-19 20:44 UTC (permalink / raw)
  To: emacs-devel

>>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> This will execute the lambda (which must *not* be byte-compiled -- in other
>> words, don't use `function' or #') in a child Emacs asynchronously.

> Can you explain why it can't be byte-compiled?

Because it becomes an un-`read'-able value if prin1'd.

> Also, I'd guess that your package could be improved if Emacs provided
> a `fork' primitive, right?

You bet.  However, I would almost never want to fork the whole environment.  I
would want the child to end up with nil values for:

    timer-list
    timer-idle-list
    kill-emacs-hook

And perhaps even a few others, if there are post-command-hooks that might
persist data, for example.  Plus, it would make no sense to fork over the
windows and frames, as the child would be headless; nor the process list.  And
what about unmodified buffers, and file locks being held by the parent?

It would probably be better if `fork' took a conservative approach, requiring
me to supply a predicate to match the variables and buffers I wanted to end up
in the child.

John



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-19  2:51 async.el: A simple asynchronous framework for Emacs John Wiegley
  2012-06-19  5:11 ` Thien-Thi Nguyen
  2012-06-19 13:29 ` Stefan Monnier
@ 2012-06-19 21:38 ` Christopher Allan Webber
  2012-06-21  8:31 ` Thierry Volpiatto
  3 siblings, 0 replies; 24+ messages in thread
From: Christopher Allan Webber @ 2012-06-19 21:38 UTC (permalink / raw)
  To: emacs-devel

I just wanted to jump in and say that this seems like a really exciting
future.  I can already imagine a lot of exciting things that could make
use of it without having any crazy complexity in threading and locking
and etc in emacs.

I hope it happens!

 - Chris

John Wiegley <jwiegley@gmail.com> writes:

> Hello,
>
> I think we've all wanted threading and asynchronicity in Emacs for some time
> now (Gnus, anyone?), and there have been many attempts to provide it.  I
> propose a module for inclusion in Emacs, async.el, which offers a very ease to
> use model for asynchronicity, without the need for any threading (and its
> attendant complexities).  It should work on every platform that supports
> asynchronous processes using `start-process'.
>
> The whole interface is two functions: `async-start' and `async-get' (of which
> the latter is optional).  Here is the basic form of use:
>
>     (async-start (lambda () ...)
>                  'function-to-call-when-done)
>
> This will execute the lambda (which must *not* be byte-compiled -- in other
> words, don't use `function' or #') in a child Emacs asynchronously.  When it's
> done, the return value is passed to `function-to-call-when-done' as an
> argument.  (If you don't care about the return value, pass the `ignore' symbol
> as the second argument).
>
> If you pass no second argument, a future is returned.  You can later call
> `async-get' on this future to obtain the value, blocking if necessary.
>
> That's it.  All you need to do asynchronous computation within Emacs.
>
> Since it's likely that you'll want the child Emacs to heavy lifting based on
> the parent Emacs' configuration, you can use `async-inject-environment' to
> pass variable definitions across the process boundary:
>
>     (async-start (lambda ()
>                    (require 'some-module)
>                    (async-inject-environment "\\`some-module-")
>                    ...))
>
> The variable definitions from the module "some-module" will be passed into the
> child.
>
> Using these facilities, here's all it takes to send e-mail asynchronously with
> smtpmail.el:
>
>     (defun async-smtpmail-send-it ()
>       (async-start
>        `(lambda ()
>           (require 'smtpmail)
>           (with-temp-buffer
>             (insert ,(buffer-substring-no-properties (point-min) (point-max)))
>             ;; Pass in the variable environment for smtpmail
>             ,(async-inject-environment "\\`\\(smtpmail\\|\\(user-\\)?mail\\)-")
>             (smtpmail-send-it)))
>        'ignore))
>
> I've also written dired-async.el, which performs copies, moves and deletes
> asynchronously.  It works great with Tramp.
>
> The files are hosted on GitHub presently:
>
>     https://github.com/jwiegley/emacs-async
>
> One thing I would love to do is to work with the authors of other modules --
> such as one of my all-time favorites, Gnus -- to see how a facility like this
> can help improve user experience.
>
> Comments welcome,
>   John
>
> p.s. This e-mail sent asynchronously with smtpmail.el :)



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-19 20:44   ` John Wiegley
@ 2012-06-19 21:46     ` Stefan Monnier
  2012-06-20  0:20       ` John Wiegley
  2012-06-20  5:34       ` John Wiegley
  0 siblings, 2 replies; 24+ messages in thread
From: Stefan Monnier @ 2012-06-19 21:46 UTC (permalink / raw)
  To: emacs-devel

>>> This will execute the lambda (which must *not* be byte-compiled -- in other
>>> words, don't use `function' or #') in a child Emacs asynchronously.
>> Can you explain why it can't be byte-compiled?
> Because it becomes an un-`read'-able value if prin1'd.

The byte-compiler uses prin1 to generate the .elc files, so I'm pretty
sure there's a way to solve this problem ;-)

>> Also, I'd guess that your package could be improved if Emacs provided
>> a `fork' primitive, right?
> You bet.  However, I would almost never want to fork the whole
> environment.  I would want the child to end up with nil values for:

>     timer-list
>     timer-idle-list
>     kill-emacs-hook

Yes, it can't just be a naive call to `fork' because it also has to
forget about all the frames and sub-processes.  Also `fork' is probably
unavailable under Windows, so the Elisp functionality should work both
with and without `fork' (i.e. we can't rely too much on `fork's
semantics, we can only use it to improve performance or save us the
trouble of "finding the same `emacs' executable in $PATH").


        Stefan



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-19 21:46     ` Stefan Monnier
@ 2012-06-20  0:20       ` John Wiegley
  2012-06-20  0:53         ` John Wiegley
  2012-06-20  5:34       ` John Wiegley
  1 sibling, 1 reply; 24+ messages in thread
From: John Wiegley @ 2012-06-20  0:20 UTC (permalink / raw)
  To: emacs-devel

Just a note, I've generalized async.el a bit further so that I can also have
async-start-process -- which uses a different process that Emacs, but keeps
the rest of the functionality the same.

So, asynchronous invocation of a lambda with callback is the same as before:

    (async-start (lambda () ... in child process ...)
                 (lambda (ret) ... handle return value from lambda ...))

Now I can get the same sort of functionality with other processes too:

    (async-start-process "cp" "/bin/cp"
                         (lambda (proc) ... use proc to get exit code ...)
                         "file1" "file2")

Further, as with `async-start', if the callback function is nil you'll get a
closure that you can inspect when you're ready.  I've also added two new
functions for working with these futures:

    (async-ready FUTURE)    ; returns t if `async-get' would not block
    (async-wait FUTURE)     ; waits for FUTURE to become ready, returns nil
    (async-get FUTURE)

Lastly, you can call `async-get' now even if you did provide a callback, it
will simply always return nil in that case (since the return value went to the
callback).

My motivation for `async-start-process' is that I want dired-async.el to use
native cp/mv/rm if all files are local (file-remote-p == nil).  But I want to
keep the same infrastructure, i.e., the simple-to-specify callbacks for
manipulating the dired buffer once the operation is complete.

Updates are on GitHub.

John



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-20  0:20       ` John Wiegley
@ 2012-06-20  0:53         ` John Wiegley
  0 siblings, 0 replies; 24+ messages in thread
From: John Wiegley @ 2012-06-20  0:53 UTC (permalink / raw)
  To: emacs-devel

Apropos of nothing, just discovered that `apply-partially' is a nice way to
run one-liners asynchronously:

  (async-start (apply-partially #'copy-file from to ok-flag preserve-time)
               (apply-partially #'message "Copy done: %s -> %s" from to))

The alternative is less consistent:

  (async-start `(lambda () (copy-file ,from ,to ,ok-flag ,preserve-time))
               (lambda () (message "Copy done: %s -> %s" from to)))

Since `form' and `to' need their values passed to the child process, while the
variables remain in scope for the callback (thanks to closures).  Of course,
the <24 version is equally consistent in its ugliness:

  (async-start `(lambda () (copy-file ,from ,to ,ok-flag ,preserve-time))
               `(lambda () (message "Copy done: %s -> %s" ,from ,to)))

John



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-19 21:46     ` Stefan Monnier
  2012-06-20  0:20       ` John Wiegley
@ 2012-06-20  5:34       ` John Wiegley
  2012-06-20  7:24         ` SAKURAI Masashi
  2012-06-20 20:36         ` Thien-Thi Nguyen
  1 sibling, 2 replies; 24+ messages in thread
From: John Wiegley @ 2012-06-20  5:34 UTC (permalink / raw)
  To: emacs-devel

>>>>> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> The byte-compiler uses prin1 to generate the .elc files, so I'm pretty sure
> there's a way to solve this problem ;-)

Actually, I was being too pessimistic.  Byte-compiled forms are handled just
fine!

What I did run into is a bug communicating with the subprocess over stdin.
It happens if I use this function:

    (defun async-smtpmail-send-it ()
      (async-start
       `(lambda ()
          (require 'smtpmail)
          (with-temp-buffer
            (insert ,(buffer-substring-no-properties (point-min) (point-max)))
            ;; Pass in the variable environment for smtpmail
            ;;,(async-inject-variables
            ;;  "\\`\\(smtpmail\\|\\(user-\\)?mail\\)-"
            ;;  (lambda (symbol)
            ;;    (or (not (functionp sym))
            ;;        (symbolp sym)))
            ;;  "\\(\\`mail-header-format-function\\|-syntax-table\\|-mode-map\\)\\'")
            (setq mail-extr-trailing-comment-start-pattern '"[-{]\\|--\\|[+@#></;]")
            (setq smtpmail-debug-info 'nil)
            (setq mail-abbrev-mode-regexp
                  '"^\\(Resent-\\)?\\(To\\|From\\|CC\\|BCC\\|Reply-to\\):")
            (setq mail-send-actions 'nil)
            (setq mail-source-imap-streams
                  '((const gssapi)
                    (const kerberos4)
                    (const tls)
                    (const ssl)
                    (const network)
                    (const shell)
                    (const starttls)))
            (smtpmail-send-it))))
       'ignore))

All I see from the stdout of the child process is around 20 ^G characters, and
then it blocks waiting for input.  However, If I delete *any one* of the setq
statements above (it doesn't matter which), it works just fine.  So it doesn't
appear to be triggered by the length, or by any special characters.  I'm
having a tough time figuring this one out.

I've also implemented a message-passing scheme so one can implement status
updates for long-running async jobs, but I'm hesitant to enable it because it
relies on communicating over the pipe.

Until then, I pass the async lambda as a program argument, which means I'm
limited on maximum length to the OS limit (around 4k on Mac, I believe).

John



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-20  5:34       ` John Wiegley
@ 2012-06-20  7:24         ` SAKURAI Masashi
  2012-06-20  7:52           ` John Wiegley
  2012-06-20 20:36         ` Thien-Thi Nguyen
  1 sibling, 1 reply; 24+ messages in thread
From: SAKURAI Masashi @ 2012-06-20  7:24 UTC (permalink / raw)
  To: jwiegley; +Cc: emacs-devel

Hi John,

I'm interesting in your async solution.
I have written async codes, deferred.el and concurrent.el,
such as the green-thread in Emacs Lisp.

At Wed, 20 Jun 2012 00:34:46 -0500,
John Wiegley wrote:
>  :
> All I see from the stdout of the child process is around 20 ^G characters, and
> then it blocks waiting for input.  However, If I delete *any one* of the setq
> statements above (it doesn't matter which), it works just fine.  So it doesn't
> appear to be triggered by the length, or by any special characters.  I'm
> having a tough time figuring this one out.
> 
> I've also implemented a message-passing scheme so one can implement status
> updates for long-running async jobs, but I'm hesitant to enable it because it
> relies on communicating over the pipe.

For the long time jobs and fast communication with other processes, 
I have written a RPC stack on the Emacs: EPC, Emacs remote Procedure Call.

kiwanami/emacs-epc · GitHub
https://github.com/kiwanami/emacs-epc

Currently, I wrote the implementations for Emacs and Perl.
Both implementations can act as client and server, such as
Emacs client - Emacs server or Emacs client - Perl server.
The protocol is the S-exp and asynchronous messaging on the 
TCP socket, like SWANK in SLIME(http://common-lisp.net/project/slime/).

The EPC may be solve the communication with the child process.


I have not written the document for EPC yet, if some people
are interesting in it, I would write some document.

The EPC is used by Emacs DBI, the database management interface.

kiwanami/emacs-edbi · GitHub
https://github.com/kiwanami/emacs-edbi

Emacs DBI (Sorry for Japanese article)
http://d.hatena.ne.jp/kiwanami/20120305/1330939440


Thank you,
--
SAKURAI, Masashi (family, given)
m.sakurai@kiwanami.net



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-20  7:24         ` SAKURAI Masashi
@ 2012-06-20  7:52           ` John Wiegley
  2012-06-20  8:13             ` John Wiegley
  0 siblings, 1 reply; 24+ messages in thread
From: John Wiegley @ 2012-06-20  7:52 UTC (permalink / raw)
  To: SAKURAI Masashi; +Cc: emacs-devel

>>>>> SAKURAI Masashi <m.sakurai@kiwanami.net> writes:

> I'm interesting in your async solution.  I have written async codes,
> deferred.el and concurrent.el, such as the green-thread in Emacs Lisp.

I remember seeing you deferred.el a few weeks ago, but it seemed just
complicated enough that I shied away.  I wanted an interface that was
literally just one function large, `async-start', since that covers nearly all
of my real-world use cases.

> For the long time jobs and fast communication with other processes, I have
> written a RPC stack on the Emacs: EPC, Emacs remote Procedure Call.
> 
> The EPC may be solve the communication with the child process.
> 
> I have not written the document for EPC yet, if some people are interesting
> in it, I would write some document.

I'm interested to hear more about this.  Although I'd like to figure out why
pipe communication in Emacs appears to be broken, since that already available
on all platforms supporting start-process, and requires no other libraries to
be loaded.

> The EPC is used by Emacs DBI, the database management interface.

This is wonderful!  Thank you so much.  I've wanted a more visual way to query
SQL in Emacs for some time.

Thanks,
  John



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-20  7:52           ` John Wiegley
@ 2012-06-20  8:13             ` John Wiegley
  2012-06-20  8:24               ` John Wiegley
  2012-06-20  9:32               ` SAKURAI Masashi
  0 siblings, 2 replies; 24+ messages in thread
From: John Wiegley @ 2012-06-20  8:13 UTC (permalink / raw)
  To: emacs-devel

>>>>> John Wiegley <jwiegley@gmail.com> writes:

> I remember seeing you deferred.el a few weeks ago, but it seemed just
> complicated enough that I shied away.  I wanted an interface that was
> literally just one function large, `async-start', since that covers nearly
> all of my real-world use cases.

Looking further, I see that deferred.el and async.el address slightly
different problem domains.  For example, `async-start' creates another Emacs
in which to call the lambda, while deferred.el put the lambda on a timer and
runs it in the parent Emacs.  This means that if it's a heavy internal
function -- like a `copy-file' of 10 gigabytes -- it will still block Emacs,
am I right?

`deferred:process' looks rather nice, and I like how easily you can create a
"pipe" by chaining process calls.  I wish I had had your library while
building Eshell!  That would have made several things much more natural in the
code.

    (deferred:$
      (deferred:process "ls" "-la")
      (deferred:nextc it
        (lambda (x) (message x))))

This is the same as:

    (async-start-process "ls" "ls"
                         (lambda (proc)
                           (message (with-current-buffer (process-buffer proc)
                                      (buffer-string))))
                         "-la")

async.el does not use timers, though it seems that `deferred:process' does,
even though it also uses process sentinels.  Am I reading that correctly?

John



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-20  8:13             ` John Wiegley
@ 2012-06-20  8:24               ` John Wiegley
  2012-06-20  9:32               ` SAKURAI Masashi
  1 sibling, 0 replies; 24+ messages in thread
From: John Wiegley @ 2012-06-20  8:24 UTC (permalink / raw)
  To: emacs-devel

>>>>> John Wiegley <jwiegley@gmail.com> writes:

>     (async-start-process "ls" "ls"
>                          (lambda (proc)
>                            (message (with-current-buffer (process-buffer proc)
>                                       (buffer-string))))
>                          "-la")

Perhaps a better syntax would be:

    (async-start-process '("ls" "-la")
                         (lambda (proc)
                           (message (with-current-buffer (process-buffer proc)
                                      (buffer-string)))))

It doesn't follow `start-process', the way that I was doing, but it mirrors
`async-start' much better.

John



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-20  8:13             ` John Wiegley
  2012-06-20  8:24               ` John Wiegley
@ 2012-06-20  9:32               ` SAKURAI Masashi
  1 sibling, 0 replies; 24+ messages in thread
From: SAKURAI Masashi @ 2012-06-20  9:32 UTC (permalink / raw)
  To: jwiegley; +Cc: emacs-devel

At Wed, 20 Jun 2012 03:13:06 -0500,
John Wiegley wrote:
> :
> Looking further, I see that deferred.el and async.el address slightly
> different problem domains.  For example, `async-start' creates another Emacs
> in which to call the lambda, while deferred.el put the lambda on a timer and
> runs it in the parent Emacs.  This means that if it's a heavy internal
> function -- like a `copy-file' of 10 gigabytes -- it will still block Emacs,
> am I right?

Yes, you are right.
A heavy task should be breaked down into small tasks in the deferred.el manner.

deferred.el just supports job controlling on the Emacs and avoiding callback hell.
Changing notation for async tasks, the async tasks becomes composable.
I think the callback style (CPS) is difficult to write and read.

> async.el does not use timers, though it seems that `deferred:process' does,
> even though it also uses process sentinels.  Am I reading that correctly?

Yes.
deferred.el uses `run-at-time` function as a simple job scheduing.
In my experience, I think that the overhead is small enough.
I feel the blocking whole Emacs is more inconvenient.

--
SAKURAI, Masashi (family, given)
m.sakurai@kiwanami.net



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-20  5:34       ` John Wiegley
  2012-06-20  7:24         ` SAKURAI Masashi
@ 2012-06-20 20:36         ` Thien-Thi Nguyen
  2012-06-20 23:52           ` John Wiegley
  1 sibling, 1 reply; 24+ messages in thread
From: Thien-Thi Nguyen @ 2012-06-20 20:36 UTC (permalink / raw)
  To: emacs-devel

() John Wiegley <jwiegley@gmail.com>
() Wed, 20 Jun 2012 00:34:46 -0500

   However, If I delete *any one* of the setq statements above
   (it doesn't matter which), it works just fine.  So it doesn't
   appear to be triggered by the length,

Are you sure?  Deleting an element changes the length of the
message.  What happens if you "semantically delete" an element
while not perturbing (much) the aggregate message length by
quoting it?

   or by any special characters.  I'm having a tough time
   figuring this one out.

Playing w/ repls to subprocs, i ran into the (IIRC) 509 byte
message limit and discovered variable ‘process-connection-type’.
Maybe that is relevant here?



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-20 20:36         ` Thien-Thi Nguyen
@ 2012-06-20 23:52           ` John Wiegley
  2012-06-21 13:17             ` Thien-Thi Nguyen
  0 siblings, 1 reply; 24+ messages in thread
From: John Wiegley @ 2012-06-20 23:52 UTC (permalink / raw)
  To: emacs-devel

>>>>> Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> Playing w/ repls to subprocs, i ran into the (IIRC) 509 byte message limit
> and discovered variable ‘process-connection-type’.  Maybe that is relevant
> here?

Yes, you found it!  Setting process-connection-type to nil made it all work.

John



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-19  2:51 async.el: A simple asynchronous framework for Emacs John Wiegley
                   ` (2 preceding siblings ...)
  2012-06-19 21:38 ` Christopher Allan Webber
@ 2012-06-21  8:31 ` Thierry Volpiatto
       [not found]   ` <m2lijg8ibx.fsf@vulcan.local.i-did-not-set--mail-host-address--so-tickle-me>
  3 siblings, 1 reply; 24+ messages in thread
From: Thierry Volpiatto @ 2012-06-21  8:31 UTC (permalink / raw)
  To: emacs-devel

John Wiegley <jwiegley@gmail.com> writes:

> I've also written dired-async.el, which performs copies, moves and deletes
> asynchronously.  It works great with Tramp.
Nice, thanks for your package.
However you will need to inject some tramp/ange-ftp user variables in
other Emacs to make it fully working, at least:

tramp-default-user-alist
ange-ftp-passive-host-alist

Otherwise it will not work when copying on some ftp servers.

(I am also using emacs-batch in helm to copy async and I have same
problem, not fixed yet)

-- 
  Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-20 23:52           ` John Wiegley
@ 2012-06-21 13:17             ` Thien-Thi Nguyen
  2012-06-22 16:01               ` Le Wang
  0 siblings, 1 reply; 24+ messages in thread
From: Thien-Thi Nguyen @ 2012-06-21 13:17 UTC (permalink / raw)
  To: emacs-devel

() John Wiegley <jwiegley@gmail.com>
() Wed, 20 Jun 2012 18:52:49 -0500

   Yes, you found it!  Setting process-connection-type to nil made it all work.

Glad to hear it.  I wonder why the default value is not nil.



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-21 13:17             ` Thien-Thi Nguyen
@ 2012-06-22 16:01               ` Le Wang
  2012-06-22 20:07                 ` Thien-Thi Nguyen
  0 siblings, 1 reply; 24+ messages in thread
From: Le Wang @ 2012-06-22 16:01 UTC (permalink / raw)
  To: Thien-Thi Nguyen; +Cc: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 423 bytes --]

On Thu, Jun 21, 2012 at 9:17 PM, Thien-Thi Nguyen <ttn@gnuvola.org> wrote:

> () John Wiegley <jwiegley@gmail.com>
> () Wed, 20 Jun 2012 18:52:49 -0500
>
>   Yes, you found it!  Setting process-connection-type to nil made it all
> work.
>
> Glad to hear it.  I wonder why the default value is not nil.
>

This is off-topic, but what a wonderful tip!  I was seeing the "^G" issue
in my customizations (also on Mac).

-- 
Le

[-- Attachment #2: Type: text/html, Size: 779 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-22 16:01               ` Le Wang
@ 2012-06-22 20:07                 ` Thien-Thi Nguyen
  2012-06-24  9:30                   ` Chong Yidong
  0 siblings, 1 reply; 24+ messages in thread
From: Thien-Thi Nguyen @ 2012-06-22 20:07 UTC (permalink / raw)
  To: Le Wang; +Cc: emacs-devel

() Le Wang <l26wang@gmail.com>
() Sat, 23 Jun 2012 00:01:47 +0800

   This is off-topic, but what a wonderful tip!

I see in (info "(elisp) Asynchronous Processes")
the introductory blurb:

   Here we describe how to create an asynchronous process.

and then, below, for ‘process-connection-type’, the paragraph:

     PTYs are usually preferable for processes visible to the
     user, as in Shell mode, because they allow job control
     (`C-c', `C-z', etc.) to work between the process and its
     children, whereas pipes do not.  For subprocesses used for
     internal purposes by programs, it is often better to use a
     pipe, because they are more efficient.  In addition, the
     total number of PTYs is limited on many systems and it is
     good not to waste them.

Probably this could be moved into the intro blurb to aid its
discovery, and furthermore modified to mention correctness as well
(and not only efficiency) for large (for a PTY) messages.  Another
idea is to move the (one) variable doc before the function docs,
but that would be a jolt in the style.  For example:

   Here we describe how to create an asynchronous process,
   including a variable that controls whether PTYs or pipes should
   be used at the time of the actual subprocess creation.

   PTYs are usually preferable for processes visible to the user,
   as in Shell mode, because they allow job control (`C-c', `C-z',
   etc.) to work between the process and its children, whereas
   pipes do not.  For subprocesses used for internal purposes by
   programs, it is often better to use a pipe, because they are
   more efficient, and immune to stray character injections that
   PTYs use for large (ca. 500 byte) messages.  In addition, the
   total number of PTYs is limited on many systems and it is good
   not to waste them.

Not so svelte (needs work), but maybe Someone will be inspired...



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-22 20:07                 ` Thien-Thi Nguyen
@ 2012-06-24  9:30                   ` Chong Yidong
  0 siblings, 0 replies; 24+ messages in thread
From: Chong Yidong @ 2012-06-24  9:30 UTC (permalink / raw)
  To: Thien-Thi Nguyen; +Cc: emacs-devel, Le Wang

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> Probably this could be moved into the intro blurb to aid its
> discovery, and furthermore modified to mention correctness as well
> (and not only efficiency) for large (for a PTY) messages.  Another
> idea is to move the (one) variable doc before the function docs,
> but that would be a jolt in the style.  For example:

Thanks.  I committed a change along the lines you suggested.



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
       [not found]             ` <87395lyam9.fsf@gmail.com>
@ 2012-06-26  0:05               ` John Wiegley
  2012-06-26  9:41                 ` Thierry Volpiatto
  0 siblings, 1 reply; 24+ messages in thread
From: John Wiegley @ 2012-06-26  0:05 UTC (permalink / raw)
  To: emacs-devel

>>>>> Thierry Volpiatto <thierry.volpiatto@gmail.com> writes:

> I must use the same form in many places for that:
>      (let ((buf (loop for b in (buffer-list)
>                       thereis (and (string-match
>                                     "\\`\\*ftp.*"
>                                     (buffer-name b)) b))))
>        (when buf (kill-buffer buf))))
>   callback))
> maybe you have an idea to make a function that return the form one time
> for all and avoid these duplicates.

I'd probably just make a function that contains your `loop' usage.

>> This is no longer necessary in the latest version of dired-async.  When an
>> async operations begin, the background of each related line is made yellow.
>> As the operations finish, the yellow is removed.
> This is true only for dired, I do not use dired, only helm, so I need a
> message to tell me when operation finish.

Ah, I understand now!

>> I would not be opposed, however, to a single async message which says,
>> (format "All asynchronous %s(s) have completed" operation).
> It is a single message that popup only once when all is finish.  So for 500
> files, it popup only at the end:

> Asynchronous copy of 500 file(s) done

> What would be great is customizing the function to use for showing message,
> using `message' by default.  One could use something else like
> `tooltip-show' or function using external program like notify-send or like.

Gotcha, this is a necessary configuration detail.  I'll add it.

John



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: async.el: A simple asynchronous framework for Emacs
  2012-06-26  0:05               ` John Wiegley
@ 2012-06-26  9:41                 ` Thierry Volpiatto
  0 siblings, 0 replies; 24+ messages in thread
From: Thierry Volpiatto @ 2012-06-26  9:41 UTC (permalink / raw)
  To: emacs-devel

John Wiegley <jwiegley@gmail.com> writes:

> I'd probably just make a function that contains your `loop' usage.
Ok done.

> Gotcha, this is a necessary configuration detail.  I'll add it.
Done also, see my branch.
I use by default a function that show message in mode-line.

-- 
  Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 




^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2012-06-26  9:41 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-06-19  2:51 async.el: A simple asynchronous framework for Emacs John Wiegley
2012-06-19  5:11 ` Thien-Thi Nguyen
2012-06-19  5:36   ` John Wiegley
2012-06-19 13:29 ` Stefan Monnier
2012-06-19 20:44   ` John Wiegley
2012-06-19 21:46     ` Stefan Monnier
2012-06-20  0:20       ` John Wiegley
2012-06-20  0:53         ` John Wiegley
2012-06-20  5:34       ` John Wiegley
2012-06-20  7:24         ` SAKURAI Masashi
2012-06-20  7:52           ` John Wiegley
2012-06-20  8:13             ` John Wiegley
2012-06-20  8:24               ` John Wiegley
2012-06-20  9:32               ` SAKURAI Masashi
2012-06-20 20:36         ` Thien-Thi Nguyen
2012-06-20 23:52           ` John Wiegley
2012-06-21 13:17             ` Thien-Thi Nguyen
2012-06-22 16:01               ` Le Wang
2012-06-22 20:07                 ` Thien-Thi Nguyen
2012-06-24  9:30                   ` Chong Yidong
2012-06-19 21:38 ` Christopher Allan Webber
2012-06-21  8:31 ` Thierry Volpiatto
     [not found]   ` <m2lijg8ibx.fsf@vulcan.local.i-did-not-set--mail-host-address--so-tickle-me>
     [not found]     ` <87a9zv1fw4.fsf@gmail.com>
     [not found]       ` <m2ehp73t8o.fsf@vulcan.local.i-did-not-set--mail-host-address--so-tickle-me>
     [not found]         ` <871ul63yky.fsf@gmail.com>
     [not found]           ` <m2zk7tzr5e.fsf@vulcan.local.i-did-not-set--mail-host-address--so-tickle-me>
     [not found]             ` <87395lyam9.fsf@gmail.com>
2012-06-26  0:05               ` John Wiegley
2012-06-26  9:41                 ` Thierry Volpiatto

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.