Jan Moringen writes: > Hi, > > I would like to start a discussion regarding a topic that has bothered > me for some time now: management and execution of (long-running) tasks > in Emacs. Instead of properly explaining what I mean by "task", I will > just paste a table from my notes that has several examples: > > | Component | Execution | Progress Indication | Cancelable > |--------------+--------------------+---------------------+----------- > | Tramp | synchronous | progress reporter | no? > | VC | sync, async | mode-line | sometimes > | Gnus | synchronous | ? | ? > | Compilation | asynchronous | mode-line | yes > | URL | synchronous, async | progress reporter | ? > | Emacs Jabber | timers, fsm | ? | ? > | SemanticDB | idle-timers | custom reporter | on input > > Each of these packages performs some kinds of potentially long running > operations. Originally, the table was intended to illustrate the > heterogeneous nature of solutions despite similar requirements and > use-cases. > > The (hopefully) upcoming thread support will probably make the situation > better and worse. Better, because long-running operations could "simply" > run in their own threads, worse because actually implementing this is > usually not that simple. The number and complexity of solutions for task > execution would probably grow further once threads become available. > > My proposal is to introduce a set of mechanisms that make task execution > more uniform for the user and easier for programmers. Here is the full > set of requirements/aspects I could come up with: > > + Scheduling/Execution > + Resource Allocation (threads) > + Synchronization (order of execution, retrieval of results) > + Canceling > + User Interface > + Graphical Indicators > + Progress/remaining Time Estimation > + Error Reporting > + Desktop Integration > + Power Management inhibition > + Desktop-wide Task Management > (Example: > http://ssickert.wordpress.com/2010/05/09/introducing-my-gsoc project/) > + Customization > + Compatibility > + Portability > + Backwards Compatibility > > Since there is so much potential for code duplication, reinventing the > wheel and divergent user interfaces, I think all of these issue should > be addressed in a general way. > > Other Environments such as Eclipse or the Evolution mail client seem to > employ such generic mechanisms since their user interfaces contain lists > of currently running tasks, which can also be interacted with. > > At this point, my general question is whether there are any plans or > related efforts regarding this topic. Of course, I would also like to > know whether people consider the approach worthwhile at all :) > > The second part of this message is about a small prototype framework I > made to explore the issue practically. > > The framework is structured like this: > > Interface Layer > +--------------------------------+-----------------------------------+ > | | | > | User Interface | Macros | > | | | > +--------------------------------+-----------------------------------+ > Backend Layer > +------------------------------+-------------------------------------+ > | | | > | Tasks +---+ Execution | > | +-----------+ +-----------+ | +-----------+ +-----------+ | > | | Strategy1 | ... | StrategyN | | | Strategy1 | ... | StrategyN | | > | +-----------+ +-----------+ | +-----------+ +-----------+ | > +----------------------------------+---------------------------------+ > > These components are discussed below: > > *Macros* > > Basically just two macros with identical syntax: > > ({do,run}-as-task NAME (TASK-SPEC) (EXECUTION-SPEC) > [DOCSTRING] > BODY) > > The difference being that the do- macro returns the result of BODY when > it is ready while the run- variant immediately returns an object from > which the result of BODY can be retrieved later. > > Examples: > > (do-as-task add-numbers () () > "Add some numbers." > (dolist (i (number-sequence 1 1000)) > (+ 1 2 3) > 'my-result)) > > + Blocks immediately > + Body may or may not start executing immediately > + Returns when the body finishes > + Returns the value returned by the body > > (let ((delayed (run-as-task add-numbers () () > "Add some numbers." > (dolist (i (number-sequence 1 1000)) > (+ 1 2 3) > 'my-result)))) > ;; do other stuff > (future-get delayed)) > > + Does not block > + Body may or may not start executing immediately > + Returns immediately > + Returns an object that implements a "future" protocol > + Result of body can be retrieved from returned object > > *Tasks Strategies* > >>From the commentary section: > > ;; What a task is > ;; + What gets executed > ;; + Execution context (functions callable from task code) > ;; + Meta information (name, description) > ;; What a task is not > ;; + how to execute (synchronous vs. asynchronous) > ;; + when to execute (delayed, lazy etc) > > (do-as-task add-numbers (progress) () > "Add some numbers, reporting progress." > (dolist (i (number-sequence 1 1000)) > (+ 1 2 3) > (progress i)) > 'my-result) > > (do-as-task add-numbers (cancelable) () > "Add some numbers, cancelable." > (dolist (i (number-sequence 1 1000)) > (+ 1 2 3) > (maybe-cancel)) > 'my-result) > > (do-as-task add-numbers (progress cancelable) () > "Add some numbers, reporting progress and cancelable." > (dolist (i (number-sequence 1 1000)) > (+ 1 2 3) > (progress i) > (maybe-cancel)) > 'my-result) > > *Execution Strategies* > > These control how and when a task is executed. The currently available > executions are (:thread is just a teaser, of course): > > (run-as-task add-numbers () (:execution blocking) > > (run-as-task add-numbers () (:execution idle) > > (run-as-task add-numbers () (:execution (timeout :delay 5)) > > (run-as-task add-numbers () (:execution thread)) > > *User Interface* > > There is also user interface code, but discussing it here would probably > provide little insight. > > The code of the framework described above is available at > > http://bazaar.launchpad.net/~scymtym/etasks/trunk/ > > Feedback and suggestions would be greatly appreciated. Sounds very interesting. I have a scanner package called "emsane" for which I developed "emsane-postop". This is basically an event driven transaction queue. you push operations on a queue, and when an event happens, the operation is popped and executed. If the the operation fails, the transaction fails. Does this fit in your framework? I dont have anywhere public to place the code at yet, so I attach the file instead, so you can have a look.