From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Jan Moringen Newsgroups: gmane.emacs.devel Subject: RFC: A framework for task management and execution in Emacs Date: Tue, 13 Jul 2010 09:02:21 +0200 Message-ID: <22158_1279004543_ZZh033Ked6tCc.00_1279004541.2415.244.camel@steed.robot-madness> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7BIT X-Trace: dough.gmane.org 1279004723 31734 80.91.229.12 (13 Jul 2010 07:05:23 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Tue, 13 Jul 2010 07:05:23 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Jul 13 09:05:20 2010 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1OYZYO-0006D7-4s for ged-emacs-devel@m.gmane.org; Tue, 13 Jul 2010 09:05:20 +0200 Original-Received: from localhost ([127.0.0.1]:51621 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OYZYN-0004gE-Ck for ged-emacs-devel@m.gmane.org; Tue, 13 Jul 2010 03:05:19 -0400 Original-Received: from [140.186.70.92] (port=32978 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OYZVc-0003Po-Jr for emacs-devel@gnu.org; Tue, 13 Jul 2010 03:02:30 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OYZVa-00058f-U0 for emacs-devel@gnu.org; Tue, 13 Jul 2010 03:02:28 -0400 Original-Received: from mux2-unibi-smtp.hrz.uni-bielefeld.de ([129.70.204.73]:47846) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OYZVa-00058F-Kp for emacs-devel@gnu.org; Tue, 13 Jul 2010 03:02:26 -0400 Original-Received: from pmxchannel-daemon.mux2-unibi-smtp.hrz.uni-bielefeld.de by mux2-unibi-smtp.hrz.uni-bielefeld.de (Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 32bit)) id <0L5H00F00I7Z4E00@mux2-unibi-smtp.hrz.uni-bielefeld.de> for emacs-devel@gnu.org; Tue, 13 Jul 2010 09:02:23 +0200 (CEST) Original-Received: from [192.168.2.100] ([92.39.24.105]) by mux2-unibi-smtp.hrz.uni-bielefeld.de (Sun Java(tm) System Messaging Server 6.3-6.03 (built Mar 14 2008; 32bit)) with ESMTPPSA id <0L5H009YAI7YT470@mux2-unibi-smtp.hrz.uni-bielefeld.de> for emacs-devel@gnu.org; Tue, 13 Jul 2010 09:02:23 +0200 (CEST) X-Mailer: Evolution 2.30.2 X-EnvFrom: jan.moringen@uni-bielefeld.de X-PMX-Version: 5.5.9.395186, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2010.7.13.65117, pmx9 X-Connecting-IP: 92.39.24.105 X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:127149 Archived-At: Hi, I would like to start a discussion regarding a topic that has bothered me for some time now: management and execution of (long-running) tasks in Emacs. Instead of properly explaining what I mean by "task", I will just paste a table from my notes that has several examples: | Component | Execution | Progress Indication | Cancelable |--------------+--------------------+---------------------+----------- | Tramp | synchronous | progress reporter | no? | VC | sync, async | mode-line | sometimes | Gnus | synchronous | ? | ? | Compilation | asynchronous | mode-line | yes | URL | synchronous, async | progress reporter | ? | Emacs Jabber | timers, fsm | ? | ? | SemanticDB | idle-timers | custom reporter | on input Each of these packages performs some kinds of potentially long running operations. Originally, the table was intended to illustrate the heterogeneous nature of solutions despite similar requirements and use-cases. The (hopefully) upcoming thread support will probably make the situation better and worse. Better, because long-running operations could "simply" run in their own threads, worse because actually implementing this is usually not that simple. The number and complexity of solutions for task execution would probably grow further once threads become available. My proposal is to introduce a set of mechanisms that make task execution more uniform for the user and easier for programmers. Here is the full set of requirements/aspects I could come up with: + Scheduling/Execution + Resource Allocation (threads) + Synchronization (order of execution, retrieval of results) + Canceling + User Interface + Graphical Indicators + Progress/remaining Time Estimation + Error Reporting + Desktop Integration + Power Management inhibition + Desktop-wide Task Management (Example: http://ssickert.wordpress.com/2010/05/09/introducing-my-gsoc project/) + Customization + Compatibility + Portability + Backwards Compatibility Since there is so much potential for code duplication, reinventing the wheel and divergent user interfaces, I think all of these issue should be addressed in a general way. Other Environments such as Eclipse or the Evolution mail client seem to employ such generic mechanisms since their user interfaces contain lists of currently running tasks, which can also be interacted with. At this point, my general question is whether there are any plans or related efforts regarding this topic. Of course, I would also like to know whether people consider the approach worthwhile at all :) The second part of this message is about a small prototype framework I made to explore the issue practically. The framework is structured like this: Interface Layer +--------------------------------+-----------------------------------+ | | | | User Interface | Macros | | | | +--------------------------------+-----------------------------------+ Backend Layer +------------------------------+-------------------------------------+ | | | | Tasks +---+ Execution | | +-----------+ +-----------+ | +-----------+ +-----------+ | | | Strategy1 | ... | StrategyN | | | Strategy1 | ... | StrategyN | | | +-----------+ +-----------+ | +-----------+ +-----------+ | +----------------------------------+---------------------------------+ These components are discussed below: *Macros* Basically just two macros with identical syntax: ({do,run}-as-task NAME (TASK-SPEC) (EXECUTION-SPEC) [DOCSTRING] BODY) The difference being that the do- macro returns the result of BODY when it is ready while the run- variant immediately returns an object from which the result of BODY can be retrieved later. Examples: (do-as-task add-numbers () () "Add some numbers." (dolist (i (number-sequence 1 1000)) (+ 1 2 3) 'my-result)) + Blocks immediately + Body may or may not start executing immediately + Returns when the body finishes + Returns the value returned by the body (let ((delayed (run-as-task add-numbers () () "Add some numbers." (dolist (i (number-sequence 1 1000)) (+ 1 2 3) 'my-result)))) ;; do other stuff (future-get delayed)) + Does not block + Body may or may not start executing immediately + Returns immediately + Returns an object that implements a "future" protocol + Result of body can be retrieved from returned object *Tasks Strategies* >From the commentary section: ;; What a task is ;; + What gets executed ;; + Execution context (functions callable from task code) ;; + Meta information (name, description) ;; What a task is not ;; + how to execute (synchronous vs. asynchronous) ;; + when to execute (delayed, lazy etc) (do-as-task add-numbers (progress) () "Add some numbers, reporting progress." (dolist (i (number-sequence 1 1000)) (+ 1 2 3) (progress i)) 'my-result) (do-as-task add-numbers (cancelable) () "Add some numbers, cancelable." (dolist (i (number-sequence 1 1000)) (+ 1 2 3) (maybe-cancel)) 'my-result) (do-as-task add-numbers (progress cancelable) () "Add some numbers, reporting progress and cancelable." (dolist (i (number-sequence 1 1000)) (+ 1 2 3) (progress i) (maybe-cancel)) 'my-result) *Execution Strategies* These control how and when a task is executed. The currently available executions are (:thread is just a teaser, of course): (run-as-task add-numbers () (:execution blocking) (run-as-task add-numbers () (:execution idle) (run-as-task add-numbers () (:execution (timeout :delay 5)) (run-as-task add-numbers () (:execution thread)) *User Interface* There is also user interface code, but discussing it here would probably provide little insight. The code of the framework described above is available at http://bazaar.launchpad.net/~scymtym/etasks/trunk/ Feedback and suggestions would be greatly appreciated. Kind regards, Jan