Thread interface issues (was Re: C level mutices and condition variables)

[Mikael Djurfeldt <mdj@kvast.blakulla.net>]

I've talked a little with Marius about this over telephone.  He asked
me to give a summary of our discussion on the list.  I'll try to
do that, but don't promise to be fair and will probably miss a lot.
(Also, Marius, there is an additional argument regarding the
"combinatorial explosion" which I've marked with ***.)

NOTE: Unfortunately, I'm *very* short of time right now, so I don't
promise to continue discussing this issue.  Therefore it's important
that those of you who support my view don't expect me to defend it (if
it needs fruther defence) but instead do that yourselves.

We talked over four connected issues:

Q1. How are we going to choose whether to use threads and which thread
    package to use?  Compile-time, link-time or run-time?

Q2. Should Guile have a "plug-in" thread interface a la glib or should
    we have multiple versions of libguile installed on the system?

Q3. Should Guile have a C level thread API?  (The present guile-devel
    news-thread arrived at the answer "no", but in my opinion the
    issue needs further thought.)

Q4. How should the C level API be implemented?  Scheme level
    implemented on top of C level or the reverse?

----------------------------------------------------------------------

Q1. How are we going to choose whether to use threads and which thread
    package to use?  Compile-time, link-time or run-time?

Currently, we make a compile-time choice of one of three thread
packages:

  I. null-threads (on systems which don't support threads or where the
     sysadmin don't want threads in Guile)

 II. coop-threads (a qthreads based cooperative thread package)

III. coop-pthreads (a pthreads based cooperative thread package)

Is it right to make this choice at libguile compile-time or should it
be at application link time or even run-time?  If run-time, should
individual Guile modules be able to independently choose which thread
library to use?

Our answer should be arrived at under consideration of our vision for
future development, which Marius and I both think is full concurrency
using pthreads.  We basically have the Stalinist view that it would be
good for the whole world to use the pthreads interface (arguments
similar to those for using libc: portability, the possibility to get
kernel-level optimizations etc).  So, it might be that we in the
future drop II and III in favour of (or keep II and III and add):

 IV. pthreads (a preemptive thread package)

Arguments for/against compile-time choice:

+ Simple.

+ Enables compile-time optimizations.  In a pthreads version, we could
  remove logic required for cooperative threading while cooperative
  thread libraries don't require mutecis for protection of common
  resources.

- The sysadmin and application developer might be different people.
  If the sysadmin has installed Guile without thread support, an
  application developer (or user for that matter!) will be in deep
  trouble if the application needs thread support.  This situation is
  unreasonable but could be resolved if we install multiple versions
  of libguile, one for each thread package.

Arguments for link-time choice:

(Link-time in this context means deciding which thread package to use
once, at initialization of libguile.)

+ Moves the choice of thread library to the application developer
  where it should be.

+ Reasonably simple.  Additional requirements are only:

   1. a plug-in interface (a struct with pointers to the thread
      library functions)

   2. glue code

Arguments against run-time choice:

- Unreasonably complex.

- Would require multiple types of mutecis and condition variables or
  overly complex united forms.  In this way the complexity intrudes
  also semantically.

I think both Marius and I agree that the only reasonable answers to Q1
are:

* link-time choice with one libguile

or

* compile-time choice with multiple libguiles (libguile,
  libguile-coop, libguile-copt etc)

This results in Q2:

----------------------------------------------------------------------

Q2. Should Guile have a "plug-in" thread interface a la glib or should
    we have multiple versions of libguile installed on the system?

An example of such a plug-in interface is included below.

Arguments for/against plug-in interface:

+ Gives true modularity by decoupling libguile from the thread
  library.

+ Gives the application developer the freedom to use other thread
  packages with Guile, including experimenting with new thread
  libraries.

  Apart from choosing between COOP threads, COOP-pthreads and
  pthreads, there might be other thread packages out there which the
  application developer might prefer.  For example, the Debian
  distribution provides State threads (libst1), LWP threads
  (liblwp2) and GNU Portable Threads (libpth; a cooperative thread
  library).

  The reason why this is a significant point is that it can be
  difficult to make different thread libraries play well together in
  the same application.  If Guile uses X-threads, the application
  usually runs into trouble if it uses Y-threads.

- (Probably insignificant) overhead due to one pointer indirection.

- Puts a restriction on the thread library capabilities that we can
  use.  (We can't continue to extend this API ad infinitum but must
  decide on a basic set of calls that we think most thread libraries
  can implement.  But we may also use the strategy of GLib: that some
  of the calls are optional and can be initialized to NULL.)

- Requires dynamic linking with an additional glue-code library if the
  thread package isn't directly written for Guile.  Such a glue-code
  library initializes the plug-in interface and provides possibly
  required extra support code.

Arguments for/against multiple versions of libguile:

+ Allows for compile-time optimizations (see Q1: compile-time choice).

- Risk for combinatorial explosion of libguile versions due to
  compilation options.

- *** Risk for combinatorial explosion of libguile support library
  versions.

  Note that we can't just draw the distinction between libguile on one
  hand which supports the application on the other hand.  We also have
  libraries like libguile-gtk which have the double role of being
  application of libguile and providing application framework for user
  applications.  While having multiple version of libguile may be
  reasonable, having, in addition, multiple versions of libguile-gtk
  et al is unreasonable.

  A partial solution is to introduce an abstraction which separates
  guile code + semi-guile code like libguile-gtk from the particular
  thread library being used so that, for example, a single
  libguile-gtk can support all versions of libguile.

  The Scheme-level thread interface provides such an abstraction so
  that Guile scheme modules are independent of the particular thread
  packages being used.  The subset of primitive functions in this
  interface can provide a C level abstraction for loadable C-level
  Guile extensions, but we could also provide a C level thread API.
  In fact, such a C level thread API has been in place since
  1998-01-30 (guile-1.2) but has just been deprecated.

  This brings us to Q3:

----------------------------------------------------------------------

Q3. Should Guile have a C level thread API?

While an application should be free to use whatever thread package it
wants by direct calls to its API, points discussed above shows that
Guile Scheme and C modules should use an abstraction in order be
independent of the thread package being used.  The C level could
either use the Scheme primitives, like this:

   SCM my_mutex = scm_make_mutex ();
   scm_lock_mutex (my_mutex);

However, for thread creation, we would still probably want to use
scm_spawn_thread, so this alterntive is rather a _partial_ C level
thread API.

In a full C level thread API, one would be able to:

   scm_t_mutex my_mutex;
   scm_mutex_init (&my_mutex);
   scm_mutex_lock (&my_mutex);

(Although, as Marius has pointed out, scm_t_mutex is a bit rigid,
since it requires knowledge of mutex size at compile-time.  In the
Glib thread interface and in the plug-in interface proposed below,
scm_mutex_init is replaced with a mutex constructor function.)

Arguments for/against C level thread API:

+ Easy programming in C.

  A major goal of Guile is to be an extension language for
  applications written in C.  Because of this, we try to provide a
  nice environment not only for Scheme but also for C programmers.

  If we do provide a C API, it should be similar to the pthread API.
  In this case, the C programmer will feel very familiar with the
  interface and it's easy to convert code written for the pthread
  API.

  If we don't provide a C API, the code needs to deal with GC issues.
  For example, it is fairly common to store mutecis in dynamic
  application datastructures.  If we go with Scheme API only, the
  application needs to make sure the mutex is GC protected.

+ Efficiency.

  In some cases, a thread package needs to be very efficient.  Thread
  package developers make great efforts to reduce overhead.  Even the
  need to switch from user space to system space is sometimes
  considered as causing significant overhead, so there are alternative
  thread libraries which run entirely in user space.

  For such applications where this matters, it seems wasteful to go by
  the indirections of the SCM data type and do unnecessary type checks
  when calling Scheme primitives from C.  Note that the C level
  interface is not only intended for the direct cooperation with
  Scheme code, in which overhead may be of less significance, but is
  also intended for time-critical C code in a loadable Guile extension
  module.

  (This point is only valid together with the Scheme on top of C
   alternative in Q4 below.)

+ Deprecation will break applications.

  The current C level thread API has been in place since 1998-01-30
  (guile-1.2) but has just been deprecated.

  Maybe only I have been using this API in several of my applications,
  but it doesn't seem nice towards application developers to remove
  such a useful feature when there is no simple replacement,
  especially when it's been in place for so long.

- Demands extra work for Guile developers to maintain it.

(Missing some - points?  They might rather belong to Q2 or Q4 and
 could be listen there.)

----------------------------------------------------------------------

Q4. How should the C level API be implemented?  Scheme level
    implemented on top of C level or reverse?

Scheme level on top of C level means that Scheme primitives call the C
level API for core thread functionality.

C level on top of Scheme level means that the Scheme primitives
implement the core thread library functionality directly and that the
C level interface is an adapter on top of this to facilitate coding
for C programmers.  (An example is provided by the currently
deprecated C level API for coop-pthreads.)

Just as regarding Q3, this may be a matter of coding philosophy:

Either one aims for a slim C level solution or one wants to take
advantage of the advanced facilities of a dynamically typed language
and is prepared to pay for it.

Arguments for Scheme level implemented on top of C level:

+ More "natural"

  Results in a natural layering of the code.  The reverse will, in
  fact, be C - Scheme - C.

Arguments for C level implemented on top of Scheme level:

+ Easier to use Scheme level facilities when implementing the thread
  packages.  For example, we might decide, at a later stage, to add
  some debugging facilities to the thread support.  We might prefer to
  write this in Scheme code.  With C implemented on top of Scheme
  level this seems easier than for the reverse.  Also, in implementing
  certain thread functionality, we might want to use Scheme facilities
  such as SCM lists.  If we have an SCM list in a mutex, how will it
  be GC protected if the mutex resides under the C API?

- We won't want to do that anyway in the envisioned pthreads
  implementation

  Since we're envisioning Guile using pthreads with full concurrency
  as a future standard thread package for Guile, it is likely that
  we won't need to implement much core thread functionality.  It's
  more likely that we just want to wrap pthreads as Scheme primitives
  which seems to harmonize better with the Scheme on top of C
  alternatives.

- Overhead

  Overhead problems will be even greater than those we'd have if we
  don't provide a C API at all.

----------------------------------------------------------------------

OK, was it difficult to figure out what I want to see in Guile? ;-)

If so, I want:

* A plug-in interface which decouples libguile from the thread
  library and allows the application developer to choose which thread
  library to use with Guile.  Preferably it should support both
  cooperative and preemptive scheduling.

* An efficient and pthreads-like C level API.  The plug-in interface
  could serve this role in addition to the decoupling role.

* Scheme primitives implemented using the low-level C API.

If you've read this far you have my respect!

Good luck with decisions and implementation,

Mikael

----------------------------------------------------------------------

Proposed plug-in interface structure:

struct scm_threads {
  /* Threads */
  scm_thread_t (*make_thread) (scm_threadattr_t *attr,
			       void * (*start_routine) (void *),
			       void *arg);
  void (*exit) (void *retval);
  int (*cancel) (scm_thread_t thread);
  int (*join) (scm_thread_t thread, void **retval);
  size_t (*free) (scm_thread_t thread); /* returns freed amount of memory */
  
  /* Cooperative threads */
  void (*yield) (void); /* optional */
  
  /* Mutices */
  scm_mutex_t * (*make_mutex) (const scm_mutexattr_t *mutexattr);
  int (*mutex_lock) (scm_mutex_t *mutex);
  int (*mutex_trylock) (scm_mutex_t *mutex);
  int (*mutex_unlock) (scm_mutex_t *mutex);
  size_t (*mutex_free) (scm_mutex_t *mutex); /* returns freed amount of memory */
  
  /* Condition variables */
  scm_cond_t * (*make_cond) (scm_condattr_t *cond_attr);
  int (*cond_signal) (scm_cond_t *cond);
  int (*cond_broadcast) (scm_cond_t *cond);
  int (*cond_wait) (scm_cond_t *cond, scm_mutex_t *mutex);
  int (*cond_timedwait) (scm_cond_t *cond,
			 scm_mutex_t *mutex,
			 const struct timespec *abstime);
  size_t (*cond_free) (scm_cond_t *cond); /* returns freed amount of memory */
  
  /* Keys */
  int (*key_create) (scm_key_t *key, void (*destr_function) (void *));
  int (*key_delete) (scm_key_t key);
  int (*setspecific) (scm_key_t key, const void *pointer);
  void * (*getspecific) (scm_key_t key);

  /* Local data
     
     This is used by Guile to store the root structure.
     It could be a custom data field in the thread data structure
     or a dedicated key.
  */
  void (*set_data) (void *data);
  void * (*data) (void);
  
  /* GC */
  void (*mark_stacks) (void);
  
  /* Parameters */
  void (*set_stack_size) (size_t size);
};


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel