Automatic value conversion in DBus bindings

Automatic value conversion in DBus bindings

[Jan Moringen]

[-- Attachment #1: Type: text/plain, Size: 1850 bytes --]

While playing around with the DBus bindings, I came across some problems
related to the automatic conversion between LISP and DBus values.

The first problem can be demonstrated by running the following code:

  (require 'dbus)

  (defun test ()
    nil)

  (dbus-register-method
   :session "org.gnu.emacs" "/org/gnu/emacs" "org.gnu.Emacs" "test"
   #'test)

  (dbus-call-method-non-blocking
   :session "org.gnu.emacs" "/org/gnu/emacs" "org.gnu.Emacs" "test")

The last call never returns. The problem is in
`dbus-call-method-non-blocking':

  (while (not (gethash key dbus-return-values-table nil))
    (read-event nil nil 0.1))

Here, the return value nil cannot be distinguished from "no return value
yet". A suggestion for a fix is in the attached patch.

I am writing to emacs-devel rather than reporting a bug because I think
the fix for this particular problem should be accompanied by a small
change of the DBus interface. The original motivation for my experiments
was writing a function that should receive DBus calls but not return a
value. This was necessary since another process (Empathy) would call the
method via DBus and complain if it returned anything.

The current LISP -> DBus conversion rules make it impossible (from what
I understand at least) to not return a value, since an empty list, which
would represent zero return values, is interpreted as a single 'false'
value. I think this special treatment of values that are null or not
lists should be removed, thus always requiring a list as the return
value of a DBus-invokable function. This change is also included in the
attached patch. Note that the documentation would need corresponding
changes.

Since the proposed changes could break existing software, a different
approach may be better. I would be glad to here opinions on the subject.

Kind regards,
Jan Moringen

[-- Attachment #2: emacs-dbus-return-values.patch --]
[-- Type: text/x-patch, Size: 1253 bytes --]

--- lisp/net/dbus.el.orig	2009-01-28 17:10:02.000000000 +0100
+++ list/net/dbus.el	2009-07-27 03:18:10.429481853 +0200
@@ -162,7 +162,7 @@
 The result will be made available in `dbus-return-values-table'."
   (puthash (list (dbus-event-bus-name last-input-event)
 		 (dbus-event-serial-number last-input-event))
-	   (if (= (length args) 1) (car args) args)
+	   args
 	   dbus-return-values-table))
 
 (defun dbus-call-method-non-blocking
@@ -184,8 +184,9 @@
 	  'dbus-call-method-non-blocking-handler args)))
     ;; Wait until `dbus-call-method-non-blocking-handler' has put the
     ;; result into `dbus-return-values-table'.
-    (while (not (gethash key dbus-return-values-table nil))
-      (read-event nil nil 0.1))
+    (let ((marker (make-symbol "marker")))
+      (while (eq (gethash key dbus-return-values-table marker) marker)
+	(read-event nil nil 0.1)))
 
     ;; Cleanup `dbus-return-values-table'.  Return the result.
     (prog1
@@ -370,7 +371,7 @@
 	  (dbus-ignore-errors
 	    (apply 'dbus-method-return-internal
 	     (nth 1 event) (nth 3 event) (nth 4 event)
-	     (if (consp result) result (list result))))))
+	     result))))
     ;; Error handling.
     (dbus-error
      ;; Return an error message when it is a message call.

Re: Automatic value conversion in DBus bindings

[Michael Albinus]

Jan Moringen <jan.moringen@uni-bielefeld.de> writes:

Hi Jan,

> While playing around with the DBus bindings, I came across some problems
> related to the automatic conversion between LISP and DBus values.
>
> The first problem can be demonstrated by running the following code:
>
>   (require 'dbus)
>
>   (defun test ()
>     nil)
>
>   (dbus-register-method
>    :session "org.gnu.emacs" "/org/gnu/emacs" "org.gnu.Emacs" "test"
>    #'test)
>
>   (dbus-call-method-non-blocking
>    :session "org.gnu.emacs" "/org/gnu/emacs" "org.gnu.Emacs" "test")

If you don't want to get a return value, you shall call

(dbus-call-method-asynchronously
 :session "org.gnu.emacs" "/org/gnu/emacs" "org.gnu.Emacs" "test" 'ignore)

> The original motivation for my experiments was writing a function that
> should receive DBus calls but not return a value. This was necessary
> since another process (Empathy) would call the method via DBus and
> complain if it returned anything.

How is your function called from Empathy? If it is
dbus_connection_send_with_reply_and_block or
dbus_connection_send_with_reply, you *must* return a value. In the case
of dbus_connection_send (that's what I suspect) it doesn't matter, I believe.

> Kind regards,
> Jan Moringen

Best regards, Michael.

Re: Automatic value conversion in DBus bindings

[Jan Moringen]

Hi Michael.

Thank you for the quick reply. Maybe I wasn't clear enough about this in
my first message. I think there are two separate problems. 

I tried to demonstrate the first with the code below.

> > While playing around with the DBus bindings, I came across some problems
> > related to the automatic conversion between LISP and DBus values.
> >
> > The first problem can be demonstrated by running the following code:
> >
> >   (require 'dbus)
> >
> >   (defun test ()
> >     nil)
> >
> >   (dbus-register-method
> >    :session "org.gnu.emacs" "/org/gnu/emacs" "org.gnu.Emacs" "test"
> >    #'test)
> >
> >   (dbus-call-method-non-blocking
> >    :session "org.gnu.emacs" "/org/gnu/emacs" "org.gnu.Emacs" "test")
> 
> If you don't want to get a return value, you shall call
> 
> (dbus-call-method-asynchronously
>  :session "org.gnu.emacs" "/org/gnu/emacs" "org.gnu.Emacs" "test" 'ignore)

In this case (the code above) the problem is not whether the caller
receives a return value or not, but whether the call returns at all. It
does return fine as long as `test' does not return nil. A value of nil
causes an infinite loop in `dbus-call-method-non-blocking' as I
explained:

> (while (not (gethash key dbus-return-values-table nil))
>     (read-event nil nil 0.1))
> 
> Here, the return value nil cannot be distinguished from "no return
> value
> yet".

The second problem is not being able to not return any values from a
DBus-invokable function.

> > The original motivation for my experiments was writing a function that
> > should receive DBus calls but not return a value. This was necessary
> > since another process (Empathy) would call the method via DBus and
> > complain if it returned anything.
> 
> How is your function called from Empathy? If it is
> dbus_connection_send_with_reply_and_block or
> dbus_connection_send_with_reply, you *must* return a value. In the case
> of dbus_connection_send (that's what I suspect) it doesn't matter, I believe.

I think that a *reply* is required, too. The problem seems to be, that
it must not have *return values* in it. I could not find the call in the
Empathy source (they seem to use gobject DBus proxies and other stuff
that I am not familiar with), but I recorded two dumps with
dbus-monitor, that hopefully illustrate the problem.

The general setup is as follows: The Emacs process takes the bus name
org.gnome.Empathy.StreamTubeHandler.x_rudel_infinote and registers an
object /org/gnome/Empathy/StreamTubeHandler/x_rudel_infinote with a
method org.gnome.Empathy.TubeHandler.HandleTube. This method is invoked
by Empathy.

The following dump shows a successful attempt in which the Emacs process
sent a method return message that does not include any return values. I
produced this with the patch from my first message applied.
----------------------------------------------------------------------
method call
  sender=:1.0 -> dest=org.gnome.Empathy.StreamTubeHandler.x_rudel_infinote 
  serial=112
  path=/org/gnome/Empathy/StreamTubeHandler/x_rudel_infinote; 
  interface=org.gnome.Empathy.TubeHandler; 
  member=HandleTube
    string ":1.4"
    object path "/org/freedesktop/Telepathy/Connection/gabble/jabber/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    object path "/org/freedesktop/Telepathy/Connection/gabble/jabber/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/StreamTubeChannel_2_1647872899"
    uint32 1
    uint32 2
method return 
  sender=:1.10 -> dest=:1.0 
  reply_serial=112
----------------------------------------------------------------------

The following dump show an unsuccessful attempt. The method call is
identical but this time the patch wasn't in place and thus the function
in the Emacs process was forced to return a value. I arbitrarily decided
to return 1. This return value is sent to Empathy as the method return
message indicates. However Empathy seems to insist on a method return
message that has no return values in it (see below).
----------------------------------------------------------------------
method call 
  sender=:1.0 -> dest=org.gnome.Empathy.StreamTubeHandler.x_rudel_infinote 
  serial=124 
  path=/org/gnome/Empathy/StreamTubeHandler/x_rudel_infinote; 
  interface=org.gnome.Empathy.TubeHandler; 
  member=HandleTube
    string ":1.4"
    object path "/org/freedesktop/Telepathy/Connection/gabble/jabber/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    object path "/org/freedesktop/Telepathy/Connection/gabble/jabber/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/StreamTubeChannel_2_459526684"
    uint32 1
    uint32 2
method return
  sender=:1.10 -> dest=:1.0
  reply_serial=124
    uint32 1
----------------------------------------------------------------------
This is the error message Empathy produces when it receives the method
return message with the return value in it:
----------------------------------------------------------------------
Unable to start application for service x_rudel_infinote: Too many
arguments in reply; expected 0, got 1
----------------------------------------------------------------------

I have no idea how I could work around this problem without the ability
to not return any values from a DBus-invoked function. If further
explanations or tests are required, I would be happy to provide them.

Thanks in advance.

Kind regards,
Jan

Re: Automatic value conversion in DBus bindings

[Michael Albinus]

Jan Moringen <jan.moringen@uni-bielefeld.de> writes:

> Hi Michael.

Hi Jan,

> In this case (the code above) the problem is not whether the caller
> receives a return value or not, but whether the call returns at all. It
> does return fine as long as `test' does not return nil. A value of nil
> causes an infinite loop in `dbus-call-method-non-blocking' as I
> explained:
>
>> (while (not (gethash key dbus-return-values-table nil))
>>     (read-event nil nil 0.1))
>> 
>> Here, the return value nil cannot be distinguished from "no return
>> value
>> yet".

Understood. I've committed to the CVS trunk a patch similar to the one
you have proposed.

> The second problem is not being able to not return any values from a
> DBus-invokable function.
>
> I think that a *reply* is required, too. The problem seems to be, that
> it must not have *return values* in it.

OK. There is another patch just committed, which allows your function to
return the special symbol `:ignore'. In this case, the reply message is
sent without any argument, as needed in your case.

Your function would look like this:

  (defun test ()
    :ignore)

> I have no idea how I could work around this problem without the ability
> to not return any values from a DBus-invoked function. If further
> explanations or tests are required, I would be happy to provide them.

Could you, please, check whether the patches solve your needs?

> Thanks in advance.
>
> Kind regards,
> Jan

Best regards, Michael.

Re: Automatic value conversion in DBus bindings

[Jan Moringen]

Hi Michael.

> > I have no idea how I could work around this problem without the ability
> > to not return any values from a DBus-invoked function. If further
> > explanations or tests are required, I would be happy to provide them.
> 
> Could you, please, check whether the patches solve your needs?

I used the :ignore symbol as you described and it worked just fine. Or
at least Empathy stopped complaining ;) Thank you for the quick fix.

Kind regards,
Jan