bug#16775: dbus interacts poorly with lisp-level event loops

[Michael Albinus <michael.albinus@gmx.de>]

Daniel Colascione <dancol@dancol.org> writes:

>>> Then why was it calling dbus-check-event on the result? I checked in a
>>> hack that addresses the immediate issue.
>>
>> Just for safety. Likely, I was not convinced the time I was writing
>> this, that it couldn't be returned by read-event. Don't remember
>> exactly.
>>
>> Does it really hurt?
>
> It certainly makes it more difficult to determine the intent of the code.

Well, so we might throw it away (as you did with your recent
patch). Thanks for this!

>>> Because secrets.el was taking a whole second to load due almost
>>> entirely to dbus delays.
>>
>> Well, that's serious. I'll check.
>
> I used time emacs -Q -batch --eval "(require 'secrets)".

Thanks. I'll have a look on it. There are other annoyances with
secrets.el on my todo, so it will be a more extensive debug
session. However, I doubt we shall touch the code during the freeze.

>> However, is it just secrets.el loading, or also normal operation in
>> secrets.el, which are delayed
>
> I don't know. Secrets has other problems that I have to fix
> separately. (By the way: dbus appears to hang if there's an error in a
> message spec and the bus disconnects from underneath us. Try
> (dbus-call-method :session "org.freedesktop.secrets"
> "/org/freedesktop/secrets/collection/login"
> "org.freedesktop.Secret.Collection" "SearchItems" '(:array
> (:dict-entry "host" ("xxxx" "xxxx")) (:dict-entry "port" "imaps"))). A
> signal of some sort pertaining to the disconnection is delivered to
> xd_read_message_1, but it drops this signal on the floor, leading
> dbus-call-method to loop until its timeout expires.

If you want to get more traces from dbusbind.c, compile Emacs with
"env MYCPPFLAGS='-DDBUS_DEBUG -Wall' make".

>> We're speaking about 0.1sec delay. Does it really hurt? (Yes, I'll check
>> the secrets.el case).
>
> Yes, 100 milliseconds is far about the threshold at which
> interactivity is visibly affected. It's completely unacceptable. It's
> worse when a single logical operation involves multiple dbus calls.
>
> No. You seem to be operating under the something interesting notion
> that it's reasonable for an interactive program to simply stop
> responding for 100ms. The threshold of human perception is widely
> regarded to be somewhere in the 30ms neighborhood.

Hmm. Problems with the dbus code were problems of being blocked, so
far. Performance hasn't been on focus (yet). I agree with you that it
shall be also tuned, but my focus seems to be different :-)

If you have (further) tuning proposals, go on!

> Right. I looked up the change history, and making dbus-call-method
> async was the right choice. The event delivery, however, needs to be
> refined. There's no reason, in principle, that dbus-call-method
> shouldn't be able to return instantly on call completion.

The code you've seen was the best compromise I could find. Of course,
one could reduce the timeouts in read-event. But OTOH, D-Bus messages
are not known to respond fast, in general. (This might change with the
upcoming kdbus, which is implemented as zero-copy).

> You misunderstand me. The code I checked in "fixes" the problem for
> both the base case and the scenario I described. The code that was
> there yesterday is similarly broken --- with respect to the 100ms
> delay ---
> for both uses. There's nothing to debug right now.

Again, I wouldn't call it "broken". It was a tradeoff between response
time of D-Bus calls, and the number of read-event calls. Of yourse, this
depends also on the underlying machine. My main development machine is 5
years old, not so fast ...

>> Definitely nothing for 24.4, we're in feature freeze. And before we're
>> adding such a patch, I recommend to discuss Stefan's proposal to add
>> another event queue in the main loop for such kind of special events.
>
> I'll have to go find that thread. I'm not sure what "another event
> queue" means, exactly, or how it would help. If it's just a different
> kind of accept-process-input, one that pulls only events of a certain
> sort out of the kbd buffer, then it's still vulnerable to the
> reentrancy problem I described in my first message.

It hasn't been discussed too much. The best reference would be
<http://thread.gmane.org/gmane.emacs.devel/169268/focus=169278>; Stefan
explains what's behind this idea.

Best regards, Michael.