Possible issue with mutexes

Possible issue with mutexes

[Herwig Hochleitner]

Hello,

I've recently created a semaphore type, based on mutex and condition 
vars, for controlling parallelism in an elisp processor.

The project went well, so far, but now that I attempted to pull out this 
semaphore implementation and create a test for it, I've noticed 
behavior, that leads me to believe, that emacs' thread and mutex 
guarantees might not hold in all cases.

When running the [test 
case](https://github.com/webnf/semaphore.el/blob/306d5a84b99526213c3a7d367f8d8041961ea52a/test/emacs-semaphore-test.el), 
you would expect `test/semaphore/active-count` to be 0 at the end of a 
run, right? For me, it's wildly inconsistent at the end of a run and 
seems to become desynchronized among worker threads. In this [example 
run](https://hastebin.com/wigiwizabi.log) from my machine, it happens to 
settle at 10.

Am I doing this wrong, somehow?

If not, could somebody help me file a report? First post on this ML ...

kind regards

Re: Possible issue with mutexes

[Herwig Hochleitner]

On 4/10/19 4:57 AM, Herwig Hochleitner wrote:
> Hello,
>
> I've recently created a semaphore type, based on mutex and condition 
> vars, for controlling parallelism in an elisp processor.
>
> The project went well, so far, but now that I attempted to pull out 
> this semaphore implementation and create a test for it, I've noticed 
> behavior, that leads me to believe, that emacs' thread and mutex 
> guarantees might not hold in all cases.
>
> When running the [test 
> case](https://github.com/webnf/semaphore.el/blob/306d5a84b99526213c3a7d367f8d8041961ea52a/test/emacs-semaphore-test.el), 
> you would expect `test/semaphore/active-count` to be 0 at the end of a 
> run, right? For me, it's wildly inconsistent at the end of a run and 
> seems to become desynchronized among worker threads. In this [example 
> run](https://hastebin.com/wigiwizabi.log) from my machine, it happens 
> to settle at 10.
>
> Am I doing this wrong, somehow?
>
> If not, could somebody help me file a report? First post on this ML ...
>
> kind regards

Argh .. found the bug in my code.

Sorry for the spam!


Since we are here already, let me ask you about this other issue, I had:

Waiting on a condition var or joining a thread on the main thread can 
freeze emacs totally dead. Like kill -9 dead.

Is this a known issue or should I investigate further?

Re: Possible issue with mutexes

[Eli Zaretskii]

> From: Herwig Hochleitner <herwig@bendlas.net>
> Date: Wed, 10 Apr 2019 05:11:36 +0200
> 
> Waiting on a condition var or joining a thread on the main thread can 
> freeze emacs totally dead. Like kill -9 dead.
> 
> Is this a known issue or should I investigate further?

Not sure if I understand correctly the situation you describe.  Would
it be possible to post a simple reproducer?

Re: Possible issue with mutexes

[Herwig Hochleitner]

On 4/10/19 5:11 PM, Eli Zaretskii wrote:

> [Why private email?]
I hit the wrong reply button, sorry.
>> From: Herwig Hochleitner <herwig@bendlas.net>
>> Date: Wed, 10 Apr 2019 17:02:35 +0200
>>
>>
>>>> Waiting on a condition var or joining a thread on the main thread can
>>>> freeze emacs totally dead. Like kill -9 dead.
>>>>
>>>> Is this a known issue or should I investigate further?
>>> Not sure if I understand correctly the situation you describe.  Would
>>> it be possible to post a simple reproducer?
>>>
>> Try evaluating this snippet (maybe not in your main emacs) and try
>> quitting out of the eval:
>>
>> (let* ((m (make-mutex))
>>          (c (make-condition-variable m)))
>>     (with-mutex m
>>       (condition-wait c)))
> How is the above "joining a thread"?

Joining a thread shows similar behavior, where you can't quit via C-g 
anymore.

(thread-join
  (make-thread
   (lambda ()
     (let* ((m (make-mutex))
            (c (make-condition-variable m)))
       (with-mutex m
         (condition-wait c))))))

> Anyway, I think what you did is wrote code that deliberately
> deadlocks, so why are you surprised it does?

That's because this is a minimal example. I'm not surprised by it 
anymore ;-)

I hit this, when working on somewhat more involved code 
https://github.com/NixOS/nixpkgs/blob/f4d71836af5dd99b5af32ace6d1775b91213858e/pkgs/applications/editors/emacs-modes/update-melpa.el

> Theres no other thread
> to signal the condition variable or signal the waiting thread.
>
> IOW, this is the expected behavior.
Yes, pardon, I'm so used to event-driven programming, I forgot that in 
Unix, everything is pull

What I'm trying to say: It would be nice, if C-g could also release 
pending condition-waits and thread-joins.

I understand, that this might be kind of a hard thing to ask, due to the 
lack of a separate input thread, as you say, but I'm not familiar with 
the internals, so I'm asking it.

kind regards