Re: (n-for-each-par-map 16 store download (reverse (iota (max-id))))) crash

[Amirouche Boubekki <amirouche@hypermove.net>]

Hi Mark,

Thanks for your support and time.

On 2018-07-05 20:30, Mark H Weaver wrote:
> Hi,
> 
> Amirouche Boubekki <amirouche@hypermove.net> writes:
> 
>> I have a program that try to download hackernews locally.
>> 
>> What it does is simple, it fetch the max identifier and
>> http-get each json value starting with the most recent
>> item. I use n-for-each-par-map with 16 threads I have
>> 8 cores.
>> 
>> Here is the full program:
> 
> [...]
> 
>> (define (store pair)
>>   (if (null? pair)
>>       (format #t "X\n")
>>       (let ((port (open-file "hn.scm" "a")))
>>         (format #t "~a\n" (car pair))
> 
> These calls to (format #t ...), which write to a port that is shared by
> multiple threads, should be performed while holding a mutex to prevent
> concurrent writes to the same port.

Ok, but it's called by:

   (n-for-each-par-map n sproc pproc lst1 lst2)

and the manual says about SPROC that :

   The sproc calls are made serially,
   in list element order, one at a time.

In my case SPROC is 'store', so my understanding is that
it should require no lock.

> 
> I/O operations in Guile do not include built-in thread synchronization,
> at least not in the fast path cases.  However, an effort was made to
> avoid _crashes_ on common architectures in the event of concurrent use
> of the same port.  Our hope was that the worst that would typically
> happen is garbled I/O.  Perhaps we failed to realize that hope.

I saw that behavior while use fibers.

>> How can I debug this?
> 
> It would be helpful to see a GDB backtrace from a crash in this 
> program.

I will retry without mutex and only gdb to reproduce the crash
and have a trace.

> Does it help to protect the shared port with a mutex?

I ran the program with gdb and a lock, it downloads more suff,
but it's also much slower.

>      Thanks,
>        Mark

Thanks again!

-- 
Amirouche ~ amz3 ~ http://www.hyperdev.fr