* bug#24357: pure guile program leaks memory
@ 2016-09-03 7:20 Amirouche Boubekki
2016-09-03 9:54 ` Andy Wingo
0 siblings, 1 reply; 5+ messages in thread
From: Amirouche Boubekki @ 2016-09-03 7:20 UTC (permalink / raw)
To: 24357
[-- Attachment #1: Type: text/plain, Size: 901 bytes --]
Using guile 2.1.3, I have a program that:
- reads urls from a text file
- download the urls using curl command via popen
- output the result to stdout
Also, it relies on n-for-each-par-map for ice-9 threads.
IMO, the most suspicious is the definition of the `curl' proc:
(define (curl url)
(let* ((port (open-input-pipe (format #f "curl -is \"~a\"" url)))
(response (read-string port)))
(close-pipe port)
response))
I fail to see how this can leak.
To reproduce the bug you need a giant list of preferably different urls.
Such a list is available at http://hyperdev.fr/data/hn/hn.urls.txt.xz
You can run the test program with the following command:
cat hn.urls.txt | guile urls-step00-fetch.scm > /dev/null
This is not the only program I tried on guile-2.1.3 that leaks but
it's the easiest to reproduce.
I will try to reproduce the bug on 2.0.12.
[-- Attachment #2: urls-step00-fetch.scm --]
[-- Type: text/plain, Size: 652 bytes --]
(use-modules (ice-9 threads))
(use-modules (ice-9 rdelim))
(use-modules (ice-9 popen))
(use-modules (ice-9 rdelim))
;;; wrapping curl command
(define (curl url)
(let* ((port (open-input-pipe (format #f "curl -is \"~a\"" url)))
(response (read-string port)))
(close-pipe port)
response))
(define (maybe-curl url)
(catch #t
(lambda () (cons url (curl url)))
(lambda _ '())))
(define urls (let loop ((line (read-line))
(out '()))
(if (eof-object? line)
out
(loop (read-line) (cons line out)))))
(n-for-each-par-map 16 write maybe-curl urls)
^ permalink raw reply [flat|nested] 5+ messages in thread
* bug#24357: pure guile program leaks memory
2016-09-03 7:20 bug#24357: pure guile program leaks memory Amirouche Boubekki
@ 2016-09-03 9:54 ` Andy Wingo
2016-09-03 18:49 ` Amirouche Boubekki
0 siblings, 1 reply; 5+ messages in thread
From: Andy Wingo @ 2016-09-03 9:54 UTC (permalink / raw)
To: Amirouche Boubekki; +Cc: 24357
On Sat 03 Sep 2016 09:20, Amirouche Boubekki <amirouche@hypermove.net> writes:
> Using guile 2.1.3, I have a program that:
>
> - reads urls from a text file
> - download the urls using curl command via popen
> - output the result to stdout
>
> Also, it relies on n-for-each-par-map for ice-9 threads.
Can you reduce it please? For example, remove the use of threads.
Andy
^ permalink raw reply [flat|nested] 5+ messages in thread
* bug#24357: pure guile program leaks memory
2016-09-03 9:54 ` Andy Wingo
@ 2016-09-03 18:49 ` Amirouche Boubekki
2016-09-12 17:47 ` Amirouche Boubekki
0 siblings, 1 reply; 5+ messages in thread
From: Amirouche Boubekki @ 2016-09-03 18:49 UTC (permalink / raw)
To: Andy Wingo; +Cc: 24357
[-- Attachment #1: Type: text/plain, Size: 1090 bytes --]
On 2016-09-03 11:54, Andy Wingo wrote:
> On Sat 03 Sep 2016 09:20, Amirouche Boubekki <amirouche@hypermove.net>
> writes:
>
>> Using guile 2.1.3, I have a program that:
>>
>> - reads urls from a text file
>> - download the urls using curl command via popen
>> - output the result to stdout
>>
>> Also, it relies on n-for-each-par-map for ice-9 threads.
>
> Can you reduce it please? For example, remove the use of threads.
>
Ok. I removed threads and only download the same url over and over
again.
Here is the error I get on stdout:
(23) Failed writing body
The program is:
(use-modules (ice-9 popen))
;;; wrapping curl command
(define (curl url)
(let* ((port (open-input-pipe (format #f "curl -is \"~a\"" url)))
(response (read-string port)))
(close-pipe port)
response))
(define (maybe-curl url)
(catch #t
(lambda ()
(display "." (current-error-port))
(write (cons url (curl url))))
(lambda _ '())))
(define urls (map (lambda _ "http://hyperdev.fr/") (iota 1000)))
(display "started")
(for-each maybe-curl urls)
[-- Attachment #2: guile-bug-24357.scm --]
[-- Type: text/plain, Size: 489 bytes --]
(use-modules (ice-9 popen))
;;; wrapping curl command
(define (curl url)
(let* ((port (open-input-pipe (format #f "curl -is \"~a\"" url)))
(response (read-string port)))
(close-pipe port)
response))
(define (maybe-curl url)
(catch #t
(lambda ()
(display "." (current-error-port))
(write (cons url (curl url))))
(lambda _ '())))
(define urls (map (lambda _ "http://hyperdev.fr/") (iota 1000)))
(display "started")
(for-each maybe-curl urls)
^ permalink raw reply [flat|nested] 5+ messages in thread
* bug#24357: pure guile program leaks memory
2016-09-03 18:49 ` Amirouche Boubekki
@ 2016-09-12 17:47 ` Amirouche Boubekki
2017-03-01 9:06 ` Andy Wingo
0 siblings, 1 reply; 5+ messages in thread
From: Amirouche Boubekki @ 2016-09-12 17:47 UTC (permalink / raw)
To: Andy Wingo; +Cc: bug-guile, 24357
On 2016-09-03 20:49, Amirouche Boubekki wrote:
> On 2016-09-03 11:54, Andy Wingo wrote:
>> On Sat 03 Sep 2016 09:20, Amirouche Boubekki <amirouche@hypermove.net>
>> writes:
>>
>>> Using guile 2.1.3, I have a program that:
>>>
>>> - reads urls from a text file
>>> - download the urls using curl command via popen
>>> - output the result to stdout
>>>
>>> Also, it relies on n-for-each-par-map for ice-9 threads.
>>
>> Can you reduce it please? For example, remove the use of threads.
>>
>
> Ok. I removed threads and only download the same url over and over
> again.
>
> Here is the error I get on stdout:
>
> (23) Failed writing body
>
Internet says this a curl issue. Please close this bug.
^ permalink raw reply [flat|nested] 5+ messages in thread
* bug#24357: pure guile program leaks memory
2016-09-12 17:47 ` Amirouche Boubekki
@ 2017-03-01 9:06 ` Andy Wingo
0 siblings, 0 replies; 5+ messages in thread
From: Andy Wingo @ 2017-03-01 9:06 UTC (permalink / raw)
To: Amirouche Boubekki; +Cc: 24357-done
On Mon 12 Sep 2016 19:47, Amirouche Boubekki <amirouche@hypermove.net> writes:
> On 2016-09-03 20:49, Amirouche Boubekki wrote:
>> On 2016-09-03 11:54, Andy Wingo wrote:
>>> On Sat 03 Sep 2016 09:20, Amirouche Boubekki
>>> <amirouche@hypermove.net> writes:
>>>
>>>> Using guile 2.1.3, I have a program that:
>>>>
>>>> - reads urls from a text file
>>>> - download the urls using curl command via popen
>>>> - output the result to stdout
>>>>
>>>> Also, it relies on n-for-each-par-map for ice-9 threads.
>>>
>>> Can you reduce it please? For example, remove the use of threads.
>>>
>>
>> Ok. I removed threads and only download the same url over and over
>> again.
>>
>> Here is the error I get on stdout:
>>
>> (23) Failed writing body
>
> Internet says this a curl issue. Please close this bug.
No problem. For reference, in the future just add "-done" after the bug
number in the mail, as I did above. Cheers :)
Andy
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2017-03-01 9:06 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-03 7:20 bug#24357: pure guile program leaks memory Amirouche Boubekki
2016-09-03 9:54 ` Andy Wingo
2016-09-03 18:49 ` Amirouche Boubekki
2016-09-12 17:47 ` Amirouche Boubekki
2017-03-01 9:06 ` Andy Wingo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).