From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Christopher Baines Newsgroups: gmane.lisp.guile.user Subject: Re: Making HTTP requests over TLS from multiple threads Date: Fri, 02 Apr 2021 22:52:28 +0100 Message-ID: <878s604an7.fsf@cbaines.net> References: <87r1kcdd6g.fsf@cbaines.net> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="40694"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.4.15; emacs 27.1 To: guile-user@gnu.org Original-X-From: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Fri Apr 02 23:53:05 2021 Return-path: Envelope-to: guile-user@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lSRiz-000ASz-66 for guile-user@m.gmane-mx.org; Fri, 02 Apr 2021 23:53:05 +0200 Original-Received: from localhost ([::1]:39900 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lSRiy-0005oP-5C for guile-user@m.gmane-mx.org; Fri, 02 Apr 2021 17:53:04 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:37140) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lSRiW-0005mw-I8 for guile-user@gnu.org; Fri, 02 Apr 2021 17:52:36 -0400 Original-Received: from mira.cbaines.net ([212.71.252.8]:54756) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lSRiU-0003kp-3K for guile-user@gnu.org; Fri, 02 Apr 2021 17:52:36 -0400 Original-Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 9917327BC5A for ; Fri, 2 Apr 2021 22:52:31 +0100 (BST) Original-Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id d1c43567 for ; Fri, 2 Apr 2021 21:52:30 +0000 (UTC) In-reply-to: <87r1kcdd6g.fsf@cbaines.net> Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.io gmane.lisp.guile.user:17387 Archived-At: --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain I've come up with a slight variant of the original script now, which for me reliably causes Guile to crash. I've included some example output below. --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=threaded-https-connections-test-guile-ports-crash.scm (use-modules (web uri) (web request) (web response) (web client) (web http) (srfi srfi-1) (ice-9 threads) (ice-9 match) (rnrs bytevectors) (srfi srfi-11) (srfi srfi-9) (srfi srfi-9 gnu) (srfi srfi-26) (gnutls) (ice-9 binary-ports) ((ice-9 ftw) #:select (scandir)) ((rnrs io ports) #:prefix rnrs-ports:)) (define* (call-with-streaming-http-request uri callback #:key (headers '())) (let* ((port (open-socket-for-uri uri)) (request (build-request uri #:method 'PUT #:version '(1 . 1) #:headers `((connection close) (Transfer-Encoding . "chunked") (Content-Type . "application/octet-stream") ,@headers) #:port port))) (set-port-encoding! port "ISO-8859-1") (let ((request (write-request request port))) (let ((chunked-output-port (make-chunked-output-port port #:buffering 128 #:keep-alive? #t))) ;; A SIGPIPE will kill Guile, so ignore it (sigaction SIGPIPE (lambda (arg) (simple-format (current-error-port) "warning: SIGPIPE\n"))) (set-port-encoding! chunked-output-port "ISO-8859-1") (callback chunked-output-port) (retry-gnutls-resource-temporarily-unavailable (lambda () (close-port chunked-output-port))) (display "\r\n" port) (force-output port)) (let ((response (read-response port))) (let ((body (read-response-body response))) (close-port port) (values response body)))))) (define (retry-gnutls-resource-temporarily-unavailable thunk) (catch 'gnutls-error thunk (lambda (key err proc . rest) (if (eq? error/again err) (begin (simple-format (current-error-port) "error/again\n") (sleep 1) (thunk)) (throw key (cons* err proc rest)))))) (define (start-thread thread-index) (call-with-new-thread (lambda () (for-each (lambda (request-index) (with-throw-handler #t (lambda () (call-with-streaming-http-request ;; The URL doesn't realy matter as the response to the ;; request doesn't matter. (peek (string->uri (if (= thread-index 1) "https://guix.cbaines.net/test" "https://www.cbaines.net/test"))) (lambda (port) (simple-format (current-error-port) "thread ~A making request\n" thread-index) (let* ((buffer-size 128) (buffer (make-bytevector buffer-size))) (for-each (lambda (index) ;; (usleep 10) (retry-gnutls-resource-temporarily-unavailable (lambda () (put-bytevector port buffer 0 buffer-size)))) (iota 20000)))))) (lambda (key . args) (simple-format #t "thread ~A: exception: ~A ~A\n" thread-index key args) (backtrace)))) (iota 1 1))))) (define threads (map start-thread (iota 6 1))) (for-each join-thread threads) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable ;;;=20 ;;; (( ;;; ;;; ;;; (((#<#< scheme: : scheme: http= s httpshttps : uhttpsnfouseri: erinfo :: #fuserinfo#f: # fh osth: sthoh= ost"guix.cbaii: n": e"wsww.wwn.wec.tbc"ab iapnorti:e n#sfe .spnath.:e n"te/= "t tt "eport ssort :t#p aquer : erys: pathme"a: th#: / ht "= ttpst f//e;;; entu:t eris erinfo( :#fe>tf##fs )f""th osti>"ost ::q uschem": hemequery: :htw#fwftps#f f wuffragment.ragment: ag= ment:: c #f##fbf>f >a)h )iost ::n neewsw.wn.ectb"a iportn: esfs .pathn: e"t/"teesorts: t" queryquery#f: = papath : fragment": /#ft>e)s t" query: #f fragment: #f>) ;;; (#< scheme: https userinfo: #f host: "www.cbaines.net" port: #f pa= th: "/test" query: #f fragment: #f>) thread 4 making request thread 3 making request thread 6 making request thread 5 making request thread 2 making request error/again error/again error/again thread 1 making request error/again error/again error/again warning: SIGPIPE thread 4: exception: gnutls-error ((# write_to_session_record_port)) Backtrace: error/again thread 6: exception: gnutls-error ((# write_to_session_record_port)) Backtrace: IInnn srfi/srfi-1.scmsrfi/srfi-1.scmsrfi/srfi-1.scm:: 634:9 634:9 7 7 (for-each # (1= )) (for-each # (1)) In ice-9/boot-9.scm: In ice-9/boot-9.scm: 1736:10 6 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) = 1736:10 6=20 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) In /home/chris/threaded-https-connections-test-guile-ports-crash.scm: In /home/chris/threaded-https-connections-test-guile-ports-crash.scm: 48:848:8 48:8 5 5 (call-with-streaming-http-request _ _ #:hea= ders _) (call-with-streaming-http-request _ _ #:headers _) In srfi/srfi-1.scm: In srfi/srfi-1.scm: 634:9 4 634:9 4 (for-each # _) (for-each # _) In /home/chris/threaded-https-connections-test-guile-ports-crash.scm: In /home/chris/threaded-https-connections-test-guile-ports-crash.scm := =20=20=20=20 71:10 3 (_ gnutls-error _ _) 71:10 3 (_ gnutls-error _ _) In ice-9/boot-9.scm: In ice-9/boot-9.scm: 1669:16 2 (raise-exception _ #:continuable? _) 1669:16 2 (raise-exception _ #:continuable? _) 1764:13 1 1764:13 1 (_ #<&compound-exception components: (#<&error> #= <&irritants irritants: ((# w= rite_to_session_record_port))=E2=80=A6>) (_ #<&compound-exception components: (#<&error> #<&irritants irritants: ((#= write_to_session_=E2=80=A6>) In=20 unknown file::n unknown file: 0 0 (backtrace #) (backtrace #)(backtrace #) IIInn srfi/srfi-1.scmsrfi/srfi-1.scm:: 634:9634:9 66 6 (for-each # (1)) (for-each # (1)) In ice-9/boot-9.scm: In ice-9/boot-9.scm: 1736:10 5 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) 1736:10 5 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) In /home/chris/threaded-https-connections-test-guile-ports-crash.scm: In /home/chris/threaded-https-connections-test-guile-ports-crash.scm: 48:8 4 48:8 4 (call-with-streaming-http-request _ _ #:headers _) (call-with-streaming-http-request _ _ #:headers _) IIInn srfi/srfi-1.scmsrfi/srfi-1.scm:srfi/srfi-1.scm: : 634:9 634:9634:9 3 3 (for-each # _) In /home/chris/threaded-https-connections-test-guile-ports-crash.scm: (for-each # _) 71:10 2 (_ gnutls-error _ _) In /home/chris/threaded-https-connections-test-guile-ports-crash.scm: 71:10 2 (_ gnutls-error _ _) In ice-9/boot-9.scm: In ice-9/boot-9.scm: 1669:16 1 (raise-exception _ #:continuable? _) 1669:16 1 (raise-exception _ #:continuable? _) 1669:16 0 (raise-exception _ #:continuable? _) 1669:16 0 (raise-exception _ #:continuable? _) ice-9/boot-9.scm:1669:16: In prooice-9/boot-9.scmc:e16691669d:urr16e: rais= e-exception: Throw to key `gnutls-error' withhI na rpgrso c`edure raise-exception: Throw tto' .kee y `gnutls-error' with args `((# write_to_session_record_port))'. guile: ports.c:2900: scm_i_write_bytes: Assertion `written =3D=3D count' fa= iled. Aborted --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmBnkhxfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XeCthAArIZGHwX/053vvZtUQqhd2FHGXtYeNa56 4QpdGJUIi3Keuax9dmW5mVLJMvIH0EiW4N81TpTBOsi8xpyUlZvfXhATJcFRGn+E j9/OO22Wf4nAH+Ln2JFHL6nXrxjfcANtwy1m3AzIK0rsykumhtsAFSVzjn96JzAQ cwfaCVLqbKOCnwccm50IB0LXK7uOD4ZSvL7OOHI155e6bjlhnnv7LAPuNzX7P1F+ ANsKY2eRPKNV9Q/cyJKvD+j5KFI5aQdREzs86OBrdMn+PgW+/crPslbTtfKekbwS h4VYgMx+uh/EUh/xkXESag+yEoCdNW4rmZmRLn8KlcgidTIoY0BeKxaDT6HfTQ58 d+ksXHP2tDq6Hz9o5KO2aKTn4jrUwaUhg/tdVv0G4E9L9JdFJXtSZyw/8zBl+qZ9 4yZOdDcGK89WhqNUK5JZMOiT7vUM0mIs/zXqBrFAXtV1f5acbDDCCODo7Rq+TceW byV5+zD+qM3fN+QywoT5FxPBVq37MuWZ8v1hG7bLFLIABl+yXwVCwUUeeFCvtYSX lpY5HrvjE4W/SAROI73Q0zTbeYXW2gkDpF88zFfEE6YDLCJ/meBg6JmLbK1Ef8Nd AdKidy+Aau9M+H/uIgEW42vRa/IcwA0IufTWmsIuGHHN9JNDRek85prvi0quJYWS HTys11AH6BM= =gApM -----END PGP SIGNATURE----- --==-=-=--