From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Newsgroups: gmane.lisp.guile.devel Subject: Re: [PATCH] Bindings for =?utf-8?B?4oCYc2VuZGZpbGXigJk=?= Date: Sun, 07 Apr 2013 21:53:26 +0200 Message-ID: <87sj32rubt.fsf@inria.fr> References: <87ip4liufs.fsf@gnu.org> <878v5hbblk.fsf@tines.lan> <87obed2iqo.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1365445708 4828 80.91.229.3 (8 Apr 2013 18:28:28 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 8 Apr 2013 18:28:28 +0000 (UTC) Cc: "Mark H. Weaver" To: guile-devel@gnu.org Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Mon Apr 08 20:28:32 2013 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1UPGnr-0000pw-D5 for guile-devel@m.gmane.org; Mon, 08 Apr 2013 20:28:27 +0200 Original-Received: from localhost ([::1]:52748 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UOvez-00043g-Vg for guile-devel@m.gmane.org; Sun, 07 Apr 2013 15:53:53 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:44206) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UOves-00040L-Fd for guile-devel@gnu.org; Sun, 07 Apr 2013 15:53:49 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UOvek-0006nu-Vv for guile-devel@gnu.org; Sun, 07 Apr 2013 15:53:46 -0400 Original-Received: from [2a01:e0b:1:123:ca0a:a9ff:fe03:271e] (port=33707 helo=xanadu.aquilenet.fr) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UOvek-0006mE-JR for guile-devel@gnu.org; Sun, 07 Apr 2013 15:53:38 -0400 Original-Received: from localhost (localhost [127.0.0.1]) by xanadu.aquilenet.fr (Postfix) with ESMTP id 5ACE1BE6C; Sun, 7 Apr 2013 21:53:28 +0200 (CEST) Original-Received: from xanadu.aquilenet.fr ([127.0.0.1]) by localhost (xanadu.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dBPetsdelbmu; Sun, 7 Apr 2013 21:53:28 +0200 (CEST) Original-Received: from pluto (reverse-83.fdn.fr [80.67.176.83]) by xanadu.aquilenet.fr (Postfix) with ESMTPSA id 663012F2D; Sun, 7 Apr 2013 21:53:27 +0200 (CEST) X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 18 Germinal an 221 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0xEA52ECF4 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 83C4 F8E5 10A3 3B4C 5BEA D15D 77DD 95E2 EA52 ECF4 X-OS: x86_64-unknown-linux-gnu In-Reply-To: <87obed2iqo.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Thu, 21 Mar 2013 10:40:47 +0100") User-Agent: Gnus/5.130005 (Ma Gnus v0.5) Emacs/24.3 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2a01:e0b:1:123:ca0a:a9ff:fe03:271e X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:16199 Archived-At: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable There were sporadic failures of the sendfile/pipe tests on Hydra. I managed to reproduce them and noticed that sometimes sendfile(2) would return 64KiB, which is much less than what we asked for. Although the man page doesn=E2=80=99t mention it, this can happen when writ= ing to a slow or limited device, like a pipe (pretty much like write(2)). I was hesitant whether to stick to libc behavior (return the number of bytes sent and let the user handle it), or to handle it directly in our =E2=80=98sendfile=E2=80=99 procedure. After discussion with Mark on IRC, I= became convinced that the latter is preferable, so here=E2=80=99s the patch. I=E2=80=99ll commit it shortly if there are no objections. Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-Change-sendfile-to-loop-until-everything-has-been-se.patch Content-Description: the patch >From 42459c382ca512c927b6228c1557afe8e10aedae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= Date: Sun, 7 Apr 2013 21:47:48 +0200 Subject: [PATCH] Change `sendfile' to loop until everything has been sent. * libguile/filesys.c (scm_sendfile)[HAVE_SYS_SENDFILE_H && HAVE_SENDFILE]: Compare RESULT with C_COUNT. Loop until C_COUNT bytes have been sent. Return SCM_UNSPECIFIED. * doc/ref/posix.texi (File System): Update the description. Explain the new semantics. --- doc/ref/posix.texi | 11 +++++++++-- libguile/filesys.c | 36 +++++++++++++++++++++++++++++------- 2 files changed, 38 insertions(+), 9 deletions(-) diff --git a/doc/ref/posix.texi b/doc/ref/posix.texi index 45f320f..13f3a48 100644 --- a/doc/ref/posix.texi +++ b/doc/ref/posix.texi @@ -806,9 +806,10 @@ The return value is unspecified. @deffn {Scheme Procedure} sendfile out in count [offset] @deffnx {C Function} scm_sendfile (out, in, count, offset) Send @var{count} bytes from @var{in} to @var{out}, both of which -are either open file ports or file descriptors. When +must be either open file ports or file descriptors. When @var{offset} is omitted, start reading from @var{in}'s current -position; otherwise, start reading at @var{offset}. +position; otherwise, start reading at @var{offset}. The return +value is unspecified. When @var{in} is a port, it is often preferable to specify @var{offset}, because @var{in}'s offset as a port may be different from the offset of @@ -824,6 +825,12 @@ In some cases, the @code{sendfile} libc function may return @code{EINVAL} or @code{ENOSYS}. In that case, Guile's @code{sendfile} procedure automatically falls back to doing a series of @code{read} and @code{write} calls. + +In other cases, the libc function may send fewer bytes than +@var{count}---for instance because @var{out} is a slow or limited +device, such as a pipe. When that happens, Guile's @code{sendfile} +automatically retries until exactly @var{count} bytes were sent or an +error occurs. @end deffn @findex rename diff --git a/libguile/filesys.c b/libguile/filesys.c index d318ae7..069c7ad 100644 --- a/libguile/filesys.c +++ b/libguile/filesys.c @@ -1111,9 +1111,10 @@ SCM_DEFINE (scm_copy_file, "copy-file", 2, 0, 0, SCM_DEFINE (scm_sendfile, "sendfile", 3, 1, 0, (SCM out, SCM in, SCM count, SCM offset), "Send @var{count} bytes from @var{in} to @var{out}, both of which " - "are either open file ports or file descriptors. When " + "must be either open file ports or file descriptors. When " "@var{offset} is omitted, start reading from @var{in}'s current " - "position; otherwise, start reading at @var{offset}.") + "position; otherwise, start reading at @var{offset}. The return " + "value is unspecified.") #define FUNC_NAME s_scm_sendfile { #define VALIDATE_FD_OR_PORT(cvar, svar, pos) \ @@ -1139,9 +1140,31 @@ SCM_DEFINE (scm_sendfile, "sendfile", 3, 1, 0, #if defined HAVE_SYS_SENDFILE_H && defined HAVE_SENDFILE /* The Linux-style sendfile(2), which is different from the BSD-style. */ - result = sendfile_or_sendfile64 (out_fd, in_fd, - SCM_UNBNDP (offset) ? NULL : &c_offset, - c_count); + { + size_t total; + off_t *offset_ptr; + + offset_ptr = SCM_UNBNDP (offset) ? NULL : &c_offset; + + /* On Linux, when OUT_FD is a file, everything is transferred at once and + RESULT == C_COUNT. However, when OUT_FD is a pipe or other "slow" + device, fewer bytes may be transferred, hence the loop. */ + for (total = 0, result = 0; total < c_count && result >= 0; ) + { + result = sendfile_or_sendfile64 (out_fd, in_fd, offset_ptr, + c_count - total); + if (result > 0) + { + total += result; + if (offset_ptr == NULL) + offset_ptr = &c_offset; + *offset_ptr = total; + } + else if (result < 0 && errno == EINTR) + /* Keep going. */ + result = 0; + } + } /* Quoting the Linux man page: "In Linux kernels before 2.6.33, out_fd must refer to a socket. Since Linux 2.6.33 it can be any file." @@ -1183,10 +1206,9 @@ SCM_DEFINE (scm_sendfile, "sendfile", 3, 1, 0, result += obtained; } - return scm_from_size_t (result); } - return scm_from_ssize_t (result); + return SCM_UNSPECIFIED; #undef VALIDATE_FD_OR_PORT } -- 1.7.10.4 --=-=-=--