From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Ken Raeburn Newsgroups: gmane.emacs.devel Subject: optimization for pop movemail Date: Sun, 03 Jul 2005 19:01:26 -0400 Message-ID: NNTP-Posting-Host: main.gmane.org X-Trace: sea.gmane.org 1120433652 24809 80.91.229.2 (3 Jul 2005 23:34:12 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Sun, 3 Jul 2005 23:34:12 +0000 (UTC) Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Jul 04 01:34:11 2005 Return-path: Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1DpDyI-0000fa-Ub for ged-emacs-devel@m.gmane.org; Mon, 04 Jul 2005 01:33:59 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1DpDzT-0000qx-Ue for ged-emacs-devel@m.gmane.org; Sun, 03 Jul 2005 19:35:11 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1DpDwx-0007uB-Jl for emacs-devel@gnu.org; Sun, 03 Jul 2005 19:32:35 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1DpDwq-0007p5-E2 for emacs-devel@gnu.org; Sun, 03 Jul 2005 19:32:28 -0400 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1DpDwk-0007RF-0k for emacs-devel@gnu.org; Sun, 03 Jul 2005 19:32:22 -0400 Original-Received: from [207.172.4.63] (helo=smtp04.mrf.mail.rcn.net) by monty-python.gnu.org with esmtp (Exim 4.34) id 1DpDYp-0001eY-Aa for emacs-devel@gnu.org; Sun, 03 Jul 2005 19:07:39 -0400 Original-Received: from 65-78-24-4.c3-0.smr-ubr1.sbo-smr.ma.cable.rcn.com (HELO raeburn.org) (65.78.24.4) by smtp04.mrf.mail.rcn.net with ESMTP; 03 Jul 2005 19:01:27 -0400 X-IronPort-AV: i="3.93,254,1115006400"; d="scan'208"; a="54686228:sNHT20483320" Original-Received: from kal-el.raeburn.org (mail@kal-el.raeburn.org [18.101.0.230]) by raeburn.org (8.12.11/8.12.11) with ESMTP id j63N1QXt026863; Sun, 3 Jul 2005 19:01:26 -0400 (EDT) Original-Received: from raeburn by kal-el.raeburn.org with local (Exim 3.36 #1 (Debian)) id 1DpDSo-0001vd-00; Sun, 03 Jul 2005 19:01:26 -0400 Original-To: emacs-devel@gnu.org X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:40287 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:40287 The POP support in movemail runs afoul of a combination of otherwise reasonable optimizations in most TCP stacks, Nagle and delayed acks. It sends (for example) "RETR 1234" and then there's a delay before it sends "\r\n", because we make two write() calls. It can add something like 0.4 seconds per message retrieved, which adds up with a lot of messages. Is this approach okay, or should I go change the code to include the \r\n at all the call sites instead? Should this wait until after the release? Ken --- lib-src/pop.c 1 Sep 2003 15:45:03 -0000 1.34 +++ lib-src/pop.c 3 Jul 2005 22:04:03 -0000 @@ -1404,10 +1393,39 @@ sendline (server, line) #define SENDLINE_ERROR "Error writing to POP server: " int ret; - ret = fullwrite (server->file, line, strlen (line)); - if (ret >= 0) - { /* 0 indicates that a blank line was written */ - ret = fullwrite (server->file, "\r\n", 2); + if (line == pop_error && strlen (line) < sizeof (pop_error) - 5) + { + /* This minor "abstraction" violation can save a fraction of a + second per message sent in a fast, reliable TCP network + environment with delayed acks and Nagle algorithm. (Movemail + writes line without \r\n, client kernel sends it, server + kernel delays ack to see if it can combine it with data, + movemail writes \r\n, client kernel waits because it has + unacked data, client kernel eventually times out and sends.) + + On a NetBSD box, this delay is 0.2 seconds per message; if + you've got a few dozen messages or so, it adds up, and if + they're small and the server is close, it can be a + significant fraction of the execution time. + + Turning off Nagle would probably change this to two packets + in rapid succession for most implementations. If we can make + it just one write call, we'll likely get one packet and keep + everybody happier. + + Fortunately, most of the formatted calls (e.g., all those + including message numbers) use pop_error as the buffer + into which the command is written. */ + strcat (line, "\r\n"); + ret = fullwrite (server->file, line, strlen (line)); + } + else + { + ret = fullwrite (server->file, line, strlen (line)); + if (ret >= 0) + { /* 0 indicates that a blank line was written */ + ret = fullwrite (server->file, "\r\n", 2); + } } if (ret < 0)