From mboxrd@z Thu Jan  1 00:00:00 1970
Path: news.gmane.org!not-for-mail
From: Paul Eggert <eggert@cs.ucla.edu>
Newsgroups: gmane.emacs.devel
Subject: Re: process.c: read_process_output: hard coded 4096 bytes read limit
Date: Sat, 22 Jun 2013 10:27:55 -0700
Message-ID: <51C5DE9B.8010607@cs.ucla.edu>
References: <kq3ssm$b00$1@ger.gmane.org>
NNTP-Posting-Host: plane.gmane.org
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-Trace: ger.gmane.org 1371922090 9665 80.91.229.3 (22 Jun 2013 17:28:10 GMT)
X-Complaints-To: usenet@ger.gmane.org
NNTP-Posting-Date: Sat, 22 Jun 2013 17:28:10 +0000 (UTC)
Cc: emacs-devel@gnu.org
To: Miguel Guedes <miguel.a.guedes@gmail.com>
Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Jun 22 19:28:09 2013
Return-path: <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>
Envelope-to: ged-emacs-devel@m.gmane.org
Original-Received: from lists.gnu.org ([208.118.235.17])
	by plane.gmane.org with esmtp (Exim 4.69)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>)
	id 1UqRbc-00022s-VZ
	for ged-emacs-devel@m.gmane.org; Sat, 22 Jun 2013 19:28:09 +0200
Original-Received: from localhost ([::1]:55867 helo=lists.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>)
	id 1UqRbc-0007Z8-3E
	for ged-emacs-devel@m.gmane.org; Sat, 22 Jun 2013 13:28:08 -0400
Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:55468)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1UqRbY-0007Z2-EH
	for emacs-devel@gnu.org; Sat, 22 Jun 2013 13:28:05 -0400
Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1UqRbX-0007mB-J6
	for emacs-devel@gnu.org; Sat, 22 Jun 2013 13:28:04 -0400
Original-Received: from smtp.cs.ucla.edu ([131.179.128.62]:50001)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <eggert@cs.ucla.edu>) id 1UqRbX-0007li-9u
	for emacs-devel@gnu.org; Sat, 22 Jun 2013 13:28:03 -0400
Original-Received: from localhost (localhost.localdomain [127.0.0.1])
	by smtp.cs.ucla.edu (Postfix) with ESMTP id 9883639E810A;
	Sat, 22 Jun 2013 10:28:00 -0700 (PDT)
X-Virus-Scanned: amavisd-new at smtp.cs.ucla.edu
Original-Received: from smtp.cs.ucla.edu ([127.0.0.1])
	by localhost (smtp.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id PdjiX72unApM; Sat, 22 Jun 2013 10:28:00 -0700 (PDT)
Original-Received: from penguin.cs.ucla.edu (Penguin.CS.UCLA.EDU [131.179.64.200])
	by smtp.cs.ucla.edu (Postfix) with ESMTPSA id 019C239E8109;
	Sat, 22 Jun 2013 10:27:59 -0700 (PDT)
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:17.0) Gecko/20130514 Thunderbird/17.0.6
In-Reply-To: <kq3ssm$b00$1@ger.gmane.org>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 131.179.128.62
X-BeenThere: emacs-devel@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "Emacs development discussions." <emacs-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-devel>,
	<mailto:emacs-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/emacs-devel>
List-Post: <mailto:emacs-devel@gnu.org>
List-Help: <mailto:emacs-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-devel>,
	<mailto:emacs-devel-request@gnu.org?subject=subscribe>
Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org
Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org
Xref: news.gmane.org gmane.emacs.devel:160891
Archived-At: <http://permalink.gmane.org/gmane.emacs.devel/160891>

On 06/22/13 03:05, Miguel Guedes wrote:

> In process.c, function read_process_output, all reads are limited to a 
> maximum of 4096 bytes.  What is the rationale behind imposing a hardcoded 
> limit of 4096 bytes per read from a process channel? (why isn't the 
> user allowed to specify this read limit when creating a process/
> connection?)

I don't know, and that limit has bothered me in the past, too.

It's a simple matter to increase it to 16 K bytes; does the
patch below help your performance?  If so, I can install it.
Letting the user specify a larger limit
might make sense, but would take a bit of thought and hacking.

=== modified file 'src/coding.h'
--- src/coding.h	2013-05-22 14:53:21 +0000
+++ src/coding.h	2013-06-22 16:53:36 +0000
@@ -399,6 +399,7 @@
   int rejected;
 };
 
+enum { CARRYOVER_BYTES_MAX = 64 };
 
 struct coding_system
 {
@@ -491,7 +492,7 @@
   /* Set to 1 if charbuf contains an annotation.  */
   unsigned annotated : 1;
 
-  unsigned char carryover[64];
+  unsigned char carryover[CARRYOVER_BYTES_MAX];
   int carryover_bytes;
 
   int default_char;

=== modified file 'src/process.c'
--- src/process.c	2013-06-22 16:43:39 +0000
+++ src/process.c	2013-06-22 17:08:46 +0000
@@ -4949,7 +4949,7 @@
    starting with our buffered-ahead character if we have one.
    Yield number of decoded characters read.
 
-   This function reads at most 4096 characters.
+   This function reads at most MAX_ALLOCA bytes.
    If you want to read all available subprocess output,
    you must call it repeatedly until it returns zero.
 
@@ -4959,16 +4959,16 @@
 static int
 read_process_output (Lisp_Object proc, register int channel)
 {
-  register ssize_t nbytes;
-  char *chars;
-  register struct Lisp_Process *p = XPROCESS (proc);
+  ssize_t nbytes;
+  char chars[MAX_ALLOCA];
+  struct Lisp_Process *p = XPROCESS (proc);
   struct coding_system *coding = proc_decode_coding_system[channel];
   int carryover = p->decoding_carryover;
-  int readmax = 4096;
+  verify (CARRYOVER_BYTES_MAX < sizeof chars);
+  int readmax = sizeof chars - carryover;
   ptrdiff_t count = SPECPDL_INDEX ();
   Lisp_Object odeactivate;
 
-  chars = alloca (carryover + readmax);
   if (carryover)
     /* See the comment above.  */
     memcpy (chars, SDATA (p->decoding_buf), carryover);