From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Paul Eggert Newsgroups: gmane.emacs.devel Subject: Re: process.c: read_process_output: hard coded 4096 bytes read limit Date: Sat, 22 Jun 2013 10:27:55 -0700 Message-ID: <51C5DE9B.8010607@cs.ucla.edu> References: NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1371922090 9665 80.91.229.3 (22 Jun 2013 17:28:10 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 22 Jun 2013 17:28:10 +0000 (UTC) Cc: emacs-devel@gnu.org To: Miguel Guedes Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Jun 22 19:28:09 2013 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1UqRbc-00022s-VZ for ged-emacs-devel@m.gmane.org; Sat, 22 Jun 2013 19:28:09 +0200 Original-Received: from localhost ([::1]:55867 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UqRbc-0007Z8-3E for ged-emacs-devel@m.gmane.org; Sat, 22 Jun 2013 13:28:08 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:55468) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UqRbY-0007Z2-EH for emacs-devel@gnu.org; Sat, 22 Jun 2013 13:28:05 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UqRbX-0007mB-J6 for emacs-devel@gnu.org; Sat, 22 Jun 2013 13:28:04 -0400 Original-Received: from smtp.cs.ucla.edu ([131.179.128.62]:50001) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UqRbX-0007li-9u for emacs-devel@gnu.org; Sat, 22 Jun 2013 13:28:03 -0400 Original-Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp.cs.ucla.edu (Postfix) with ESMTP id 9883639E810A; Sat, 22 Jun 2013 10:28:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at smtp.cs.ucla.edu Original-Received: from smtp.cs.ucla.edu ([127.0.0.1]) by localhost (smtp.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PdjiX72unApM; Sat, 22 Jun 2013 10:28:00 -0700 (PDT) Original-Received: from penguin.cs.ucla.edu (Penguin.CS.UCLA.EDU [131.179.64.200]) by smtp.cs.ucla.edu (Postfix) with ESMTPSA id 019C239E8109; Sat, 22 Jun 2013 10:27:59 -0700 (PDT) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130514 Thunderbird/17.0.6 In-Reply-To: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 131.179.128.62 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:160891 Archived-At: On 06/22/13 03:05, Miguel Guedes wrote: > In process.c, function read_process_output, all reads are limited to a > maximum of 4096 bytes. What is the rationale behind imposing a hardcoded > limit of 4096 bytes per read from a process channel? (why isn't the > user allowed to specify this read limit when creating a process/ > connection?) I don't know, and that limit has bothered me in the past, too. It's a simple matter to increase it to 16 K bytes; does the patch below help your performance? If so, I can install it. Letting the user specify a larger limit might make sense, but would take a bit of thought and hacking. === modified file 'src/coding.h' --- src/coding.h 2013-05-22 14:53:21 +0000 +++ src/coding.h 2013-06-22 16:53:36 +0000 @@ -399,6 +399,7 @@ int rejected; }; +enum { CARRYOVER_BYTES_MAX = 64 }; struct coding_system { @@ -491,7 +492,7 @@ /* Set to 1 if charbuf contains an annotation. */ unsigned annotated : 1; - unsigned char carryover[64]; + unsigned char carryover[CARRYOVER_BYTES_MAX]; int carryover_bytes; int default_char; === modified file 'src/process.c' --- src/process.c 2013-06-22 16:43:39 +0000 +++ src/process.c 2013-06-22 17:08:46 +0000 @@ -4949,7 +4949,7 @@ starting with our buffered-ahead character if we have one. Yield number of decoded characters read. - This function reads at most 4096 characters. + This function reads at most MAX_ALLOCA bytes. If you want to read all available subprocess output, you must call it repeatedly until it returns zero. @@ -4959,16 +4959,16 @@ static int read_process_output (Lisp_Object proc, register int channel) { - register ssize_t nbytes; - char *chars; - register struct Lisp_Process *p = XPROCESS (proc); + ssize_t nbytes; + char chars[MAX_ALLOCA]; + struct Lisp_Process *p = XPROCESS (proc); struct coding_system *coding = proc_decode_coding_system[channel]; int carryover = p->decoding_carryover; - int readmax = 4096; + verify (CARRYOVER_BYTES_MAX < sizeof chars); + int readmax = sizeof chars - carryover; ptrdiff_t count = SPECPDL_INDEX (); Lisp_Object odeactivate; - chars = alloca (carryover + readmax); if (carryover) /* See the comment above. */ memcpy (chars, SDATA (p->decoding_buf), carryover);