From mboxrd@z Thu Jan  1 00:00:00 1970
Path: news.gmane.org!not-for-mail
From: Miguel Guedes <miguel.a.guedes@gmail.com>
Newsgroups: gmane.emacs.devel
Subject: Re: process.c: read_process_output: hard coded 4096 bytes read limit
Date: Sun, 23 Jun 2013 08:59:11 +0000 (UTC)
Message-ID: <kq6dcu$mlm$1@ger.gmane.org>
References: <kq3ssm$b00$1@ger.gmane.org> <51C5DE9B.8010607@cs.ucla.edu>
NNTP-Posting-Host: plane.gmane.org
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Trace: ger.gmane.org 1371977978 14914 80.91.229.3 (23 Jun 2013 08:59:38 GMT)
X-Complaints-To: usenet@ger.gmane.org
NNTP-Posting-Date: Sun, 23 Jun 2013 08:59:38 +0000 (UTC)
To: emacs-devel@gnu.org
Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Jun 23 10:59:38 2013
Return-path: <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>
Envelope-to: ged-emacs-devel@m.gmane.org
Original-Received: from lists.gnu.org ([208.118.235.17])
	by plane.gmane.org with esmtp (Exim 4.69)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>)
	id 1Uqg91-0005NU-DV
	for ged-emacs-devel@m.gmane.org; Sun, 23 Jun 2013 10:59:35 +0200
Original-Received: from localhost ([::1]:58164 helo=lists.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>)
	id 1Uqg91-0006E2-3o
	for ged-emacs-devel@m.gmane.org; Sun, 23 Jun 2013 04:59:35 -0400
Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:57912)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ged-emacs-devel@m.gmane.org>) id 1Uqg8y-0006DP-3z
	for emacs-devel@gnu.org; Sun, 23 Jun 2013 04:59:33 -0400
Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <ged-emacs-devel@m.gmane.org>) id 1Uqg8x-00071T-5n
	for emacs-devel@gnu.org; Sun, 23 Jun 2013 04:59:32 -0400
Original-Received: from plane.gmane.org ([80.91.229.3]:50802)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <ged-emacs-devel@m.gmane.org>) id 1Uqg8w-00071G-VA
	for emacs-devel@gnu.org; Sun, 23 Jun 2013 04:59:31 -0400
Original-Received: from list by plane.gmane.org with local (Exim 4.69)
	(envelope-from <ged-emacs-devel@m.gmane.org>) id 1Uqg8v-0005F7-5t
	for emacs-devel@gnu.org; Sun, 23 Jun 2013 10:59:29 +0200
Original-Received: from 176.24.186.11 ([176.24.186.11])
	by main.gmane.org with esmtp (Gmexim 0.1 (Debian))
	id 1AlnuQ-0007hv-00
	for <emacs-devel@gnu.org>; Sun, 23 Jun 2013 10:59:29 +0200
Original-Received: from miguel.a.guedes by 176.24.186.11 with local (Gmexim 0.1
	(Debian)) id 1AlnuQ-0007hv-00
	for <emacs-devel@gnu.org>; Sun, 23 Jun 2013 10:59:29 +0200
X-Injected-Via-Gmane: http://gmane.org/
Original-Lines: 39
Original-X-Complaints-To: usenet@ger.gmane.org
X-Gmane-NNTP-Posting-Host: 176.24.186.11
User-Agent: Pan/0.139 (Sexual Chocolate; GIT bf56508 git://git.gnome.org/pan2)
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-Received-From: 80.91.229.3
X-BeenThere: emacs-devel@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "Emacs development discussions." <emacs-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-devel>,
	<mailto:emacs-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/emacs-devel>
List-Post: <mailto:emacs-devel@gnu.org>
List-Help: <mailto:emacs-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-devel>,
	<mailto:emacs-devel-request@gnu.org?subject=subscribe>
Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org
Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org
Xref: news.gmane.org gmane.emacs.devel:160903
Archived-At: <http://permalink.gmane.org/gmane.emacs.devel/160903>

On Sat, 22 Jun 2013 10:27:55 -0700, Paul Eggert wrote:

> I don't know, and that limit has bothered me in the past, too.
> 
> It's a simple matter to increase it to 16 K bytes; does the patch below
> help your performance?  If so, I can install it. Letting the user
> specify a larger limit might make sense, but would take a bit of thought
> and hacking.
> 

Thank you for your reply, Paul!

I am unable to apply and test the patch at the moment but I'm sure it 
will help with performance, though not entirely fix it.  I have this 
minor mode that I'm developing that receives code completion data 
asynchronously from a server bound to localhost.  The actual server side 
processing of code completion always takes less than 100ms and the 
results are sent to the client in full.  It is then ironic (as well as 
frustrating) that the bottleneck (latency lasting many seconds) I'm 
experiencing is being caused by emacs when just receiving the raw data 
before even any processing takes place in the client side (which is very 
fast anyway)!

There are a couple of ways I can use to mitigate the latency issues 
mentioned above so I already have a fix, however IMHO there is a problem 
in the way emacs reads data from process channels that needs to be fixed, 
which I wouldn't mind contributing to.  I assume you're an active emacs 
developer (my own knowledge of emacs' internal code and architecture is 
extremely limited), how would you tackle this, Paul?  

Apart from raising the hardcoded limit, one (perhaps simpler) way would 
be to allow the user to specify the buffer size when spawning a process/
network connection, however that still leaves open the problem of the 
waits between reads.  So perhaps the best strategy to fix this, though 
probably more complicated, would be to tackle the waits between reads 
head on?  How problematic with this be?

Apologies for not understanding but by "I can install it", did you mean 
you can commit it?