From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Miguel Guedes Newsgroups: gmane.emacs.devel Subject: Re: process.c: read_process_output: hard coded 4096 bytes read limit Date: Sun, 23 Jun 2013 08:59:11 +0000 (UTC) Message-ID: References: <51C5DE9B.8010607@cs.ucla.edu> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Trace: ger.gmane.org 1371977978 14914 80.91.229.3 (23 Jun 2013 08:59:38 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 23 Jun 2013 08:59:38 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Jun 23 10:59:38 2013 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Uqg91-0005NU-DV for ged-emacs-devel@m.gmane.org; Sun, 23 Jun 2013 10:59:35 +0200 Original-Received: from localhost ([::1]:58164 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Uqg91-0006E2-3o for ged-emacs-devel@m.gmane.org; Sun, 23 Jun 2013 04:59:35 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:57912) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Uqg8y-0006DP-3z for emacs-devel@gnu.org; Sun, 23 Jun 2013 04:59:33 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Uqg8x-00071T-5n for emacs-devel@gnu.org; Sun, 23 Jun 2013 04:59:32 -0400 Original-Received: from plane.gmane.org ([80.91.229.3]:50802) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Uqg8w-00071G-VA for emacs-devel@gnu.org; Sun, 23 Jun 2013 04:59:31 -0400 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Uqg8v-0005F7-5t for emacs-devel@gnu.org; Sun, 23 Jun 2013 10:59:29 +0200 Original-Received: from 176.24.186.11 ([176.24.186.11]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 23 Jun 2013 10:59:29 +0200 Original-Received: from miguel.a.guedes by 176.24.186.11 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 23 Jun 2013 10:59:29 +0200 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 39 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 176.24.186.11 User-Agent: Pan/0.139 (Sexual Chocolate; GIT bf56508 git://git.gnome.org/pan2) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 80.91.229.3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:160903 Archived-At: On Sat, 22 Jun 2013 10:27:55 -0700, Paul Eggert wrote: > I don't know, and that limit has bothered me in the past, too. > > It's a simple matter to increase it to 16 K bytes; does the patch below > help your performance? If so, I can install it. Letting the user > specify a larger limit might make sense, but would take a bit of thought > and hacking. > Thank you for your reply, Paul! I am unable to apply and test the patch at the moment but I'm sure it will help with performance, though not entirely fix it. I have this minor mode that I'm developing that receives code completion data asynchronously from a server bound to localhost. The actual server side processing of code completion always takes less than 100ms and the results are sent to the client in full. It is then ironic (as well as frustrating) that the bottleneck (latency lasting many seconds) I'm experiencing is being caused by emacs when just receiving the raw data before even any processing takes place in the client side (which is very fast anyway)! There are a couple of ways I can use to mitigate the latency issues mentioned above so I already have a fix, however IMHO there is a problem in the way emacs reads data from process channels that needs to be fixed, which I wouldn't mind contributing to. I assume you're an active emacs developer (my own knowledge of emacs' internal code and architecture is extremely limited), how would you tackle this, Paul? Apart from raising the hardcoded limit, one (perhaps simpler) way would be to allow the user to specify the buffer size when spawning a process/ network connection, however that still leaves open the problem of the waits between reads. So perhaps the best strategy to fix this, though probably more complicated, would be to tackle the waits between reads head on? How problematic with this be? Apologies for not understanding but by "I can install it", did you mean you can commit it?