From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.bugs Subject: bug#2790: emacs 22.1.1 cannot open 5GB file on 64GB 64-bit GNU/Linux box Date: Fri, 27 Mar 2009 22:12:54 -0400 Message-ID: References: <3c6c07c20903260850r180e942dscb2c61d1096793f8@mail.gmail.com> <3c6c07c20903270927y5292e32as857233aa1fd75737@mail.gmail.com> Reply-To: Stefan Monnier , 2790@emacsbugs.donarmstrong.com NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1238207048 16270 80.91.229.12 (28 Mar 2009 02:24:08 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 28 Mar 2009 02:24:08 +0000 (UTC) Cc: 2790@emacsbugs.donarmstrong.com To: Mike Coleman Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Mar 28 03:25:26 2009 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1LnOEe-0000OM-4M for geb-bug-gnu-emacs@m.gmane.org; Sat, 28 Mar 2009 03:25:24 +0100 Original-Received: from localhost ([127.0.0.1]:47123 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LnODG-00011i-LF for geb-bug-gnu-emacs@m.gmane.org; Fri, 27 Mar 2009 22:23:58 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1LnODB-00011d-QJ for bug-gnu-emacs@gnu.org; Fri, 27 Mar 2009 22:23:53 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1LnOD6-0000vL-8X for bug-gnu-emacs@gnu.org; Fri, 27 Mar 2009 22:23:52 -0400 Original-Received: from [199.232.76.173] (port=49695 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LnOD6-0000vC-60 for bug-gnu-emacs@gnu.org; Fri, 27 Mar 2009 22:23:48 -0400 Original-Received: from rzlab.ucr.edu ([138.23.92.77]:53818) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1LnOD5-0003wO-M5 for bug-gnu-emacs@gnu.org; Fri, 27 Mar 2009 22:23:47 -0400 Original-Received: from rzlab.ucr.edu (rzlab.ucr.edu [127.0.0.1]) by rzlab.ucr.edu (8.13.8/8.13.8/Debian-3) with ESMTP id n2S2NjUq027542; Fri, 27 Mar 2009 19:23:45 -0700 Original-Received: (from debbugs@localhost) by rzlab.ucr.edu (8.13.8/8.13.8/Submit) id n2S2K2NX026340; Fri, 27 Mar 2009 19:20:02 -0700 X-Loop: owner@emacsbugs.donarmstrong.com Resent-From: Stefan Monnier Resent-To: bug-submit-list@donarmstrong.com Resent-CC: Emacs Bugs Resent-Date: Sat, 28 Mar 2009 02:20:02 +0000 Resent-Message-ID: Resent-Sender: owner@emacsbugs.donarmstrong.com X-Emacs-PR-Message: followup 2790 X-Emacs-PR-Package: emacs X-Emacs-PR-Keywords: Original-Received: via spool by 2790-submit@emacsbugs.donarmstrong.com id=B2790.123820638525002 (code B ref 2790); Sat, 28 Mar 2009 02:20:02 +0000 Original-Received: (at 2790) by emacsbugs.donarmstrong.com; 28 Mar 2009 02:13:05 +0000 X-Spam-Bayes: score:0.5 Bayes not run. spammytokens:Tokens not available. hammytokens:Tokens not available. Original-Received: from ironport2-out.teksavvy.com (ironport2-out.pppoe.ca [206.248.154.182]) by rzlab.ucr.edu (8.13.8/8.13.8/Debian-3) with ESMTP id n2S2D1d8024993 for <2790@emacsbugs.donarmstrong.com>; Fri, 27 Mar 2009 19:13:02 -0700 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvgEAAQlzUlFxIfy/2dsb2JhbACBUc5Vg3cGhHU X-IronPort-AV: E=Sophos;i="4.38,436,1233550800"; d="scan'208";a="35797682" Original-Received: from 69-196-135-242.dsl.teksavvy.com (HELO pastel.home) ([69.196.135.242]) by ironport2-out.teksavvy.com with ESMTP; 27 Mar 2009 22:12:55 -0400 Original-Received: by pastel.home (Postfix, from userid 20848) id 1B1588682; Fri, 27 Mar 2009 22:12:54 -0400 (EDT) In-Reply-To: <3c6c07c20903270927y5292e32as857233aa1fd75737@mail.gmail.com> (Mike Coleman's message of "Fri, 27 Mar 2009 11:27:23 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.91 (gnu/linux) X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6 (newer, 3) Resent-Date: Fri, 27 Mar 2009 22:23:52 -0400 X-BeenThere: bug-gnu-emacs@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:26694 Archived-At: > Okay, tried it (emacs-23.0.91), but no luck. Looks very nice, but > finding that large file produced the same error. The value of > 'most-positive-fixnum' prints correctly, though (which is different). There was an incorrect check that limited the size to INT_MAX/4 (i.e. 512MB for systems where ints are 32bit). I've removed this check in the CVS code (see patch below). I am now able to open an 800MB file with this check removed. OTOH with a 2GB file, I got some unjustified "memory exhausted" error, but I haven't tracked it down yet. Note also that when you open large files, it's worthwhile to use find-file-literally to be sure it's opened in unibyte mode; otherwise it gets decoded which takes ages. Also if the file has many lines (my 800MB file was made up by copying a C file many times, so it had millions of lines), turning off line-number-mode is is needed to recover responsiveness when navigating near the end of the buffer. Stefan Index: src/fileio.c =================================================================== RCS file: /sources/emacs/emacs/src/fileio.c,v retrieving revision 1.651 retrieving revision 1.652 diff -u -r1.651 -r1.652 --- src/fileio.c 24 Mar 2009 14:14:54 -0000 1.651 +++ src/fileio.c 28 Mar 2009 02:06:08 -0000 1.652 @@ -3300,7 +3300,11 @@ overflow. The calculations below double the file size twice, so check that it can be multiplied by 4 safely. */ if (XINT (end) != st.st_size - || st.st_size > INT_MAX / 4) + /* Actually, it should test either INT_MAX or LONG_MAX + depending on which one is used for EMACS_INT. But in + any case, in practice, this test is redundant with the + one above. + || st.st_size > INT_MAX / 4 */) error ("Maximum buffer size exceeded"); /* The file size returned from stat may be zero, but data