From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.devel Subject: Re: map-file-lines Date: Tue, 03 Feb 2009 07:57:47 -0600 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: <864ozb7hd0.fsf@lifelogs.com> References: <86wsc87o3c.fsf@lifelogs.com> <86skmw7ig6.fsf@lifelogs.com> <86fxiw7e5w.fsf@lifelogs.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1233669707 16894 80.91.229.12 (3 Feb 2009 14:01:47 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 3 Feb 2009 14:01:47 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Feb 03 15:03:02 2009 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1LULrG-0005L6-I3 for ged-emacs-devel@m.gmane.org; Tue, 03 Feb 2009 15:02:34 +0100 Original-Received: from localhost ([127.0.0.1]:43267 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LULpx-0000QR-Fp for ged-emacs-devel@m.gmane.org; Tue, 03 Feb 2009 09:01:13 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1LULlD-0006O5-GO for emacs-devel@gnu.org; Tue, 03 Feb 2009 08:56:20 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1LULlC-0006NI-Ii for emacs-devel@gnu.org; Tue, 03 Feb 2009 08:56:18 -0500 Original-Received: from [199.232.76.173] (port=52865 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LULlA-0006N7-5h for emacs-devel@gnu.org; Tue, 03 Feb 2009 08:56:16 -0500 Original-Received: from main.gmane.org ([80.91.229.2]:39182 helo=ciao.gmane.org) by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1LULl9-0001hB-Gb for emacs-devel@gnu.org; Tue, 03 Feb 2009 08:56:15 -0500 Original-Received: from list by ciao.gmane.org with local (Exim 4.43) id 1LULl9-0001eY-3m for emacs-devel@gnu.org; Tue, 03 Feb 2009 13:56:15 +0000 Original-Received: from 38.98.147.130 ([38.98.147.130]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 03 Feb 2009 13:56:15 +0000 Original-Received: from tzz by 38.98.147.130 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 03 Feb 2009 13:56:15 +0000 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 54 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 38.98.147.130 X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6; d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.0.60 (gnu/linux) Cancel-Lock: sha1:JTh9jjQuf+CTQZwg75wXQEQdlkg= X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:108677 Archived-At: On Mon, 02 Feb 2009 17:42:31 -0500 Stefan Monnier wrote: SM> How would you use map-file-lines to implement the "large file viewing SM> and searching"? ... SM> I know about the large-file problem, obviously, but I wonder what kind SM> of UI you expect to provide, in order for it to be able to work just one SM> line at a time. ... SM> I don't se how such a "line-at-a-time" or "chunk-at-a-time" processing SM> (i.e. stream processing) will enable Emacs to let you conveniently edit SM> a large binary file. ... SM> OK, that's indeed how I imagine it as well, but I fail to see how this SM> relates to map-file-lines. All you need for that is to use the BEG and SM> END args of insert-file-contents (and maybe also to extend those args SM> so they can be floats, in case Emacs's ints are too limited). On Tue, 03 Feb 2009 08:27:01 +0100 joakim@verona.se wrote: j> That being said, how would the insert-file-contents solution work in j> practice? Has something been done along these lines already? Would it be j> possible to make some kind of generic solution that would make for j> instance hexl mode work on large files? I tried to condense the various messages into one reply. map-file-lines as it stands is just a stream processor, and not useful to *view* a large file. Unfortunately Emacs does almost everything in the buffer, so true large-file view and edit requires core-level work to make the buffer able to address the whole file. In addition, almost every Emacs package assumes that the buffer is small and can be scanned quickly; imagine how slow it will be to open and edit a 20GB gzipped file. map-file-lines was much easier to implement, and (just like `grep' and many other utilities) can be useful on its own despite the single-line limitations. So to answer your and Joakim's questions, based on what I know so far, the best approach is to have a special narrowing mode to view large files. It won't use map-file-lines, but like it, it will fetch the next chunk only when needed. It needs to be special because the normal narrow/widen calls should not widen to the whole file. It's best, in fact, if the decision to move the "window" into the file back and forth is only left to the user and to special motion commands, and normal packages can not move that window. Writing inserted text, in particular, is very slow with large files. A binary editor is not so hard to implement, but inserting text may need special permission (like the motion commands above) to make the user experience bearable. Ted