Re: Strange performance hit for insert-file-contents-literally()

[Nordlöw <per.nordlow@gmail.com>]

On Nov 27, 9:26 am, Nordlöw <per.nord...@gmail.com> wrote:
> On Nov 27, 8:58 am, Nordlöw <per.nord...@gmail.com> wrote:
>
>
>
>
>
> > I am implementing a package for doing recursive file system magic-
> > detect and scan/search/grep for pattern.
> > In this process I have stumbled across a strange performance problem;
>
> > If I want to grep for raw binary (literal) patterns, I thought that
> >   (benchmark-run 1 (with-temp-buffer (buffer-disable-undo) (insert-
> > file-contents-literally "~/pnw.tar.gz" nil)))
>
> > would given optimal performance but it turns out that
> >   (benchmark-run 1 (save-excursion (set-buffer (find-file-noselect "~/
> > pnw.tar.gz" t t))))
>
> > is about 11 times faster on my laptop PC (the first time its called of
> > course, the second time it reuses the open buffer so that doesn't
> > count)!.
>
> > As find-file-noselect() uses insert-file-contents-literally() it must
> > disable some stuff that find-file-literally() doesn't. The question is
> > what?
>
> > I would prefer no to be forced to use find-file-noselect() as this
> > does other unneccessary stuff I don't need.
> > It also reuses the buffer if its already open which is not always what
> > I want with compressed files for example.
> > My logic namely detects the file magic headers for compressed files
> > and then reopens them non-literally using jka-compr in order to detect
> > their underlying (decompressed) type like the unix program "file"
> > does.
>
> > Is this maybe a bug?
>
> > I am currently using Emacs CVS.
>
> > Thanks in advance for all tips,
> > Nordlöw
>
> I also need the sub-file aspect (BEG END) of  insert-file-contents-
> literally().
>
> I could get by with
> - find-file-noselect() when I want to scan the whole file a la grep
> and
> - insert-file-contents-literally() when I only want to open a part
> (typically beginning or end) when I want to detect its contents/type a
> al file.
>
> How can I detect if a file is already opened before calling find-file-
> noselect() on it?
> Reason: I don't want files that don't match my grep pattern to remain
> opened (kill its buffer).
>
> Please help! This package I am writing will soon be release to the
> public (EmacsWIki) and will be really useful!
> It is an elegant (I think) merge of the UNIX command line tools find,
> tree, file, grep and file all implemented in pure emacs lisp (making
> it independent of underlying OS read it will work on windows) with its
> own nice colored presentation/hit-navigation mode. It is compatible
> with TRAMP and does auto-compression (jka) of files that are being
> searched and supports emacs-style regexps (of course) making it simply
> the ultimate file system scanning tool for Emacs users.
>
> /Nordlöw

Note: The observed difference is for really large files ~250Mb in my
case. Not for files of size ~1Mb. I don't what the size limit for the
difference is.

/Nordlöw