Strange performance hit for insert-file-contents-literally()

Strange performance hit for insert-file-contents-literally()

[Nordlöw]

I am implementing a package for doing recursive file system magic-
detect and scan/search/grep for pattern.
In this process I have stumbled across a strange performance problem;

If I want to grep for raw binary (literal) patterns, I thought that
  (benchmark-run 1 (with-temp-buffer (buffer-disable-undo) (insert-
file-contents-literally "~/pnw.tar.gz" nil)))

would given optimal performance but it turns out that
  (benchmark-run 1 (save-excursion (set-buffer (find-file-noselect "~/
pnw.tar.gz" t t))))

is about 11 times faster on my laptop PC (the first time its called of
course, the second time it reuses the open buffer so that doesn't
count)!.

As find-file-noselect() uses insert-file-contents-literally() it must
disable some stuff that find-file-literally() doesn't. The question is
what?

I would prefer no to be forced to use find-file-noselect() as this
does other unneccessary stuff I don't need.
It also reuses the buffer if its already open which is not always what
I want with compressed files for example.
My logic namely detects the file magic headers for compressed files
and then reopens them non-literally using jka-compr in order to detect
their underlying (decompressed) type like the unix program "file"
does.

Is this maybe a bug?

I am currently using Emacs CVS.

Thanks in advance for all tips,
Nordlöw

Re: Strange performance hit for insert-file-contents-literally()

[Nordlöw]

On Nov 27, 8:58 am, Nordlöw <per.nord...@gmail.com> wrote:
> I am implementing a package for doing recursive file system magic-
> detect and scan/search/grep for pattern.
> In this process I have stumbled across a strange performance problem;
>
> If I want to grep for raw binary (literal) patterns, I thought that
>   (benchmark-run 1 (with-temp-buffer (buffer-disable-undo) (insert-
> file-contents-literally "~/pnw.tar.gz" nil)))
>
> would given optimal performance but it turns out that
>   (benchmark-run 1 (save-excursion (set-buffer (find-file-noselect "~/
> pnw.tar.gz" t t))))
>
> is about 11 times faster on my laptop PC (the first time its called of
> course, the second time it reuses the open buffer so that doesn't
> count)!.
>
> As find-file-noselect() uses insert-file-contents-literally() it must
> disable some stuff that find-file-literally() doesn't. The question is
> what?
>
> I would prefer no to be forced to use find-file-noselect() as this
> does other unneccessary stuff I don't need.
> It also reuses the buffer if its already open which is not always what
> I want with compressed files for example.
> My logic namely detects the file magic headers for compressed files
> and then reopens them non-literally using jka-compr in order to detect
> their underlying (decompressed) type like the unix program "file"
> does.
>
> Is this maybe a bug?
>
> I am currently using Emacs CVS.
>
> Thanks in advance for all tips,
> Nordlöw

I also need the sub-file aspect (BEG END) of  insert-file-contents-
literally().

I could get by with
- find-file-noselect() when I want to scan the whole file a la grep
and
- insert-file-contents-literally() when I only want to open a part
(typically beginning or end) when I want to detect its contents/type a
al file.

How can I detect if a file is already opened before calling find-file-
noselect() on it?
Reason: I don't want files that don't match my grep pattern to remain
opened (kill its buffer).

Please help! This package I am writing will soon be release to the
public (EmacsWIki) and will be really useful!
It is an elegant (I think) merge of the UNIX command line tools find,
tree, file, grep and file all implemented in pure emacs lisp (making
it independent of underlying OS read it will work on windows) with its
own nice colored presentation/hit-navigation mode. It is compatible
with TRAMP and does auto-compression (jka) of files that are being
searched and supports emacs-style regexps (of course) making it simply
the ultimate file system scanning tool for Emacs users.

/Nordlöw

Re: Strange performance hit for insert-file-contents-literally()

[Nordlöw]

On Nov 27, 9:26 am, Nordlöw <per.nord...@gmail.com> wrote:
> On Nov 27, 8:58 am, Nordlöw <per.nord...@gmail.com> wrote:
>
>
>
>
>
> > I am implementing a package for doing recursive file system magic-
> > detect and scan/search/grep for pattern.
> > In this process I have stumbled across a strange performance problem;
>
> > If I want to grep for raw binary (literal) patterns, I thought that
> >   (benchmark-run 1 (with-temp-buffer (buffer-disable-undo) (insert-
> > file-contents-literally "~/pnw.tar.gz" nil)))
>
> > would given optimal performance but it turns out that
> >   (benchmark-run 1 (save-excursion (set-buffer (find-file-noselect "~/
> > pnw.tar.gz" t t))))
>
> > is about 11 times faster on my laptop PC (the first time its called of
> > course, the second time it reuses the open buffer so that doesn't
> > count)!.
>
> > As find-file-noselect() uses insert-file-contents-literally() it must
> > disable some stuff that find-file-literally() doesn't. The question is
> > what?
>
> > I would prefer no to be forced to use find-file-noselect() as this
> > does other unneccessary stuff I don't need.
> > It also reuses the buffer if its already open which is not always what
> > I want with compressed files for example.
> > My logic namely detects the file magic headers for compressed files
> > and then reopens them non-literally using jka-compr in order to detect
> > their underlying (decompressed) type like the unix program "file"
> > does.
>
> > Is this maybe a bug?
>
> > I am currently using Emacs CVS.
>
> > Thanks in advance for all tips,
> > Nordlöw
>
> I also need the sub-file aspect (BEG END) of  insert-file-contents-
> literally().
>
> I could get by with
> - find-file-noselect() when I want to scan the whole file a la grep
> and
> - insert-file-contents-literally() when I only want to open a part
> (typically beginning or end) when I want to detect its contents/type a
> al file.
>
> How can I detect if a file is already opened before calling find-file-
> noselect() on it?
> Reason: I don't want files that don't match my grep pattern to remain
> opened (kill its buffer).
>
> Please help! This package I am writing will soon be release to the
> public (EmacsWIki) and will be really useful!
> It is an elegant (I think) merge of the UNIX command line tools find,
> tree, file, grep and file all implemented in pure emacs lisp (making
> it independent of underlying OS read it will work on windows) with its
> own nice colored presentation/hit-navigation mode. It is compatible
> with TRAMP and does auto-compression (jka) of files that are being
> searched and supports emacs-style regexps (of course) making it simply
> the ultimate file system scanning tool for Emacs users.
>
> /Nordlöw

Note: The observed difference is for really large files ~250Mb in my
case. Not for files of size ~1Mb. I don't what the size limit for the
difference is.

/Nordlöw

RE: Strange performance hit for insert-file-contents-literally()

[Drew Adams]

> How can I detect if a file is already opened before calling find-file-
> noselect() on it?

`C-h f buffer-file-name'
`C-h f buffer-list'

RE: Strange performance hit for insert-file-contents-literally()

[Drew Adams]

> Note: The observed difference is for really large files ~250Mb in my
> case. Not for files of size ~1Mb. I don't what the size limit for the
> difference is.

`C-h f file-attributes', attribute number 7 (8th attribute) is file size.