Re: Emacs opens only first 16384 bytes of file?!

[Christoph Groth <christoph@grothesque.org>]

Ruijie Yu wrote:
> Óscar Fuentes <ofv@wanadoo.es> writes:
>
> > [...]
> > As of today (Emacs 30, current development branch) the same is true.
> >
> > /proc/cpuinfo is not a regular file, in the sense that its content
> > is not stored in a device. AFAIK it is generated on-the-fly when it
> > is read.
> >
> > You can do some simple observations:
> >
> > $ ls -l /proc/cpuinfo
> > -r--r--r-- 1 root root 0 feb  7 03:34 /proc/cpuinfo
> > $ du /proc/cpuinfo
> > 0       /proc/cpuinfo
> >
> > Here those tools are saying that the file's size is 0.
>
> Thanks for elaborating on the part that I missed.  So, to reproduce
> the issue, we need to find a file which does not show size, and whose
> contents are generated on-the-fly when read -- in the original report,
> it is /proc/cpuinfo, and I have also found /proc/kallsyms which
> contains around 12M of data.  In this case, I have reproduced the
> issue on my Emacs 29 build.

Yes, that’s it, thanks for examining this further!

> > I guess that Emacs detects that the file is special and reads its
> > contents following some heuristics. What surprises me is that M-x
> > revert-file actually reads all the content.
> >
> > Of course, looking at the sources would be enlightening, but why do the
> > effort of actually clearing the matter when it is so cheap to throw
> > speculation? ;-)
>
> I think the limit of 16384 is probably caused by fileio.c:3919
> (907fd1f7ff4 somewhere on master, inside DEFUN "insert-file-contents")
> which declares a read buffer.  The constant READ_BUF_SIZE is
> indirectly set as 16 * 1024 = 16384.  Didn't read further there, nor
> inside `revert-buffer'.

Indeed this seems to be related to the advertised file size being zero.

However, no other programs (including editors) seem to have a problem
with this.  So it really must be Emacs trying to be clever.

Finding files under /proc/ is, of course, not a very relevant
application of Emacs, but Elisp code gets posted that relies on this:
https://nullprogram.com/blog/2015/10/14/

I had something similar in my init.el to determine the value to use with
"make -j" as a default compile command - this is how I noticed this
issue in the first place.  (Now I changed that to executing the command
nproc.)

What worries me more is that there may be other, more sneaky, cases
where this backfires.  For example, perhaps opening a file that is being
written (so that its size increases), is broken as well in some cases?

Should I report this to bug-gnu-emacs@gnu.org, or is posting on this
list enough?  Or should I open a ticket with debbugs.gnu.org?

Cheers
Christoph