bug#69492: File too big for Emacs?

bug#69492: File too big for Emacs?

[Robert Boyer]

[-- Attachment #1: Type: text/plain, Size: 5635 bytes --]

Dear Emacs bug exterminators,

The 1 gb file for which I give a url below does not seem to work in Emacs.

The file enumerates the primes below 10^9, so it would be very handy to
have around.

I can get the file into an Emacs buffer in literal mode.  Thanks for that.

But then I cannot move to the bottom, i.e., using M->.  In fact, in the
attempt to
move to the bottom, things go so badly that I have to reboot my $100
Lenovo Chromebook.

Is it just too big for Emacs to handle?


https://drive.google.com/file/d/1IaRNZ1rUQAZ72A7rJYpescmnJhpuGliA/view?usp=sharing

Thanks,

Bob

P. S.  Maybe this is some kind of joke.  In 1972, J Moore and I coded the
'77 editor', which could handle
a file of any size, using the 'pieces' approach that Microsoft later
adopted.  I'll try to see what
I can do with Word, if I can afford a copy.  Alas, the ICL 4130 on which
the 77 editor ran is no longer available.
I think that ICL was absorbed into Fujitsu, the cause of the British postal
scandal.

Here's the stuff you like with bug reports.

From: bob <bob@penguin>
To: bug-gnu-emacs@gnu.org
Subject: 28.2; File too big?
--text follows this line--




In GNU Emacs 28.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.37,
cairo version 1.16.0)
 of 2023-05-13, modified by Debian built on x86-ubc-01
Windowing system distributor 'The X.Org Foundation', version 11.0.12014000
System Description: Debian GNU/Linux 12 (bookworm)

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/libexec
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/28.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/28.2/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils
 --with-native-compilation --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/libexec
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/28.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/28.2/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils
 --with-native-compilation --with-cairo --with-x=yes
 --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -ffile-prefix-map=/build/emacs-mPr7Vr/emacs-28.2+1=.
 -fstack-protector-strong -Wformat -Werror=format-security -Wall'
 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF
TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Text

Minor modes in effect:
  shell-dirtrack-mode: t
  display-time-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny rfc822 mml mml-sec epa
derived epg rfc6068 epg-config gnus-util text-property-search mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail time-date help-fns radix-tree cl-print debug backtrace
help-mode find-func dired-aux cus-edit pp cus-load wid-edit trace
sh-script smie executable dired dired-loaddefs cal-menu calendar
cal-loaddefs ange-ftp shell pcomplete comint ansi-color ring benchmark
time rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums mm-util mail-prsvr
mail-utils face-remap finder-inf package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf
url-util mailcap url-handlers url-parse auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
url-vars seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo move-toolbar gtk x-toolkit x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 135967 8344)
 (symbols 48 11618 0)
 (strings 32 36783 2556)
 (string-bytes 1 1163174)
 (vectors 16 21452)
 (vector-slots 8 422610 15484)
 (floats 8 52 74)
 (intervals 56 625 0)
 (buffers 992 19))

[-- Attachment #2: Type: text/html, Size: 6452 bytes --]

bug#69492: File too big for Emacs?

[Eli Zaretskii]

> Cc: J Moore <moore@cs.utexas.edu>, Alan Bundy <bundy@ed.ac.uk>,
>  Grant Passmore <grant@imandra.ai>, rms@gnu.org,
>  Stas Boukarev <stassats@gmail.com>
> From: Robert Boyer <robertstephenboyer@gmail.com>
> Date: Fri, 1 Mar 2024 17:30:56 -0600
> 
> Dear Emacs bug exterminators,
> 
> The 1 gb file for which I give a url below does not seem to work in Emacs.
> 
> The file enumerates the primes below 10^9, so it would be very handy to
> have around.
> 
> I can get the file into an Emacs buffer in literal mode.  Thanks for that.
> 
> But then I cannot move to the bottom, i.e., using M->.  In fact, in the attempt to
> move to the bottom, things go so badly that I have to reboot my $100 
> Lenovo Chromebook.
> 
> Is it just too big for Emacs to handle?

Since your Emacs is a 64-bit build, a 1GB file shouldn't be too big,
provided that your system has enough virtual memory to support reading
it into memory (which might require more than 1GB, perhaps up to 2GB).
If Emacs says "Memory exhausted" when you visit that file, try
enlarging the size of swap for your system.

bug#69492: File too big for Emacs?

[Robert Boyer]

[-- Attachment #1: Type: text/plain, Size: 1463 bytes --]

I did not seem to have trouble finding the file, literally.

It showed up ok in a buffer, as I said in my bug report.

The problem I had was that I could not move to the bottom of the buffer
with M->.

Can you?

Bob


On Sat, Mar 2, 2024 at 12:55 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > Cc: J Moore <moore@cs.utexas.edu>, Alan Bundy <bundy@ed.ac.uk>,
> >  Grant Passmore <grant@imandra.ai>, rms@gnu.org,
> >  Stas Boukarev <stassats@gmail.com>
> > From: Robert Boyer <robertstephenboyer@gmail.com>
> > Date: Fri, 1 Mar 2024 17:30:56 -0600
> >
> > Dear Emacs bug exterminators,
> >
> > The 1 gb file for which I give a url below does not seem to work in
> Emacs.
> >
> > The file enumerates the primes below 10^9, so it would be very handy to
> > have around.
> >
> > I can get the file into an Emacs buffer in literal mode.  Thanks for
> that.
> >
> > But then I cannot move to the bottom, i.e., using M->.  In fact, in the
> attempt to
> > move to the bottom, things go so badly that I have to reboot my $100
> > Lenovo Chromebook.
> >
> > Is it just too big for Emacs to handle?
>
> Since your Emacs is a 64-bit build, a 1GB file shouldn't be too big,
> provided that your system has enough virtual memory to support reading
> it into memory (which might require more than 1GB, perhaps up to 2GB).
> If Emacs says "Memory exhausted" when you visit that file, try
> enlarging the size of swap for your system.
>

[-- Attachment #2: Type: text/html, Size: 2331 bytes --]

bug#69492: File too big for Emacs?

[Eli Zaretskii]

> From: Robert Boyer <robertstephenboyer@gmail.com>
> Date: Sat, 2 Mar 2024 02:54:47 -0600
> Cc: 69492@debbugs.gnu.org, moore@cs.utexas.edu, bundy@ed.ac.uk, 
> 	grant@imandra.ai, rms@gnu.org, stassats@gmail.com
> 
> I did not seem to have trouble finding the file, literally.
> 
> It showed up ok in a buffer, as I said in my bug report.  
> 
> The problem I had was that I could not move to the bottom of the buffer with M->.

What exactly is the problem?  My guess is that the problem is the same
I mentioned: your system doesn't have enough VM.  What does the
command 'free' display if you invoke it from the shell prompt?

bug#69492: File too big for Emacs?

[Robert Boyer]

[-- Attachment #1: Type: text/plain, Size: 1046 bytes --]

> What does the command 'free' display if you invoke it from the shell
prompt?

> free
               total        used        free      shared  buff/cache
available
Mem:         2827236     1172052     1010804       22736      644380
1655184
Swap:              0           0           0
>

On Sat, Mar 2, 2024 at 5:10 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Robert Boyer <robertstephenboyer@gmail.com>
> > Date: Sat, 2 Mar 2024 02:54:47 -0600
> > Cc: 69492@debbugs.gnu.org, moore@cs.utexas.edu, bundy@ed.ac.uk,
> >       grant@imandra.ai, rms@gnu.org, stassats@gmail.com
> >
> > I did not seem to have trouble finding the file, literally.
> >
> > It showed up ok in a buffer, as I said in my bug report.
> >
> > The problem I had was that I could not move to the bottom of the buffer
> with M->.
>
> What exactly is the problem?  My guess is that the problem is the same
> I mentioned: your system doesn't have enough VM.  What does the
> command 'free' display if you invoke it from the shell prompt?
>

[-- Attachment #2: Type: text/html, Size: 1906 bytes --]

bug#69492: File too big for Emacs?

[Robert Boyer]

[-- Attachment #1: Type: text/plain, Size: 2261 bytes --]

I suspect that I have a Chromebook/Gnu-Linux problem, not an Emacs problem.

Using 'crosh', the terminal for the Chromebook, I did:

   swap enable 4000

and then rebooted.

After the reboot, Chrosh reports a lot of swap space:

crosh> free
               total        used        free      shared  buff/cache
available
Mem:         3881392     1615268       48408     1896264     2217716
 299596
Swap:        4095996     2636800     1459196
crosh>

However, the free command when typed into the Emacs shell, running under
Linux, still says:

> free
               total        used        free      shared  buff/cache
available
Mem:         2827236     1178476     1075032       22224      573728
1648760
Swap:              0           0           0
>

It seems possible that my problem may have a solution, namely to wipe my
Linux clean and start over from scratch.

I thank you for your wise and very kind help, and I will let you know if I
make any progress.

Bob

P. S. I think you may close this bug report, saying 'not an Emacs problem'.

On Sat, Mar 2, 2024 at 6:15 AM Robert Boyer <robertstephenboyer@gmail.com>
wrote:

> > What does the command 'free' display if you invoke it from the shell
> prompt?
>
> > free
>                total        used        free      shared  buff/cache
> available
> Mem:         2827236     1172052     1010804       22736      644380
> 1655184
> Swap:              0           0           0
> >
>
> On Sat, Mar 2, 2024 at 5:10 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > From: Robert Boyer <robertstephenboyer@gmail.com>
>> > Date: Sat, 2 Mar 2024 02:54:47 -0600
>> > Cc: 69492@debbugs.gnu.org, moore@cs.utexas.edu, bundy@ed.ac.uk,
>> >       grant@imandra.ai, rms@gnu.org, stassats@gmail.com
>> >
>> > I did not seem to have trouble finding the file, literally.
>> >
>> > It showed up ok in a buffer, as I said in my bug report.
>> >
>> > The problem I had was that I could not move to the bottom of the buffer
>> with M->.
>>
>> What exactly is the problem?  My guess is that the problem is the same
>> I mentioned: your system doesn't have enough VM.  What does the
>> command 'free' display if you invoke it from the shell prompt?
>>
>

[-- Attachment #2: Type: text/html, Size: 3601 bytes --]

bug#69492: File too big for Emacs?

[Eli Zaretskii]

> From: Robert Boyer <robertstephenboyer@gmail.com>
> Date: Sat, 2 Mar 2024 06:15:18 -0600
> Cc: 69492@debbugs.gnu.org, moore@cs.utexas.edu, bundy@ed.ac.uk, 
> 	grant@imandra.ai, rms@gnu.org, stassats@gmail.com
> 
> > What does the command 'free' display if you invoke it from the shell prompt?
> 
> > free
>                total        used        free      shared  buff/cache   available
> Mem:         2827236     1172052     1010804       22736      644380     1655184
> Swap:              0           0           0

I don't think this is enough for visiting a 1GB file in Emacs.

bug#69492: File too big for Emacs?

[Robert Boyer]

[-- Attachment #1: Type: text/plain, Size: 745 bytes --]

You are so right.

Thanks,

Bod


On Sat, Mar 2, 2024 at 7:38 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Robert Boyer <robertstephenboyer@gmail.com>
> > Date: Sat, 2 Mar 2024 06:15:18 -0600
> > Cc: 69492@debbugs.gnu.org, moore@cs.utexas.edu, bundy@ed.ac.uk,
> >       grant@imandra.ai, rms@gnu.org, stassats@gmail.com
> >
> > > What does the command 'free' display if you invoke it from the shell
> prompt?
> >
> > > free
> >                total        used        free      shared  buff/cache
>  available
> > Mem:         2827236     1172052     1010804       22736      644380
>  1655184
> > Swap:              0           0           0
>
> I don't think this is enough for visiting a 1GB file in Emacs.
>

[-- Attachment #2: Type: text/html, Size: 1608 bytes --]

bug#69492: File too big for Emacs?

[Stefan Kangas]

Robert Boyer <robertstephenboyer@gmail.com> writes:

> You are so right.

No further comments within a couple of months, so I'm closing this bug
now.  If this conclusion is incorrect and this is still an issue, please
reply to this email (use "Reply to all" in your email client) and we can
reopen the bug report.