Re: Crash in handling tar files (Was: GDB debugger mode for Emacs in ELPA)

[Eli Zaretskii <eliz@gnu.org>]

> Date: Mon, 30 Jun 2008 09:25:04 +0530 (IST)
> From: filerz-emacs@yahoo.com
> Cc: emacs-devel@gnu.org
> 
>  Do we really need a different allocation method for M$?

How do you mean ``different''?  REL_ALLOC is used not only on
MS-Windows.

> Looks like the REL_ALLOC is a pool based allocator. Could someone
> comment on benefits seen in using this pool based allocation?

REL_ALLOC (actually, ralloc.c) is a relocatable memory allocator.  Its
benefit is that, unlike most (maybe all?) malloc implementations, it
can recover from memory fragmentation by relocating some of the Lisp
objects (strings and buffers, IIRC) in order to bring small fragmented
chunks of free memory into larger and less scattered ones.  Thus, when
large memory buffers are requested by alloc.c, ralloc.c can serve
those requests for longer time without asking more core from the OS.

Without ralloc.c, Emacs's memory footprint tends to grow much faster
and does not always level out.  You can try it: just recompile Emacs
without REL_ALLOC defined, and run it in a long, real-life editing
session; then compare the memory footprints.

> If this is used only in M$ platform, we could create multiple heaps at startup and use them. Once the heap becomes highly fragmented (we can use the low fragmentation heap), we can create a new heap, wait till all references to the fragmented heap are freed and destroy the fragmented heap. Since Emacs is single threaded, we can create a heap with no serialization, that will further speedup the allocations on M$.

Do we really want to develop a new memory allocation scheme from
scratch?  ralloc.c is used on many platforms for many years (GNU/Linux
stopped using it because they use mmap and Doug Lea's allocator (again
IIRC), which is even better than ralloc.c).  There's no reason to
believe ralloc.c causes any real trouble, certainly not because of
some petty bug in tar-mode.el or thereabouts!

> Even the second scenario (without loading the 'tar-mode') does not happen on GNU/Linux (as REL_ALLOC stuff is not used there I guess)

You are assuming that this is ralloc.c's fault.  But it's much more
probable that it's some memory corruption elsewhere, and if that is
the case, other memory allocation mechanisms will see it as well.

> I could reproduce by creating a tar file with 'src/w32.c' as "tar cvf crash.tar src/w32.c". Here is the backtrace:
> 
> (gdb) bt
> #0  0x0119eaf3 in re_search_2 (bufp=0x1376420, str1=0x388a160 "ostfix modifiers\"\n \"quail/latin-alt\")\n(register-input-method\n \"latin-3-alt-postfix\" \"Latin-3\" 'quail-use-package\n \"3<\" \"Latin-3 char

This is strange text!  What is your value of auto-coding-regexp-alist?
Do you have any idea why auto-coding-regexp-alist-lookup ended up
looking at the string pointed to by str1 above?