please set both MALLOC_PERTURB_ and MALLOC_CHECK_ envvars

please set both MALLOC_PERTURB_ and MALLOC_CHECK_ envvars

[Jim Meyering]

Now, bootstrapping emacs on Fedora 15 x86_64
(with env settings as below) fails with a segfault.

If you develop emacs, or even if you just build from sources regularly,
on a glibc-based system, do us all a favor and build with these envvar
settings:

    export MALLOC_PERTURB_=$((RANDOM % 255 + 1))
    export MALLOC_CHECK_=3

Why?  Because that helps you expose malloc-related problems far earlier.
I've found numerous bugs that way.  The trouble is that you have to
be ready to debug, or at least ready to retry with

    env MALLOC_PERTURB_=0 MALLOC_CHECK_=0 make ...

to see if things work better without protection.

Until recently, I've been building like this to work around
a problem that I traced back to a commit in emacs several years ago:

    make bootstrap RUN_TEMACS='MALLOC_CHECK_=0 ./temacs'

However, today, even with that, make bootstrap is failing with a
segfault (in various places) that goes away when I turn off malloc
robustness checking via both variables for the whole process
and not just one of them when running temacs:

    MALLOC_PERTURB_=0 MALLOC_CHECK_=0 make bootstrap

That suggests a very recent change (within the last 24 hrs)
makes emacs act to differently when MALLOC_PERTURB_=N causes glibc
to scribble nonzero (N) bytes into each just freed buffer.

Re: please set both MALLOC_PERTURB_ and MALLOC_CHECK_ envvars

[Jim Meyering]

Jim Meyering wrote:
> Now, bootstrapping emacs on Fedora 15 x86_64
> (with env settings as below) fails with a segfault.
>
> If you develop emacs, or even if you just build from sources regularly,
> on a glibc-based system, do us all a favor and build with these envvar
> settings:
>
>     export MALLOC_PERTURB_=$((RANDOM % 255 + 1))
>     export MALLOC_CHECK_=3
>
> Why?  Because that helps you expose malloc-related problems far earlier.
> I've found numerous bugs that way.  The trouble is that you have to
> be ready to debug, or at least ready to retry with
>
>     env MALLOC_PERTURB_=0 MALLOC_CHECK_=0 make ...
>
> to see if things work better without protection.
>
> Until recently, I've been building like this to work around
> a problem that I traced back to a commit in emacs several years ago:
>
>     make bootstrap RUN_TEMACS='MALLOC_CHECK_=0 ./temacs'
>
> However, today, even with that, make bootstrap is failing with a
> segfault (in various places) that goes away when I turn off malloc
> robustness checking via both variables for the whole process
> and not just one of them when running temacs:
>
>     MALLOC_PERTURB_=0 MALLOC_CHECK_=0 make bootstrap
>
> That suggests a very recent change (within the last 24 hrs)
> makes emacs act to differently when MALLOC_PERTURB_=N causes glibc
> to scribble nonzero (N) bytes into each just freed buffer.

Most of that argument is valid, but I have to confess this time
a compiler problem appears to be at fault.  To get "make" to succeed,
I have to build like this:

    env MALLOC_CHECK_=0 make CFLAGS=-O0

Use CFLAGS=-O or MALLOC_CHECK_=4 (or anything else in 1..255)
and the build segfaults when using the just-built temacs
to compile .el files.

Re: please set both MALLOC_PERTURB_ and MALLOC_CHECK_ envvars

[Thien-Thi Nguyen]

() Jim Meyering <jim@meyering.net>
() Fri, 27 May 2011 23:57:40 +0200

   this time a compiler problem appears

Please describe specifics of this compiler (name, version, etc).

Re: please set both MALLOC_PERTURB_ and MALLOC_CHECK_ envvars

[Jim Meyering]

Thien-Thi Nguyen wrote:
> () Jim Meyering <jim@meyering.net>
> () Fri, 27 May 2011 23:57:40 +0200
>
>    this time a compiler problem appears
>
> Please describe specifics of this compiler (name, version, etc).

It was the gcc from Fedora 15.

    $ rpm -q gcc
    gcc-4.6.0-7.fc15.x86_64

Using gcc built from the very latest (yesterday),

    gcc version 4.7.0 20110528 (experimental) (GCC)

everything works fine, and yes!, even with these envvar settings:

    MALLOC_PERTURB_=117
    MALLOC_CHECK_=3

So at least for a little while, I'll be using that.

Bottom line: using the very latest gcc-4.7.x, all is well.

Re: please set both MALLOC_PERTURB_ and MALLOC_CHECK_ envvars

[Jim Meyering]

Jim Meyering wrote:
> Thien-Thi Nguyen wrote:
>> () Jim Meyering <jim@meyering.net>
>> () Fri, 27 May 2011 23:57:40 +0200
>>
>>    this time a compiler problem appears
>>
>> Please describe specifics of this compiler (name, version, etc).
>
> It was the gcc from Fedora 15.
>
>     $ rpm -q gcc
>     gcc-4.6.0-7.fc15.x86_64
>
> Using gcc built from the very latest (yesterday),
>
>     gcc version 4.7.0 20110528 (experimental) (GCC)
>
> everything works fine, and yes!, even with these envvar settings:
>
>     MALLOC_PERTURB_=117
>     MALLOC_CHECK_=3
>
> So at least for a little while, I'll be using that.
>
> Bottom line: using the very latest gcc-4.7.x, all is well.

Here's an update.

Maybe that was just luck, because bootstrap failed this morning with
the aggressive MALLOC_* settings, even when using the latest version of gcc.

However, when I turned off MALLOC_CHECK_, it succeeded:

  MALLOC_CHECK_=0 MALLOC_PERTURB_=91 make bootstrap

Re: please set both MALLOC_PERTURB_ and MALLOC_CHECK_ envvars

[Andreas Schwab]

MALLOC_CHECK_ has never worked on the dumped emacs.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: please set both MALLOC_PERTURB_ and MALLOC_CHECK_ envvars

[Chong Yidong]

Jim Meyering <jim@meyering.net> writes:

>> If you develop emacs, or even if you just build from sources regularly,
>> on a glibc-based system, do us all a favor and build with these envvar
>> settings:
>>
>>     export MALLOC_PERTURB_=$((RANDOM % 255 + 1))
>>     export MALLOC_CHECK_=3
>>
>> Why?  Because that helps you expose malloc-related problems far earlier.
>> I've found numerous bugs that way.
>
> Most of that argument is valid, but I have to confess this time
> a compiler problem appears to be at fault.
> [...]
>     gcc version 4.7.0 20110528 (experimental) (GCC)
>
> everything works fine, and yes!, even with these envvar settings:
>
>     MALLOC_PERTURB_=117
>     MALLOC_CHECK_=3
>
> So at least for a little while, I'll be using that.

Maybe we should add these environment variables to the Hydra build.  But
what to use for MALLOC_PERTURB---117, or the value in your previous
message?

Re: please set both MALLOC_PERTURB_ and MALLOC_CHECK_ envvars

[Jim Meyering]

Chong Yidong wrote:

> Jim Meyering <jim@meyering.net> writes:
>
>>> If you develop emacs, or even if you just build from sources regularly,
>>> on a glibc-based system, do us all a favor and build with these envvar
>>> settings:
>>>
>>>     export MALLOC_PERTURB_=$((RANDOM % 255 + 1))
>>>     export MALLOC_CHECK_=3
>>>
>>> Why?  Because that helps you expose malloc-related problems far earlier.
>>> I've found numerous bugs that way.
>>
>> Most of that argument is valid, but I have to confess this time
>> a compiler problem appears to be at fault.
>> [...]
>>     gcc version 4.7.0 20110528 (experimental) (GCC)
>>
>> everything works fine, and yes!, even with these envvar settings:
>>
>>     MALLOC_PERTURB_=117
>>     MALLOC_CHECK_=3
>>
>> So at least for a little while, I'll be using that.
>
> Maybe we should add these environment variables to the Hydra build.

Good idea.

> But
> what to use for MALLOC_PERTURB---117, or the value in your previous
> message?

You can use any value between 1..255 for MALLOC_PERTURB_
That chooses the byte that glibc will use to memset all freed buffers.
I set it like this in a shell start-up file:

    export MALLOC_PERTURB_=$(($RANDOM % 255 + 1))

Using a random value is slightly better than using a fixed one
in case your fixed value is someday just the right/wrong value
to mask a problem.  At least with a random value, if you rerun
the test in a different shell, the odds are good you won't use
the unfortunate setting again.

Re: please set both MALLOC_PERTURB_ and MALLOC_CHECK_ envvars

[Paul Eggert]

On 05/27/11 14:50, Jim Meyering wrote:
>     export MALLOC_PERTURB_=$((RANDOM % 255 + 1))
>     export MALLOC_CHECK_=3
> 

I tried that, on Fedora 14 x86-64, and the Emacs trunk build failed
as follows:

  Compiling language/thai-word.el

  In toplevel form:
  language/thai-word.el:10738:5:Error: Memory exhausted--use C-x s then exit and restart Emacs
  make[2]: *** [language/thai-word.elc] Error 1
  make[2]: Leaving directory `/home/eggert/src/gnu/emacs/int-hash/lisp'

As near as I can discover, Emacs was fine, but the malloc debugging
caused it to use so much more memory that Emacs ran out of memory
trying to compile thai-word.el.  This is on a host with
8 GiB of RAM.

Perhaps there is a real Emacs bug in there somewhere, but I spent
a reasonable amount of time looking for it unsuccessfully.

Re: please set both MALLOC_PERTURB_ and MALLOC_CHECK_ envvars

[Jim Meyering]

Paul Eggert wrote:

> On 05/27/11 14:50, Jim Meyering wrote:
>>     export MALLOC_PERTURB_=$((RANDOM % 255 + 1))
>>     export MALLOC_CHECK_=3
>>
>
> I tried that, on Fedora 14 x86-64, and the Emacs trunk build failed
> as follows:
>
>   Compiling language/thai-word.el
>
>   In toplevel form:
>   language/thai-word.el:10738:5:Error: Memory exhausted--use C-x s
> then exit and restart Emacs
>   make[2]: *** [language/thai-word.elc] Error 1
>   make[2]: Leaving directory `/home/eggert/src/gnu/emacs/int-hash/lisp'
>
> As near as I can discover, Emacs was fine, but the malloc debugging
> caused it to use so much more memory that Emacs ran out of memory
> trying to compile thai-word.el.  This is on a host with
> 8 GiB of RAM.
>
> Perhaps there is a real Emacs bug in there somewhere, but I spent
> a reasonable amount of time looking for it unsuccessfully.

Same here.  I even bisected back through several years of commits to
find the one that *appeared* to introduce this problem:
    http://thread.gmane.org/gmane.emacs.devel/137942
But all that was assuming a reliable compiler.

With those envvar settings, I've been bootstrapping emacs for
months, but only with this kludge:

     make bootstrap RUN_TEMACS='MALLOC_CHECK_=0 ./temacs'

Yesterday, I found that I can bootstrap with no kludge at all as long
as I use the latest gcc-4.7.0.  I.e., with that compiler, a plain
"make bootstrap" now succeeds in spite of my aggressive envvar settings.