feature/asan-gc-poisoning: Better memory checks using AddressSanitizer

feature/asan-gc-poisoning: Better memory checks using AddressSanitizer

[Vibhav Pant]

As part of trying to debug the deluge of GC borks in
scratch/comp-static-data, I took a slight detour through Emacs' memory
management code and the sanity checks that it optionally comes with.
The changes in feature/asan-gc-poisoning attempt to augment the
latter, specifically when Emacs is built with '-fsanitize=address'.

When enabled, almost all memory management operations are instrumented
with calls to ASan's "(un)poisoning" functions, which allow ASan to
mark allocated but not freed addresses as "poisoned", triggering
errors whenever they have been accessed. The idea behind this is to
add another layer of checks against code that makes use of free lists
and arena based allocators, only allowing access to regions of the
memory that are actually intended to be accessible (aka "unpoisoned"
in ASan parlance) by non alloc-related C code. An instance of this is
any object on a free list, or a vectorlike with the type PVEC_FREE (we
do abort whenever we come across such a vector, but this ideally
catches the bug way earlier in the flow, and therefore closer to the
source of the actual problem). Additional documentation explaining
this feature is available both on src/alloc.c and the "Running Emacs
with address sanitization" section in etc/DEBUG.

The branch bootstraps successfully both with native-comp enabled and
disabled, and passes everything on the test suite on my machine.
Because enabling Address Sanitizer comes with a significant
performance + memory overhead, using it as a daily driver isn't
without a decent bit of sluggishness, although building with -O2 and
native compilation seems to help, at least a little.
I plan to get install this in `master` soon, so both feedback and
testing from other users would be greatly appreciated. The current
codebase quite likely does everything alloc related correctly, so an
ASan error might be indicative of a snag in this branch instead of
Emacs itself (fingers crossed for the opposite, however :)).

Thanks,
Vibhav
-- 
Vibhav Pant
vibhavp@gmail.com
GPG: 7ED1 D48C 513C A024 BE3A  785F E3FB 28CB 6AB5 9598

Re: feature/asan-gc-poisoning: Better memory checks using AddressSanitizer

[Vibhav Pant]

[-- Attachment #1: Type: text/plain, Size: 2352 bytes --]

Unless there are any objections, I plan to merge this branch into master
today.

Thanks,
Vibhav

Vibhav Pant
vibhavp@gmail.com

On Tue, Dec 6, 2022, 02:05 Vibhav Pant <vibhavp@gmail.com> wrote:

> As part of trying to debug the deluge of GC borks in
> scratch/comp-static-data, I took a slight detour through Emacs' memory
> management code and the sanity checks that it optionally comes with.
> The changes in feature/asan-gc-poisoning attempt to augment the
> latter, specifically when Emacs is built with '-fsanitize=address'.
>
> When enabled, almost all memory management operations are instrumented
> with calls to ASan's "(un)poisoning" functions, which allow ASan to
> mark allocated but not freed addresses as "poisoned", triggering
> errors whenever they have been accessed. The idea behind this is to
> add another layer of checks against code that makes use of free lists
> and arena based allocators, only allowing access to regions of the
> memory that are actually intended to be accessible (aka "unpoisoned"
> in ASan parlance) by non alloc-related C code. An instance of this is
> any object on a free list, or a vectorlike with the type PVEC_FREE (we
> do abort whenever we come across such a vector, but this ideally
> catches the bug way earlier in the flow, and therefore closer to the
> source of the actual problem). Additional documentation explaining
> this feature is available both on src/alloc.c and the "Running Emacs
> with address sanitization" section in etc/DEBUG.
>
> The branch bootstraps successfully both with native-comp enabled and
> disabled, and passes everything on the test suite on my machine.
> Because enabling Address Sanitizer comes with a significant
> performance + memory overhead, using it as a daily driver isn't
> without a decent bit of sluggishness, although building with -O2 and
> native compilation seems to help, at least a little.
> I plan to get install this in `master` soon, so both feedback and
> testing from other users would be greatly appreciated. The current
> codebase quite likely does everything alloc related correctly, so an
> ASan error might be indicative of a snag in this branch instead of
> Emacs itself (fingers crossed for the opposite, however :)).
>
> Thanks,
> Vibhav
> --
> Vibhav Pant
> vibhavp@gmail.com
> GPG: 7ED1 D48C 513C A024 BE3A  785F E3FB 28CB 6AB5 9598
>

[-- Attachment #2: Type: text/html, Size: 3057 bytes --]

Re: feature/asan-gc-poisoning: Better memory checks using AddressSanitizer

[Eli Zaretskii]

> From: Vibhav Pant <vibhavp@gmail.com>
> Date: Sun, 18 Dec 2022 01:38:33 +0530
> 
> Unless there are any objections, I plan to merge this branch into master today. 

No objections, but please make the ASAN_POISON_* functions macros that
expand either to corresponding asan_poison_* functions or to nothing,
depending on whether GC_ASAN_POISON_* is defined.  That's because I'd
like to avoid unnecessary overhead of a function call in a "normal"
build without optimizations.

Thanks.

Re: feature/asan-gc-poisoning: Better memory checks using AddressSanitizer

[Vibhav Pant]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

On Sat, 2022-12-17 at 22:24 +0200, Eli Zaretskii wrote:
> > From: Vibhav Pant <vibhavp@gmail.com>
> > Date: Sun, 18 Dec 2022 01:38:33 +0530
> > 
> > Unless there are any objections, I plan to merge this branch into
> > master today. 
> 
> No objections, but please make the ASAN_POISON_* functions macros
> that
> expand either to corresponding asan_poison_* functions or to nothing,
> depending on whether GC_ASAN_POISON_* is defined.  That's because I'd
> like to avoid unnecessary overhead of a function call in a "normal"
> build without optimizations.
> 
> Thanks.

Sure, Eli.

Thanks,
Vibhav
-- 
Vibhav Pant
vibhavp@gmail.com
GPG: 7ED1 D48C 513C A024 BE3A  785F E3FB 28CB 6AB5 9598

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]