bug#24751: 26.0.50; Regex stack overflow not detected properly (gets "Variable binding depth exceeds max-specpdl-size")

[Eli Zaretskii <eliz@gnu.org>]

> From: npostavs@users.sourceforge.net
> Cc: 24751@debbugs.gnu.org
> Date: Sun, 13 Nov 2016 00:39:39 -0500
> 
> > I think the patch can be simplified, where we now multiply by the size
> > of fail_stack_elt_t and then divide by it: simply remove both the
> > multiplication and the division.  That will make the code easier to
> > read, and will make the units of each variable clear, something that I
> > think is at the heart of this issue.
> 
> Ah, right.

Thanks, LGTM.

> >> but effectively increases the size of the failure stack (so the
> >> sample file size has to be increased 8-fold to get a regex stack
> >> overflow).
> >
> > Which IMO is exactly TRT, since re_max_failures was computed given the
> > runtime stack size of 8MB, so having it bail out after merely 800KB
> > doesn't sound right to me, don't you agree?
> 
> Yes, I suppose we should also try to make use of the stack, rather than
> calling malloc, right?  Something like this:
> 
> diff --git i/src/regex.c w/src/regex.c
> index d23ba01..dcabde5 100644
> --- i/src/regex.c
> +++ w/src/regex.c
> @@ -447,7 +447,11 @@ init_syntax_once (void)
>  #else /* not REGEX_MALLOC  */
>  
>  # ifdef emacs
> -#  define REGEX_USE_SAFE_ALLOCA USE_SAFE_ALLOCA
> +#  define REGEX_USE_SAFE_ALLOCA                                         \
> +  ptrdiff_t sa_avail = re_max_failures                                  \
> +    * TYPICAL_FAILURE_SIZE * sizeof (fail_stack_elt_t);                 \
> +  ptrdiff_t sa_count = SPECPDL_INDEX (); bool sa_must_free = false
> +

Yes.  And please also add a comment there saying that this replaces
USE_SAFE_ALLOCA.

> -size_t re_max_failures = 40000;
> +size_t re_max_failures = 20;
>  # else
>  size_t re_max_failures = 4000;
>  # endif
> 
> 
> Actually I find Emacs still compiles if I removed that line completely,
> there's just a compile warning saying
> 
>     regex.o: In function `re_match_2_internal':
>     /home/npostavs/src/emacs/emacs-master/lib-src/../src/regex.c:5529: warning: the 're_max_failures' variable is obsolete and will go away.
> 
> I guess there's some kind of definition of it in libc?

Most probably.  You should be able to see that using "nm -A".  If
that's indeed so, I think we should rename that variable to something
like emacs_re_max_failures, to avoid stomping on the libc variable..