How and when to use GCPRO?

How and when to use GCPRO?

[Leo]

Hello,

I wonder if anyone can point me to a more detailed explanation of how
and when to use GCPRO other than what's in the elisp manual.

Is there any symptom in a gdb session that one should suspect a GCPRO is
missing?

For example, Could someone comment on the following function I just
wrote, which tries to add support for feature expressions like in common
lisp?

static Lisp_Object
eval_feature_expression (form)
     Lisp_Object form;
{
  Lisp_Object val, exp;
  struct gcpro gcpro1;

  if (! NILP(Fatom(form)))
    return NILP(Fmemq(form, Vfeatures)) ? Qnil:Qt;

  if (EQ (Fcar(form), Qnot))
    {
      if (XINT(Flength(form)) != 2)
        error ("Invalid feature expression");
      val = eval_feature_expression (Fcar(Fcdr(form)));
      return NILP(val) ? Qt:Qnil;
    }

  if (EQ (Fcar(form), Qand))
    {
      if (NILP (Fcdr (form)))
        error ("Invalid feature expression");
      val = eval_feature_expression (Fcar (Fcdr (form)));
      if (NILP (Fcdr (Fcdr (form))))
        return val;
      else if (NILP(val))
        return Qnil;
      else
        {
          GCPRO1(exp);
          exp = Fcons (Qand, (Fcdr(Fcdr(form))));
          val = eval_feature_expression (exp);
          UNGCPRO;
          return val;
        }
    }

  if (EQ (Fcar(form), Qor))
    {
      if (NILP (Fcdr (form)))
        error ("Invalid feature expression");
      val = eval_feature_expression (Fcar (Fcdr (form)));
      if (NILP (Fcdr (Fcdr (form))))
        return val;
      else if (NILP (val))
        {
          GCPRO1(exp);
          exp = Fcons (Qor, (Fcdr(Fcdr(form))));
          UNGCPRO;
          return eval_feature_expression (exp);
        }
      else
        return Qt;
    }
  error ("Invalid feature expression");
}

I am also reading the XEmacs internals manual on GCPROing but I have
little idea what is still true for GNU Emacs.

Thanks in advance and happy holidays.

Leo

Re: How and when to use GCPRO?

[Andreas Schwab]

Whenever you call a function that can GC and you are using a reference
to a Lisp object around that call.

None of the functions you call in your example can GC.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: How and when to use GCPRO?

[Leo]

On 2010-12-27 10:15 +0000, Andreas Schwab wrote:
> Whenever you call a function that can GC and you are using a reference
> to a Lisp object around that call.
>
> None of the functions you call in your example can GC.
>
> Andreas.

Thanks, Andreas. I'll keep that in mind.

If you pass a Lisp_Object as an argument to a function that does GC but
you don't need to use that object after the call, do you need to GCPRO
it?

For example, in `require' (fns.c line 2976), strictly speaking is
GCPROing `filename' absolutely necessary?

Thanks.
Leo

Re: How and when to use GCPRO?

[Andreas Schwab]

Leo <sdl.web@gmail.com> writes:

> For example, in `require' (fns.c line 2976), strictly speaking is
> GCPROing `filename' absolutely necessary?

I don't think so, and neither is feature.  Both values are already
protected through the caller.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: How and when to use GCPRO?

[Stefan Monnier]

> Whenever you call a function that can GC and you are using a reference
> to a Lisp object around that call.
> None of the functions you call in your example can GC.

Actually, both Fcar and Fcdr can GC (by signalling an error which
triggers the debugger).


        Stefan "wondering why eval_feature_expression would need to be
                written in C rather than in Elisp"

Re: How and when to use GCPRO?

[Andreas Schwab]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Actually, both Fcar and Fcdr can GC (by signalling an error which
> triggers the debugger).

But the debugger will not return.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: How and when to use GCPRO?

[Stefan Monnier]

>> Actually, both Fcar and Fcdr can GC (by signalling an error which
>> triggers the debugger).
> But the debugger will not return.

Oh, right,


        Stefan

Re: How and when to use GCPRO?

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 559 bytes --]

On 12/27/10 5:01 PM, Stefan Monnier wrote:
>>> Actually, both Fcar and Fcdr can GC (by signalling an error which
>>> triggers the debugger).
>> But the debugger will not return.
> 
> Oh, right,
> 

Discussions like this worry me. What if the function is later changed to
call something that can GC? What if it's used in some new context? If
it's not utterly performance-critical code, isn't it better to be safe
than sorry and GCPRO anyway? It's not as if it's an expensive operation.
Premature optimization is the root of all evil, after all.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: How and when to use GCPRO?

[Stefan Monnier]

>>>> Actually, both Fcar and Fcdr can GC (by signalling an error which
>>>> triggers the debugger).
>>> But the debugger will not return.
>> Oh, right,
> Discussions like this worry me. What if the function is later changed to
> call something that can GC? What if it's used in some new context? If
> it's not utterly performance-critical code, isn't it better to be safe
> than sorry and GCPRO anyway? It's not as if it's an expensive operation.
> Premature optimization is the root of all evil, after all.

On most platforms, GCPRO is a no-op, and on those where it's not, it's
never been a major performance issue, AFAIK.  The problem is just that
it makes the code that much more verbose and painful to
write/read/maintain.


        Stefan

Re: How and when to use GCPRO?

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 998 bytes --]

On 12/27/10 6:17 PM, Stefan Monnier wrote:
>>>>> Actually, both Fcar and Fcdr can GC (by signalling an error which
>>>>> triggers the debugger).
>>>> But the debugger will not return.
>>> Oh, right,
>> Discussions like this worry me. What if the function is later changed to
>> call something that can GC? What if it's used in some new context? If
>> it's not utterly performance-critical code, isn't it better to be safe
>> than sorry and GCPRO anyway? It's not as if it's an expensive operation.
>> Premature optimization is the root of all evil, after all.
> 
> On most platforms, GCPRO is a no-op, and on those where it's not, it's
> never been a major performance issue, AFAIK.  The problem is just that
> it makes the code that much more verbose and painful to
> write/read/maintain.

Hrm. You learn something every day --- I thought Emacs was doing precise
GC the way it has since time immemorial. How much of a win was the
GC_MARK_STACK conservative scanning approach?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: How and when to use GCPRO?

[Stefan Monnier]

> Hrm. You learn something every day --- I thought Emacs was doing precise
> GC the way it has since time immemorial.  How much of a win was the
> GC_MARK_STACK conservative scanning approach?

As long as we still support platforms that use GCPROs, it's not a win
at all.  Tho it may have beneficial effects on the performance, I don't
know (it could also be detrimental to performance, by the way, because
of extra work it has to do every time we allocate memory for Lisp
objects in order to later on be able to conservatively tell whether an
integer happens to point to an allocated memory area).


        Stefan

Conservative scanning (was: Re: How and when to use GCPRO?)

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1517 bytes --]

On 12/27/10 8:15 PM, Stefan Monnier wrote:
>> Hrm. You learn something every day --- I thought Emacs was doing precise
>> GC the way it has since time immemorial.  How much of a win was the
>> GC_MARK_STACK conservative scanning approach?
> 
> As long as we still support platforms that use GCPROs, it's not a win
> at all.  Tho it may have beneficial effects on the performance, I don't
> know (it could also be detrimental to performance, by the way, because
> of extra work it has to do every time we allocate memory for Lisp
> objects in order to later on be able to conservatively tell whether an
> integer happens to point to an allocated memory area).
> 
> 
>         Stefan

Of course, since insertion in a RB tree is O(log N), N here being the
number of allocated memory blocks, my gut tells me that consing time
isn't hurt all that much. The whole approach basically moves work from
the whole runtime (in the form of GCPRO) to allocation and deallocation.
I don't know whether that's a win.

Why bother with the conservative scanning at all? If it's on by default
almost everywhere, then it might be hiding the presence of GCPRO bugs.
Conservative scanning seems more useful as a debugging aide
(GC_MARK_STACK_CHECK_GCPROS and CHECK_ZOMBIES) than as the primary GC
strategy.

Usually, conservative GC is what you do when there's not enough
information for precise GC, but that's not the case in the Emacs core.

Has anyone run benchmarks lately? Eight years is a long time.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: How and when to use GCPRO?

[Richard Stallman]

There is a simple general rule for which Emacs C functions can GC:
those that can directly or indirectly call eval.  To make Fcar call
eval would be a rather shocking and drastic change.  There is no need
to take precautions today against the possibility of such a drastic
change in the future.


-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org

Re: How and when to use GCPRO?

[Leo]

On 2010-12-27 16:00 +0000, Stefan Monnier wrote:
>         Stefan "wondering why eval_feature_expression would need to be
>                 written in C rather than in Elisp"

-- 
Oracle is the new evil

Common Lisp like feature expressions (was: How and when to use GCPRO?)

[Leo]

On 2010-12-27 16:00 +0000, Stefan Monnier wrote:
>         Stefan "wondering why eval_feature_expression would need to be
>                 written in C rather than in Elisp"

I hadn't thought of that :(

I put together a small patch for anyone who would like to try it:

diff --git a/lisp/subr.el b/lisp/subr.el
index 03f7e04..92f3b0a 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -1719,6 +1719,31 @@ FILE should be the name of a library, with no directory name."
   (eval-after-load file (read)))
 (make-obsolete 'eval-next-after-load `eval-after-load "23.2")
 \f
+(defun eval-feature-expression (form)
+  (cond
+   ((atom form) (featurep form nil))
+   ;; ensure form is a proper list
+   ((condition-case nil
+        (and (length form) nil)
+      (error (error "Invalid feature expression: %s" form))))
+   ((eq (car form) 'not)
+    (if (= (length form) 2)
+        (not (eval-feature-expression (cadr form)))
+      (error "Invalid feature expression: %s" form)))
+   ((= (length form) 2)
+    (eval-feature-expression (cadr form)))
+   ((eq (car form) 'and)
+    (if (= (length form) 1)
+        t
+      (and (eval-feature-expression (cadr form))
+           (eval-feature-expression (cons 'and (cddr form))))))
+   ((eq (car form) 'or)
+    (if (= (length form) 1)
+        nil
+      (or (eval-feature-expression (cadr form))
+          (eval-feature-expression (cons 'or (cddr form))))))
+   (t (error "Invalid feature expression: %s" form))))
+\f
 ;;;; Process stuff.
 
 (defun process-lines (program &rest args)
diff --git a/src/lread.c b/src/lread.c
index 72c01ed..32c36ee 100644
--- a/src/lread.c
+++ b/src/lread.c
@@ -96,6 +96,9 @@ Lisp_Object Qinhibit_file_name_operation;
 Lisp_Object Qeval_buffer_list, Veval_buffer_list;
 Lisp_Object Qfile_truename, Qdo_after_load_evaluation; /* ACM 2006/5/16 */
 
+/* Used in feature expression */
+Lisp_Object Qobarray, Qeval_feature_expression;
+
 /* Used instead of Qget_file_char while loading *.elc files compiled
    by Emacs 21 or older.  */
 static Lisp_Object Qget_emacs_mule_file_char;
@@ -2426,6 +2429,24 @@ read1 (readcharfun, pch, first_in_list)
 	  UNREAD (c);
 	  invalid_syntax ("#", 1);
 	}
+      /* feature expressions as in Common Lisp */
+      if (c == '+' || c == '-')
+        {
+          Lisp_Object fexp = read0(readcharfun);
+          fexp = call1 (Qeval_feature_expression, fexp);
+          if (c == '-')
+            fexp = NILP (fexp) ? Qt : Qnil;
+          if (NILP (fexp))
+            {
+              int count = SPECPDL_INDEX ();
+              Lisp_Object tem = Fmake_vector (make_number(17),make_number(0));
+              /* use a temporary obarray */
+              specbind (Qobarray, tem);
+              read0 (readcharfun);
+              unbind_to (count, Qnil);
+            }
+          goto retry;
+        }
       if (c == '^')
 	{
 	  c = READCHAR;
@@ -4536,6 +4557,11 @@ to load.  See also `load-dangerous-libraries'.  */);
   Qcomma_dot = intern_c_string (",.");
   staticpro (&Qcomma_dot);
 
+  Qeval_feature_expression = intern_c_string ("eval-feature-expression");
+  staticpro (&Qeval_feature_expression);
+  Qobarray = intern_c_string ("obarray");
+  staticpro(&Qobarray);
+  
   Qinhibit_file_name_operation = intern_c_string ("inhibit-file-name-operation");
   staticpro (&Qinhibit_file_name_operation);
 

Re: Common Lisp like feature expressions (was: How and when to use GCPRO?)

[Andreas Schwab]

Leo <sdl.web@gmail.com> writes:

> +(defun eval-feature-expression (form)
> +  (cond
> +   ((atom form) (featurep form nil))
> +   ;; ensure form is a proper list
> +   ((condition-case nil
> +        (and (length form) nil)

Nice quadratic behaviour.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Common Lisp like feature expressions (was: How and when to use GCPRO?)

[Leo]

On 2010-12-27 19:38 +0000, Andreas Schwab wrote:
> Leo <sdl.web@gmail.com> writes:
>
>> +(defun eval-feature-expression (form)
>> +  (cond
>> +   ((atom form) (featurep form nil))
>> +   ;; ensure form is a proper list
>> +   ((condition-case nil
>> +        (and (length form) nil)
>
> Nice quadratic behaviour.
>
> Andreas.

How about something like this:

(defun eval-feature-expression (form)
  (cond
   ((atom form) (featurep form nil))
   ((eq (car form) 'not)
    (not (eval-feature-expression (cadr form))))
   ((eq (car form) 'and)
    (if (not (cdr form))
        t
      (and (eval-feature-expression (cadr form))
           (eval-feature-expression (cons 'and (cddr form))))))
   ((eq (car form) 'or)
    (if (not (cdr form))
        nil
      (or (eval-feature-expression (cadr form))
          (eval-feature-expression (cons 'or (cddr form))))))
   (t (error "Invalid feature expression: %s" form))))

Leo

Re: Common Lisp like feature expressions

[Stefan Monnier]

> (defun eval-feature-expression (form)
>   (cond
>    ((atom form) (featurep form nil))
>    ((eq (car form) 'not)
>     (not (eval-feature-expression (cadr form))))
>    ((eq (car form) 'and)
>     (if (not (cdr form))
>         t
>       (and (eval-feature-expression (cadr form))
>            (eval-feature-expression (cons 'and (cddr form))))))
>    ((eq (car form) 'or)
>     (if (not (cdr form))
>         nil
>       (or (eval-feature-expression (cadr form))
>           (eval-feature-expression (cons 'or (cddr form))))))
>    (t (error "Invalid feature expression: %s" form))))

Just for fun, here's a version using pcase:

 (defun eval-feature-expression (form)
   (pcase form
    ((pred atom) (featurep form nil))
    (`(not ,x) (not (eval-feature-expression x)))
    (`(and) t)
    (`(and ,form1 . ,forms)
     (and (eval-feature-expression form1)
          (eval-feature-expression (cons 'and forms))))
    (`(or) nil)
    (`(or ,form1 . ,forms)
     (or (eval-feature-expression form1)
         (eval-feature-expression (cons 'or forms))))
    (t (error "Invalid feature expression: %s" form)))))


-- Stefan

Re: Common Lisp like feature expressions

[Leo]

On 2010-12-28 01:01 +0000, Stefan Monnier wrote:
> Just for fun, here's a version using pcase:
>
>  (defun eval-feature-expression (form)
>    (pcase form
>     ((pred atom) (featurep form nil))
>     (`(not ,x) (not (eval-feature-expression x)))
>     (`(and) t)
>     (`(and ,form1 . ,forms)
>      (and (eval-feature-expression form1)
>           (eval-feature-expression (cons 'and forms))))
>     (`(or) nil)
>     (`(or ,form1 . ,forms)
>      (or (eval-feature-expression form1)
>          (eval-feature-expression (cons 'or forms))))
>     (t (error "Invalid feature expression: %s" form)))))

That appears to be an excellent example for pcase. Maybe it should be
added to the header of pcase.el or where it is documented? Thanks.

Leo