Re: cc-vars.el

Re: cc-vars.el

[Kenichi Handa]

In article <rzqwunberel.fsf@albion.dl.ac.uk>, Dave Love <d.love@dl.ac.uk> writes:
> Richard Stallman <rms@gnu.org> writes:
>>      + 2002-11-14  Kenichi Handa  <handa@m17n.org>
>>      + 
>>      + 	* progmodes/cc-vars.el: Don't cc-bytecomp-defun char-table-p.
>>      + 
>>  
>>  I think that is the right fix--this cc-bytecomp-defun is a bad kludge.
>>  I am thinking that maybe we should get rid of it because it is so
>>  risky.
>>  
>>  I had undone Dave's change in set-buffer-file-coding-system to fix this,
>>  but I restored that change today after concluding that cc-bytecomp-defun
>>  is the real culprit.

> What's all this about?  I haven't intentionally touched anything
> related to cc-mode.

Dave, there's nothing wrong with your change.  It just
revealed a bug of cc-bytecomp-defun.

That bug is just fixed by Martin in HEAD as below:

2002-11-16  Martin Stjernholm  <bug-cc-mode@gnu.org>

	* progmodes/cc-bytecomp.el (cc-bytecomp-defun): Fixed bug that
	caused existing function definitions to be overridden by
	phonies when the bytecomp environment is restored.

So, I've just undone my change to cc-vars.el.  I confirmed
that it is surely fixed now by doing bootstrap.

But, Richard is against cc-bytecomp-defun.  He wrote:

Richard Stallman <rms@gnu.org> writes:
>     + 2002-11-14  Kenichi Handa  <handa@m17n.org>
>     + 
>     + 	* progmodes/cc-vars.el: Don't cc-bytecomp-defun char-table-p.
>     + 

> I think that is the right fix--this cc-bytecomp-defun is a bad kludge.
> I am thinking that maybe we should get rid of it because it is so
> risky.

> I had undone Dave's change in set-buffer-file-coding-system to fix this,
> but I restored that change today after concluding that cc-bytecomp-defun
> is the real culprit.

Martin, could you discuss this issue with Richard from now
on.  I don't have any strong opinion on this issue.

---
Ken'ichi HANDA
handa@m17n.org

Re: cc-vars.el

[Richard Stallman]

cc-bytecomp-defun is still a horrible kludge, but if it now really
only changes functions that were undefined, I don't insist on
getting rid of it.

Re: cc-vars.el

[Dave Love]

Kenichi Handa <handa@m17n.org> writes:

> But, Richard is against cc-bytecomp-defun.  He wrote:
> 
> Richard Stallman <rms@gnu.org> writes:
> >     + 2002-11-14  Kenichi Handa  <handa@m17n.org>
> >     + 
> >     + 	* progmodes/cc-vars.el: Don't cc-bytecomp-defun char-table-p.
> >     + 
> 
> > I think that is the right fix--this cc-bytecomp-defun is a bad kludge.
> > I am thinking that maybe we should get rid of it because it is so
> > risky.
> 
> > I had undone Dave's change in set-buffer-file-coding-system to fix this,
> > but I restored that change today after concluding that cc-bytecomp-defun
> > is the real culprit.
> 
> Martin, could you discuss this issue with Richard from now
> on.  I don't have any strong opinion on this issue.

I still don't understand how what I did affected this, but then I
didn't try to use cc-mode in the bootstrapped result.  However, I
strongly agree with rms.  I didn't realize cc-mode was doing that, but
I'm sure it shouldn't.

I've previously fixed Gnus and W3 to avoid special compilation
mechanisms like that.  Apart from being probably unsuitable for
installing in Emacs, they actually hid bugs.  There are typically
still some compilation warnings if you do portability stuff cleanly,
but they're not usually excessive.  Changes to the compiler would
help, e.g. to avoid warnings from

 (unless (fboundp 'fred)
   (define fred ...))

A change for that I once suggested greatly reduced the noise and I
don't think it's a big issue to re-write a few things to forms such a
compiler check would be documented to recognize.

I see a comment in cc-bytecomp implying that compilation is supposed
to produce byte code that's portable between Emacs and XEmacs, but
that's a lost cause.  They are (now) basically incompatible,
e.g. XEmacs byte code can crash Emacs if you force it to be loaded.
This means that the compiler can reasonably eliminate code conditional
on `(featurep 'xemacs)', for instance, reducing spurious warnings
again and perhaps improving efficiency in a few cases.

I also think it's unwise to try to support ancient versions of
(X)Emacs.  That just makes life difficult and takes resources that
could be put into improvements for current versions, especially taking
advantage of new features.

Sorry if that sounds like just a gripe, but it's from a fair amount of
experience. :-/

Re: cc-vars.el

[Martin Stjernholm]

Dave Love <d.love@dl.ac.uk> wrote:

> /.../ There are typically still some compilation warnings if you do
> portability stuff cleanly, but they're not usually excessive.

I consider anything more than two warnings all-in-all to be too
excessive. After that it's too easy to get accustomed to them and stop
looking carefully enough.

> I see a comment in cc-bytecomp implying that compilation is supposed
> to produce byte code that's portable between Emacs and XEmacs, but
> that's a lost cause.

True, but it's more valid within different versions of the same
branch, e.g. between Emacs 20 and 21, and even more so between minor
versions. I had at one point a simple macro system that tested all
this at compile time and made completely tailored byte code, and I got
a complaint from a user then. I don't remember between what exact
versions he tried to port it, but a single complaint isn't much of
test data anyway. From it I can only draw the conclusion that the
problem exists, not at all how big or small it is.

The only way to avoid compiler warnings cleanly (in some sense) is to
do the tests with macros at compile time, but that is apparently not a
good solution. Thus, the only resort is to make these kludgy symbol
and variable bindings.

> /.../ Changes to the compiler would help, e.g. to avoid warnings
> from
> 
>  (unless (fboundp 'fred)
>    (define fred ...))

It's possible to do something like this without any compiler warnings
at all; that's essentially what cc-bytecomp-defun does (and now it
hopefully does it correctly too). The problem is not there but rather
that it's necessary to actually bind the symbols to get rid of the
warnings since there is no other way to hook into the compiler to
control them.

> A change for that I once suggested greatly reduced the noise and I
> don't think it's a big issue to re-write a few things to forms such a
> compiler check would be documented to recognize.

Don't you rather mean code like this?

    (if (fboundp 'fred)
        (fred ...))

It'd be cool if the compiler did enough flow analysis to cope with
that, but it might be hard to get it to work well, and if it doesn't
work well then I'd prefer to have the possibility to control the
warnings explicitly with some pragma-like system.

It's really the lack of such a system in the byte compiler that is the
problem here.

An interesting angle is if something like the following can be used
(although I'd consider this too a workaround rather than a solution):

    (if (boundp 'fido)
        (symbol-value 'fido))
    (if (fboundp 'fred)
        (funcall 'fred ...))

Someone with insight in the inner workings of the compiler can perhaps
advice on how effectively it manages to optimize such things.

> I also think it's unwise to try to support ancient versions of
> (X)Emacs.  That just makes life difficult and takes resources that
> could be put into improvements for current versions, especially taking
> advantage of new features.

That is a valid point to some extent, but it's a choice I prefer to
make myself when it comes to my own "upstream" development CC Mode. I
approach it this way: If there's some _specific_ feature in newer
emacsen that would give a real benefit, and if I think it's too hard
to implement compatibility kludges for it, then and only then do I
scrap compatibility. (Thus I don't consider an argument that a kludge
just looks big and ugly as a valid reason for dropping it.)

Anyway, this is besides the point regarding cc-bytecomp, since that
one is necessary to keep it working and compiling without warnings on
both Emacs and XEmacs. It's actually the support for both flavors that
is responsible for approximately 90% of all the time I spend on
compatibility kludges. It also means I can't use a new feature until
it's so old that it exist in the other flavor, in at least an
approximately similar form. (Such a feature is the syntax-table text
property, which I intend to make greater use of and which will be the
specific reason that CC Mode soon will ditch support for Emacs 19 and
XEmacs 19 and 20.)

As for the distribution in Emacs I don't really have any objections if
someone decides to rip out all the compatibility code, except for one
thing: The ripping-out itself is a possible source of errors, and I
wouldn't like to have to spend time on hunting down bugs due to that
which gets reported to me.

Re: cc-vars.el

[Stefan Monnier]

> I still don't understand how what I did affected this, but then I
> didn't try to use cc-mode in the bootstrapped result.  However, I
> strongly agree with rms.  I didn't realize cc-mode was doing that, but
> I'm sure it shouldn't.
> 
> I've previously fixed Gnus and W3 to avoid special compilation
> mechanisms like that.  Apart from being probably unsuitable for
> installing in Emacs, they actually hid bugs.  There are typically
> still some compilation warnings if you do portability stuff cleanly,
> but they're not usually excessive.  Changes to the compiler would
> help, e.g. to avoid warnings from
> 
>  (unless (fboundp 'fred)
>    (define fred ...))
> 
> A change for that I once suggested greatly reduced the noise and I
> don't think it's a big issue to re-write a few things to forms such a
> compiler check would be documented to recognize.
> 
> I see a comment in cc-bytecomp implying that compilation is supposed
> to produce byte code that's portable between Emacs and XEmacs, but
> that's a lost cause.  They are (now) basically incompatible,
> e.g. XEmacs byte code can crash Emacs if you force it to be loaded.
> This means that the compiler can reasonably eliminate code conditional
> on `(featurep 'xemacs)', for instance, reducing spurious warnings
> again and perhaps improving efficiency in a few cases.
> 
> I also think it's unwise to try to support ancient versions of
> (X)Emacs.  That just makes life difficult and takes resources that
> could be put into improvements for current versions, especially taking
> advantage of new features.
> 
> Sorry if that sounds like just a gripe, but it's from a fair amount of
> experience. :-/

(with-aol-mode Me too!)


	Stefan

Re: cc-vars.el

[Miles Bader]

Martin Stjernholm <mast@lysator.liu.se> writes:
> Dave Love <d.love@dl.ac.uk> wrote:
> > A change for that I once suggested greatly reduced the noise and I
> > don't think it's a big issue to re-write a few things to forms such a
> > compiler check would be documented to recognize.
>
> Don't you rather mean code like this?
> 
>     (if (fboundp 'fred)
>         (fred ...))
> 
> It'd be cool if the compiler did enough flow analysis to cope with
> that, but it might be hard to get it to work well, and if it doesn't
> work well then I'd prefer to have the possibility to control the
> warnings explicitly with some pragma-like system.

The emacs compiler isn't particularly clever, but it could match certain
specified forms like the above without much trouble.  Such forms would
be explicitly defined to have this effect, and it would be up to
programmers to use the correct form (since the only effect of not doing
it right is a warning, it's not a big problem to be so picky).

E.g. when it sees (if (fboundp 'SYMBOL) ACTION ...), it could push
SYMBOL on a list of functions not to warn about, while it was compiling
ACTION.  [other forms that macroexpand into `if' without modifying the
condition would work too]

_That_ sort of change is pretty easy, and has the advantage of working
even with old compilers that don't special case it (they'll just emit a
warning like before, but the generated code will be the same).

[Think of it as a pragma expressed in code...]

-Miles
-- 
Yo mama's so fat when she gets on an elevator it HAS to go down.

Re: cc-vars.el

[Kai Großjohann]

Martin Stjernholm <mast@lysator.liu.se> writes:

> An interesting angle is if something like the following can be used
> (although I'd consider this too a workaround rather than a solution):
>
>     (if (boundp 'fido)
>         (symbol-value 'fido))
>     (if (fboundp 'fred)
>         (funcall 'fred ...))

I use this approach in Tramp.  It's ugly, but there are no warnings.
I haven't thought (much) about efficiency in Tramp, as I'm working
under the assumption that the runtime is dominated by network traffic
anyway.

But I would really appreciate it if the byte-compiler could grok
(when (fboundp 'f) (f bla)).

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: cc-vars.el

[Martin Stjernholm]

Miles Bader <miles@lsi.nec.co.jp> wrote:

> Martin Stjernholm <mast@lysator.liu.se> writes:
> > Dave Love <d.love@dl.ac.uk> wrote:
> > > A change for that I once suggested greatly reduced the noise and I
> > > don't think it's a big issue to re-write a few things to forms such a
> > > compiler check would be documented to recognize.
/.../
> E.g. when it sees (if (fboundp 'SYMBOL) ACTION ...), it could push
> SYMBOL on a list of functions not to warn about, while it was compiling
> ACTION.  [other forms that macroexpand into `if' without modifying the
> condition would work too]
> 
> _That_ sort of change is pretty easy, and has the advantage of working
> even with old compilers that don't special case it (they'll just emit a
> warning like before, but the generated code will be the same).

That's a good property. It can however be worked around anyway with
something like this:

    (eval-when-compile
      (unless (fboundp 'bytecomp-nowarn-funbound)
        (defmacro bytecomp-nowarn-funbound (dummy form) form)))

> [Think of it as a pragma expressed in code...]

That's the problem; you can't get the pragma without getting the code
then. E.g. if ACTION gets very large and I decide to split it up in
several functions I had to add otherwise completely unnecessary
fboundp checks in them just to silence the compiler.

Your suggestion will work most of the time, but it still needs to be
complemented with some pragma forms that doesn't have other effects.

Re: cc-vars.el

[Dave Love]

Martin Stjernholm <mast@lysator.liu.se> writes:

> True, but it's more valid within different versions of the same
> branch, e.g. between Emacs 20 and 21,

You definitely can't rely on Emacs 21 byte code running in Emacs 20.

> and even more so between minor versions.

There's no guarantee of that either, particularly if macro expansions
change to use new features.

> I had at one point a simple macro system that tested all
> this at compile time and made completely tailored byte code, and I got
> a complaint from a user then.

I don't understand that.  I wouldn't worry about details of the
compiled code.

> The only way to avoid compiler warnings cleanly (in some sense) is to
> do the tests with macros at compile time,

I don't think that's true.  Definitely not if compilation is done in
batch, file-by-file so that stuff in `eval-when-compile' doesn't
affect other compilations.  

> Don't you rather mean code like this?
> 
>     (if (fboundp 'fred)
>         (fred ...))

Yes, sorry.

> It'd be cool if the compiler did enough flow analysis to cope with
> that, but it might be hard to get it to work well, and if it doesn't
> work well then I'd prefer to have the possibility to control the
> warnings explicitly with some pragma-like system.

It doesn't do data-flow analysis, and doesn't need to in order to do
this sort of thing usefully.  It was a simple change to the function
that compiles `if'.

> An interesting angle is if something like the following can be used
> (although I'd consider this too a workaround rather than a solution):
> 
>     (if (boundp 'fido)
>         (symbol-value 'fido))
>     (if (fboundp 'fred)
>         (funcall 'fred ...))
> 
> Someone with insight in the inner workings of the compiler can perhaps
> advice on how effectively it manages to optimize such things.

(equal (byte-compile (lambda () (if (fboundp 'fred) (funcall 'fred))))
       (byte-compile (lambda () (if (fboundp 'fred) (fred)))))
  => t

I think CL used such a trick, but it's not (now) effective.

> Anyway, this is besides the point regarding cc-bytecomp, since that
> one is necessary to keep it working and compiling without warnings on
> both Emacs and XEmacs.

I don't see what the real difference is between cc-mode and Gnus in
this respect, but then I'm not maintaining cc-mode.

Re: cc-vars.el

[Miles Bader]

On Tue, Nov 19, 2002 at 04:45:19PM +0100, Martin Stjernholm wrote:
> > E.g. when it sees (if (fboundp 'SYMBOL) ACTION ...), it could push
> > SYMBOL on a list of functions not to warn about, while it was compiling
> > ACTION.  [other forms that macroexpand into `if' without modifying the
> > condition would work too]
> > 
> > _That_ sort of change is pretty easy, and has the advantage of working
> > even with old compilers that don't special case it (they'll just emit a
> > warning like before, but the generated code will be the same).
> 
> > [Think of it as a pragma expressed in code...]
> 
> That's the problem; you can't get the pragma without getting the code
> then.  E.g. if ACTION gets very large and I decide to split it up in
> several functions I had to add otherwise completely unnecessary
> fboundp checks in them just to silence the compiler.

Why does it matter?  A few extra fboundp checks at load-time are so utterly
inconsequential that it's hardly worth worrying about the possibility that
they might be generated in some cases.

Do you have _real_ examples of a case where this method causes problems?
If not, we shouldn't waste time wondering if there _might_ be some -- this
simply isn't a very important issue.

-Miles
-- 
We are all lying in the gutter, but some of us are looking at the stars.
-Oscar Wilde

Re: cc-vars.el

[Dave Love]

Miles Bader <miles@lsi.nec.co.jp> writes:

> _That_ sort of change is pretty easy, and has the advantage of working
> even with old compilers that don't special case it (they'll just emit a
> warning like before, but the generated code will be the same).

Exactly.  This was it.  I think it was the only diff needed, but I'm
not sure it didn't depend on another change.

As far as I remember, someone didn't like it because it wasn't
general, i.e. it doesn't work with all conditionals.  I don't think
that matters if it's documented.

*** bytecomp.el	23 Jul 2001 16:02:49 -0000	2.84
--- bytecomp.el	19 Nov 2002 11:29:15 -0000
*************** If FORM is a lambda or a macro, byte-com
*** 3213,3231 ****
  
  (defun byte-compile-if (form)
    (byte-compile-form (car (cdr form)))
!   (if (null (nthcdr 3 form))
!       ;; No else-forms
!       (let ((donetag (byte-compile-make-tag)))
! 	(byte-compile-goto-if nil for-effect donetag)
! 	(byte-compile-form (nth 2 form) for-effect)
! 	(byte-compile-out-tag donetag))
!     (let ((donetag (byte-compile-make-tag)) (elsetag (byte-compile-make-tag)))
!       (byte-compile-goto 'byte-goto-if-nil elsetag)
!       (byte-compile-form (nth 2 form) for-effect)
!       (byte-compile-goto 'byte-goto donetag)
!       (byte-compile-out-tag elsetag)
!       (byte-compile-body (cdr (cdr (cdr form))) for-effect)
!       (byte-compile-out-tag donetag)))
    (setq for-effect nil))
  
  (defun byte-compile-cond (clauses)
--- 3213,3258 ----
  
  (defun byte-compile-if (form)
    (byte-compile-form (car (cdr form)))
!   ;; Check whether we have `(if (fboundp ...' or `(if (boundp ...'
!   (let* ((clause (nth 1 form))
! 	 (fbound (if (eq 'fboundp (car-safe clause))
! 		     (and (eq 'quote (car-safe (nth 1 clause)))
! 			  (nth 1 (nth 1 clause))))) ; the relevant symbol
! 	 (bound (if (eq 'boundp (car-safe clause))
! 		    (and (eq 'quote (car-safe (nth 1 clause)))
! 			 (nth 1 (nth 1 clause)))))) ; the relevant symbol
!     (if (null (nthcdr 3 form))
! 	;; No else-forms
! 	(let ((donetag (byte-compile-make-tag)))
! 	  (byte-compile-goto-if nil for-effect donetag)
! 	  (cond (bound (let ((byte-compile-bound-variables
! 			      (cons bound
! 				    byte-compile-bound-variables)))
! 			 (byte-compile-form (nth 2 form) for-effect)))
! 		(fbound (byte-compile-form (nth 2 form) for-effect)
! 			(setq byte-compile-unresolved-functions
! 			      (delq (assq fbound
! 					  byte-compile-unresolved-functions)
! 				    byte-compile-unresolved-functions)))
! 		(t (byte-compile-form (nth 2 form) for-effect)))
! 	  (byte-compile-out-tag donetag))
!       (let ((donetag (byte-compile-make-tag))
! 	    (elsetag (byte-compile-make-tag)))
! 	(byte-compile-goto 'byte-goto-if-nil elsetag)
! 	(cond (bound (let ((byte-compile-bound-variables
! 			    (cons bound
! 				  byte-compile-bound-variables)))
! 		       (byte-compile-form (nth 2 form) for-effect)))
! 	      (fbound (byte-compile-form (nth 2 form) for-effect)
! 		      (setq byte-compile-unresolved-functions
! 			    (delq (assq fbound
! 					byte-compile-unresolved-functions)
! 				  byte-compile-unresolved-functions)))
! 	      (t (byte-compile-form (nth 2 form) for-effect)))
! 	(byte-compile-goto 'byte-goto donetag)
! 	(byte-compile-out-tag elsetag)
! 	(byte-compile-body (cdr (cdr (cdr form))) for-effect)
! 	(byte-compile-out-tag donetag))))
    (setq for-effect nil))
  
  (defun byte-compile-cond (clauses)

Re: cc-vars.el

[Dave Love]

"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:

> (with-aol-mode Me too!)

Error: not top posted.

Re: cc-vars.el

[Martin Stjernholm]

Dave Love <d.love@dl.ac.uk> wrote:

> > An interesting angle is if something like the following can be used
> > (although I'd consider this too a workaround rather than a solution):
> > 
> >     (if (boundp 'fido)
> >         (symbol-value 'fido))
> >     (if (fboundp 'fred)
> >         (funcall 'fred ...))
> > 
> > Someone with insight in the inner workings of the compiler can perhaps
> > advice on how effectively it manages to optimize such things.
> 
> (equal (byte-compile (lambda () (if (fboundp 'fred) (funcall 'fred))))
>        (byte-compile (lambda () (if (fboundp 'fred) (fred)))))
>   => t

So simple. ;) This is a good trick to know, but the same doesn't hold
for variables. Of course, that's no proof that the symbol-value thing
is (measurably) slower. I doubt it is.

There are still warnings that aren't addressed by these tricks,
though: The one about obsolete variables, and the one about incorrect
number of arguments (which cc-bytecomp-defun doesn't solve either).

> I think CL used such a trick, but it's not (now) effective.

What is not effective about it?

> > Anyway, this is besides the point regarding cc-bytecomp, since that
> > one is necessary to keep it working and compiling without warnings on
> > both Emacs and XEmacs.
> 
> I don't see what the real difference is between cc-mode and Gnus in
> this respect, but then I'm not maintaining cc-mode.

I've not investigated Gnus so I don't know either. Perhaps they use
symbol-value and funcall everywhere. Besides, cc-bytecomp also solves
another matter, namely to make it compile cleanly even in the presence
of an earlier version of CC Mode being loaded (another issue that is
irrelevant when it's distributed with Emacs). From the look of it,
Gnus hasn't addressed that at all.

Re: cc-vars.el

[Martin Stjernholm]

Miles Bader <miles@gnu.org> wrote:

> /.../  A few extra fboundp checks at load-time are so utterly
> inconsequential that it's hardly worth worrying about the possibility that
> they might be generated in some cases.

What makes you think they are restricted to load time code?

> Do you have _real_ examples of a case where this method causes problems?
> If not, we shouldn't waste time wondering if there _might_ be some -- this
> simply isn't a very important issue.

I got bugged by users enough to come up with the whole cc-bytecomp
kludge, at least. What would be the problem of adding a macro in the
bytecomp environment to control the warning aspect only? That has to
be simpler than recognizing the if-form. It's not like it's an
either-or issue between two irreconcilable solutions.

Re: cc-vars.el

[Miles Bader]

Martin Stjernholm <mast@lysator.liu.se> writes:
> > /.../  A few extra fboundp checks at load-time are so utterly
> > inconsequential that it's hardly worth worrying about the possibility that
> > they might be generated in some cases.
> 
> What makes you think they are restricted to load time code?

The idioms I'm thinking of are:

  (if (fboundp 'foo)
      (foo bar)
    (foo-work-around bar))

which will work for `when' too, due to macroexpansion:

  (when (fboundp 'foo)
    ...do stuff...
    (foo bar)
    ...do more stuff...)

I'd probably make it work for `and' too, so you could do:

  (and (fboundp 'foo) (foo bar))

What case do you have in mind where these aren't sufficient?
Can you give examples?

> What would be the problem of adding a macro in the bytecomp
> environment to control the warning aspect only?  That has to be
> simpler than recognizing the if-form.  It's not like it's an either-or
> issue between two irreconcilable solutions.

No, but let's not add both unless it's actually necessary.

-Miles
-- 
Come now, if we were really planning to harm you, would we be waiting here, 
 beside the path, in the very darkest part of the forest?

Re: cc-vars.el

[Kai Großjohann]

Martin Stjernholm <mast@lysator.liu.se> writes:

> I've not investigated Gnus so I don't know either. Perhaps they use
> symbol-value and funcall everywhere.

I think Gnus follows the Emacs model most of the time and uses compat
functions for XEmacs.  This means lots of warnings on XEmacs.

I'm not sure that this is the right approach...

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: cc-vars.el

[Stefan Monnier]

> Martin Stjernholm <mast@lysator.liu.se> writes:
> > > /.../  A few extra fboundp checks at load-time are so utterly
> > > inconsequential that it's hardly worth worrying about the possibility that
> > > they might be generated in some cases.
> > 
> > What makes you think they are restricted to load time code?
> 
> The idioms I'm thinking of are:
> 
>   (if (fboundp 'foo)
>       (foo bar)
>     (foo-work-around bar))
> 
> which will work for `when' too, due to macroexpansion:
> 
>   (when (fboundp 'foo)
>     ...do stuff...
>     (foo bar)
>     ...do more stuff...)
> 
> I'd probably make it work for `and' too, so you could do:
> 
>   (and (fboundp 'foo) (foo bar))
> 
> What case do you have in mind where these aren't sufficient?
> Can you give examples?

Obviously your example are in run-time code, not in load-time code,
so you seem to agree with him that the fboundp checks will not just occur
at load-time.

> > What would be the problem of adding a macro in the bytecomp
> > environment to control the warning aspect only?  That has to be
> > simpler than recognizing the if-form.  It's not like it's an either-or
> > issue between two irreconcilable solutions.
> No, but let's not add both unless it's actually necessary.

There's already a macro to turn off warnings for variables: (defvar foo).
I suggested to do the same for (defun bar), but IIRC Richard vetoed it
because he thinks pragmas to turn off some warnings are just wrong.

The fact is that people do want them and end up cobbling up hackish
ones (as we've seen with cc-bytecomp; I myself used something like
(eval-when-compile (autoload 'foo "bar")), so we might as well provide
clean ones.  This way we could even add an "ignore pragmas" flag for
people who like to get all the warnings.

As for whether or not pragmas will ever be made unnecessary, my opinion
is that it seems highly unlikely that we'll ever get the byte-compiler
to understand all the cases where a function call is safe.

But I also completely agree that we should try to make sure that pragmas
as needed as rarely as possible, so I wholeheartedly support Dave's
suggestion which will at least reduce the amount of useless warnings.


	Stefan

Re: cc-vars.el

[Stefan Monnier]

> > But I also completely agree that we should try to make sure that pragmas
> > as needed as rarely as possible, so I wholeheartedly support Dave's
> > suggestion which will at least reduce the amount of useless warnings.
> 
> One of the problems I've seen is with code which is conditioned to
> differentiate between either gnu emacs and xemacs, or between different
> os-types.

This is irrelevant: Dave's patch doesn't impact those cases at all.

> In the first case, many packages seem to have their own "xyz-xemacs-p"
> function or variable, and the bytecompiler will surely have a hard time
> determining whether a piece of code isn't used at all on this platform.

And it shouldn't: the code should simply be fixed.  In 90% of the
time an fboundp check would be better (and Dave's patch would then have
a chance of understanding what's going on).

> A new built-in variable `emacs-type' (value = 'gnu-emacs or e.g. 'xemacs)
> would fix this problem; for portability, packages may add the following
> snippet to support older versions of gnu and xemacs:

We don't want to encourage people to use brain dead tests like those.
There's already (featurep 'xemacs) for such things.
Maybe we should add to byte-opt.el the following snippet:

   (put 'featurep 'byte-optimizer 'byte-optimize-featurep)
   (defun byte-optimize-featurep (form)
     (if (equal '(featurep 'xemacs) form) nil form))

since Emacs-21's byte-code won't work on XEmacs anyway, there's no
point keeping XEmacs code around.  It makes the code shorter and
gets rid of a few warnings.


	Stefan

Re: cc-vars.el

[Kim F. Storm]

"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:

> 
> But I also completely agree that we should try to make sure that pragmas
> as needed as rarely as possible, so I wholeheartedly support Dave's
> suggestion which will at least reduce the amount of useless warnings.
> 

One of the problems I've seen is with code which is conditioned to
differentiate between either gnu emacs and xemacs, or between different
os-types.

In the first case, many packages seem to have their own "xyz-xemacs-p"
function or variable, and the bytecompiler will surely have a hard time
determining whether a piece of code isn't used at all on this platform.

A new built-in variable `emacs-type' (value = 'gnu-emacs or e.g. 'xemacs)
would fix this problem; for portability, packages may add the following snippet
to support older versions of gnu and xemacs:

        (unless (boundp 'emacs-type)
          (setq emacs-type
             (cond ((string-match "XEmacs" (emacs-version)) 'xemacs)
                   (t 'gnu-emacs))))

In this case, I think it is ok for the byte-compiler to simply ignore
all errors in the "other type of emacs" branch.

In the second case, code may do one of the following:
        (if (eq system-type 'msdos)  ;; or using cond
           (msdos-function ...))

or
        (if (fboundp 'msdos-function)
           (msdos-function ...))

The problem here is that the byte-compiled file must actually work on
msdos even when byte-compiled on, say, gnu/linux.  Maybe we could add
a list of prototypes for built-in platform specific functions in the
byte-compiler.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: cc-vars.el

[Dave Love]

Martin Stjernholm <mast@lysator.liu.se> writes:

> That's the problem; you can't get the pragma without getting the code
> then. E.g. if ACTION gets very large and I decide to split it up in
> several functions I had to add otherwise completely unnecessary
> fboundp checks in them just to silence the compiler.

You can put a conditional around the lot, but I don't remember that
sort of situation actually occurring in code I've worked on.

> Your suggestion will work most of the time, but it still needs to be
> complemented with some pragma forms that doesn't have other effects.

As I said, it's easy to suppress undefined function warnings safely if
you compile file-by-file in batch, for instance with dummy autoloads
for them inside eval-when-compile.  There's probably a case for
something like `(defun fred)' being treated by the optimizer like
`(defvar fred)', but I'm surprised this is such a big deal.

Re: cc-vars.el

[Kim F. Storm]

"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:

> > > But I also completely agree that we should try to make sure that pragmas
> > > as needed as rarely as possible, so I wholeheartedly support Dave's
> > > suggestion which will at least reduce the amount of useless warnings.
> > 
> > One of the problems I've seen is with code which is conditioned to
> > differentiate between either gnu emacs and xemacs, or between different
> > os-types.
> 
> This is irrelevant: Dave's patch doesn't impact those cases at all.

Off-topic, but not irrelevant IMO.

> 
> > In the first case, many packages seem to have their own "xyz-xemacs-p"
> > function or variable, and the bytecompiler will surely have a hard time
> > determining whether a piece of code isn't used at all on this platform.
> 
> And it shouldn't: the code should simply be fixed.  In 90% of the
> time an fboundp check would be better (and Dave's patch would then have
> a chance of understanding what's going on).

True, but an fboundp check doesn't work in the cases where emacs and xemacs
functions have different parameter lists, such as display-completion-list.

> 
> > A new built-in variable `emacs-type' (value = 'gnu-emacs or e.g. 'xemacs)
> > would fix this problem; for portability, packages may add the following
> > snippet to support older versions of gnu and xemacs:
> 
> We don't want to encourage people to use brain dead tests like those.
> There's already (featurep 'xemacs) for such things.

That's good to know, thanks!!!

A quick scan shows that the following packages don't use the proper test:

        bookmark.el, ediff.el, filesets.el, follow.el, ido.el,
        iswitchb.el, ps-print.el, speedbar.el [partially], term.el,
        type-break.el, edt, viper, eshell, net/eudc-bob.el, ada-mode.el,
        cperl-mode.el, hideshow.el, octave-mod.el, flyspell.el, ispell.el

Even loaddefs.el has (defconst xemacsp ...) which comes from ispell.el

So the tests may be brain dead, but very popular...

> Maybe we should add to byte-opt.el the following snippet:
> 
>    (put 'featurep 'byte-optimizer 'byte-optimize-featurep)
>    (defun byte-optimize-featurep (form)
>      (if (equal '(featurep 'xemacs) form) nil form))
> 
> since Emacs-21's byte-code won't work on XEmacs anyway, there's no
> point keeping XEmacs code around.  It makes the code shorter and
> gets rid of a few warnings.

That's the right approach, yes.  Please install it!

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: cc-vars.el

[Dave Love]

Martin Stjernholm <mast@lysator.liu.se> writes:

> So simple. ;) This is a good trick to know, but the same doesn't hold
> for variables.

I'm surprised it doesn't -- that's probably a minor omission -- but
you avoid warnings about unbound variables with `(eval-when-compile
(defvar fred))'.

> Of course, that's no proof that the symbol-value thing
> is (measurably) slower. I doubt it is.

Of course not since it generates identical code, but the fact that
it's optimized away means you don't suppress any warning.

> There are still warnings that aren't addressed by these tricks,
> though: The one about obsolete variables, and the one about incorrect
> number of arguments (which cc-bytecomp-defun doesn't solve either).

You can turn off such warnings in `byte-compile-warnings', but I think
it's best to get the warnings in a small amount of compatibility code.

> > I think CL used such a trick, but it's not (now) effective.
> 
> What is not effective about it?

It's optimized away.

> I've not investigated Gnus so I don't know either. Perhaps they use
> symbol-value and funcall everywhere.

No.  There are compatibility layers and (X)Emacs-specific files, and
it doesn't attempt to eliminate all warnings.

> Besides, cc-bytecomp also solves another matter, namely to make it
> compile cleanly even in the presence of an earlier version of CC
> Mode being loaded

I don't think it should worry about such things.  For builds of
packages, I maintain that compilation should be done file-by-file in
batch.  (If you're developing them and not restarting Emacs, you
probably want unload-feature to work properly with them.)

Re: cc-vars.el

[Dave Love]

Martin Stjernholm <mast@lysator.liu.se> writes:

> I got bugged by users enough to come up with the whole cc-bytecomp
> kludge, at least.

Why do users care about how the package is compiled?  If they worry
about compilation warnings, they should be told not to unless they've
determined that they really indicate a bug.

> What would be the problem of adding a macro in the
> bytecomp environment to control the warning aspect only?

You can already turn off warnings (and CL actually has `declare'), but
doing that covered up bugs in W3, for instance.

Re: cc-vars.el

[Richard Stallman]

    ! 		(fbound (byte-compile-form (nth 2 form) for-effect)
    ! 			(setq byte-compile-unresolved-functions
    ! 			      (delq (assq fbound
    ! 					  byte-compile-unresolved-functions)
    ! 				    byte-compile-unresolved-functions)))

If the function was in the list before, it should not
be removed now.  Something like

  (let ((old byte-compile-unresolved-functions))
    (byte-compile-form (nth 2 form) for-effect)
    (unless (assq fbound old)
      (delq (assq fbound
		  byte-compile-unresolved-functions))))

Re: cc-vars.el

[Miles Bader]

On Wed, Nov 20, 2002 at 08:58:54AM -0500, Stefan Monnier wrote:
> This way we could even add an "ignore pragmas" flag for
> people who like to get all the warnings.

[we can do this regardless]

> As for whether or not pragmas will ever be made unnecessary, my opinion
> is that it seems highly unlikely that we'll ever get the byte-compiler
> to understand all the cases where a function call is safe.

No one ever said we could; clearly _that's_ hard.  My question is whether we
we need `explicit pragmas' as opposed to `implied pragmas' (as illustrated).
If not, then we can just document `if you don't want warnings, use one of
these simple code patterns in this case'.

I know that in my own usage of fboundp checks, at least, is pretty limited
to a small set of common pattersn like the ones I listed.  So are there some
other uses that don't easily fit?  What are they?  

It's not that I hate explicit pragmas with a passion or anything; if someone
can get something past rms, I guess that's fine with me; but no one's
presented much evidence in this thread that they're actually needed, and it'd
be nice to see what things they _are_ needed for -- and anyway, we need that
to design the pragmas anyway...  Did I miss something?

-Miles
-- 
P.S.  All information contained in the above letter is false,
      for reasons of military security.

Re: cc-vars.el

[Miles Bader]

That code seems to majorly obfuscate `byte-compile-if'; is it really
necessary to do that?

It be nice to structure it so that the amount of intrusion into the `normal'
function was very small, with most of the work being done by helper
function(s); that would make it easier to add this feature to other forms.

Thanks,

-Miles
-- 
P.S.  All information contained in the above letter is false,
      for reasons of military security.

Re: cc-vars.el

[Stefan Monnier]

> > This way we could even add an "ignore pragmas" flag for
> > people who like to get all the warnings.
> [we can do this regardless]

I think you misunderstood: when people use tricks as is done in cc-bytecomp,
the byte-compiler has no clue what's going on and so you can't tell him to
ignore those hacks and give the warnings that they're trying to hide.
Whereas with well-defined pragmas like (defun foo), the byte-compiler
(meaning the user who byte-compiles a file ;-) can decide whether to
obey the pragmas or not.

> I know that in my own usage of fboundp checks, at least, is pretty limited
> to a small set of common pattersn like the ones I listed.  So are there some
> other uses that don't easily fit?  What are they?  

Agreed in general: supporting the (if (fboundp foo) bar baz) construct will
take care of some of the cases and will even cover a large part of the
cases, provided the code is rewritten to take advantage of it.

Cases it doesn't but can cover:

	(if (and (fboundp foo) bla) bar baz)

Cases it can't cover without adding harmful run-time checks:

	(if (fboundp 'make-extent)
	    (let ((x (make-extent a b)))
	      (set-extent-property x e f)))

By "harmful" I mean that not only the (fboundp 'set-extent-property) test
is wasting resources, but more importantly there's nothing else you can do
if it fails.

> It's not that I hate explicit pragmas with a passion or anything; if someone
> can get something past rms, I guess that's fine with me; but no one's
> presented much evidence in this thread that they're actually needed, and it'd
> be nice to see what things they _are_ needed for -- and anyway, we need that
> to design the pragmas anyway...  Did I miss something?

I don't think you missed anything.  I completely agree that we should
first concentrate on solution that don't require pragmas, especially
since it seems that they are more likely to be accepted by Richard.

Any objection to my little (featurep 'xemacs) hack and/or to Dave's
(if (fboundp ..) ..) patch ?


	Stefan

Re: cc-vars.el

[Richard Stallman]

    I think Gnus follows the Emacs model most of the time and uses compat
    functions for XEmacs.  This means lots of warnings on XEmacs.

This is good cooperation with Emacs.

Re: cc-vars.el

[Richard Stallman]

    There's already a macro to turn off warnings for variables: (defvar foo).
    I suggested to do the same for (defun bar), but IIRC Richard vetoed it
    because he thinks pragmas to turn off some warnings are just wrong.

I don't think it is wrong necessarily.  However, what would (defun bar)
actually do?  Would it be a magic flag just for the compiler?

    As for whether or not pragmas will ever be made unnecessary, my opinion
    is that it seems highly unlikely that we'll ever get the byte-compiler
    to understand all the cases where a function call is safe.

I see no reason to try to get even close to that.  It would be wasted
effort.  If there are a few standard constructions that avoid warnings,
programmers can use those constructions.  That is easy all around.

Re: cc-vars.el

[Stefan Monnier]

>     There's already a macro to turn off warnings for variables: (defvar foo).
>     I suggested to do the same for (defun bar), but IIRC Richard vetoed it
>     because he thinks pragmas to turn off some warnings are just wrong.
> 
> I don't think it is wrong necessarily.  However, what would (defun bar)
> actually do?  Would it be a magic flag just for the compiler?

Yes, just like (defvar foo): it wouldn't define the function but would
instead just declare that there exist such a function somewhere and
that the byte-compiler should consequently not emit any warning about it.

But it's more important to focus on the (if (fboundp ...) ...) case.


	Stefan

Re: cc-vars.el

[Martin Stjernholm]

Miles Bader <miles@gnu.org> wrote:

> It's not that I hate explicit pragmas with a passion or anything; if someone
> can get something past rms, I guess that's fine with me; but no one's
> presented much evidence in this thread that they're actually needed, and it'd
> be nice to see what things they _are_ needed for -- and anyway, we need that
> to design the pragmas anyway...  Did I miss something?

I've looked through the things I've used the cc-bytecomp kludges for
in CC Mode, and sure enough, simple patterns for if and cond would
take care of most things.

However, some rather large parts exist only to add language knowledge
to various external packages that are optional and might not be
available at all. Putting conditionals outside the whole set of
defconsts, defuns etc would then be necessary to silence the compiler.
But doing so would give load order dependencies (i.e. it wouldn't work
if the optional packages gets loaded later). And coping with that
would require an eval-after-load around the whole thing..

Another more design-wise argument for providing them is that
programming tools should be orthogonal to make sure all possible cases
are covered. The stance that pragmas can get implemented when someone
needs them aren't good since it'd be a considerable obstacle if a
non-emacs core developer actually encounters such a situation. It's
probably more likely that an ugly kludge is devised instead.

Re: cc-vars.el

[Martin Stjernholm]

"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> wrote:

> Yes, just like (defvar foo): it wouldn't define the function but would
> instead just declare that there exist such a function somewhere and
> that the byte-compiler should consequently not emit any warning about it.

This would suit me well, at least.

Are these declarations local to the current compilation? If not then I
think it'd be a good idea to make them so, to avoid that the warnings
are covered in packages that are compiled later on.

Re: cc-vars.el

[Dave Love]

"Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:

> We don't want to encourage people to use brain dead tests like those.
> There's already (featurep 'xemacs) for such things.
> Maybe we should add to byte-opt.el the following snippet:

I meant to say the same thing.  I've long used an equivalent
implementation which may have slightly coloured my impression of
warnings, but it's not in what I'm currently running.

I'd also note [not to Stefan] that you can always eliminate code at
compile time if that's what you want:

(disassemble (byte-compile (lambda () 
			     (if (eval-when-compile (and (fboundp 'fred)
							 (featurep 'xemacs)))
				 t)))
	     (current-buffer))
  => nil
byte code:
  args: nil
0	constant  nil
1	return	  

Re: cc-vars.el

[Dave Love]

storm@cua.dk (Kim F. Storm) writes:

> So the tests may be brain dead, but very popular...

They're typically in externally-maintained packages, and it's possible
that the authors insist on supporting versions of XEmacs predating the
feature, misguided though I think that is.

> That's the right approach, yes.  Please install it!

That's a fairly fundamental change to install blithely.  I think it's
OK, but only after extensive use.  (It would need documenting
clearly.)

Re: cc-vars.el

[Dave Love]

Richard Stallman <rms@gnu.org> writes:

> If the function was in the list before, it should not
> be removed now.

Indeed.  It may be that it got added back somehow else.  I don't
remember, but I'll have another look at it sometime.

Re: cc-vars.el

[Dave Love]

Miles Bader <miles@gnu.org> writes:

> That code seems to majorly obfuscate `byte-compile-if'; is it really
> necessary to do that?

I don't know.

> It be nice to structure it so that the amount of intrusion into the
> `normal' function was very small, with most of the work being done
> by helper function(s);

Feel free.

> that would make it easier to add this feature to other forms.

I'm not convinced that's worthwhile, but if you want to do it I won't
bother revisiting the change.