Avoiding 'reference to free variable' warnings while splitting elisp source files

Avoiding 'reference to free variable' warnings while splitting elisp source files

[Joe Riel]

Is there a sane way to split the source of an elisp file
while avoiding the 'reference to free variable' warnings
when byte-compiling?

The problem arises when a variable defined in one file
is used in another.   How does one avoid those errors?
I have half a dozen source files that are all required
by one package.  A variable may be used in several files
(generally it is used mainly in one file, which is where
it is defined (with defvar), but, must be used in another.
I've tried adding 

(eval-when-compile
  (require 'other-source-file))

to avoid the error; that sort of works, but not
when two files each uses a variable defined in the other.

There has to be a better way.



-- 
Joe Riel

RE: Avoiding 'reference to free variable' warnings while splittingelisp source files

[Drew Adams]

> Is there a sane way to split the source of an elisp file
> while avoiding the 'reference to free variable' warnings
> when byte-compiling?
> 
> The problem arises when a variable defined in one file
> is used in another.   How does one avoid those errors?

Use a vacuous definition: (defvar the-variable)
See (elisp) `Defining Variables':

 This form defines `foo' but does not initialize it:
   (defvar foo) => foo

Re: Avoiding 'reference to free variable' warnings while splittingelisp source files

[Stefan Monnier]

>  This form defines `foo' but does not initialize it:
>    (defvar foo) => foo

Oops that should be "declares `foo'" since it doesn't define it.


        Stefan

RE: Avoiding 'reference to free variable' warnings whilesplittingelisp source files

[Drew Adams]

> >  This form defines `foo' but does not initialize it:
> >    (defvar foo) => foo
> 
> Oops that should be "declares `foo'" since it doesn't define it.

It does (Emacs, so far) or it doesn't (Stefan).  It depends on what one means by
"defining".

The node (elisp) `Defining Variables' starts by saying what Emacs means by a
"variable definition": it "announces your intention to use a symbol as a global
variable."  That is probably pretty close to what you mean by "declaring".  It
does not mention value initialization (which is optional).

What you presumably want to point out here is that this (a vacuous `defvar')
does not define the variable's _value_.  (And it does not define the variable's
doc.)

But the Emacs approach of calling _any_ `defvar' (whether or not it defines the
value and doc) a variable _definition_ is consistent.

If you want to identify that defining with "declaring", that would be fine too.

But if you want to replace that use of "defining" with "declaring" then please
be sure to do so everywhere, and make it clear that from now on "defining"
always means declaring _and initializing_.  IOW, make clear that a `defvar' that
does not specify a value is no longer considered a vacuous "definition" etc.

To me, it is clear enough the way Emacs has always spoken: Defining means
declaring and optionally value-initializing.  That is, a variable definition
need not also give the variable a value.

  A variable definition serves three purposes.  First, it
  informs people who read the code that the symbol is _intended_
  to be used a certain way (as a variable).  Second, it informs
  the Lisp system of this, optionally supplying an initial value
  and a documentation string.  Third, it provides information to
  programming tools such as `etags', allowing them to find where
  the variable was defined.

FWIW - There are currently many, *many* occurrences of "declar" in the Elisp
manual, and *none* of them have to do with `defvar' or `defconst'.  Over 99% of
them have to do with functions and macros, and a few have to do with
"customization declarations": `defcustom', `defgroup', and `defface'.

Re: Avoiding 'reference to free variable' warnings whilesplittingelisp source files

[Stefan Monnier]

> The node (elisp) `Defining Variables' starts by saying what Emacs
> means by a "variable definition": it "announces your intention to use
> a symbol as a global variable."  That is probably pretty close to what
> you mean by "declaring".  It does not mention value initialization
> (which is optional).

Hmm... indeed.  That's too bad: it's really a misuse of the concept of
"definition": not only is it different from most other programming
language's use of "definition" and "declaration" but it's also different
from the everyday use of "definition".


        Stefan

RE: Avoiding 'reference to free variable' warningswhilesplittingelisp source files

[Drew Adams]

> Hmm... indeed.  That's too bad: it's really a misuse of the concept of
> "definition": not only is it different from most other programming
> language's use of "definition" and "declaration" but it's 
> also different from the everyday use of "definition".

Not to belabor this ;-), but there is really no conflict with everyday use, IMO.

One can distinguish defining something partially from defining it completely.
You can decide to reserve the word "define" for defining completely.  Or not.

(In my own view, all definition is partial in practice, if only because we can
allow for redefinition in any context, and a complete definition would then mean
over all time.)

When you create something like a variable, which at runtime refers to a
container/location that can optionally have contents/value, you _define_ it, at
least partially.  At least in terms of everyday use of the word - IMO.

Since the value is optional you might even consider that it is _fully_ defined
without a value.  But at least one should accord that it is partially defined in
that case.

A variable is not the same thing as its value at a given time or even the same
thing as its history of values over all time.  We agree about that, I expect.
But people sometimes speak loosely of a variable being "undefined" when they
really mean that its value is undefined.

More generally, since it is a handy pointer and there is typically little risk
of confusion, we often mistakenly speak about a variable when we really mean its
value (at a given time).

No real harm in that, in general.  It's a pardonable "misuse" of terminology -
or a handy abbreviation, depending on your point of view.

Whether a variable that has no known value is considered to be defined comes
down to defining "define" for the given context (e.g. the given language, such
as Emacs Lisp).

There is nothing wrong with some language spec stating that by "undefined
variable" it means a variable whose _value_ is undefined.  (Or that it means
either that or else a variable that has never even been declared/mentioned.)

On the other hand, there is nothing wrong with a language considering that such
declarations are definitions.

And that is the general tradition for Lisp: `defun, `defvar', `defconst',
`defstruct', `defmacro',...  That little "def" prefix stands for "define".
(Scheme even uses just `define'.)

That's just naming, of course; it doesn't prove that these are actually
definitions, according to some external criteria.  But it shows that Lisp and
its original designers consider them to be definitions.

You might consider all of that to be "misuse of the concept of 'definition."  I
don't agree (though there is also room for multiple concepts of 'definition'),
but it's not important.

Someone could decide to revamp all of this in Emacs Lisp, for functions,
structures, faces, and all the rest, preferring the term "declare" to "define"
everywhere.

Today, Emacs Lisp uses (you would say "misuses") "define" most everywhere, with
the few exceptions I listed, most of which have specific meaning as
declarations/suggestions to the compiler (and most of which are concerned with
functions).

But if you want to change to using "declare" everywhere instead of "define",
have fun doing so.  Personally, I don't think you will gain much in the
exercise.  (And it might even become harder to locate and talk about those
compiler declarations we have today.)


[There are also the various times/contexts of declaration/definition/creation to
consider: in source code (for a human), in byte-compiled code (for tools), and
in executing code (e.g. in memory).

In one sense (for a human reader), just writing a variable
declaration/definition declares/defines it.  In another sense, it is not
declared/defined until a memory location is allocated for it at runtime.  In
another sense, it is not fully defined until the content of that location has
been filled explicitly by program (it always has some content).

And then there are math variables as opposed to programming
variables-as-locations.  But even for math (and functional language) variables,
a text can talk about a given variable without its value being known.  Even
there, the variable is something different from its value: it is a name for the
value, just as the numeral `4' is a name for the number four.

The only difference there from the usual programming case of a variable as a
container with contents changeable over time is the changeable part.

We still can talk about "defining" a variable (partially) when all that has been
written so far is "Let X be an integer."  Naming is defining, at least to some
extent.  Of course, "Let X = 2" is a more complete definition, and we do
sometimes speak of X being "undefined" (an "unknown") in the first case.

There is a lot of flexibility in the terminology, and yes, some misuse.]

Re: Avoiding 'reference to free variable' warningswhilesplittingelisp source files

[Andreas Röhler]

Am 23.03.2013 23:41, schrieb Drew Adams:
>> Hmm... indeed.  That's too bad: it's really a misuse of the concept of
>> "definition": not only is it different from most other programming
>> language's use of "definition" and "declaration" but it's
>> also different from the everyday use of "definition".
>
> Not to belabor this ;-), but there is really no conflict with everyday use, IMO.
>
> One can distinguish defining something partially from defining it completely.
> You can decide to reserve the word "define" for defining completely.  Or not.
>
> (In my own view, all definition is partial in practice, if only because we can
> allow for redefinition in any context, and a complete definition would then mean
> over all time.)
>
> When you create something like a variable, which at runtime refers to a
> container/location that can optionally have contents/value, you _define_ it, at
> least partially.  At least in terms of everyday use of the word - IMO.
>
> Since the value is optional you might even consider that it is _fully_ defined
> without a value.  But at least one should accord that it is partially defined in
> that case.
>
> A variable is not the same thing as its value at a given time or even the same
> thing as its history of values over all time.  We agree about that, I expect.
> But people sometimes speak loosely of a variable being "undefined" when they
> really mean that its value is undefined.
>
> More generally, since it is a handy pointer and there is typically little risk
> of confusion, we often mistakenly speak about a variable when we really mean its
> value (at a given time).
>
> No real harm in that, in general.  It's a pardonable "misuse" of terminology -
> or a handy abbreviation, depending on your point of view.
>
> Whether a variable that has no known value is considered to be defined comes
> down to defining "define" for the given context (e.g. the given language, such
> as Emacs Lisp).
>
> There is nothing wrong with some language spec stating that by "undefined
> variable" it means a variable whose _value_ is undefined.  (Or that it means
> either that or else a variable that has never even been declared/mentioned.)
>
> On the other hand, there is nothing wrong with a language considering that such
> declarations are definitions.
>
> And that is the general tradition for Lisp: `defun, `defvar', `defconst',
> `defstruct', `defmacro',...  That little "def" prefix stands for "define".
> (Scheme even uses just `define'.)
>
> That's just naming, of course; it doesn't prove that these are actually
> definitions, according to some external criteria.  But it shows that Lisp and
> its original designers consider them to be definitions.
>
> You might consider all of that to be "misuse of the concept of 'definition."  I
> don't agree (though there is also room for multiple concepts of 'definition'),
> but it's not important.
>
> Someone could decide to revamp all of this in Emacs Lisp, for functions,
> structures, faces, and all the rest, preferring the term "declare" to "define"
> everywhere.
>
> Today, Emacs Lisp uses (you would say "misuses") "define" most everywhere, with
> the few exceptions I listed, most of which have specific meaning as
> declarations/suggestions to the compiler (and most of which are concerned with
> functions).
>
> But if you want to change to using "declare" everywhere instead of "define",
> have fun doing so.  Personally, I don't think you will gain much in the
> exercise.  (And it might even become harder to locate and talk about those
> compiler declarations we have today.)
>
>
> [There are also the various times/contexts of declaration/definition/creation to
> consider: in source code (for a human), in byte-compiled code (for tools), and
> in executing code (e.g. in memory).
>
> In one sense (for a human reader), just writing a variable
> declaration/definition declares/defines it.  In another sense, it is not
> declared/defined until a memory location is allocated for it at runtime.  In
> another sense, it is not fully defined until the content of that location has
> been filled explicitly by program (it always has some content).
>
> And then there are math variables as opposed to programming
> variables-as-locations.  But even for math (and functional language) variables,
> a text can talk about a given variable without its value being known.  Even
> there, the variable is something different from its value: it is a name for the
> value, just as the numeral `4' is a name for the number four.
>
> The only difference there from the usual programming case of a variable as a
> container with contents changeable over time is the changeable part.
>
> We still can talk about "defining" a variable (partially) when all that has been
> written so far is "Let X be an integer."  Naming is defining, at least to some
> extent.  Of course, "Let X = 2" is a more complete definition, and we do
> sometimes speak of X being "undefined" (an "unknown") in the first case.
>
> There is a lot of flexibility in the terminology, and yes, some misuse.]
>
>
>

Hi Drew,

let me take occasion to thank you of the numerous contributions making Emacs more user-friendly.
At the very case, as we may and must make an agreement how to understand words,
IMO Stefan's proposal is useful.

What about to understand "define" in a wider sense, making more comprehensive assigments, while
"declare" is understood as a beginning of a defining-process, undertaking its initial part.

Probably "initiate" would be es good a "declare", it's just to agree a convention, a possibility.

Best,

Andreas

RE: Avoiding 'reference to free variable' warningswhilesplittingelisp source files

[Drew Adams]

> At the very case, as we may and must make an agreement how to 
> understand words, IMO Stefan's proposal is useful.
> 
> What about to understand "define" in a wider sense, making 
> more comprehensive assigments, while "declare" is understood
> as a beginning of a defining-process, undertaking its
> initial part.
> 
> Probably "initiate" would be es good a "declare", it's just 
> to agree a convention, a possibility.

Sorry, I don't have anything particular to add to what I already wrote.

As I said, I have no great objection to a replacement of "define/definition"
everywhere" by "declare/declaration", if done consistently and with attention.
It is tilting at windmills IMO, but go for it if you think it improves
something.

But see what I wrote earlier for a more nuanced response.
If what I've written doesn't help you, please ignore it.