Lisp help: problem with uninterned symbols

Lisp help: problem with uninterned symbols

[Lars Brinkhoff]

Hello,

I have a problem with symbols that aren't interned in any Emacs
obarray.  I intern the symbols in my own tables, but I would still
like to have the symbols as constants in my code, and have them
compare eq as usual.

Here is a simplified version of my code:

    ;;; file my-intern.el
    (defvar symbol-table nil)
    (defun my-intern (string)
      (let ((x (assoc string symbol-table)))
        (if x
            (cdr x)
            (let ((sym (make-symbol string)))
              (push (cons string sym) symbol-table)
              sym))))

Now, to include my-interned symbols as constants, I may use a macro or
eval-when-compile:

  ;;; file foobar.el
  (defun foo () (eval-when-compile (my-intern "foo")))
  (defun bar () (eval-when-compile (my-intern "foo")))

If I load this file and call (eq (foo) (bar)), the result is t as I
like.  However, if I compile the file the result is:

  ;;; file foobar.elc
  [...]
  (defalias 'foo #[nil "\300\207" [#1=#:foo] 1])
  (defalias 'bar #[nil "\300\207" [#1=#:foo] 1])

As you can see, there are two non-eq #:foo in the compiled code, so
(eq (foo) (bar)) will return nil.

If I trick the byte compiler into compiling the functions as one
top-level form, say like

  ;;; file foobar.el
  (let ()
    (defun foo () (eval-when-compile (my-intern "foo")))
    (defun bar () (eval-when-compile (my-intern "foo"))))

the result is:

  ;;; file foobar.elc
  [...]
  (byte-code "\300\301M\210\302\303M\207"
             [foo #[nil "\300\207" [#1=#:foo] 1] bar #[nil "\300\207" [#1#] 1]]
             2)

Here, there's only one #:foo which is referenced a second time with
#1#.  Great, (eq (foo) (bar)) works again.  However, that trick
doesn't work if foo and bar are defined in two separate file.

Any idea how to solve this?

-- 
Lars Brinkhoff,         Services for Unix, Linux, GCC, HTTP
Brinkhoff Consulting    http://www.brinkhoff.se/

Re: Lisp help: problem with uninterned symbols

[Pascal Bourguignon]

Lars Brinkhoff <lars@nocrew.org> writes:
>   ;;; file foobar.el
>   (let ()
>     (defun foo () (eval-when-compile (my-intern "foo")))
>     (defun bar () (eval-when-compile (my-intern "foo"))))
> 
> the result is:
> 
>   ;;; file foobar.elc
>   [...]
>   (byte-code "\300\301M\210\302\303M\207"
>              [foo #[nil "\300\207" [#1=#:foo] 1] bar #[nil "\300\207" [#1#] 1]]
>              2)
> 
> Here, there's only one #:foo which is referenced a second time with
> #1#.  Great, (eq (foo) (bar)) works again.  However, that trick
> doesn't work if foo and bar are defined in two separate file.
> 
> Any idea how to solve this?

Doctor, when I do this, it hurts!

You have the solution: encapsulate your compilation units in a progn.


-- 
__Pascal_Bourguignon__                              .  *   * . * .* .
http://www.informatimago.com/                        .   *   .   .*
There is no worse tyranny than to force             * .  . /\  ()  . *
a man to pay for what he does not                    . .  / .\   . * .
want merely because you think it                    .*.  / *  \  . .
would be good for him. -- Robert Heinlein             . /*   o \     .
http://www.theadvocates.org/                        *   '''||'''   .
SCO Spam-magnet: postmaster@sco.com                 ******************

Re: Lisp help: problem with uninterned symbols

[Lars Brinkhoff]

Pascal Bourguignon <spam@thalassa.informatimago.com> writes:
> Doctor, when I do this, it hurts!

I still need to do it, and I'm willing to endure some pain.

> You have the solution: encapsulate your compilation units in a progn.

Not quite.  As I wrote:

> > that trick doesn't work if foo and bar are defined in two separate
> > file[s].

-- 
Lars Brinkhoff,         Services for Unix, Linux, GCC, HTTP
Brinkhoff Consulting    http://www.brinkhoff.se/

Re: Lisp help: problem with uninterned symbols

[Pascal Bourguignon]

Lars Brinkhoff <lars@nocrew.org> writes:

> Pascal Bourguignon <spam@thalassa.informatimago.com> writes:
> > Doctor, when I do this, it hurts!
> 
> I still need to do it, and I'm willing to endure some pain.
> 
> > You have the solution: encapsulate your compilation units in a progn.
> 
> Not quite.  As I wrote:
> 
> > > that trick doesn't work if foo and bar are defined in two separate
> > > file[s].

It seems  to me that  it's a "feature"  of the byte compiler.  It just
does not  search an uninterned symbols amongst  all uninterned symbols
ever created to find if there's another occurence.

One question: why don't you use an obarray for you symbols?
(I doubt that would change anything thought).

What about  interning all your  symbols as normal emacs  lisp symbols.
Of course, you may want to prefix their names with a "package" name.

Otherwise, I'm afraid you'll have to extend the emacs byte compiler to
allow it to work with various sources of "interned" symbols.


-- 
__Pascal_Bourguignon__                              .  *   * . * .* .
http://www.informatimago.com/                        .   *   .   .*
There is no worse tyranny than to force             * .  . /\  ()  . *
a man to pay for what he does not                    . .  / .\   . * .
want merely because you think it                    .*.  / *  \  . .
would be good for him. -- Robert Heinlein             . /*   o \     .
http://www.theadvocates.org/                        *   '''||'''   .
SCO Spam-magnet: postmaster@sco.com                 ******************

Re: Lisp help: problem with uninterned symbols

[Stefan Monnier]

> I have a problem with symbols that aren't interned in any Emacs
> obarray.  I intern the symbols in my own tables, but I would still
> like to have the symbols as constants in my code, and have them
> compare eq as usual.

I think you're out of luck.  Same thing with any other object, by the way:
dumping a `cons' cell twice in two different forms will lose the sharing
information and will thus result in having two distinct cons cells when
read back.

Two solutions:
- somehow figure out a way to dump your data in a single form
- avoid dumping the raw data and instead force the use of my-intern when
  loading the file.  Maybe with something like

    (defvar my-item (my-intern "foo"))
    (defun foo () my-item)
    (defun bar () my-item)

  of course I strongly suspect that this example code is very different from
  your actual code and turning one into the other my be non-trivial.
  If you can't do it easily, post more details here and we'll happily make
  fun of you.


-- Stefan

Re: Lisp help: problem with uninterned symbols

[Lars Brinkhoff]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> I think you're out of luck.

Good to know I didn't miss something obvious.

> Same thing with any other object, by the way: dumping a `cons' cell
> twice in two different forms will lose the sharing information and
> will thus result in having two distinct cons cells when read back.

Yes, I noticed that too, but that haven't ran into that problem yet.

> - avoid dumping the raw data and instead force the use of my-intern
> when loading the file.  Maybe with something like
> 
>     (defvar my-item (my-intern "foo"))
>     (defun foo () my-item)
>     (defun bar () my-item)

That would be good enough I guess.

> of course I strongly suspect that this example code is very
> different from your actual code and turning one into the other my be
> non-trivial.

Umm, not really, I think.  My example had enough of the essence of the
problem.

> If you can't do it easily, post more details here and we'll happily
> make fun of you.

All details are here:
  http://www.nocrew.org/cgi-bin/cvsweb/cvsweb.cgi/emacs-cl/
mostly in cl-packages.el and especially the kw macro in utils.el.

-- 
Lars Brinkhoff,         Services for Unix, Linux, GCC, HTTP
Brinkhoff Consulting    http://www.brinkhoff.se/

Re: Lisp help: problem with uninterned symbols

[Lars Brinkhoff]

Pascal Bourguignon <spam@thalassa.informatimago.com> writes:
> One question: why don't you use an obarray for you symbols?

I may do that I the future, but one problem (I believe) is that it's
not possible to intern the same symbol in more than one obarray.

> (I doubt that would change anything thought).

No, I don't think so.

> Otherwise, I'm afraid you'll have to extend the emacs byte compiler
> to allow it to work with various sources of "interned" symbols.

Something like Common Lisp's load-time-value would be nice.  My
problem would then be solved by wrapping (load-time-value ...) around
the difficult parts.

Building on Stefan's suggestion, how about this?

  (defvar load-time-value-list)
  
  (defmacro start-load-time-values ()
    "This macro must be called before the first use of load-time-value."
    (setq load-time-value-list nil))
  
  (defmacro end-load-time-values ()
    "This macro must be called after the last use of load-time-value."
    `(progn ,@(mapcar (lambda (x) `(defvar ,@x))
                      load-time-value-list)))
  
  (defmacro load-time-value (form &optional read-only-p)
    (if byte-compile-current-form
        (let ((sym (gentemp)))
          (push (list sym form) load-time-value-list)
          sym)
        form))

Note the use of gentemp instead of gensym.

-- 
Lars Brinkhoff,         Services for Unix, Linux, GCC, HTTP
Brinkhoff Consulting    http://www.brinkhoff.se/

Re: Lisp help: problem with uninterned symbols

[Lars Brinkhoff]

Lars Brinkhoff <lars@nocrew.org> writes:
> Something like Common Lisp's load-time-value would be nice.

I see Emacs' CL package already defines load-time-value.  Problem
solved!

-- 
Lars Brinkhoff,         Services for Unix, Linux, GCC, HTTP
Brinkhoff Consulting    http://www.brinkhoff.se/