Re: Circular records: how do I best handle them? (The new correct warning position branch now bootstraps in native compilation!)

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> >> >> Hmm... circularity is quite normal in data structures, yes.
>> >> >> But presumably this is only applied to source code where circularity is
>> >> >> very rare.  Could it be that you end up recursing in elements which
>> >> >> actually aren't part of the source code (and hence can't have
>> >> >> symbols-with-positions)?
>> >> > I honestly don't know at the moment.
>> >> I think it's worth the effort to try and track this down.  Maybe we can
>> >> completely circumvent the problem.
>> > I don't think there are any such cases.
>> Hmm... I thought this whole circular records thread started because you
>> bumped into such a case.  I feel like I'm misunderstanding something.
> What I bumped into was circularly linked vectors in the source code
> being compiled.

Then my question above turns into: what is this source code?

> I've amended the reader so that it doesn't put positions on symbols
> which are read as components of other structures such as byte compiled
> functions, text property lists in strings, and so on.  (Actually, there
> was very little to amend.).

OK.

>> > The positions get stripped before the code is dumped to the .elc.
>> Why bother?  You can just have a `print-symbols-without-position` which
>> you let-bind around the printing code.
> I think I've got that already, though it's a long time since I looked at
> it.

So why do you need to strip the positions before dumping the code into
the `.elc`?

>>     (defmacro foobar-really (arg1 arg2)
>>       (puthash arg1 arg2 foobar-remember)
>>       `(progn (do-something ,arg1) (do-something-else ,arg2)))
[...]
> In that (puthash arg1 arg2 foobar-remember), if the key is a symbol with
> position, it is stripped.  I think the value will keep its positions.
> This might still be a problem.

`put` and `puthash` are just some of the ways a macro's arg can
"escape".  A macro may also something like

    (push arg my-list-of-stuff)

Having to strip symbol positions in `put` and `puthash` (i.e. having
this implementation detail leak to those places which aren't directly
related to compilation) is pretty ugly.  Do we really want to extend
that to `setq`, `aset`, and whatnot?

Maybe we should "bite the bullet" and expect macros to announce whether
they support sympos or not and if they don't we strip the positions
before calling them (we can try to be a bit more clever by using the
Edebug spec to find args where sympos will probably be harmless).

>> > It's used all over the place.  In eval-when/and-compile, it is used
>> > before the evaluation.  It is used before dumping the byte compiled code
>> > to the file.elc, and before passing this code to the native compiler.
>> > Several (?most) of the byte-compile-file-form-... functions use it.
>> > It's used in the newish keymap functions near the end of bytecomp.el, in
>> > byte-compile-annotate-call-tree, etc.  Also in cl-define-compiler-macro,
>> > and internal-macro-expand-for-load.
>> Interesting.  Why do you need it at so many places?
>> What is it usually used for?
> Stipping positions from compiled code before dumping it to an .elc file,

That sounds like "one place"

> and also before passing the compiled code to the native compiler.

And that sounds like "a second place".
In contrast above you list all kinds of *other* places.
Why do we need to strip positions in those other places?
Could we instead change the code (e.g. byte-compile-annotate-call-tree)
so it works with sympos?

> The fact is these positions on the symbols are unwanted for most uses of
> symbols around compilation, being needed only in the analysis phase of
> the source code.

So you're saying the problem is that your compiler doesn't separate the
front end from the backend?  That's indeed an inconvenient of the
current bytecompiler code.



        Stefan