Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Francesco Potorti`]

   Oh, ok.  For each file which you are going to generate there will be
   one chunk, possibly quite small, started by one or more of @o, @a or
   @c (other dialects will use other start-sequences).  This then
   references chunk-names [...]

I looked at http://w3.pppl.gov/%7ekrommes/fweb.html#SEC16, where named
modules are explained, and I was convinced that etags would need to
implement most of the functionality of a .web processor.

Does not make sense, given that we have the alternative of parsing the
generated files and generate tags that point to the .web file by having
etags look at #line directives.

So we must, in certain cases, parse #line directives.

Now we have these cases (jump to the conclusion if you do not care):

1) .web file and .c file
   In this case etags cannot parse the .web, but can parse the .c and
   make tags pointing to .web.  This is the most reasonable behaviour,
   and should ideally be the default one, or even the only possible one.

2) .y file alone
   Etags already handles this beacuse it can parse .y files.

3) .c file alone, generated by .y
   Etags should not honour #line, because we do not have a .y.  But is
   this case interesting?  Maybe not at all.  If people really need
   that, they can remove #line directives.  Or else etags could be
   instructed to behave like it does now, that is, by ignoring #line.

4) .c together with .y
   Currently Etags parses both, each on their own.  If we let etags by
   default (or always) honour #line, then we will have two sets of
   identical tags produced in the TAGS file.  We can live with this, or
   implement the logic that in this case, only one set should be
   produced.

5) .c together with .y on a TAGS file generated using etags --append
   In this case, to implement the logic that only one set should be
   produced, etags should read the TAGS file before appending to it.
   Easy. 


Conclusion:

1) etags will honour the #line directive by default

2) if problems will arise, there will be an option to disable this
   behaviour and use the current one instead, where #line is ignored

3) additionally, and as a separate improvement, when told to parse files
   X and Y, where Y contains #line directives pointing to X, do not tag
   X.  This allows to use `etags zz.y zz.c', and etags will only tag
   zz.c, and generate tags pointing to zz.y.  It will also be possible
   to use `etags xx.web xx.c' and etags will only tag xx.c with tags
   pointing to xx.web.

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Kim F. Storm]

Francesco Potorti` <pot@gnu.org> writes:

> 1) .web file and .c file
>    In this case etags cannot parse the .web, but can parse the .c and
>    make tags pointing to .web.  This is the most reasonable behaviour,
>    and should ideally be the default one, or even the only possible one.

I'm still not sure I understand all the implication of etags looking
at #line lines.

In the case of the C-preprocessor, there is a 1:1 correpondance
between the line numbers in the source file and the target file,
meaning that if it output a #line 1000 while at line 1000 (of course),
then if you advance 25 lines in the target file, it corresponds to
line 1025 of the source file (provided there are no other #line in
those 25 lines).  

However, this will often NOT be the case when #line directives are
found in files generated by other programs (there may be a 1:100 or a
100:1 correspondance) -- but you don't know.  So etags will have to 
put the #line directive's line number directly into TAGS.

So what are the implications of etags doing that?


> 3) additionally, and as a separate improvement, when told to parse files
>    X and Y, where Y contains #line directives pointing to X, do not tag
>    X.  This allows to use `etags zz.y zz.c', and etags will only tag
>    zz.c, and generate tags pointing to zz.y.  It will also be possible
>    to use `etags xx.web xx.c' and etags will only tag xx.c with tags
>    pointing to xx.web.

Suppose the source file is in written in YLAG which contains
a the following declaration on line 100:
         FUNC f::x(a,b) { a /?/ b }

The resulting myfile.c file will contain code corresponding to this
looks something like this:

        #line "myfile.ylag" 100
        _YLp *
        _YLfunc_x__f_(_YLp *a, _YLp *b)
        {
          return _YLop9(a,b);
        }

So, what does etags put into the TAGS file about f::x ?

It writes:
        _YLfunc_x__f_ is at line 100 in myfile.ylag

Supposing that etags understood YLAG syntax, I would - as a user of
etags - like to use   M-. f::x RET   to jump to the definition of f::x

But I cannot do that, since etags according to your rule didn't parse
myfile.ylag at all.  Is that really acceptable?


On the other hand, I can do M-. _YLfunc_x__f_ RET which (surprise)
takes me to the declaration of f::x in myfile.ylag.

That may be ok in some (most?) cases, but it may also be the wrong
thing - if I was actually looking for that function trying to
understand what is *really* going on at a low level.


I don't know how to fix these problems (all solutions seem to
counter-act each other).  Here are some ideas:

It has been mentioned before to put dual references into TAGS:
one for the actual file, and one for the file from the #line
directive.  This is simple.

Etags could somehow correlate the two files to see if they use
"identical" identifiers, i.e. whether it really makes sense to not tag
the source file.  This seems non-trivial, as you really don't know
what program was used to generate the target file -- even if those
files are both, say, .c files, you really don't know if they share any
identifiers (or other similarities).  Eg. it could be that the
generation of the target file specifically involved changing all
identifiers.

We could have a command which can go backwards in a file looking for a
#line tag and jump to the referenced file and line (optionally plus
the offset to that line).  This would not really mess with (or even
require) etags as all!  Maybe such a function already exists?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk


_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Sven Utcke]

Hello Kim,

> I'm still not sure I understand all the implication of etags looking
> at #line lines.
> 
> In the case of the C-preprocessor, there is a 1:1 correpondance
> between the line numbers in the source file and the target file,
> meaning that if it output a #line 1000 while at line 1000 (of course),
> then if you advance 25 lines in the target file, it corresponds to
> line 1025 of the source file (provided there are no other #line in
> those 25 lines).  

Yes.

> However, this will often NOT be the case when #line directives are
> found in files generated by other programs (there may be a 1:100 or a
> 100:1 correspondance) -- but you don't know.  So etags will have to 
> put the #line directive's line number directly into TAGS.

I do not think this is correct.  The program generating the #line
directives can, I think, be assumed to output a new #line directive
whenever there is no 1:1 correspondence.  So I don't think there
should be a problem.  This might look like this:

--- snip ---
/* 3: */
#line 210 "test-homology.web"

int main(int argc,char**argv)
{
/* 6: */
#line 323 "test-homology.web"

int c;
extern char*optarg;
char*bitangent_file= NULL;
/* 8: */
#line 349 "test-homology.web"

bitangent_array bitangents;
/* 10: */
#line 113 "test-homology.web"

outline_list_pointer outlines= NULL;
outline_list_pointer the_outline= NULL;
int num_outlines= 0;
int i;
--- snip ---

Note that the line-number need not be monotonically increasing though
(something the gdb handles badly, btw --- maybe I should post a
bug-report :-)

> So, what does etags put into the TAGS file about f::x ?
> 
> It writes:
>         _YLfunc_x__f_ is at line 100 in myfile.ylag
> 
> Supposing that etags understood YLAG syntax, I would - as a user of
> etags - like to use   M-. f::x RET   to jump to the definition of f::x
> 
> But I cannot do that, since etags according to your rule didn't parse
> myfile.ylag at all.  Is that really acceptable?

No.  If etags understands YLAG, then it should preferably parse the
original file, not the derived one.

> We could have a command which can go backwards in a file looking for a
> #line tag and jump to the referenced file and line (optionally plus
> the offset to that line).  This would not really mess with (or even
> require) etags as all!  Maybe such a function already exists?

Sounds like a viable workaround, but I still believe that etags should
honour #line directives (that's what they are there for, after all).
Personally, I also believe that when both *.c and *.y were given to
etags, it _should_ create dual references, both pointing to the *.y
file...

Sven
-- 
 _  __                     The Cognitive Systems Group
| |/ /___  __ _ ___                                       University of Hamburg
| ' </ _ \/ _` (_-<  phone:    +49 (0)40 42883-2576      Vogt-Koelln-Strasse 30
|_|\_\___/\__, /__/  fax  :    +49 (0)40 42883-2572             D-22527 Hamburg
          |___/ http://kogs-www.informatik.uni-hamburg.de/~utcke/home.html

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Sven Utcke]

Hello Richard, hello all,

>     It has been mentioned before to put dual references into TAGS:
>     one for the actual file, and one for the file from the #line
>     directive.  This is simple.
> 
> Having both files in the TAGS table is what we want to AVOID.
> It is wasteful.

Not in the case to which the sentence above was alluding.  There you
would have a line

FUNC f::x(a,b) { a /?/ b }

in the .ylag file, but a line

_YLfunc_x__f_(_YLp *a, _YLp *b)

in the generated .c file.  I believe the only way to reasonably handle
this would be _four_ references altogether:  One for each file, and
one for each functionname.  So looking up f::a would get you to the
*.ylag file first, and to the *.c file next (Ok, this would probably
be hard to implement :-), while looking up _YLfunc_x__f_ (why would
anybody want to do that?) would get you to the *.c file first and to
the *.ylag next.  

I guess.  But then, I have no clue about ylag, only web.  And for the
latter _one_ reference (to the *.web) would certainly be enough...

Sven
-- 
 _  __                     The Cognitive Systems Group
| |/ /___  __ _ ___                                       University of Hamburg
| ' </ _ \/ _` (_-<  phone:    +49 (0)40 42883-2576      Vogt-Koelln-Strasse 30
|_|\_\___/\__, /__/  fax  :    +49 (0)40 42883-2572             D-22527 Hamburg
          |___/ http://kogs-www.informatik.uni-hamburg.de/~utcke/home.html

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Sven Utcke]

>     However, this will often NOT be the case when #line directives are
>     found in files generated by other programs (there may be a 1:100 or a
>     100:1 correspondance) -- but you don't know.  So etags will have to 
>     put the #line directive's line number directly into TAGS.
> 
> That is doing things the hard way.  Wouldn't it be easier to handle
> #line in Lisp inside find-tag?

Well, you would then need to load the .c file first (which I never
load, and which just might confuse people if it showed up among the
buffers), only to parse that and load the referenced *.web file.  And
the TAGS file does save a line-number anyway, or am I missing
something?  And what about other tools using the TAGS-file --- are
there other tolls using it?

Sven
-- 
 _  __                     The Cognitive Systems Group
| |/ /___  __ _ ___                                       University of Hamburg
| ' </ _ \/ _` (_-<  phone:    +49 (0)40 42883-2576      Vogt-Koelln-Strasse 30
|_|\_\___/\__, /__/  fax  :    +49 (0)40 42883-2572             D-22527 Hamburg
          |___/ http://kogs-www.informatik.uni-hamburg.de/~utcke/home.html

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Francesco Potorti`]

   > However, this will often NOT be the case when #line directives are
   > found in files generated by other programs (there may be a 1:100 or a
   > 100:1 correspondance) -- but you don't know.  So etags will have to 
   > put the #line directive's line number directly into TAGS.
   
   I do not think this is correct.  The program generating the #line
   directives can, I think, be assumed to output a new #line directive
   whenever there is no 1:1 correspondence.

I do believe that Sven is right.  That's the precise purpose of the
#line directive.

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Francesco Potorti`]

       3) additionally, and as a separate improvement, when told to parse files
          X and Y, where Y contains #line directives pointing to X, do not tag
          X.  This allows to use `etags zz.y zz.c', and etags will only tag
          zz.c, and generate tags pointing to zz.y.
   
   That is backwards.  It should only tag zz.y, not zz.c.
   
   The reason is that zz.y may have tags that are not reflected in zz.c.

Argh.  Right.   
   						  It will also be possible
          to use `etags xx.web xx.c' and etags will only tag xx.c with tags
          pointing to xx.web.
   
   That is reasonable in this case because xx.web is in a format that
   can't be processed by etags.
   
So etags should distinguish the two cases.  I'd like to find a more
linear logic, however.

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Francesco Potorti`]

Richard Stallman <rms@gnu.org> writes:
   That is doing things the hard way.  Wouldn't it be easier to handle
   #line in Lisp inside find-tag?

Okay, let's try to change the approach completely.  We should be able to
do that all inside find-tag, without even changing the user interface.

Using C-u M-. one can search for more tags.  We (not me, please!) can
change find-tag so that, before looking in the TAGS buffer for more
tags, the current buffer is searched backwards from point for a #line
directive.  If that is found, jump to the correponding location in the
file referenced by #line. 

Etags is left unchanged.

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Sven Utcke]

> Richard Stallman <rms@gnu.org> writes:
>    That is doing things the hard way.  Wouldn't it be easier to handle
>    #line in Lisp inside find-tag?
> 
> Okay, let's try to change the approach completely.  We should be able to
> do that all inside find-tag, without even changing the user interface.
> 
> Using C-u M-. one can search for more tags.  We (not me, please!) can
> change find-tag so that, before looking in the TAGS buffer for more
> tags, the current buffer is searched backwards from point for a #line
> directive.  If that is found, jump to the correponding location in the
> file referenced by #line. 

Fine.  So now we have a workaround which nearly works.  It has the
disadvantage of showing the .c file first (which is either nearly
unreadable or, if it isn't, might no get edited accidentially).  It
Requires one more key-sequence to get to the right place.  But, worst
of all:

> Etags is left unchanged.

And therefore, in my opinion, left buggy, since I reall believe etags
should honour #line directives.  Not the interface to etags, but etags
itself...

Sven
-- 
 _  __                     The Cognitive Systems Group
| |/ /___  __ _ ___                                       University of Hamburg
| ' </ _ \/ _` (_-<  phone:    +49 (0)40 42883-2576      Vogt-Koelln-Strasse 30
|_|\_\___/\__, /__/  fax  :    +49 (0)40 42883-2572             D-22527 Hamburg
          |___/ http://kogs-www.informatik.uni-hamburg.de/~utcke/home.html

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Sven Utcke]

>    That is backwards.  It should only tag zz.y, not zz.c.
> 
>    The reason is that zz.y may have tags that are not reflected in zz.c.
> 
> Argh.  Right.   
>    						  It will also be possible
>           to use `etags xx.web xx.c' and etags will only tag xx.c with tags
>           pointing to xx.web.
> 
>    That is reasonable in this case because xx.web is in a format that
>    can't be processed by etags.
> 
> So etags should distinguish the two cases.  I'd like to find a more
> linear logic, however.

Personally, I do not think we should burden ourselves with this at
all.  I certainly would not give a .web file to etags, and _not_
giving .web files to etags is easily automated :-) So I _believe_ that
the right thing to do would be to create references for all files
given to etags, and to obey #line.  So if someone gives a .y and .c
file to etags, it _should_ create duplicate references to the .y file.

Of course one could optionally have etags remove duplicate entries, or
(maybe even better), if both the .y and .c file are given than etags
could take this as a hint to not obey #line for this particular .c
file, but I believe the best approach for now might be to simply
generate the duplicate references and be done...

Sven
-- 
 _  __                     The Cognitive Systems Group
| |/ /___  __ _ ___                                       University of Hamburg
| ' </ _ \/ _` (_-<  phone:    +49 (0)40 42883-2576      Vogt-Koelln-Strasse 30
|_|\_\___/\__, /__/  fax  :    +49 (0)40 42883-2572             D-22527 Hamburg
          |___/ http://kogs-www.informatik.uni-hamburg.de/~utcke/home.html

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Stefan Monnier]

> Personally, I do not think we should burden ourselves with this at
> all.  I certainly would not give a .web file to etags, and _not_
> giving .web files to etags is easily automated :-) So I _believe_ that
> the right thing to do would be to create references for all files
> given to etags, and to obey #line.  So if someone gives a .y and .c
> file to etags, it _should_ create duplicate references to the .y file.
> 
> Of course one could optionally have etags remove duplicate entries, or
> (maybe even better), if both the .y and .c file are given than etags
> could take this as a hint to not obey #line for this particular .c
> file, but I believe the best approach for now might be to simply
> generate the duplicate references and be done...

(eval-when-compile (require 'aol))
(with-aol-mode (complete-agreement))


	Stefan


_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Richard Stallman]

    Not in the case to which the sentence above was alluding.  There you
    would have a line

    FUNC f::x(a,b) { a /?/ b }

    in the .ylag file, but a line

    _YLfunc_x__f_(_YLp *a, _YLp *b)

The second is an internal artifact of compilation, not part of the
source code.  Only the first is a real tag.  So etags should record
only the first one.


_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Richard Stallman]

    Using C-u M-. one can search for more tags.  We (not me, please!) can
    change find-tag so that, before looking in the TAGS buffer for more
    tags, the current buffer is searched backwards from point for a #line
    directive.  If that is found, jump to the correponding location in the
    file referenced by #line. 

    Etags is left unchanged.

This might be the right way to handle #line for this case, where the
#line refers to a real source file which etags cannot or did not scan.
Although I think it should go straight to the real source file;
it should not "find" any tags in the generated file at all.

However, for the .c and .y case, it should be etags that recognizes
that it should not scan the .c file if the .y file is going to
be scanned.

Please consider this question decided.

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Francesco Potorti`]

I made etags aware of the #line directives that can be found in
automatically generated sources, for example .y-->.c, or .web-->.[hc].

A consequence of my changes is that the memory footprint of etags
increased dramatically for big file, as it needs to keep all tags in
memory until all files are parsed.  In general this shouldn't be a
problem: tags files are relatively small.

Those of you who are interested, please try the new version in the CVS
or at <ftp://fly.cnuce.cnr.it/pub/etags.c.gz>.

There is an undocumented option for obtaining the old behaviour of
ignoring #line directives, but I won't tell you, so you will be forced
to read the source if you need that, and while cursing me you'll
remember to tell me that you had a problem :-)

Next change to do is the "don't parse xxx.c when both xxx.c and xxx.y
are given".

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Sven Utcke]

Hello Francesco,

> I made etags aware of the #line directives that can be found in
> automatically generated sources, for example .y-->.c, or .web-->.[hc].

Great, thanks!

Hmm.  One problem though, although I'm not sure how to handle this:  

I create the tag-file using something like:

find . -name "*.c" -exec etags -a -I -o /home/utcke/src/StreetSigns/TAGS {} \;

probably not the most elegant way to do this (something featuring
xargs would probably be preferred), but it works.  Only: the TAGS file
now contains entries like the following (I'm replacing the non-ascii
characters with a look-alike ascii representation...) 

^L
Canny/Khoros/kedgelio.c,21
#define MIN(^?MIN^A1,0
^L
kedgelio.web,226
static char rcsid[^?597,
boolean kread_edgels(^?54,
[...]

This is because the #line directives in Canny/Khoros/kedgelio.c read:

#line 590 "kedgelio.web"

But of course in the TAGS file it should read
Canny/Khoros/kedgelio.web instead of kedgelio.web.  Possible
solutions:

a) Get fweb to output absolute paths.
b) Have etags prefix the filename from the #Line-directive with the
   current path.
c) Something else?

I'm not sure what would be the best thing to do here, as both a) and
b) would prevent me from seperating the *.c from the *.web...

Sven
-- 
 _  __                     The Cognitive Systems Group
| |/ /___  __ _ ___                                       University of Hamburg
| ' </ _ \/ _` (_-<  phone:    +49 (0)40 42883-2576      Vogt-Koelln-Strasse 30
|_|\_\___/\__, /__/  fax  :    +49 (0)40 42883-2572             D-22527 Hamburg
          |___/ http://kogs-www.informatik.uni-hamburg.de/~utcke/home.html

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Bug Report (Feature request?) etags (GNU Emacs 21.1)

[Eli Zaretskii]

On Tue, 5 Mar 2002, Sven Utcke wrote:

> I create the tag-file using something like:
> 
> find . -name "*.c" -exec etags -a -I -o /home/utcke/src/StreetSigns/TAGS {} \;
> 
> probably not the most elegant way to do this (something featuring
> xargs would probably be preferred), but it works.

As Francesco pointed out in the past, etags can read the file names from 
stdin, so just pipe find's output to etags, and you'll have a much more 
efficient command.

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel