identifiing strings and comments in program source files (to skip them over)

identifiing strings and comments in program source files (to skip them over)

[Peter Tury]

Hi,

isn't it a good idea to use the face property (get-text-property (point)
'face) to decide if the point is after a string or comment?

I have a mode what uses font-lock-mode and highlights everything nicely. I
think I could use this (= the face set by font-lock-mode) for identifiing
if e.g. a search stopped in a string or comment. (I know font-lock-mode
must be enabled for this.) This way I could omit reparsing the buffer (with
parse-partial-sexp) e.g. in a function what searches something "real" (=
strings and comments should be skipped).

What do you think: can a robust solution be built using this approach?

Thanks,
P

Re: identifiing strings and comments in program source files (to skip them over)

[Thien-Thi Nguyen]

Peter Tury <tury.peter@gmail.com> writes:

> What do you think: can a robust solution be built using this approach?

yes, if you now how (and how much) to wait for font lock to do its job:
if the scanning you have in mind occurs before font lock is done, it may
give incorrect results.  this can be difficult to debug because by the
time you notice the incorrect results, font lock may have finished.

thi

Re: identifiing strings and comments in program source files (to skip them over)

[Stefan Monnier]

> I have a mode what uses font-lock-mode and highlights everything nicely. I
> think I could use this (= the face set by font-lock-mode) for identifiing
> if e.g. a search stopped in a string or comment. (I know font-lock-mode
> must be enabled for this.) This way I could omit reparsing the buffer (with
> parse-partial-sexp) e.g. in a function what searches something "real" (=
> strings and comments should be skipped).

You can use this (I've used it in several occasions).
In Emacs-CVS you can also use syntax-ppss which is like parse-partial-sexp
except it does caching (and it's used by font-lock so the cache is already
filled by the font-lock activity).


        Stefan

identifiing strings and comments in program source files (to skip them over)

[martin rudalics]

 > Hi,
 >
 > isn't it a good idea to use the face property (get-text-property (point)
 > 'face) to decide if the point is after a string or comment?
 >
 > I have a mode what uses font-lock-mode and highlights everything nicely.

Font-lock support modes like jit-lock try to avoid assigning faces to
parts of a buffer that are not displayed.  Hence, a default Emacs
usually does not highlight everything nicely.

 > I
 > think I could use this (= the face set by font-lock-mode) for identifiing
 > if e.g. a search stopped in a string or comment. (I know font-lock-mode
 > must be enabled for this.) This way I could omit reparsing the buffer (with
 > parse-partial-sexp) e.g. in a function what searches something "real" (=
 > strings and comments should be skipped).
 >
 > What do you think: can a robust solution be built using this approach?

Not really.  Fontification is synchronized with redisplay.  Determining
whether a search stopped in a string or comment, on the other hand,
should work independently of whether redisplay happened.  In particular,
contextual fontification - the thing you see happening when you insert a
single double-quote in an elisp buffer - occurs by default not earlier
than .5 seconds after you inserted the double-quote - so your search
would have to wait until that has been done.  Deferred fontification and
syntax-table properties get things more complicated even.

You could explicitly fontify the area you want to search through, but
this might require fontifying the entire buffer and thus would not
provide any advantage: After all font-lock would have to reparse the
entire buffer for this purpose.  To check whether searching stopped in a
comment or string you could parse from `beginning-of-defun' or something
similar instead - `syntax-ppss' does that.

Re: identifiing strings and comments in program source files (to skip them over)

[Peter Tury]

On Wed, 26 Apr 2006 05:53:01 +0200, Thien-Thi Nguyen wrote:

> Peter Tury <tury.peter@gmail.com> writes:
> 
>> What do you think: can a robust solution be built using this approach?
> 
> yes, if you now how (and how much) to wait for font lock to do its job:

Yes, I had a feeling this could be a problem.
When I eval (text-properties-at (point)) I see e.g. (face
font-lock-comment-face fontified t). Can I use this "fontified" for
checking if font lock is done? Or this remains always t if fontified once?
What is this "fontified" exactly? How can I get more info? Please point me
some place in the manual.

Anyway, I would like to get some hideshow ;-)) (I couldn't use hideshow.el
until now for this major mode; now I plan to reuse it somehow, with
modifications.) So I hope fontification will be ready when I search (for
hide/show.)

Thanks!
P

Re: identifiing strings and comments in program source files (to skip them over)

[Peter Tury]

On Wed, 26 Apr 2006 02:01:53 -0400, Stefan Monnier wrote:

> In Emacs-CVS you can also use syntax-ppss which is like parse-partial-sexp
> except it does caching (and it's used by font-lock so the cache is already
> filled by the font-lock activity).

Thanks! I will check this syntax-ppss also!
P

Re: identifiing strings and comments in program source files (to skip them over)

[Peter Tury]

On Wed, 26 Apr 2006 08:38:52 +0200, martin rudalics wrote:

> To check whether searching stopped in a
> comment or string you could parse from `beginning-of-defun' or something
> similar instead - `syntax-ppss' does that.

Thanks for this detailed explanation also! My problem is that I have legacy
code (with ~1 MB source files sometimes) and I can't really imagine
anything for beginning-of-defun. E.g. C-like /*...*/ works for commenting
and whole subprograms can be commented out... In other words: I think I
should have to start parsing everytime from the beginning of the buffer...

Any idea? Or 1MB source files are not too big for this on today's PCs?

Br,
P