unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Re: shell mode syntax higlighting problem with here documents
       [not found] <mailman.11923.1129741494.20277.bug-gnu-emacs@gnu.org>
@ 2005-10-20  3:48 ` Stefan Monnier
  2005-10-20  5:12   ` Nicholas Sushkin
  2005-10-20 21:12   ` Nicholas Sushkin
  0 siblings, 2 replies; 9+ messages in thread
From: Stefan Monnier @ 2005-10-20  3:48 UTC (permalink / raw)
  Cc: emacs-devel

> #!/bin/bash
> cat << EOF | \
>     tee -a bug.txt

> This is a test case for a bug in bash shell mode text highlighting
> EOF

If people who hack on SH scripts with lots of heredoc thingies could try the
patch below, it would help.  It's relative to the Emacs-CVS code.
It seems "safe" and appears to do the right thing with the above example,
but I'd like to get some positive feedback before installing it,


        Stefan


--- orig/lisp/progmodes/sh-script.el
+++ mod/lisp/progmodes/sh-script.el
@@ -869,7 +869,18 @@
 (defconst sh-st-symbol (string-to-syntax "_"))
 (defconst sh-here-doc-syntax (string-to-syntax "|")) ;; generic string
 
-(defconst sh-here-doc-open-re "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\).*\\(\n\\)")
+(defconst sh-escaped-line-re
+  (concat
+   "\\(?:"
+   ;; First the real regexp.
+   "\\(?:.*[^\\\n]\\)?\\(?:\\(?:\\\\\\\\\\)*\\\\\n\\(?:.*[^\\\n]\\)?\\)*"
+   ;; Then a failsafe fallback: in case the search boundary prevents us from
+   ;; finding the actual end of line.
+   "\\|.*\\)"))
+
+(defconst sh-here-doc-open-re
+  (concat "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\)"
+          sh-escaped-line-re "\\(\n\\)"))
 
 (defvar sh-here-doc-markers nil)
 (make-variable-buffer-local 'sh-here-doc-markers)
@@ -883,7 +894,9 @@
          ;; A rough regexp that should find the opening <<EOF back.
 	 (sre (concat "<<\\(-?\\)\\s-*['\"\\]?"
 		      ;; Use \s| to cheaply check it's an open-heredoc.
-		      eof-re "['\"]?\\([ \t|;&)<>].*\\)?\\s|"))
+		      eof-re "['\"]?\\([ \t|;&)<>]"
+                      sh-escaped-line-re
+                      "\\)?\\s|"))
 	 ;; A regexp that will find other EOFs.
 	 (ere (concat "^" (if indented "[ \t]*") eof-re "\n"))
 	 (start (save-excursion
@@ -922,7 +935,8 @@
 START is the position of <<.
 STRING is the actual word used as delimiter (f.ex. \"EOF\").
 INDENTED is non-nil if the here document's content (and the EOF mark) can
-be indented (i.e. a <<- was used rather than just <<)."
+be indented (i.e. a <<- was used rather than just <<).
+Point is at the beginning of the next line."
   (unless (or (memq (char-before start) '(?< ?>))
 	      (sh-in-comment-or-string start))
     ;; We're looking at <<STRING, so we add "^STRING$" to the syntactic
@@ -933,6 +947,20 @@
 	(setq sh-here-doc-re
 	      (concat sh-here-doc-open-re "\\|^\\([ \t]*\\)"
 		      (regexp-opt sh-here-doc-markers t) "\\(\n\\)"))))
+    (let ((ppss (save-excursion (syntax-ppss (1- (point))))))
+      (if (nth 4 ppss)
+          ;; The \n not only starts the heredoc but also closes a comment.
+          ;; Let's close the comment just before the \n.
+          (put-text-property (1- (point)) (point) 'syntax-table '(12))) ;">"
+      (if (or (> (count-lines start (point)) 1) (nth 5 ppss))
+          ;; If the sh-escaped-line-re part of sh-here-doc-re has matched
+          ;; several lines, make sure we refontify them together.
+          ;; Furthermore, if (nth 5 ppss) is non-nil (i.e. the \n is
+          ;; escaped), it means we haven't actually found the right \n.
+          ;; Don't bother fixing it now, but place a multiline property so
+          ;; that when jit-lock-context-* refontifies the rest of the
+          ;; buffer, it also refontifies the current line with it.
+          (put-text-property start (point) 'font-lock-multiline t)))
     sh-here-doc-syntax))
 
 (defun sh-font-lock-here-doc (limit)

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: shell mode syntax higlighting problem with here documents
  2005-10-20  3:48 ` shell mode syntax higlighting problem with here documents Stefan Monnier
@ 2005-10-20  5:12   ` Nicholas Sushkin
  2005-10-20 21:12   ` Nicholas Sushkin
  1 sibling, 0 replies; 9+ messages in thread
From: Nicholas Sushkin @ 2005-10-20  5:12 UTC (permalink / raw)
  Cc: emacs-devel

Stefan,

Merci. I'll try to test the patch within the next couple of days and let you 
know.

On Wednesday 19 October 2005 23:48, Stefan Monnier wrote: 

> If people who hack on SH scripts with lots of heredoc thingies could try
> the patch below, it would help.  It's relative to the Emacs-CVS code. It
> seems "safe" and appears to do the right thing with the above example,
> but I'd like to get some positive feedback before installing it,

-- 
Nicholas Sushkin, Senior Software Engineer
Open Finance

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: shell mode syntax higlighting problem with here documents
  2005-10-20  3:48 ` shell mode syntax higlighting problem with here documents Stefan Monnier
  2005-10-20  5:12   ` Nicholas Sushkin
@ 2005-10-20 21:12   ` Nicholas Sushkin
  2005-10-20 21:22     ` Nick Roberts
  1 sibling, 1 reply; 9+ messages in thread
From: Nicholas Sushkin @ 2005-10-20 21:12 UTC (permalink / raw)
  Cc: emacs-devel

Stefan, I am trying to test in Emacs 21.3.1 your revision 1.169, but getting 
"Invalid escape character syntax" in the statement below. Is my emacs too 
old or it's really a bug? 

Thanks

(defvar sh-skeleton-pair-default-alist '((?( _ ?)) (?\))
          (?[ ?\s _ ?\s ?]) (?\])
          (?{ _ ?}) (?\}))
  "Value to use for `skeleton-pair-default-alist' in Shell-Script mode.")




On Wednesday 19 October 2005 23:48, Stefan Monnier wrote: 

> > #!/bin/bash
> > cat << EOF | \
> >     tee -a bug.txt
> >
> > This is a test case for a bug in bash shell mode text highlighting
> > EOF
>
> If people who hack on SH scripts with lots of heredoc thingies could try
> the patch below, it would help.  It's relative to the Emacs-CVS code. It
> seems "safe" and appears to do the right thing with the above example,
> but I'd like to get some positive feedback before installing it,
>
>
>         Stefan
>
>
> --- orig/lisp/progmodes/sh-script.el
> +++ mod/lisp/progmodes/sh-script.el
> @@ -869,7 +869,18 @@
>  (defconst sh-st-symbol (string-to-syntax "_"))
>  (defconst sh-here-doc-syntax (string-to-syntax "|")) ;; generic string
>
> -(defconst sh-here-doc-open-re
> "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\).*\\(\n\\)")
> +(defconst sh-escaped-line-re
> +  (concat
> +   "\\(?:"
> +   ;; First the real regexp.
> +  
> "\\(?:.*[^\\\n]\\)?\\(?:\\(?:\\\\\\\\\\)*\\\\\n\\(?:.*[^\\\n]\\)?\\)*" + 
>  ;; Then a failsafe fallback: in case the search boundary prevents us
> from +   ;; finding the actual end of line.
> +   "\\|.*\\)"))
> +
> +(defconst sh-here-doc-open-re
> +  (concat "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\)"
> +          sh-escaped-line-re "\\(\n\\)"))
>
>  (defvar sh-here-doc-markers nil)
>  (make-variable-buffer-local 'sh-here-doc-markers)
> @@ -883,7 +894,9 @@
>           ;; A rough regexp that should find the opening <<EOF back.
>  	 (sre (concat "<<\\(-?\\)\\s-*['\"\\]?"
>  		      ;; Use \s| to cheaply check it's an open-heredoc.
> -		      eof-re "['\"]?\\([ \t|;&)<>].*\\)?\\s|"))
> +		      eof-re "['\"]?\\([ \t|;&)<>]"
> +                      sh-escaped-line-re
> +                      "\\)?\\s|"))
>  	 ;; A regexp that will find other EOFs.
>  	 (ere (concat "^" (if indented "[ \t]*") eof-re "\n"))
>  	 (start (save-excursion
> @@ -922,7 +935,8 @@
>  START is the position of <<.
>  STRING is the actual word used as delimiter (f.ex. \"EOF\").
>  INDENTED is non-nil if the here document's content (and the EOF mark)
> can -be indented (i.e. a <<- was used rather than just <<)."
> +be indented (i.e. a <<- was used rather than just <<).
> +Point is at the beginning of the next line."
>    (unless (or (memq (char-before start) '(?< ?>))
>  	      (sh-in-comment-or-string start))
>      ;; We're looking at <<STRING, so we add "^STRING$" to the syntactic
> @@ -933,6 +947,20 @@
>  	(setq sh-here-doc-re
>  	      (concat sh-here-doc-open-re "\\|^\\([ \t]*\\)"
>  		      (regexp-opt sh-here-doc-markers t) "\\(\n\\)"))))
> +    (let ((ppss (save-excursion (syntax-ppss (1- (point))))))
> +      (if (nth 4 ppss)
> +          ;; The \n not only starts the heredoc but also closes a
> comment. +          ;; Let's close the comment just before the \n.
> +          (put-text-property (1- (point)) (point) 'syntax-table '(12)))
> ;">" +      (if (or (> (count-lines start (point)) 1) (nth 5 ppss))
> +          ;; If the sh-escaped-line-re part of sh-here-doc-re has
> matched +          ;; several lines, make sure we refontify them
> together. +          ;; Furthermore, if (nth 5 ppss) is non-nil (i.e. the
> \n is +          ;; escaped), it means we haven't actually found the
> right \n. +          ;; Don't bother fixing it now, but place a multiline
> property so +          ;; that when jit-lock-context-* refontifies the
> rest of the +          ;; buffer, it also refontifies the current line
> with it. +          (put-text-property start (point) 'font-lock-multiline
> t))) sh-here-doc-syntax))
>
>  (defun sh-font-lock-here-doc (limit)

-- 
Nicholas Sushkin, Senior Software Engineer
Open Finance, Chelsea Piers Pier 62 Suite 316, New York NY 10011
Tel +1 646 723 2790 Fax +1 646 723 2789 nsushkin@openfinance.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: shell mode syntax higlighting problem with here documents
  2005-10-20 21:12   ` Nicholas Sushkin
@ 2005-10-20 21:22     ` Nick Roberts
  2005-10-20 21:56       ` Nicholas Sushkin
  0 siblings, 1 reply; 9+ messages in thread
From: Nick Roberts @ 2005-10-20 21:22 UTC (permalink / raw)
  Cc: emacs-devel

Nicholas Sushkin writes:
 > Stefan, I am trying to test in Emacs 21.3.1 your revision 1.169, but getting 
 > "Invalid escape character syntax" in the statement below. Is my emacs too 
 > old or it's really a bug? 

Your emacs is too old.  I guess there might be some value in testing against
an old version but there's a lot more value in testing with the current one.
If you can't build Emacs from CVS then perhaps you can download the pretest
which should be available shortly.

Nick

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: shell mode syntax higlighting problem with here documents
  2005-10-20 21:22     ` Nick Roberts
@ 2005-10-20 21:56       ` Nicholas Sushkin
  2005-10-21  3:14         ` Stefan Monnier
  0 siblings, 1 reply; 9+ messages in thread
From: Nicholas Sushkin @ 2005-10-20 21:56 UTC (permalink / raw)
  Cc: emacs-devel

Ok, I tried the CVS version as of half an hour ago.

It seems that on the initial load, it highlights correctly. However, if I 
insert a couple of spaces at the beginning of the line that contains << 
EOF, the highlighting after the final EOF breaks. Kill and reload the file 
fixes it again.

On Thursday 20 October 2005 17:22, Nick Roberts wrote: 

> Nicholas Sushkin writes:
>  > Stefan, I am trying to test in Emacs 21.3.1 your revision 1.169, but
>  > getting "Invalid escape character syntax" in the statement below. Is
>  > my emacs too old or it's really a bug?
>
> Your emacs is too old.  I guess there might be some value in testing
> against an old version but there's a lot more value in testing with the
> current one. If you can't build Emacs from CVS then perhaps you can
> download the pretest which should be available shortly.
>
> Nick

-- 
Nicholas Sushkin, Senior Software Engineer
Open Finance

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: shell mode syntax higlighting problem with here documents
  2005-10-20 21:56       ` Nicholas Sushkin
@ 2005-10-21  3:14         ` Stefan Monnier
  2005-10-21 17:03           ` Nicholas Sushkin
  0 siblings, 1 reply; 9+ messages in thread
From: Stefan Monnier @ 2005-10-21  3:14 UTC (permalink / raw)
  Cc: Nick Roberts, emacs-devel

> Ok, I tried the CVS version as of half an hour ago.
> It seems that on the initial load, it highlights correctly. However, if I 
> insert a couple of spaces at the beginning of the line that contains << 
> EOF, the highlighting after the final EOF breaks. Kill and reload the file 
> fixes it again.

I've installed a slightly fixed variant of the patch I sent.
Can you try with the latest CVS code?  It seemed to work on all my tests.


        Stefan

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: shell mode syntax higlighting problem with here documents
  2005-10-21  3:14         ` Stefan Monnier
@ 2005-10-21 17:03           ` Nicholas Sushkin
  2005-10-21 17:42             ` Stefan Monnier
  0 siblings, 1 reply; 9+ messages in thread
From: Nicholas Sushkin @ 2005-10-21 17:03 UTC (permalink / raw)
  Cc: Nick Roberts, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 575 bytes --]

On Thursday 20 October 2005 23:14, Stefan Monnier wrote: 

> I've installed a slightly fixed variant of the patch I sent.
> Can you try with the latest CVS code?  It seemed to work on all my tests.

I still get the same behavior. I load the test case below. It starts 
highlighted properly (snapshot 1). I insert a space before the command that 
starts the here document, highlighting breaks (snapshot 2). I remove the 
inserted space, the highlighting is still broken (snapshot 3).

Do you have the same behavior?
-- 
Nicholas Sushkin, Senior Software Engineer
Open Finance

[-- Attachment #2: emacs-shhighlight-bug-1.png --]
[-- Type: image/png, Size: 26071 bytes --]

[-- Attachment #3: emacs-shhighlight-bug-2.png --]
[-- Type: image/png, Size: 24313 bytes --]

[-- Attachment #4: emacs-shhighlight-bug-3.png --]
[-- Type: image/png, Size: 24310 bytes --]

[-- Attachment #5: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: shell mode syntax higlighting problem with here documents
  2005-10-21 17:03           ` Nicholas Sushkin
@ 2005-10-21 17:42             ` Stefan Monnier
  2005-10-21 19:03               ` Nicholas Sushkin
  0 siblings, 1 reply; 9+ messages in thread
From: Stefan Monnier @ 2005-10-21 17:42 UTC (permalink / raw)
  Cc: Nick Roberts, emacs-devel

>> I've installed a slightly fixed variant of the patch I sent.
>> Can you try with the latest CVS code?  It seemed to work on all my tests.

> I still get the same behavior. I load the test case below. It starts 
> highlighted properly (snapshot 1). I insert a space before the command that 
> starts the here document, highlighting breaks (snapshot 2). I remove the 
> inserted space, the highlighting is still broken (snapshot 3).

> Do you have the same behavior?

I've just installed a patch to font-lock.el which I've been using "for ever"
but somehow forgot to install until now.  It should work better now.


        Stefan

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: shell mode syntax higlighting problem with here documents
  2005-10-21 17:42             ` Stefan Monnier
@ 2005-10-21 19:03               ` Nicholas Sushkin
  0 siblings, 0 replies; 9+ messages in thread
From: Nicholas Sushkin @ 2005-10-21 19:03 UTC (permalink / raw)
  Cc: Nick Roberts, emacs-devel

On Friday 21 October 2005 13:42, Stefan Monnier wrote: 

> I've just installed a patch to font-lock.el which I've been using "for
> ever" but somehow forgot to install until now.  It should work better
> now.

It does! It seems to work right now. Thanks a lot, Stefan!

-- 
Nicholas Sushkin, Senior Software Engineer
Open Finance

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2005-10-21 19:03 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <mailman.11923.1129741494.20277.bug-gnu-emacs@gnu.org>
2005-10-20  3:48 ` shell mode syntax higlighting problem with here documents Stefan Monnier
2005-10-20  5:12   ` Nicholas Sushkin
2005-10-20 21:12   ` Nicholas Sushkin
2005-10-20 21:22     ` Nick Roberts
2005-10-20 21:56       ` Nicholas Sushkin
2005-10-21  3:14         ` Stefan Monnier
2005-10-21 17:03           ` Nicholas Sushkin
2005-10-21 17:42             ` Stefan Monnier
2005-10-21 19:03               ` Nicholas Sushkin

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).