* Re: shell mode syntax higlighting problem with here documents [not found] <mailman.11923.1129741494.20277.bug-gnu-emacs@gnu.org> @ 2005-10-20 3:48 ` Stefan Monnier 2005-10-20 5:12 ` Nicholas Sushkin 2005-10-20 21:12 ` Nicholas Sushkin 0 siblings, 2 replies; 9+ messages in thread From: Stefan Monnier @ 2005-10-20 3:48 UTC (permalink / raw) Cc: emacs-devel > #!/bin/bash > cat << EOF | \ > tee -a bug.txt > This is a test case for a bug in bash shell mode text highlighting > EOF If people who hack on SH scripts with lots of heredoc thingies could try the patch below, it would help. It's relative to the Emacs-CVS code. It seems "safe" and appears to do the right thing with the above example, but I'd like to get some positive feedback before installing it, Stefan --- orig/lisp/progmodes/sh-script.el +++ mod/lisp/progmodes/sh-script.el @@ -869,7 +869,18 @@ (defconst sh-st-symbol (string-to-syntax "_")) (defconst sh-here-doc-syntax (string-to-syntax "|")) ;; generic string -(defconst sh-here-doc-open-re "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\).*\\(\n\\)") +(defconst sh-escaped-line-re + (concat + "\\(?:" + ;; First the real regexp. + "\\(?:.*[^\\\n]\\)?\\(?:\\(?:\\\\\\\\\\)*\\\\\n\\(?:.*[^\\\n]\\)?\\)*" + ;; Then a failsafe fallback: in case the search boundary prevents us from + ;; finding the actual end of line. + "\\|.*\\)")) + +(defconst sh-here-doc-open-re + (concat "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\)" + sh-escaped-line-re "\\(\n\\)")) (defvar sh-here-doc-markers nil) (make-variable-buffer-local 'sh-here-doc-markers) @@ -883,7 +894,9 @@ ;; A rough regexp that should find the opening <<EOF back. (sre (concat "<<\\(-?\\)\\s-*['\"\\]?" ;; Use \s| to cheaply check it's an open-heredoc. - eof-re "['\"]?\\([ \t|;&)<>].*\\)?\\s|")) + eof-re "['\"]?\\([ \t|;&)<>]" + sh-escaped-line-re + "\\)?\\s|")) ;; A regexp that will find other EOFs. (ere (concat "^" (if indented "[ \t]*") eof-re "\n")) (start (save-excursion @@ -922,7 +935,8 @@ START is the position of <<. STRING is the actual word used as delimiter (f.ex. \"EOF\"). INDENTED is non-nil if the here document's content (and the EOF mark) can -be indented (i.e. a <<- was used rather than just <<)." +be indented (i.e. a <<- was used rather than just <<). +Point is at the beginning of the next line." (unless (or (memq (char-before start) '(?< ?>)) (sh-in-comment-or-string start)) ;; We're looking at <<STRING, so we add "^STRING$" to the syntactic @@ -933,6 +947,20 @@ (setq sh-here-doc-re (concat sh-here-doc-open-re "\\|^\\([ \t]*\\)" (regexp-opt sh-here-doc-markers t) "\\(\n\\)")))) + (let ((ppss (save-excursion (syntax-ppss (1- (point)))))) + (if (nth 4 ppss) + ;; The \n not only starts the heredoc but also closes a comment. + ;; Let's close the comment just before the \n. + (put-text-property (1- (point)) (point) 'syntax-table '(12))) ;">" + (if (or (> (count-lines start (point)) 1) (nth 5 ppss)) + ;; If the sh-escaped-line-re part of sh-here-doc-re has matched + ;; several lines, make sure we refontify them together. + ;; Furthermore, if (nth 5 ppss) is non-nil (i.e. the \n is + ;; escaped), it means we haven't actually found the right \n. + ;; Don't bother fixing it now, but place a multiline property so + ;; that when jit-lock-context-* refontifies the rest of the + ;; buffer, it also refontifies the current line with it. + (put-text-property start (point) 'font-lock-multiline t))) sh-here-doc-syntax)) (defun sh-font-lock-here-doc (limit) ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-20 3:48 ` shell mode syntax higlighting problem with here documents Stefan Monnier @ 2005-10-20 5:12 ` Nicholas Sushkin 2005-10-20 21:12 ` Nicholas Sushkin 1 sibling, 0 replies; 9+ messages in thread From: Nicholas Sushkin @ 2005-10-20 5:12 UTC (permalink / raw) Cc: emacs-devel Stefan, Merci. I'll try to test the patch within the next couple of days and let you know. On Wednesday 19 October 2005 23:48, Stefan Monnier wrote: > If people who hack on SH scripts with lots of heredoc thingies could try > the patch below, it would help. It's relative to the Emacs-CVS code. It > seems "safe" and appears to do the right thing with the above example, > but I'd like to get some positive feedback before installing it, -- Nicholas Sushkin, Senior Software Engineer Open Finance ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-20 3:48 ` shell mode syntax higlighting problem with here documents Stefan Monnier 2005-10-20 5:12 ` Nicholas Sushkin @ 2005-10-20 21:12 ` Nicholas Sushkin 2005-10-20 21:22 ` Nick Roberts 1 sibling, 1 reply; 9+ messages in thread From: Nicholas Sushkin @ 2005-10-20 21:12 UTC (permalink / raw) Cc: emacs-devel Stefan, I am trying to test in Emacs 21.3.1 your revision 1.169, but getting "Invalid escape character syntax" in the statement below. Is my emacs too old or it's really a bug? Thanks (defvar sh-skeleton-pair-default-alist '((?( _ ?)) (?\)) (?[ ?\s _ ?\s ?]) (?\]) (?{ _ ?}) (?\})) "Value to use for `skeleton-pair-default-alist' in Shell-Script mode.") On Wednesday 19 October 2005 23:48, Stefan Monnier wrote: > > #!/bin/bash > > cat << EOF | \ > > tee -a bug.txt > > > > This is a test case for a bug in bash shell mode text highlighting > > EOF > > If people who hack on SH scripts with lots of heredoc thingies could try > the patch below, it would help. It's relative to the Emacs-CVS code. It > seems "safe" and appears to do the right thing with the above example, > but I'd like to get some positive feedback before installing it, > > > Stefan > > > --- orig/lisp/progmodes/sh-script.el > +++ mod/lisp/progmodes/sh-script.el > @@ -869,7 +869,18 @@ > (defconst sh-st-symbol (string-to-syntax "_")) > (defconst sh-here-doc-syntax (string-to-syntax "|")) ;; generic string > > -(defconst sh-here-doc-open-re > "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\).*\\(\n\\)") > +(defconst sh-escaped-line-re > + (concat > + "\\(?:" > + ;; First the real regexp. > + > "\\(?:.*[^\\\n]\\)?\\(?:\\(?:\\\\\\\\\\)*\\\\\n\\(?:.*[^\\\n]\\)?\\)*" + > ;; Then a failsafe fallback: in case the search boundary prevents us > from + ;; finding the actual end of line. > + "\\|.*\\)")) > + > +(defconst sh-here-doc-open-re > + (concat "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\)" > + sh-escaped-line-re "\\(\n\\)")) > > (defvar sh-here-doc-markers nil) > (make-variable-buffer-local 'sh-here-doc-markers) > @@ -883,7 +894,9 @@ > ;; A rough regexp that should find the opening <<EOF back. > (sre (concat "<<\\(-?\\)\\s-*['\"\\]?" > ;; Use \s| to cheaply check it's an open-heredoc. > - eof-re "['\"]?\\([ \t|;&)<>].*\\)?\\s|")) > + eof-re "['\"]?\\([ \t|;&)<>]" > + sh-escaped-line-re > + "\\)?\\s|")) > ;; A regexp that will find other EOFs. > (ere (concat "^" (if indented "[ \t]*") eof-re "\n")) > (start (save-excursion > @@ -922,7 +935,8 @@ > START is the position of <<. > STRING is the actual word used as delimiter (f.ex. \"EOF\"). > INDENTED is non-nil if the here document's content (and the EOF mark) > can -be indented (i.e. a <<- was used rather than just <<)." > +be indented (i.e. a <<- was used rather than just <<). > +Point is at the beginning of the next line." > (unless (or (memq (char-before start) '(?< ?>)) > (sh-in-comment-or-string start)) > ;; We're looking at <<STRING, so we add "^STRING$" to the syntactic > @@ -933,6 +947,20 @@ > (setq sh-here-doc-re > (concat sh-here-doc-open-re "\\|^\\([ \t]*\\)" > (regexp-opt sh-here-doc-markers t) "\\(\n\\)")))) > + (let ((ppss (save-excursion (syntax-ppss (1- (point)))))) > + (if (nth 4 ppss) > + ;; The \n not only starts the heredoc but also closes a > comment. + ;; Let's close the comment just before the \n. > + (put-text-property (1- (point)) (point) 'syntax-table '(12))) > ;">" + (if (or (> (count-lines start (point)) 1) (nth 5 ppss)) > + ;; If the sh-escaped-line-re part of sh-here-doc-re has > matched + ;; several lines, make sure we refontify them > together. + ;; Furthermore, if (nth 5 ppss) is non-nil (i.e. the > \n is + ;; escaped), it means we haven't actually found the > right \n. + ;; Don't bother fixing it now, but place a multiline > property so + ;; that when jit-lock-context-* refontifies the > rest of the + ;; buffer, it also refontifies the current line > with it. + (put-text-property start (point) 'font-lock-multiline > t))) sh-here-doc-syntax)) > > (defun sh-font-lock-here-doc (limit) -- Nicholas Sushkin, Senior Software Engineer Open Finance, Chelsea Piers Pier 62 Suite 316, New York NY 10011 Tel +1 646 723 2790 Fax +1 646 723 2789 nsushkin@openfinance.com ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-20 21:12 ` Nicholas Sushkin @ 2005-10-20 21:22 ` Nick Roberts 2005-10-20 21:56 ` Nicholas Sushkin 0 siblings, 1 reply; 9+ messages in thread From: Nick Roberts @ 2005-10-20 21:22 UTC (permalink / raw) Cc: emacs-devel Nicholas Sushkin writes: > Stefan, I am trying to test in Emacs 21.3.1 your revision 1.169, but getting > "Invalid escape character syntax" in the statement below. Is my emacs too > old or it's really a bug? Your emacs is too old. I guess there might be some value in testing against an old version but there's a lot more value in testing with the current one. If you can't build Emacs from CVS then perhaps you can download the pretest which should be available shortly. Nick ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-20 21:22 ` Nick Roberts @ 2005-10-20 21:56 ` Nicholas Sushkin 2005-10-21 3:14 ` Stefan Monnier 0 siblings, 1 reply; 9+ messages in thread From: Nicholas Sushkin @ 2005-10-20 21:56 UTC (permalink / raw) Cc: emacs-devel Ok, I tried the CVS version as of half an hour ago. It seems that on the initial load, it highlights correctly. However, if I insert a couple of spaces at the beginning of the line that contains << EOF, the highlighting after the final EOF breaks. Kill and reload the file fixes it again. On Thursday 20 October 2005 17:22, Nick Roberts wrote: > Nicholas Sushkin writes: > > Stefan, I am trying to test in Emacs 21.3.1 your revision 1.169, but > > getting "Invalid escape character syntax" in the statement below. Is > > my emacs too old or it's really a bug? > > Your emacs is too old. I guess there might be some value in testing > against an old version but there's a lot more value in testing with the > current one. If you can't build Emacs from CVS then perhaps you can > download the pretest which should be available shortly. > > Nick -- Nicholas Sushkin, Senior Software Engineer Open Finance ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-20 21:56 ` Nicholas Sushkin @ 2005-10-21 3:14 ` Stefan Monnier 2005-10-21 17:03 ` Nicholas Sushkin 0 siblings, 1 reply; 9+ messages in thread From: Stefan Monnier @ 2005-10-21 3:14 UTC (permalink / raw) Cc: Nick Roberts, emacs-devel > Ok, I tried the CVS version as of half an hour ago. > It seems that on the initial load, it highlights correctly. However, if I > insert a couple of spaces at the beginning of the line that contains << > EOF, the highlighting after the final EOF breaks. Kill and reload the file > fixes it again. I've installed a slightly fixed variant of the patch I sent. Can you try with the latest CVS code? It seemed to work on all my tests. Stefan ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-21 3:14 ` Stefan Monnier @ 2005-10-21 17:03 ` Nicholas Sushkin 2005-10-21 17:42 ` Stefan Monnier 0 siblings, 1 reply; 9+ messages in thread From: Nicholas Sushkin @ 2005-10-21 17:03 UTC (permalink / raw) Cc: Nick Roberts, emacs-devel [-- Attachment #1: Type: text/plain, Size: 575 bytes --] On Thursday 20 October 2005 23:14, Stefan Monnier wrote: > I've installed a slightly fixed variant of the patch I sent. > Can you try with the latest CVS code? It seemed to work on all my tests. I still get the same behavior. I load the test case below. It starts highlighted properly (snapshot 1). I insert a space before the command that starts the here document, highlighting breaks (snapshot 2). I remove the inserted space, the highlighting is still broken (snapshot 3). Do you have the same behavior? -- Nicholas Sushkin, Senior Software Engineer Open Finance [-- Attachment #2: emacs-shhighlight-bug-1.png --] [-- Type: image/png, Size: 26071 bytes --] [-- Attachment #3: emacs-shhighlight-bug-2.png --] [-- Type: image/png, Size: 24313 bytes --] [-- Attachment #4: emacs-shhighlight-bug-3.png --] [-- Type: image/png, Size: 24310 bytes --] [-- Attachment #5: Type: text/plain, Size: 142 bytes --] _______________________________________________ Emacs-devel mailing list Emacs-devel@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-21 17:03 ` Nicholas Sushkin @ 2005-10-21 17:42 ` Stefan Monnier 2005-10-21 19:03 ` Nicholas Sushkin 0 siblings, 1 reply; 9+ messages in thread From: Stefan Monnier @ 2005-10-21 17:42 UTC (permalink / raw) Cc: Nick Roberts, emacs-devel >> I've installed a slightly fixed variant of the patch I sent. >> Can you try with the latest CVS code? It seemed to work on all my tests. > I still get the same behavior. I load the test case below. It starts > highlighted properly (snapshot 1). I insert a space before the command that > starts the here document, highlighting breaks (snapshot 2). I remove the > inserted space, the highlighting is still broken (snapshot 3). > Do you have the same behavior? I've just installed a patch to font-lock.el which I've been using "for ever" but somehow forgot to install until now. It should work better now. Stefan ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-21 17:42 ` Stefan Monnier @ 2005-10-21 19:03 ` Nicholas Sushkin 0 siblings, 0 replies; 9+ messages in thread From: Nicholas Sushkin @ 2005-10-21 19:03 UTC (permalink / raw) Cc: Nick Roberts, emacs-devel On Friday 21 October 2005 13:42, Stefan Monnier wrote: > I've just installed a patch to font-lock.el which I've been using "for > ever" but somehow forgot to install until now. It should work better > now. It does! It seems to work right now. Thanks a lot, Stefan! -- Nicholas Sushkin, Senior Software Engineer Open Finance ^ permalink raw reply [flat|nested] 9+ messages in thread
![[-- Attachment #2: emacs-shhighlight-bug-1.png --] [-- Type: image/png, Size: 26071 bytes --]](../../200510211303.09656.nsushkin@openfinance.com/2-emacs-shhighlight-bug-1.png){kind=link}
![[-- Attachment #3: emacs-shhighlight-bug-2.png --] [-- Type: image/png, Size: 24313 bytes --]](../../200510211303.09656.nsushkin@openfinance.com/3-emacs-shhighlight-bug-2.png){kind=link}
![[-- Attachment #4: emacs-shhighlight-bug-3.png --] [-- Type: image/png, Size: 24310 bytes --]](../../200510211303.09656.nsushkin@openfinance.com/4-emacs-shhighlight-bug-3.png){kind=link}
end of thread, other threads:[~2005-10-21 19:03 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <mailman.11923.1129741494.20277.bug-gnu-emacs@gnu.org>
2005-10-20 3:48 ` shell mode syntax higlighting problem with here documents Stefan Monnier
2005-10-20 5:12 ` Nicholas Sushkin
2005-10-20 21:12 ` Nicholas Sushkin
2005-10-20 21:22 ` Nick Roberts
2005-10-20 21:56 ` Nicholas Sushkin
2005-10-21 3:14 ` Stefan Monnier
2005-10-21 17:03 ` Nicholas Sushkin
2005-10-21 17:42 ` Stefan Monnier
2005-10-21 19:03 ` Nicholas Sushkin
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).