* Re: shell mode syntax higlighting problem with here documents [not found] <mailman.11923.1129741494.20277.bug-gnu-emacs@gnu.org> @ 2005-10-20 3:48 ` Stefan Monnier 2005-10-20 5:12 ` Nicholas Sushkin 2005-10-20 21:12 ` Nicholas Sushkin 0 siblings, 2 replies; 10+ messages in thread From: Stefan Monnier @ 2005-10-20 3:48 UTC (permalink / raw) Cc: emacs-devel > #!/bin/bash > cat << EOF | \ > tee -a bug.txt > This is a test case for a bug in bash shell mode text highlighting > EOF If people who hack on SH scripts with lots of heredoc thingies could try the patch below, it would help. It's relative to the Emacs-CVS code. It seems "safe" and appears to do the right thing with the above example, but I'd like to get some positive feedback before installing it, Stefan --- orig/lisp/progmodes/sh-script.el +++ mod/lisp/progmodes/sh-script.el @@ -869,7 +869,18 @@ (defconst sh-st-symbol (string-to-syntax "_")) (defconst sh-here-doc-syntax (string-to-syntax "|")) ;; generic string -(defconst sh-here-doc-open-re "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\).*\\(\n\\)") +(defconst sh-escaped-line-re + (concat + "\\(?:" + ;; First the real regexp. + "\\(?:.*[^\\\n]\\)?\\(?:\\(?:\\\\\\\\\\)*\\\\\n\\(?:.*[^\\\n]\\)?\\)*" + ;; Then a failsafe fallback: in case the search boundary prevents us from + ;; finding the actual end of line. + "\\|.*\\)")) + +(defconst sh-here-doc-open-re + (concat "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\)" + sh-escaped-line-re "\\(\n\\)")) (defvar sh-here-doc-markers nil) (make-variable-buffer-local 'sh-here-doc-markers) @@ -883,7 +894,9 @@ ;; A rough regexp that should find the opening <<EOF back. (sre (concat "<<\\(-?\\)\\s-*['\"\\]?" ;; Use \s| to cheaply check it's an open-heredoc. - eof-re "['\"]?\\([ \t|;&)<>].*\\)?\\s|")) + eof-re "['\"]?\\([ \t|;&)<>]" + sh-escaped-line-re + "\\)?\\s|")) ;; A regexp that will find other EOFs. (ere (concat "^" (if indented "[ \t]*") eof-re "\n")) (start (save-excursion @@ -922,7 +935,8 @@ START is the position of <<. STRING is the actual word used as delimiter (f.ex. \"EOF\"). INDENTED is non-nil if the here document's content (and the EOF mark) can -be indented (i.e. a <<- was used rather than just <<)." +be indented (i.e. a <<- was used rather than just <<). +Point is at the beginning of the next line." (unless (or (memq (char-before start) '(?< ?>)) (sh-in-comment-or-string start)) ;; We're looking at <<STRING, so we add "^STRING$" to the syntactic @@ -933,6 +947,20 @@ (setq sh-here-doc-re (concat sh-here-doc-open-re "\\|^\\([ \t]*\\)" (regexp-opt sh-here-doc-markers t) "\\(\n\\)")))) + (let ((ppss (save-excursion (syntax-ppss (1- (point)))))) + (if (nth 4 ppss) + ;; The \n not only starts the heredoc but also closes a comment. + ;; Let's close the comment just before the \n. + (put-text-property (1- (point)) (point) 'syntax-table '(12))) ;">" + (if (or (> (count-lines start (point)) 1) (nth 5 ppss)) + ;; If the sh-escaped-line-re part of sh-here-doc-re has matched + ;; several lines, make sure we refontify them together. + ;; Furthermore, if (nth 5 ppss) is non-nil (i.e. the \n is + ;; escaped), it means we haven't actually found the right \n. + ;; Don't bother fixing it now, but place a multiline property so + ;; that when jit-lock-context-* refontifies the rest of the + ;; buffer, it also refontifies the current line with it. + (put-text-property start (point) 'font-lock-multiline t))) sh-here-doc-syntax)) (defun sh-font-lock-here-doc (limit) ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-20 3:48 ` shell mode syntax higlighting problem with here documents Stefan Monnier @ 2005-10-20 5:12 ` Nicholas Sushkin 2005-10-20 21:12 ` Nicholas Sushkin 1 sibling, 0 replies; 10+ messages in thread From: Nicholas Sushkin @ 2005-10-20 5:12 UTC (permalink / raw) Cc: emacs-devel Stefan, Merci. I'll try to test the patch within the next couple of days and let you know. On Wednesday 19 October 2005 23:48, Stefan Monnier wrote: > If people who hack on SH scripts with lots of heredoc thingies could try > the patch below, it would help. It's relative to the Emacs-CVS code. It > seems "safe" and appears to do the right thing with the above example, > but I'd like to get some positive feedback before installing it, -- Nicholas Sushkin, Senior Software Engineer Open Finance ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-20 3:48 ` shell mode syntax higlighting problem with here documents Stefan Monnier 2005-10-20 5:12 ` Nicholas Sushkin @ 2005-10-20 21:12 ` Nicholas Sushkin 2005-10-20 21:22 ` Nick Roberts 1 sibling, 1 reply; 10+ messages in thread From: Nicholas Sushkin @ 2005-10-20 21:12 UTC (permalink / raw) Cc: emacs-devel Stefan, I am trying to test in Emacs 21.3.1 your revision 1.169, but getting "Invalid escape character syntax" in the statement below. Is my emacs too old or it's really a bug? Thanks (defvar sh-skeleton-pair-default-alist '((?( _ ?)) (?\)) (?[ ?\s _ ?\s ?]) (?\]) (?{ _ ?}) (?\})) "Value to use for `skeleton-pair-default-alist' in Shell-Script mode.") On Wednesday 19 October 2005 23:48, Stefan Monnier wrote: > > #!/bin/bash > > cat << EOF | \ > > tee -a bug.txt > > > > This is a test case for a bug in bash shell mode text highlighting > > EOF > > If people who hack on SH scripts with lots of heredoc thingies could try > the patch below, it would help. It's relative to the Emacs-CVS code. It > seems "safe" and appears to do the right thing with the above example, > but I'd like to get some positive feedback before installing it, > > > Stefan > > > --- orig/lisp/progmodes/sh-script.el > +++ mod/lisp/progmodes/sh-script.el > @@ -869,7 +869,18 @@ > (defconst sh-st-symbol (string-to-syntax "_")) > (defconst sh-here-doc-syntax (string-to-syntax "|")) ;; generic string > > -(defconst sh-here-doc-open-re > "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\).*\\(\n\\)") > +(defconst sh-escaped-line-re > + (concat > + "\\(?:" > + ;; First the real regexp. > + > "\\(?:.*[^\\\n]\\)?\\(?:\\(?:\\\\\\\\\\)*\\\\\n\\(?:.*[^\\\n]\\)?\\)*" + > ;; Then a failsafe fallback: in case the search boundary prevents us > from + ;; finding the actual end of line. > + "\\|.*\\)")) > + > +(defconst sh-here-doc-open-re > + (concat "<<-?\\s-*\\\\?\\(\\(?:['\"][^'\"]+['\"]\\|\\sw\\)+\\)" > + sh-escaped-line-re "\\(\n\\)")) > > (defvar sh-here-doc-markers nil) > (make-variable-buffer-local 'sh-here-doc-markers) > @@ -883,7 +894,9 @@ > ;; A rough regexp that should find the opening <<EOF back. > (sre (concat "<<\\(-?\\)\\s-*['\"\\]?" > ;; Use \s| to cheaply check it's an open-heredoc. > - eof-re "['\"]?\\([ \t|;&)<>].*\\)?\\s|")) > + eof-re "['\"]?\\([ \t|;&)<>]" > + sh-escaped-line-re > + "\\)?\\s|")) > ;; A regexp that will find other EOFs. > (ere (concat "^" (if indented "[ \t]*") eof-re "\n")) > (start (save-excursion > @@ -922,7 +935,8 @@ > START is the position of <<. > STRING is the actual word used as delimiter (f.ex. \"EOF\"). > INDENTED is non-nil if the here document's content (and the EOF mark) > can -be indented (i.e. a <<- was used rather than just <<)." > +be indented (i.e. a <<- was used rather than just <<). > +Point is at the beginning of the next line." > (unless (or (memq (char-before start) '(?< ?>)) > (sh-in-comment-or-string start)) > ;; We're looking at <<STRING, so we add "^STRING$" to the syntactic > @@ -933,6 +947,20 @@ > (setq sh-here-doc-re > (concat sh-here-doc-open-re "\\|^\\([ \t]*\\)" > (regexp-opt sh-here-doc-markers t) "\\(\n\\)")))) > + (let ((ppss (save-excursion (syntax-ppss (1- (point)))))) > + (if (nth 4 ppss) > + ;; The \n not only starts the heredoc but also closes a > comment. + ;; Let's close the comment just before the \n. > + (put-text-property (1- (point)) (point) 'syntax-table '(12))) > ;">" + (if (or (> (count-lines start (point)) 1) (nth 5 ppss)) > + ;; If the sh-escaped-line-re part of sh-here-doc-re has > matched + ;; several lines, make sure we refontify them > together. + ;; Furthermore, if (nth 5 ppss) is non-nil (i.e. the > \n is + ;; escaped), it means we haven't actually found the > right \n. + ;; Don't bother fixing it now, but place a multiline > property so + ;; that when jit-lock-context-* refontifies the > rest of the + ;; buffer, it also refontifies the current line > with it. + (put-text-property start (point) 'font-lock-multiline > t))) sh-here-doc-syntax)) > > (defun sh-font-lock-here-doc (limit) -- Nicholas Sushkin, Senior Software Engineer Open Finance, Chelsea Piers Pier 62 Suite 316, New York NY 10011 Tel +1 646 723 2790 Fax +1 646 723 2789 nsushkin@openfinance.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-20 21:12 ` Nicholas Sushkin @ 2005-10-20 21:22 ` Nick Roberts 2005-10-20 21:56 ` Nicholas Sushkin 0 siblings, 1 reply; 10+ messages in thread From: Nick Roberts @ 2005-10-20 21:22 UTC (permalink / raw) Cc: emacs-devel Nicholas Sushkin writes: > Stefan, I am trying to test in Emacs 21.3.1 your revision 1.169, but getting > "Invalid escape character syntax" in the statement below. Is my emacs too > old or it's really a bug? Your emacs is too old. I guess there might be some value in testing against an old version but there's a lot more value in testing with the current one. If you can't build Emacs from CVS then perhaps you can download the pretest which should be available shortly. Nick ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-20 21:22 ` Nick Roberts @ 2005-10-20 21:56 ` Nicholas Sushkin 2005-10-21 3:14 ` Stefan Monnier 0 siblings, 1 reply; 10+ messages in thread From: Nicholas Sushkin @ 2005-10-20 21:56 UTC (permalink / raw) Cc: emacs-devel Ok, I tried the CVS version as of half an hour ago. It seems that on the initial load, it highlights correctly. However, if I insert a couple of spaces at the beginning of the line that contains << EOF, the highlighting after the final EOF breaks. Kill and reload the file fixes it again. On Thursday 20 October 2005 17:22, Nick Roberts wrote: > Nicholas Sushkin writes: > > Stefan, I am trying to test in Emacs 21.3.1 your revision 1.169, but > > getting "Invalid escape character syntax" in the statement below. Is > > my emacs too old or it's really a bug? > > Your emacs is too old. I guess there might be some value in testing > against an old version but there's a lot more value in testing with the > current one. If you can't build Emacs from CVS then perhaps you can > download the pretest which should be available shortly. > > Nick -- Nicholas Sushkin, Senior Software Engineer Open Finance ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-20 21:56 ` Nicholas Sushkin @ 2005-10-21 3:14 ` Stefan Monnier 2005-10-21 17:03 ` Nicholas Sushkin 0 siblings, 1 reply; 10+ messages in thread From: Stefan Monnier @ 2005-10-21 3:14 UTC (permalink / raw) Cc: Nick Roberts, emacs-devel > Ok, I tried the CVS version as of half an hour ago. > It seems that on the initial load, it highlights correctly. However, if I > insert a couple of spaces at the beginning of the line that contains << > EOF, the highlighting after the final EOF breaks. Kill and reload the file > fixes it again. I've installed a slightly fixed variant of the patch I sent. Can you try with the latest CVS code? It seemed to work on all my tests. Stefan ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-21 3:14 ` Stefan Monnier @ 2005-10-21 17:03 ` Nicholas Sushkin 2005-10-21 17:42 ` Stefan Monnier 0 siblings, 1 reply; 10+ messages in thread From: Nicholas Sushkin @ 2005-10-21 17:03 UTC (permalink / raw) Cc: Nick Roberts, emacs-devel [-- Attachment #1: Type: text/plain, Size: 575 bytes --] On Thursday 20 October 2005 23:14, Stefan Monnier wrote: > I've installed a slightly fixed variant of the patch I sent. > Can you try with the latest CVS code? It seemed to work on all my tests. I still get the same behavior. I load the test case below. It starts highlighted properly (snapshot 1). I insert a space before the command that starts the here document, highlighting breaks (snapshot 2). I remove the inserted space, the highlighting is still broken (snapshot 3). Do you have the same behavior? -- Nicholas Sushkin, Senior Software Engineer Open Finance [-- Attachment #2: emacs-shhighlight-bug-1.png --] [-- Type: image/png, Size: 26071 bytes --] [-- Attachment #3: emacs-shhighlight-bug-2.png --] [-- Type: image/png, Size: 24313 bytes --] [-- Attachment #4: emacs-shhighlight-bug-3.png --] [-- Type: image/png, Size: 24310 bytes --] [-- Attachment #5: Type: text/plain, Size: 142 bytes --] _______________________________________________ Emacs-devel mailing list Emacs-devel@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-devel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-21 17:03 ` Nicholas Sushkin @ 2005-10-21 17:42 ` Stefan Monnier 2005-10-21 19:03 ` Nicholas Sushkin 0 siblings, 1 reply; 10+ messages in thread From: Stefan Monnier @ 2005-10-21 17:42 UTC (permalink / raw) Cc: Nick Roberts, emacs-devel >> I've installed a slightly fixed variant of the patch I sent. >> Can you try with the latest CVS code? It seemed to work on all my tests. > I still get the same behavior. I load the test case below. It starts > highlighted properly (snapshot 1). I insert a space before the command that > starts the here document, highlighting breaks (snapshot 2). I remove the > inserted space, the highlighting is still broken (snapshot 3). > Do you have the same behavior? I've just installed a patch to font-lock.el which I've been using "for ever" but somehow forgot to install until now. It should work better now. Stefan ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: shell mode syntax higlighting problem with here documents 2005-10-21 17:42 ` Stefan Monnier @ 2005-10-21 19:03 ` Nicholas Sushkin 0 siblings, 0 replies; 10+ messages in thread From: Nicholas Sushkin @ 2005-10-21 19:03 UTC (permalink / raw) Cc: Nick Roberts, emacs-devel On Friday 21 October 2005 13:42, Stefan Monnier wrote: > I've just installed a patch to font-lock.el which I've been using "for > ever" but somehow forgot to install until now. It should work better > now. It does! It seems to work right now. Thanks a lot, Stefan! -- Nicholas Sushkin, Senior Software Engineer Open Finance ^ permalink raw reply [flat|nested] 10+ messages in thread
* shell mode syntax higlighting problem with here documents @ 2005-10-19 17:04 Nicholas Sushkin 0 siblings, 0 replies; 10+ messages in thread From: Nicholas Sushkin @ 2005-10-19 17:04 UTC (permalink / raw) This bug report will be sent to the Free Software Foundation, not to your local site managers! Please write in English, because the Emacs maintainers do not have translators to read other languages for them. Your bug report will be posted to the bug-gnu-emacs@gnu.org mailing list, and to the gnu.emacs.bug news group. In GNU Emacs 21.3.1 (i386-redhat-linux-gnu, X toolkit, Xaw3d scroll bars) of 2005-02-04 on bugs.build.redhat.com configured using `configure --build=i386-redhat-linux --host=i386-redhat-linux --target=i386-redhat-linux-gnu --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --with-pop --with-sound' Important settings: value of $LC_ALL: nil value of $LC_COLLATE: nil value of $LC_CTYPE: nil value of $LC_MESSAGES: nil value of $LC_MONETARY: nil value of $LC_NUMERIC: nil value of $LC_TIME: nil value of $LANG: en_US.UTF-8 locale-coding-system: utf-8 default-enable-multibyte-characters: t Please describe exactly what actions triggered the bug and the precise symptoms of the bug: In the following shell script, the shell script mode syntax highlighting is broken. #!/bin/bash cat << EOF | \ tee -a bug.txt This is a test case for a bug in bash shell mode text highlighting EOF echo "Highlighting is screwed up now" if [ 1 = 1 ] then echo "screwed up" fi Recent input: <down> <down> <up> s c r e w e d SPC u p " <return> f i <return> C-x 4 b C-g C-x s <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <return> <S-up> <down> <up> # @ <S-backspace> ! / b i n / b a s h C-x s <down> <down> <down> <down-mouse-1> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <mouse-movement> <drag-mouse-1> <down-mouse-1> <mouse-1> <down-mouse-1> <mouse-1> <help-echo> <help-echo> <help-echo> <help-echo> <help-echo> <help-echo> <help-echo> <help-echo> <help-echo> <help-echo> <help-echo> <help-echo> <help-echo> <help-echo> <help-echo> <menu-bar> <help-menu> <report-emacs-bug> Recent messages: Indentation setup for shell type bash Mark set [3 times] Auto-saving...done Wrote /tmp/bug.sh Quit Wrote /tmp/bug.sh line-move: Beginning of buffer [2 times] Mark set Wrote /tmp/bug.sh Loading emacsbug...done ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2005-10-21 19:03 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- [not found] <mailman.11923.1129741494.20277.bug-gnu-emacs@gnu.org> 2005-10-20 3:48 ` shell mode syntax higlighting problem with here documents Stefan Monnier 2005-10-20 5:12 ` Nicholas Sushkin 2005-10-20 21:12 ` Nicholas Sushkin 2005-10-20 21:22 ` Nick Roberts 2005-10-20 21:56 ` Nicholas Sushkin 2005-10-21 3:14 ` Stefan Monnier 2005-10-21 17:03 ` Nicholas Sushkin 2005-10-21 17:42 ` Stefan Monnier 2005-10-21 19:03 ` Nicholas Sushkin 2005-10-19 17:04 Nicholas Sushkin
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.