bug#18405: 24.3.93; bat-mode.el

bug#18405: 24.3.93; bat-mode.el

[Achim Gratz]

In GNU Emacs 24.3.93.1 (i686-pc-linux-gnu, GTK+ Version 3.12.2)
 of 2014-08-24 on Rainer.invalid
Windowing system distributor `The X.Org Foundation', version 11.0.11403901
System Description:	openSUSE 13.1 (Bottle) (i586)

The fontlock pattern for environment variables ins bat-mode.el is

        ("%\\(\\w+\\)%?"
         (1 font-lock-variable-name-face))
        ("!\\(\\w+\\)!?"                ; delayed-expansion !variable!
         (1 font-lock-variable-name-face))

For CMD.EXE at least from Windows 7 onward (I've also tested with the
cmd.exe that comes with Wine), the name of an environment variable can
actually be any character including shell special characters (which
you'd then have to quote separately).  While it's possible to define a
variable that has a "%" character in its name, it can not be used it in
cmd itself since it never gets expanded, so the easiest replacement
pattern would be

        ("%\\([^%]+\\)%?"
         (1 font-lock-variable-name-face))
        ("!\\([^!%]+\\)!?"                ; delayed-expansion !variable!
         (1 font-lock-variable-name-face))

I've noticed this discrepancy when using variables with underscores in
their name, which only fontified the part up to the first underscore
correctly.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

bug#18405: 24.3.93; bat-mode.el

[Stefan Monnier]

>         ("%\\([^%]+\\)%?"
>          (1 font-lock-variable-name-face))
>         ("!\\([^!%]+\\)!?"                ; delayed-expansion !variable!
>          (1 font-lock-variable-name-face))

I know nothing of cmd.exe's syntax and the general thrust of the change
sounds good, but: the above matches "%foo\nbar", which is probably not
right.  When can variable refs be of the form %name or !name
(i.e. without the terminating % or !)?


        Stefan

bug#18405: 24.3.93; bat-mode.el

[Achim Gratz]

Stefan Monnier writes:
> I know nothing of cmd.exe's syntax and the general thrust of the change
> sounds good, but: the above matches "%foo\nbar", which is probably not
> right.  When can variable refs be of the form %name or !name
> (i.e. without the terminating % or !)?

I haven't tried embedded newlines (or rather \r\n as it were) in
variable names.  Just like the embedded '%' character they probably are
possible somehow, but mostly useless.  So it would make sense to exclude
them (embedded whitespace is also somewhat iffy, but it tends to work
surprisingly well within CMD itself).  The syntax for variable names in
CMD does not seem to be specified formally, at least I haven't found
anything.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

bug#18405: 24.3.93; bat-mode.el

[Eli Zaretskii]

> Date: Thu, 04 Sep 2014 16:39:25 -0400
> Cc: 18405@debbugs.gnu.org
> 
> When can variable refs be of the form %name or !name (i.e. without
> the terminating % or !)?

In the FOR loops.  The simplest example is (words in caps are
keywords, but don't need to appear in caps):

   FOR %f in (*.c) DO something %f

bug#18405: 24.3.93; bat-mode.el

[Eli Zaretskii]

> Date: Fri, 05 Sep 2014 10:03:00 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: Stromeko@nexgo.de, 18405@debbugs.gnu.org
> 
> > When can variable refs be of the form %name or !name (i.e. without
> > the terminating % or !)?
> 
> In the FOR loops.

And, of course, in references to batch file arguments, like %1, %2,
etc.

bug#18405: 24.3.93; bat-mode.el

[Achim Gratz]

Stefan Monnier writes:
> When can variable refs be of the form %name or !name
> (i.e. without the terminating % or !)?

These are so-called parameter variables; more specifically loop
variables and arguments, these must be single alpha character or
numerical.  There are also local variables when you enable extended
syntax, but they localize the environment variables and thus keep their
syntax.

I think that's the best syntax description (and most concise) I've found
so far:

http://ss64.com/nt/syntax-variables.html
http://ss64.com/nt/syntax-args.html


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

bug#18405: 24.3.93; bat-mode.el

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 699 bytes --]

On 09/05/2014 11:39 AM, Achim Gratz wrote:
> Stefan Monnier writes:
>> When can variable refs be of the form %name or !name
>> (i.e. without the terminating % or !)?
> 
> These are so-called parameter variables; more specifically loop
> variables and arguments, these must be single alpha character or
> numerical.  There are also local variables when you enable extended
> syntax, but they localize the environment variables and thus keep their
> syntax.
> 
> I think that's the best syntax description (and most concise) I've found
> so far:
> 

Mind just importing http://www.emacswiki.org/emacs/ntcmd.el into trunk?
It handles a bunch of nasty cmd.exe syntactic corner cases.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#18405: 24.3.93; bat-mode.el

[Eli Zaretskii]

> Date: Sat, 06 Sep 2014 20:24:46 -0700
> From: Daniel Colascione <dancol@dancol.org>
> 
> Mind just importing http://www.emacswiki.org/emacs/ntcmd.el into trunk?

Would you mind some comments about it?

bug#18405: 24.3.93; bat-mode.el

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 798 bytes --]

On 09/07/2014 08:17 AM, Eli Zaretskii wrote:
>> Date: Sat, 06 Sep 2014 20:24:46 -0700
>> From: Daniel Colascione <dancol@dancol.org>
>>
>> Mind just importing http://www.emacswiki.org/emacs/ntcmd.el into trunk?
> 
> Would you mind some comments about it?

Sorry --- I should have been more specific. I wrote ntcmd a while ago
because I was frustrated with existing modes for working with batch
files. ntcmd uses a parser to deal with cmd.exe's very weird syntax;
it's important to do that because cmd's syntax is highly irregular and
using regular font-lock patterns to match it is doomed --- in
particular, matches involving loop variables are complicated. (See
ntcmd-forward-generic-cmd and ntcmd-forward-for-cmd for some interesting
special cases and fossilized cmd.exe bugs.)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#18405: 24.3.93; bat-mode.el

[Eli Zaretskii]

> Date: Sun, 07 Sep 2014 18:04:15 -0700
> From: Daniel Colascione <dancol@dancol.org>
> CC: Stromeko@nexgo.de, 18405@debbugs.gnu.org
> 
> On 09/07/2014 08:17 AM, Eli Zaretskii wrote:
> >> Date: Sat, 06 Sep 2014 20:24:46 -0700
> >> From: Daniel Colascione <dancol@dancol.org>
> >>
> >> Mind just importing http://www.emacswiki.org/emacs/ntcmd.el into trunk?
> > 
> > Would you mind some comments about it?
> 
> Sorry --- I should have been more specific. I wrote ntcmd a while ago
> because I was frustrated with existing modes for working with batch
> files. ntcmd uses a parser to deal with cmd.exe's very weird syntax;
> it's important to do that because cmd's syntax is highly irregular and
> using regular font-lock patterns to match it is doomed --- in
> particular, matches involving loop variables are complicated. (See
> ntcmd-forward-generic-cmd and ntcmd-forward-for-cmd for some interesting
> special cases and fossilized cmd.exe bugs.)

I've read the code, so yes, I've seen that.

My point was that I saw a few of its aspects that might benefit from
improvements, and if we are going to add that package to Emacs, I'd
like to see those happen.

bug#18405: 24.3.93; bat-mode.el

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1396 bytes --]



On 09/07/2014 07:42 PM, Eli Zaretskii wrote:
>> Date: Sun, 07 Sep 2014 18:04:15 -0700
>> From: Daniel Colascione <dancol@dancol.org>
>> CC: Stromeko@nexgo.de, 18405@debbugs.gnu.org
>>
>> On 09/07/2014 08:17 AM, Eli Zaretskii wrote:
>>>> Date: Sat, 06 Sep 2014 20:24:46 -0700
>>>> From: Daniel Colascione <dancol@dancol.org>
>>>>
>>>> Mind just importing http://www.emacswiki.org/emacs/ntcmd.el into trunk?
>>>
>>> Would you mind some comments about it?
>>
>> Sorry --- I should have been more specific. I wrote ntcmd a while ago
>> because I was frustrated with existing modes for working with batch
>> files. ntcmd uses a parser to deal with cmd.exe's very weird syntax;
>> it's important to do that because cmd's syntax is highly irregular and
>> using regular font-lock patterns to match it is doomed --- in
>> particular, matches involving loop variables are complicated. (See
>> ntcmd-forward-generic-cmd and ntcmd-forward-for-cmd for some interesting
>> special cases and fossilized cmd.exe bugs.)
> 
> I've read the code, so yes, I've seen that.
> 
> My point was that I saw a few of its aspects that might benefit from
> improvements, and if we are going to add that package to Emacs, I'd
> like to see those happen.

Like what? In 2011, you just mentioned that we should be using or
enhancing bat-generic-mode, which, as I explained, is insufficient.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#18405: 24.3.93; bat-mode.el

[Eli Zaretskii]

> Date: Sun, 07 Sep 2014 19:47:52 -0700
> From: Daniel Colascione <dancol@dancol.org>
> CC: Stromeko@nexgo.de, 18405@debbugs.gnu.org
> 
> > My point was that I saw a few of its aspects that might benefit from
> > improvements, and if we are going to add that package to Emacs, I'd
> > like to see those happen.
> 
> Like what?

See below.

> In 2011, you just mentioned that we should be using or enhancing
> bat-generic-mode

My goodness, I hope you don't expect me to forever hold my peace about
this just because I happened to write something 3 years ago.

Here are my comments:

 . There are some strange decisions about faces, for example:

   . the switch letters in Unix-style -x switches are highlighted in
     font-lock-type-face
   . redirection characters are highlighted in font-lock-constant-face
   . opening parenthesis in "echo" commands are given
     font-lock-warning-face
   . variable definitions in for loops (the 'x' part in "for %%x") are
     highlighted with different face from their references in the "do"
     part of the loop

 . Text after "echo" (up to redirection character, if any) is always a
   simple string, except if it's "on" or "off", but is not highlighted
   as a string, which then causes some random highlighting of the
   parts of the string that just happen to match other possible
   syntaxes

 . Something like this confuses fontification of the redirection
   character, probably because of the parentheses (the opening paren
   is given the font-lock-warning-face):

      echo int main()           >junk.c

 . In general, I don't understand what was intended to be the handling
   of parentheses, they seem to be either not highlighted at all or
   given the warning face.

 . A command of this form:

     if exist somefile dosomethingwith somefile

   has "dosomethingwith" sometimes in the font-lock-builtin-face and
   sometimes with no face at all, not sure why.  Looks like when
   'somefile' includes slashes, the command is not fontified.

 . The value to which a variable is set in "set foo-bar" doesn't have
   any face, which causes some funny fontifications if that value
   happens to match unrelated syntaxes, as in "set foo=-bar=baz"

 . The equals sign in "if foo == bar" is sometimes given the
   font-lock-keyword-face and sometimes no face at all.  Here's one
   example of the latter:

    if not (%svgsupport%) == (Y) goto :svgDone

 . This line has its fontifications after "(%distfiles%)" screwed:

    for %%d in (%distfiles%) do if not exist %%d set fileNotFound=%%d

   In particular, built-in commands are not fontified as such.

 . Some built-in commands are missing from ntcmd-shell-builtins:

     assoc, call, mklink

 . "dir" and "rem" appear twice in the list in ntcmd-shell-builtins

 . The support of "set /a" command is incomplete, e.g. try

     set /a y+=140

Thanks.

bug#18405: 24.3.93; bat-mode.el

[Stefan Monnier]

>    . variable definitions in for loops (the 'x' part in "for %%x") are
>      highlighted with different face from their references in the "do"
>      part of the loop

FWIW in sh-mode, for something like "for f in echo $f", we also
highlight the two occurrences of "f" differently.  I'm not saying it's
a feature, but it's definitely a common behavior to highlight the
"introduction" occurrence differently from the "reference" occurrences.


        Stefan

bug#18405: 24.3.93; bat-mode.el

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Daniel Colascione <dancol@dancol.org>, Stromeko@nexgo.de,
>         18405@debbugs.gnu.org
> Date: Tue, 09 Sep 2014 09:45:50 -0400
> 
> >    . variable definitions in for loops (the 'x' part in "for %%x") are
> >      highlighted with different face from their references in the "do"
> >      part of the loop
> 
> FWIW in sh-mode, for something like "for f in echo $f", we also
> highlight the two occurrences of "f" differently.  I'm not saying it's
> a feature, but it's definitely a common behavior to highlight the
> "introduction" occurrence differently from the "reference" occurrences.

Except that in the ntcmd case, that's not the intended behavior,
AFAICT.

bug#18405: 24.3.93; bat-mode.el

[Achim Gratz]

Daniel Colascione writes:
> Mind just importing http://www.emacswiki.org/emacs/ntcmd.el into trunk?
> It handles a bunch of nasty cmd.exe syntactic corner cases.

I've briefly looked at it and this seems like a better approach in the
long run.  However, I'd still like to have that particular bug in
bat-mode fixed in the next release, while ntcmd.el looks like something
that would have to be done in trunk.  So I suggest that this should be
separate bug report or split it off from the original one if possible.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

bug#28311: 25.2; Improve syntax highlighting in bat-mode

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 5170 bytes --]

Hi!

I've noticed some problems with syntax highlighting in bat-mode. In
addition, I have two improvement suggestions.

Problem 1:

bat-mode highlights variables surrounded by "%":s, as in "%alpha%".
However, when the variable contains an underscore, as in "%alpha_beta%",
only "alpha" is highlighted.

Problem 2:

Similarly, bat-mode highlights variables defined using "set". As with
problem 1 above, it does not handle words with underscore. Concretely, in
"set alpha_beta=value", only "alpha" is highlighted.

Problem 3:

bat-mode highlights keywords like "in". Unfortunately, it also highlights
the word "in" when used as a file extension. For example "xxx alpha.in".

Improvement suggestion 1:

In bat scripts, the command "echo" can be used to print strings to the
console. Unlike in shell scripts, the argument to "echo" does not require
quotes. It would be nice if Emacs would highlight anything after "echo" as
a string (using font-lock-string-face).

Improvement suggestion 2:

In bat scripts, variable substitution is performed inside quoted strings,
for example:

    xxx "hello %alpha goodbye"

Unfortunately, emacs does not highlight this, which I hereby propose that
it should.


I had plans to fix all of the above myself. Unfortunately, my Emacs
todo-list is too long for my available time, so Richard S. suggested that I
report this here, in case someone else would have more time than I do.

Anyway, problem 1 and 2 can probably be fixed by adapting a regexp. Problem
3 should easily be fixed by using a custom font-lock syntax table, where
"." is seen as a word character, or possibly a symbol character.
Improvement suggestion 1 could probably be handled by a simple rule
matching echo + space + whatever and highlight "whatever" using
font-lock-string-face. All of these are easy to fix.

Improvement suggestion 2 raises the bar to "medium", as it would require
replacing the current rexexp rule with one written in elisp. The rule will
need to use the "prepend" (not "t") override flag to add
font-lock-variable-name-face to the variable name in top of the string
face. However, the code must ensure that it does not do this for comments.

Side note: Typically, the "prepend" override flag is preferred over "t", as
the end result will mix face properties from both the new and old faces.
Concretely, I use a string face with a red foreground and a grey
background. With the "prepend" flag, the variable name will be drawn using
yellow from font-lock-variable-name-face but the background will still be
grey, matching the rest of the string.

    -- Anders Lindgren



In GNU Emacs 25.2.1 (x86_64-apple-darwin13.4.0, NS appkit-1265.21 Version
10.9.5 (Build 13F1911))
 of 2017-04-21 built on builder10-9.porkrind.org
Windowing system distributor 'Apple', version 10.3.1504
Configured using:
 'configure --with-ns '--enable-locallisppath=/Library/Application
 Support/Emacs/${version}/site-lisp:/Library/Application
 Support/Emacs/site-lisp' --with-modules'

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES

Important settings:
  value of $LC_CTYPE: UTF-8
  value of $LANG: en_SE.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Bat

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
next-line: End of buffer

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired format-spec rfc822 mml
mml-sec password-cache epg epg-config gnus-util mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util help-fns help-mode cl-loaddefs pcase
cl-lib mail-prsvr mail-utils vc-dispatcher vc-svn bat-mode easymenu
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel ns-win ucs-normalize term/common-win tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame
cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese charscript
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote kqueue cocoa ns
multi-tty make-network-process emacs)

Memory information:
((conses 16 196862 6811)
 (symbols 48 19646 0)
 (miscs 40 53 163)
 (strings 32 15415 5200)
 (string-bytes 1 442636)
 (vectors 16 33051)
 (vector-slots 8 651956 5657)
 (floats 8 162 94)
 (intervals 56 216 0)
 (buffers 976 19))

[-- Attachment #2: Type: text/html, Size: 6842 bytes --]

bug#28311: 25.2; Improve syntax highlighting in bat-mode

[Achim Gratz]

Anders Lindgren writes:
> I've noticed some problems with syntax highlighting in bat-mode.

Consider joining this report with bug#18405.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

bug#28311: 25.2; Improve syntax highlighting in bat-mode

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Thu, 31 Aug 2017 21:27:43 +0200
> 
> Anders Lindgren writes:
> > I've noticed some problems with syntax highlighting in bat-mode.
> 
> Consider joining this report with bug#18405.

I've picked the lower-hanging fruit and fixed some of the problems
reported in these two bug reports on the emacs-26 branch.

Thanks.

bug#18405: bug#28311: 25.2; Improve syntax highlighting in bat-mode

[Glenn Morris]

Running 5 tests (2017-09-22 09:57:54-0700)
   passed  1/5  bat-test-fill-paragraph-comment
hfy batch mode ( *temp*-973592:bat-mode)
Test bat-test-fontification-iter-var-1 backtrace:
  signal(ert-test-failed (((should (equal (bat-test-fontify "echo %%a\
  ert-fail(((should (equal (bat-test-fontify "echo %%a\necho %%~dp1\ne
  (if (unwind-protect (setq value-17 (apply fn-15 args-16)) (setq form
  (let (form-description-19) (if (unwind-protect (setq value-17 (apply
  (let ((value-17 'ert-form-evaluation-aborted-18)) (let (form-descrip
  (let* ((fn-15 (function equal)) (args-16 (condition-case err (let ((
  (closure (t) nil (let* ((fn-15 (function equal)) (args-16 (condition
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name bat-test-fontification-iter-var-1 :do
  ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test 
  ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
  ert-run-tests-batch(nil)
  ert-run-tests-batch-and-exit(nil)
  eval((ert-run-tests-batch-and-exit nil))
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/progmodes/bat-mode-t
  command-line()
  normal-top-level()
Test bat-test-fontification-iter-var-1 condition:
    (ert-test-failed
     ((should
       (equal
	(bat-test-fontify "echo %%a
echo %%~dp1
echo %%~$PATH:I")
	"<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">a</span>
<span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</span>
<span class=\"builtin\">echo</span> %%~$<span class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</span>"))
      :form
      (equal
       #("<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">a</span>
<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~dp1</span>
<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~$PATH:I</span>" 22 26
(face font-lock-builtin-face)
64 65
(face font-lock-variable-name-face)
95 99
(face font-lock-builtin-face)
137 141
(face font-lock-variable-name-face)
171 175
(face font-lock-builtin-face)
213 221
(face font-lock-variable-name-face))
       "<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">a</span>
<span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</span>
<span class=\"builtin\">echo</span> %%~$<span class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</span>")
      :value nil :explanation
      (arrays-of-different-length 228 263
				  #("<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">a</span>
<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~dp1</span>
<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~$PATH:I</span>" 22 26
(face font-lock-builtin-face)
64 65
(face font-lock-variable-name-face)
95 99
(face font-lock-builtin-face)
137 141
(face font-lock-variable-name-face)
171 175
(face font-lock-builtin-face)
213 221
(face font-lock-variable-name-face))
				  "<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">a</span>
<span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</span>
<span class=\"builtin\">echo</span> %%~$<span class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</span>" first-mismatch-at 109)))
   FAILED  2/5  bat-test-fontification-iter-var-1
hfy batch mode ( *temp*-454955:bat-mode)
   passed  3/5  bat-test-fontification-var-decl
hfy batch mode ( *temp*-493544:bat-mode)
   passed  4/5  bat-test-fontification-var-delayed-exp
hfy batch mode ( *temp*-832943:bat-mode)
   passed  5/5  bat-test-fontification-var-exp

Ran 5 tests, 4 results as expected, 1 unexpected (2017-09-22 09:57:54-0700)

1 unexpected results:
   FAILED  bat-test-fontification-iter-var-1

bug#18405: bug#28311: 25.2; Improve syntax highlighting in bat-mode

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 5708 bytes --]

Hi!

Unfortunately, I can't comment on the actual error found, as I wasn't
involved in the fix...

Anyway, I am positively glad that you have tests for font-lock rules.
However, using htmlfontify for this doesn't seem like the best tool for
this. 1) The output of htmlfontify may change in the future, in which case
you will need to rewrite all your tests, 2) HTML is hard to read, and 3)
The ert explainer doesn't really help you here. If example, in this case
all it tells you is that the resulting html strings have different lengths.
On other words, it may work for snippets like this, but it would not scale
well.

A couple of years ago I wrote a tool, faceup (https://github.com/
Lindydancer/faceup), specifically for testing highlighting in Emacs using
ert. It uses a simple markup language, so the result is easier to read than
when using HTML. The reference text can be used inline (like in your test)
or stored in a corresponding ".faceup" file, which makes it possible to
test files thousands of lines long. When a test fails, the ert explainer
will tell you which lines contained errors, what that lines contained, and
what they should contain, so it's much easier to find errors. I have
successfully used this is several of my font-lock packages. In addition,
I've accumulated a number of source files in various programming languages
in a regression test suite for font-lock (https://github.com/
Lindydancer/font-lock-regression-suite).

If you would like to modernise your font-lock test, I would be happy to
contribute it to Emacs (as I suggested in bug#16063).

    -- Anders

On Fri, Sep 22, 2017 at 7:01 PM, Glenn Morris <rgm@gnu.org> wrote:

>
> Running 5 tests (2017-09-22 09:57:54-0700)
>    passed  1/5  bat-test-fill-paragraph-comment
> hfy batch mode ( *temp*-973592:bat-mode)
> Test bat-test-fontification-iter-var-1 backtrace:
>   signal(ert-test-failed (((should (equal (bat-test-fontify "echo %%a\
>   ert-fail(((should (equal (bat-test-fontify "echo %%a\necho %%~dp1\ne
>   (if (unwind-protect (setq value-17 (apply fn-15 args-16)) (setq form
>   (let (form-description-19) (if (unwind-protect (setq value-17 (apply
>   (let ((value-17 'ert-form-evaluation-aborted-18)) (let (form-descrip
>   (let* ((fn-15 (function equal)) (args-16 (condition-case err (let ((
>   (closure (t) nil (let* ((fn-15 (function equal)) (args-16 (condition
>   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
>   ert-run-test(#s(ert-test :name bat-test-fontification-iter-var-1 :do
>   ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test
>   ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
>   ert-run-tests-batch(nil)
>   ert-run-tests-batch-and-exit(nil)
>   eval((ert-run-tests-batch-and-exit nil))
>   command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/progmodes/bat-mode-t
>   command-line()
>   normal-top-level()
> Test bat-test-fontification-iter-var-1 condition:
>     (ert-test-failed
>      ((should
>        (equal
>         (bat-test-fontify "echo %%a
> echo %%~dp1
> echo %%~$PATH:I")
>         "<span class=\"builtin\">echo</span> %%<span
> class=\"variable-name\">a</span>
> <span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</
> span>
> <span class=\"builtin\">echo</span> %%~$<span
> class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</
> span>"))
>       :form
>       (equal
>        #("<span class=\"builtin\">echo</span> %%<span
> class=\"variable-name\">a</span>
> <span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~dp1</
> span>
> <span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~$PATH:I</span>"
> 22 26
> (face font-lock-builtin-face)
> 64 65
> (face font-lock-variable-name-face)
> 95 99
> (face font-lock-builtin-face)
> 137 141
> (face font-lock-variable-name-face)
> 171 175
> (face font-lock-builtin-face)
> 213 221
> (face font-lock-variable-name-face))
>        "<span class=\"builtin\">echo</span> %%<span
> class=\"variable-name\">a</span>
> <span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</
> span>
> <span class=\"builtin\">echo</span> %%~$<span
> class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</
> span>")
>       :value nil :explanation
>       (arrays-of-different-length 228 263
>                                   #("<span class=\"builtin\">echo</span>
> %%<span class=\"variable-name\">a</span>
> <span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~dp1</
> span>
> <span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~$PATH:I</span>"
> 22 26
> (face font-lock-builtin-face)
> 64 65
> (face font-lock-variable-name-face)
> 95 99
> (face font-lock-builtin-face)
> 137 141
> (face font-lock-variable-name-face)
> 171 175
> (face font-lock-builtin-face)
> 213 221
> (face font-lock-variable-name-face))
>                                   "<span class=\"builtin\">echo</span>
> %%<span class=\"variable-name\">a</span>
> <span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</
> span>
> <span class=\"builtin\">echo</span> %%~$<span
> class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</span>"
> first-mismatch-at 109)))
>    FAILED  2/5  bat-test-fontification-iter-var-1
> hfy batch mode ( *temp*-454955:bat-mode)
>    passed  3/5  bat-test-fontification-var-decl
> hfy batch mode ( *temp*-493544:bat-mode)
>    passed  4/5  bat-test-fontification-var-delayed-exp
> hfy batch mode ( *temp*-832943:bat-mode)
>    passed  5/5  bat-test-fontification-var-exp
>
> Ran 5 tests, 4 results as expected, 1 unexpected (2017-09-22 09:57:54-0700)
>
> 1 unexpected results:
>    FAILED  bat-test-fontification-iter-var-1
>
>

[-- Attachment #2: Type: text/html, Size: 7652 bytes --]

bug#28311: 25.2; Improve syntax highlighting in bat-mode

[Eli Zaretskii]

> From: Anders Lindgren <andlind@gmail.com>
> Date: Fri, 22 Sep 2017 21:04:19 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>, Achim Gratz <Stromeko@nexgo.de>, 28311@debbugs.gnu.org, 
> 	18405@debbugs.gnu.org
> 
> If you would like to modernise your font-lock test, I would be happy to contribute it to Emacs (as I suggested in
> bug#16063).

Why wouldn't we want that?  Please go ahead, and thanks.

bug#28311: 25.2; Improve syntax highlighting in bat-mode

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 352 bytes --]

>
> > If you would like to modernise your font-lock test, I would be happy to
> contribute it to Emacs (as I suggested in
> > bug#16063).
>
> Why wouldn't we want that?  Please go ahead, and thanks.
>

Thanks! In which branch should I add it, master or Emacs-26?

I guess it should go into lisp/emacs-lisp, as "ert.el" is located there.

    -- Anders

[-- Attachment #2: Type: text/html, Size: 800 bytes --]

bug#28311: 25.2; Improve syntax highlighting in bat-mode

[Eli Zaretskii]

> From: Anders Lindgren <andlind@gmail.com>
> Date: Sat, 23 Sep 2017 14:50:27 +0200
> Cc: Glenn Morris <rgm@gnu.org>, Stromeko@nexgo.de, 28311@debbugs.gnu.org, 
> 	18405@debbugs.gnu.org
> 
>  Why wouldn't we want that? Please go ahead, and thanks.
> 
> Thanks! In which branch should I add it, master or Emacs-26?

How large is it?  Also, is it supposed to be used outside of the ert
context?

> I guess it should go into lisp/emacs-lisp, as "ert.el" is located there.

Probably.  Actually, why not make it part of ert, say, ert-faces.el?

bug#18405: bug#28311: 25.2; Improve syntax highlighting in bat-mode

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

>
> > Thanks! In which branch should I add it, master or Emacs-26?
>
> How large is it?  Also, is it supposed to be used outside of the ert
> context?
>

The main source file is about 45 KB, with additional 15KB of test files.

It can be used outside the ERT context, as the markup language is generic.
I can think of many applications for it, like use it as the storage format
for a word-processor like mode, where the user can apply bold, italics etc.


> I guess it should go into lisp/emacs-lisp, as "ert.el" is located there.
>
> Probably.  Actually, why not make it part of ert, say, ert-faces.el?
>

As it can be used outside of ert, I would prefer the current name.

     -- Anders

[-- Attachment #2: Type: text/html, Size: 1260 bytes --]

bug#18405: bug#28311: 25.2; Improve syntax highlighting in bat-mode

[Eli Zaretskii]

> From: Anders Lindgren <andlind@gmail.com>
> Date: Sat, 23 Sep 2017 20:05:35 +0200
> Cc: Glenn Morris <rgm@gnu.org>, Achim Gratz <Stromeko@nexgo.de>, 28311@debbugs.gnu.org, 
> 	18405@debbugs.gnu.org
> 
>  > Thanks! In which branch should I add it, master or Emacs-26?
> 
>  How large is it? Also, is it supposed to be used outside of the ert
>  context?
> 
> The main source file is about 45 KB, with additional 15KB of test files.
> 
> It can be used outside the ERT context, as the markup language is generic. I can think of many applications
> for it, like use it as the storage format for a word-processor like mode, where the user can apply bold, italics
> etc.

In that case, I guess master is the right place.

Thanks.

bug#18405: 24.3.93; bat-mode.el

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> The main source file is about 45 KB, with additional 15KB of test files.
>> 
>> It can be used outside the ERT context, as the markup language is generic. I can think of many applications
>> for it, like use it as the storage format for a word-processor like mode, where the user can apply bold, italics
>> etc.
>
> In that case, I guess master is the right place.

This was added to Emacs a year later, as a0e5a02125a6.

A quick skim of this bug report seems to indicate that most of the
issues described here were fixed, so I'm closing this bug report.  If
there's anything more that should be done in bat-mode, then filing new
bug reports for any remaining issues would be the way forward.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no