bug#7485: 23.2; Fix removing unrecognized ANSI sequences in ansi-color-apply

[Leo <sdl.web@gmail.com>]

On 2010-11-26 18:55 +0000, Stefan Monnier wrote:
>> In ansi-color-apply, (string-match "\033" string start) finds the wrong
>> portion of context if unrecognized ANSI sequences is not removed before
>> the match.
>
> You mean, because \033 can appear in that unrecognized ANSI sequence?

Yes.

>> This can cause, for example, eshell's prompt to disappear if
>> ansi-color-apply is used in eshell-preoutput-filter-functions.
>> The attached patch tries to fix this.
>
> I don't quite understand your patch.  And your saying "tries to fix"
> doesn't make me more confident.

which means I only briefly tested the patch and it seems not to make
things worse while fixing a bug that annoys me ;)

>> diff --git a/lisp/ansi-color.el b/lisp/ansi-color.el
>> index 00162c9..40c0066 100644
>> *** a/lisp/ansi-color.el
>> --- b/lisp/ansi-color.el
>> ***************
>> *** 341,352 ****
>>         (put-text-property start (length string) 'ansi-color t string)
>>         (put-text-property start (length string) 'face face string))
>>       ;; save context, add the remainder of the string to the result
>> !     (let (fragment)
>> !       (if (string-match "\033" string start)
>>   	  (let ((pos (match-beginning 0)))
>> ! 	    (setq fragment (substring string pos))
>> ! 	    (push (substring string start pos) result))
>> ! 	(push (substring string start) result))
>>         (if (or face fragment)
>>   	  (setq ansi-color-context (list face fragment))
>>   	(setq ansi-color-context nil)))
>> --- 341,355 ----
>>         (put-text-property start (length string) 'ansi-color t string)
>>         (put-text-property start (length string) 'face face string))
>>       ;; save context, add the remainder of the string to the result
>> !     (let ((remaining (substring string start))
>> ! 	  fragment)
>> !       (while (string-match ansi-color-drop-regexp remaining)
>> ! 	(setq remaining (replace-match "" nil nil remaining)))
>> !       (if (string-match "\033" remaining)
>>   	  (let ((pos (match-beginning 0)))
>> ! 	    (setq fragment (substring remaining pos))
>> ! 	    (push (substring remaining 0 pos) result))
>> ! 	(push remaining result))
>>         (if (or face fragment)
>>   	  (setq ansi-color-context (list face fragment))
>>   	(setq ansi-color-context nil)))
>
> This appears to "drop control sequences" even tho they weren't dropped
> before, so it does more than "ignore false-positive \033 from
> unrecognized control sequences".  IIUC those unrecognized control
> sequences are dropped elsewhere, right?  So are maybe suggesting that
> they are currently not dropped at the right time (i.e. dropped too
> late), or that the \033 handling takes place too early, or that we
> currently forget to drop those control sequences or ... ?
>
>
>         Stefan

Unrecognized control sequences also begin with char \033. The FRAGMENT
part of ansi-color-context wasn't supposed to contain actual text,
otherwise they might get lost. It should always be a fragment of a whole
ansi control sequence if non-nil.

Let me show how this stops eshell prompt from showing.

Assume ansi-color-apply is in eshell-preoutput-filter-functions. Suppose
the output of shell command 'ack linux' ends with an unrecognized
control sequence ^[[K. The whole output string that goes through
ansi-color-apply is

   output of 'ack linux' + 'eshell prompt'.

After 'ack linux' finishes, the FRAGMENT of ansi-color-context becomes:

#("^[[K\n~/.emacs.d $ " 4 17
   (read-only t face eshell-prompt rear-nonsticky
              (face read-only)))

The string after being processed by ansi-color-apply is thus without an
eshell prompt.

Leo