bug#35129: Unsupported escape sequences for progress in shell

bug#35129: Unsupported escape sequences for progress in shell

[Juri Linkov]

Some escape sequences are unhandled in shell mode.
Most often they appear when running `apt install'
that shows its progress bar that is supposed to stay
on the same line, but spreads over several lines,
so the output looks like a mess (where ^[7 and ^[8
are escape sequences currently still unhandled):

  debconf: unable to initialize frontend: Dialog
  debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
  debconf: falling back to frontend: Readline
  ^[7Progress: [  0%] [................................................................................................] ^[8
  ^[7Progress: [  6%] [######..........................................................................................] ^[8
  ^[7Progress: [ 13%] [############....................................................................................] ^[8
  ^[7Progress: [ 19%] [##################..............................................................................] ^[8
  ^[7Progress: [ 25%] [########################........................................................................] ^[8
  ^[7Progress: [ 31%] [##############################..................................................................] ^[8
  ^[7Progress: [ 38%] [####################################............................................................] ^[8
  ^[7Progress: [ 44%] [##########################################......................................................] ^[8
  ^[7Progress: [ 50%] [################################################................................................] ^[8
  ^[7Progress: [ 56%] [######################################################..........................................] ^[8
  ^[7Progress: [ 63%] [############################################################....................................] ^[8
  ^[7Progress: [ 69%] [##################################################################..............................] ^[8
  ^[7Progress: [ 75%] [########################################################################........................] ^[8
  ^[7Progress: [ 81%] [##############################################################################..................] ^[8
  ^[7Progress: [ 88%] [####################################################################################............] ^[8
  ^[7Progress: [ 94%] [##########################################################################################......] ^[8

I'm making this report because escape sequences for colors are handled fine,
and visually these indicators have the green background.

bug#35129: Unsupported escape sequences for progress in shell

[Juri Linkov]

> Some escape sequences are unhandled in shell mode.
> Most often they appear when running `apt install'
> that shows its progress bar that is supposed to stay
> on the same line, but spreads over several lines,
> so the output looks like a mess (where ^[7 and ^[8
> are escape sequences currently still unhandled):

After more research, I found an interesting post
https://oremacs.com/2019/03/24/shell-apt/
that proposes a promising way to handle
these escape sequences.

So maybe this should be customizable?

bug#35129: Unsupported escape sequences for progress in shell

[Michael Albinus]

Juri Linkov <juri@linkov.net> writes:

Hi Juri,

>> Some escape sequences are unhandled in shell mode.
>> Most often they appear when running `apt install'
>> that shows its progress bar that is supposed to stay
>> on the same line, but spreads over several lines,
>> so the output looks like a mess (where ^[7 and ^[8
>> are escape sequences currently still unhandled):
>
> After more research, I found an interesting post
> https://oremacs.com/2019/03/24/shell-apt/
> that proposes a promising way to handle
> these escape sequences.
>
> So maybe this should be customizable?

Looks interesting. However, we have `progress-reporter'. The code from
above shall be made available via this.

This would also improve Tramp; there are requests to integrate the
progress reporter from external commands.

Best regards, Michael.

bug#35129: Unsupported escape sequences for progress in shell

[Juri Linkov]

>>> Some escape sequences are unhandled in shell mode.
>>> Most often they appear when running `apt install'
>>> that shows its progress bar that is supposed to stay
>>> on the same line, but spreads over several lines,
>>> so the output looks like a mess (where ^[7 and ^[8
>>> are escape sequences currently still unhandled):
>>
>> After more research, I found an interesting post
>> https://oremacs.com/2019/03/24/shell-apt/
>> that proposes a promising way to handle
>> these escape sequences.
>>
>> So maybe this should be customizable?
>
> Looks interesting. However, we have `progress-reporter'. The code from
> above shall be made available via this.

I'm not sure if we can provide min-value/max-value for
make-progress-reporter.  If not, then it uses spinning/pulsing.
It seems progress-reporter doesn't support arbitrary strings
such as that come from underlying process like ^[7Progress: [ 0%]^[8

> This would also improve Tramp; there are requests to integrate the
> progress reporter from external commands.

Could you show an example of a progress reporter from an external command.
Do they use arbitrary strings?

bug#35129: Unsupported escape sequences for progress in shell

[Michael Albinus]

Juri Linkov <juri@linkov.net> writes:

Hi Juri,

>> Looks interesting. However, we have `progress-reporter'. The code from
>> above shall be made available via this.
>
> I'm not sure if we can provide min-value/max-value for
> make-progress-reporter.  If not, then it uses spinning/pulsing.
> It seems progress-reporter doesn't support arbitrary strings
> such as that come from underlying process like ^[7Progress: [ 0%]^[8

progress-reporter should be improved to accept also strings like "45%"
to be displayed literally.

>> This would also improve Tramp; there are requests to integrate the
>> progress reporter from external commands.
>
> Could you show an example of a progress reporter from an external command.
> Do they use arbitrary strings?

Some scp or pscp versions do it. For example:

--8<---------------cut here---------------start------------->8---
$ pscp -v localhost:/net/ford/Multimedia/Eigene\ Videos/Videos/20150829_145255.mp4 /tmp/foo
[...]
foo                       | 203369 kB | 5810.6 kB/s | ETA: 00:00:00 | 84%
--8<---------------cut here---------------end--------------->8---

Best regards, Michael.

bug#35129: Unsupported escape sequences for progress in shell

[Juri Linkov]

>>> Looks interesting. However, we have `progress-reporter'. The code from
>>> above shall be made available via this.
>>
>> I'm not sure if we can provide min-value/max-value for
>> make-progress-reporter.  If not, then it uses spinning/pulsing.
>> It seems progress-reporter doesn't support arbitrary strings
>> such as that come from underlying process like ^[7Progress: [ 0%]^[8
>
> progress-reporter should be improved to accept also strings like "45%"
> to be displayed literally.

Another problem is that there is no clear moment to start/stop
progress-reporter with calls to make-progress-reporter and
progress-reporter-done.

>>> This would also improve Tramp; there are requests to integrate the
>>> progress reporter from external commands.
>>
>> Could you show an example of a progress reporter from an external command.
>> Do they use arbitrary strings?
>
> Some scp or pscp versions do it. For example:
>
> $ pscp -v localhost:/net/ford/Multimedia/Eigene\ Videos/Videos/20150829_145255.mp4 /tmp/foo
> [...]
> foo                       | 203369 kB | 5810.6 kB/s | ETA: 00:00:00 | 84%

Progress on copying large files would be needed for local file operations
as well.

bug#35129: Unsupported escape sequences for progress in shell

[Michael Albinus]

Juri Linkov <juri@linkov.net> writes:

Hi Juri,

>>> I'm not sure if we can provide min-value/max-value for
>>> make-progress-reporter.  If not, then it uses spinning/pulsing.
>>> It seems progress-reporter doesn't support arbitrary strings
>>> such as that come from underlying process like ^[7Progress: [ 0%]^[8
>>
>> progress-reporter should be improved to accept also strings like "45%"
>> to be displayed literally.
>
> Another problem is that there is no clear moment to start/stop
> progress-reporter with calls to make-progress-reporter and
> progress-reporter-done.

At least for my use case, calling external p?scp commands in Tramp,
there is no problem. But in general you might be right.

>>> Could you show an example of a progress reporter from an external command.
>>> Do they use arbitrary strings?
>>
>> Some scp or pscp versions do it. For example:
>>
>> $ pscp -v localhost:/net/ford/Multimedia/Eigene\ Videos/Videos/20150829_145255.mp4 /tmp/foo
>> [...]
>> foo                       | 203369 kB | 5810.6 kB/s | ETA: 00:00:00 | 84%
>
> Progress on copying large files would be needed for local file operations
> as well.

Yes. But local copying is not implemented via external commands; this
needs another progress reporter call.

Best regards, Michael.