Display corruption with binary files

Display corruption with binary files

[August Karlstrom]

Hi,

When I open any binary file with Emacs in console mode (emacs -nw) the 
display gets corrupted (the menu and parts of the mode line disappears). 
In GUI mode it works as expected. I use GNU Emacs version 21.4.1 and 
gnome-terminal 2.16.1 under Ubuntu 6.10.

Any clues?


Regards,

August

Re: Display corruption with binary files

[Robert Thorpe]

August Karlstrom wrote:
> Hi,
>
> When I open any binary file with Emacs in console mode (emacs -nw) the
> display gets corrupted (the menu and parts of the mode line disappears).
> In GUI mode it works as expected. I use GNU Emacs version 21.4.1 and
> gnome-terminal 2.16.1 under Ubuntu 6.10.
>
> Any clues?

Bits of the binary file being interpreted at terminal control
characters is a likely culprit.
If that is the problem then Emacs should not send such characters to
the terminal.  It may be a terminal problem though.  I'd report it as
an Emacs bug with that caveat.

Re: Display corruption with binary files

[August Karlstrom]

Robert Thorpe skrev:
> August Karlstrom wrote:
>> Hi,
>>
>> When I open any binary file with Emacs in console mode (emacs -nw) the
>> display gets corrupted (the menu and parts of the mode line disappears).
>> In GUI mode it works as expected. I use GNU Emacs version 21.4.1 and
>> gnome-terminal 2.16.1 under Ubuntu 6.10.
>>
>> Any clues?
> 
> Bits of the binary file being interpreted at terminal control
> characters is a likely culprit.

OK, but isn't a binary file as well as a text file really just a 
sequence of arbitrary bytes?

> If that is the problem then Emacs should not send such characters to
> the terminal.  It may be a terminal problem though.  I'd report it as
> an Emacs bug with that caveat.

Eli Zaretskii's answer seems to indicate that this is a known problem.


Thanks,

August

Re: Display corruption with binary files

[Eli Zaretskii]

> From: August Karlstrom <fusionfive@comhem.se>
> Date: Sat, 25 Nov 2006 10:30:16 GMT
> > 
> > Bits of the binary file being interpreted at terminal control
> > characters is a likely culprit.
> 
> OK, but isn't a binary file as well as a text file really just a 
> sequence of arbitrary bytes?
> 
> > If that is the problem then Emacs should not send such characters to
> > the terminal.  It may be a terminal problem though.  I'd report it as
> > an Emacs bug with that caveat.
> 
> Eli Zaretskii's answer seems to indicate that this is a known problem.

Yes, it's a known problem that the Unix terminal interprets certain
sequences of characters as commands.  I'm not sure Emacs can do
anything to solve this, but suggestions are welcome.

Re: Display corruption with binary files

[Perry Smith]

[-- Attachment #1.1: Type: text/plain, Size: 1503 bytes --]


On Nov 25, 2006, at 5:04 AM, Eli Zaretskii wrote:

>> From: August Karlstrom <fusionfive@comhem.se>
>> Date: Sat, 25 Nov 2006 10:30:16 GMT
>>>
>>> Bits of the binary file being interpreted at terminal control
>>> characters is a likely culprit.
>>
>> OK, but isn't a binary file as well as a text file really just a
>> sequence of arbitrary bytes?
>>
>>> If that is the problem then Emacs should not send such characters to
>>> the terminal.  It may be a terminal problem though.  I'd report  
>>> it as
>>> an Emacs bug with that caveat.
>>
>> Eli Zaretskii's answer seems to indicate that this is a known  
>> problem.
>
> Yes, it's a known problem that the Unix terminal interprets certain
> sequences of characters as commands.  I'm not sure Emacs can do
> anything to solve this, but suggestions are welcome.

I have not been closely watching this but I'm confused.

emacs of old (I thought) would 'quote' the characters.  So, if the  
file had a control-C, emacs would display ^C and a single forward- 
character while the cursor is sitting on the ^ would move two screeen  
spaces up to the next character.  So, while emacs would send escape  
sequences itself to control the terminal, the data of the file would  
be sent properly.  (I'm not sure I'm saying clearly what I mean.)

Is this not done?  Didn't emacs use to do it?

Perry Smith ( pedz@easesoftware.com )
Ease Software, Inc. ( http://www.easesoftware.com )

Low cost SATA Disk Systems for IBMs p5, pSeries, and RS/6000 AIX systems



[-- Attachment #1.2: Type: text/html, Size: 7417 bytes --]

[-- Attachment #2: Type: text/plain, Size: 152 bytes --]

_______________________________________________
help-gnu-emacs mailing list
help-gnu-emacs@gnu.org
http://lists.gnu.org/mailman/listinfo/help-gnu-emacs

Re: Display corruption with binary files

[Eli Zaretskii]

> Cc: help-gnu-emacs@gnu.org
> From: Perry Smith <pedz@easesoftware.com>
> Date: Sat, 25 Nov 2006 10:38:04 -0600
> 
> emacs of old (I thought) would 'quote' the characters.  So, if the  
> file had a control-C, emacs would display ^C and a single forward- 
> character while the cursor is sitting on the ^ would move two screeen  
> spaces up to the next character.

I don't think Emacs ever did that.

> Is this not done?  Didn't emacs use to do it?

Only if you use hexl, which is why I recommended it to the OP.

Re: Display corruption with binary files

[Perry Smith]

[-- Attachment #1.1: Type: text/plain, Size: 1548 bytes --]

On Nov 25, 2006, at 3:02 PM, Eli Zaretskii wrote:

>> Cc: help-gnu-emacs@gnu.org
>> From: Perry Smith <pedz@easesoftware.com>
>> Date: Sat, 25 Nov 2006 10:38:04 -0600
>>
>> emacs of old (I thought) would 'quote' the characters.  So, if the
>> file had a control-C, emacs would display ^C and a single forward-
>> character while the cursor is sitting on the ^ would move two screeen
>> spaces up to the next character.
>
> I don't think Emacs ever did that.
>
>> Is this not done?  Didn't emacs use to do it?
>
> Only if you use hexl, which is why I recommended it to the OP.

We must be talking about two different things or something.

On my Mac system, there is a version of emacs 21.2.1 in /usr/bin/ 
emacs.  I have a bzip tar file.  I type:

emacs foo.tar.bz2

from a Mac "terminal" window (not an X11 window) and it comes up just  
fine: inside the terminal.  I can move around just like I remember.

The teminal is pretending to be an xterm.  In fact, you can do the  
same thing inside an xterm, just unset DISPLAY -- otherwise emacs  
will become an X11 client.

I tried doing the same thing to the GUI style emacs 22.0.50 version  
but if I do it from the terminal, it complains that it can't find  
things: encoded-kb... I probably need to set my load-path on the  
command line somehow.

Anyhow... I know it worked as of emacs 21.  I can't test it on emacs  
22 today.

Perry Smith ( pedz@easesoftware.com )
Ease Software, Inc. ( http://www.easesoftware.com )

Low cost SATA Disk Systems for IBMs p5, pSeries, and RS/6000 AIX systems



[-- Attachment #1.2: Type: text/html, Size: 7419 bytes --]

[-- Attachment #2: Type: text/plain, Size: 152 bytes --]

_______________________________________________
help-gnu-emacs mailing list
help-gnu-emacs@gnu.org
http://lists.gnu.org/mailman/listinfo/help-gnu-emacs

Re: Display corruption with binary files

[Eli Zaretskii]

> Cc: help-gnu-emacs@gnu.org
> From: Perry Smith <pedz@easesoftware.com>
> Date: Sat, 25 Nov 2006 18:34:04 -0600
> 
> On my Mac system, there is a version of emacs 21.2.1 in /usr/bin/ 
> emacs.  I have a bzip tar file.  I type:
> 
> emacs foo.tar.bz2
> 
> from a Mac "terminal" window (not an X11 window) and it comes up just  
> fine: inside the terminal.  I can move around just like I remember.

Maybe whoever built that version of Emacs for the Mac did something
special to it.

Re: Display corruption with binary files

[Perry Smith]

On Nov 25, 2006, at 10:09 PM, Eli Zaretskii wrote:

>> Cc: help-gnu-emacs@gnu.org
>> From: Perry Smith <pedz@easesoftware.com>
>> Date: Sat, 25 Nov 2006 18:34:04 -0600
>>
>> On my Mac system, there is a version of emacs 21.2.1 in /usr/bin/
>> emacs.  I have a bzip tar file.  I type:
>>
>> emacs foo.tar.bz2
>>
>> from a Mac "terminal" window (not an X11 window) and it comes up just
>> fine: inside the terminal.  I can move around just like I remember.
>
> Maybe whoever built that version of Emacs for the Mac did something
> special to it.

I get the same results with the emacs on the RS/6000 that I built  
personally.  I simple did configure and make.  That emacs is 21.4.1.


Perry Smith ( pedz@easesoftware.com )
Ease Software, Inc. ( http://www.easesoftware.com )

Low cost SATA Disk Systems for IBMs p5, pSeries, and RS/6000 AIX systems

Re: Display corruption with binary files

[Arjen Wiersma]

Hi,

>>> Cc: help-gnu-emacs@gnu.org
>>> From: Perry Smith <pedz@easesoftware.com>
>>> Date: Sat, 25 Nov 2006 18:34:04 -0600
>>>
>>> On my Mac system, there is a version of emacs 21.2.1 in /usr/bin/
>>> emacs.  I have a bzip tar file.  I type:
>>>
>>> emacs foo.tar.bz2
>>>
>>> from a Mac "terminal" window (not an X11 window) and it comes up  
>>> just
>>> fine: inside the terminal.  I can move around just like I remember.
>>
>> Maybe whoever built that version of Emacs for the Mac did something
>> special to it.
>
> I get the same results with the emacs on the RS/6000 that I built  
> personally.  I simple did configure and make.  That emacs is 21.4.1.

I tried both Emacs 21.2.1 and 22.0.90.1 (2006-11-12) and they both  
escape the characters to they ^-ed versions

To illustrate, first line from /bin/ls.

\376\355\372\316^@^@^@^R^@^@^@^@^@^@^@^B^@^@^@^K^@^@^FH^@^@^@ 
\205^@^@^@^A^@^@^@8__PAGEZERO^@^@^@^@^@^@^@^@^@^@^@^@^P^@^@^@^@^@^@^@ 
\^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^D^@^@^@^A^@^@^BX__TEXT^@^@^@^@^@^@^@ 
^@^@^@^@^@^P^@^@^@`^@^@^@^@^@^@^@`^@^@^@^@^G^@^@^@^E^@^@^@^ 
\H^@^@^@^@__text^@^@^@^@^@^@^@^@^@^@__TEXT^@^@^@^@^@^@^@^@^@^@^@^@^Z 
\304^@^@F\350^@^@

It does this both in terminal and gui (started with -q)

Regards,

Arjen

Re: Display corruption with binary files

[Eli Zaretskii]

> Cc: help-gnu-emacs@gnu.org
> From: Perry Smith <pedz@easesoftware.com>
> Date: Sat, 25 Nov 2006 22:55:45 -0600
> 
> On Nov 25, 2006, at 10:09 PM, Eli Zaretskii wrote:
> 
> >> Cc: help-gnu-emacs@gnu.org
> >> From: Perry Smith <pedz@easesoftware.com>
> >> Date: Sat, 25 Nov 2006 18:34:04 -0600
> >>
> >> On my Mac system, there is a version of emacs 21.2.1 in /usr/bin/
> >> emacs.  I have a bzip tar file.  I type:
> >>
> >> emacs foo.tar.bz2
> >>
> >> from a Mac "terminal" window (not an X11 window) and it comes up just
> >> fine: inside the terminal.  I can move around just like I remember.
> >
> > Maybe whoever built that version of Emacs for the Mac did something
> > special to it.
> 
> I get the same results with the emacs on the RS/6000 that I built  
> personally.  I simple did configure and make.  That emacs is 21.4.1.

Okay, I see that I've misunderstood the point and responded in a way
that could cause a terrible confusion.  Let me try to clarify.

Yes, Emacs does display control characters as ^c or \XXX.  However, it
does so only for certain characters (mainly, 8-bit characters and
low-end 7-bit characters).  It is quite possible that an arbitrary
binary garbage could be interpreted by Emacs as non-ASCII characters
and cause some terminal control sequence be written to the display.
Also, I think control characters that come from a display table (for
those who know what that is) are output verbatim.

So, to summarize, Emacs mostly does TRT with unprintable characters,
but not always, and visiting a binary file in the default multibyte
mode could plausibly cause related problems.

Sorry for any confusion I could have caused with my earlier messages
in this thread.

Re: Display corruption with binary files

[Robert Thorpe]

Perry Smith wrote:
> On Nov 25, 2006, at 5:04 AM, Eli Zaretskii wrote:
>
> >> From: August Karlstrom <fusionfive@comhem.se>
> >> Date: Sat, 25 Nov 2006 10:30:16 GMT
> >>>
> >>> Bits of the binary file being interpreted at terminal control
> >>> characters is a likely culprit.
> >>
> >> OK, but isn't a binary file as well as a text file really just a
> >> sequence of arbitrary bytes?
> >>
> >>> If that is the problem then Emacs should not send such characters to
> >>> the terminal.  It may be a terminal problem though.  I'd report
> >>> it as
> >>> an Emacs bug with that caveat.
> >>
> >> Eli Zaretskii's answer seems to indicate that this is a known
> >> problem.
> >
> > Yes, it's a known problem that the Unix terminal interprets certain
> > sequences of characters as commands.  I'm not sure Emacs can do
> > anything to solve this, but suggestions are welcome.
>
> I have not been closely watching this but I'm confused.
>
> emacs of old (I thought) would 'quote' the characters.  So, if the
> file had a control-C, emacs would display ^C and a single forward-
> character while the cursor is sitting on the ^ would move two screeen
> spaces up to the next character.  So, while emacs would send escape
> sequences itself to control the terminal, the data of the file would
> be sent properly.  (I'm not sure I'm saying clearly what I mean.)
>
> Is this not done?  Didn't emacs use to do it?

It may have been a terminal setup that did this.  I think I've seen
this behaviour in the past before too.

Re: Display corruption with binary files

[Peter Dyballa]

Am 25.11.2006 um 10:30 schrieb August Karlstrom:

> OK, but isn't a binary file as well as a text file really just a  
> sequence of arbitrary bytes?

A text file, binary or multinary, does not contain control  
characters. Control characters are for example ASCII NUL ...  
INFORMATION SEPARATOR ONE (U+0000...U+001F) and DELETE ...  
APPLICATION PROGRAM COMMAND (U+007F...U+009F), plus some more, that  
for example determine whether a text goes right-to-left or left-to- 
right, or up-down or ...

--
Greetings

   Pete

Time is an illusion. Lunchtime, doubly so.

Re: Display corruption with binary files

[August Karlstrom]

Peter Dyballa skrev:
> 
> Am 25.11.2006 um 10:30 schrieb August Karlstrom:
> 
>> OK, but isn't a binary file as well as a text file really just a 
>> sequence of arbitrary bytes?
> 
> A text file, binary or multinary, does not contain control characters. 
> Control characters are for example ASCII NUL ... INFORMATION SEPARATOR 
> ONE (U+0000...U+001F) and DELETE ... APPLICATION PROGRAM COMMAND 
> (U+007F...U+009F), plus some more, that for example determine whether a 
> text goes right-to-left or left-to-right, or up-down or ...

OK, thanks for the info.


August

Re: Display corruption with binary files

[Stefan Monnier]

> Yes, it's a known problem that the Unix terminal interprets certain
> sequences of characters as commands.  I'm not sure Emacs can do
> anything to solve this, but suggestions are welcome.

IIRC the problem was reported and fixed in Emacs-CVS a few months ago
(eight-bit-* chars were sent as raw uninterpreted bytes to the terminal
without going through coding-system: this was OK for eight bit encodings,
but not with utf-8).


        Stefan

Re: Display corruption with binary files

[Eli Zaretskii]

> From: August Karlstrom <fusionfive@comhem.se>
> Date: Thu, 23 Nov 2006 22:05:45 GMT
> 
> When I open any binary file with Emacs in console mode (emacs -nw) the 
> display gets corrupted (the menu and parts of the mode line disappears). 
> In GUI mode it works as expected. I use GNU Emacs version 21.4.1 and 
> gnome-terminal 2.16.1 under Ubuntu 6.10.
> 
> Any clues?

Try "M-x hexl-find-file RET", it is meant for editing binary files.

Re: Display corruption with binary files

[August Karlstrom]

Eli Zaretskii skrev:
>> From: August Karlstrom <fusionfive@comhem.se>
>> Date: Thu, 23 Nov 2006 22:05:45 GMT
>>
>> When I open any binary file with Emacs in console mode (emacs -nw) the 
>> display gets corrupted (the menu and parts of the mode line disappears). 
>> In GUI mode it works as expected. I use GNU Emacs version 21.4.1 and 
>> gnome-terminal 2.16.1 under Ubuntu 6.10.
>>
>> Any clues?
> 
> Try "M-x hexl-find-file RET", it is meant for editing binary files.

Works like a charm.


Thanks,

August

Re: Display corruption with binary files

[August Karlstrom]

August Karlstrom skrev:
> Eli Zaretskii skrev:
>>> From: August Karlstrom <fusionfive@comhem.se>
>>> Date: Thu, 23 Nov 2006 22:05:45 GMT
>>>
>>> When I open any binary file with Emacs in console mode (emacs -nw) 
>>> the display gets corrupted (the menu and parts of the mode line 
>>> disappears). In GUI mode it works as expected. I use GNU Emacs 
>>> version 21.4.1 and gnome-terminal 2.16.1 under Ubuntu 6.10.
>>>
>>> Any clues?
>>
>> Try "M-x hexl-find-file RET", it is meant for editing binary files.
> 
> Works like a charm.

One thing I've noticed with `hexl-find-file' though is that read-only 
permission of a file is not respected (I'm able to edit a read-only 
binary file). This is not the case if I open the file in Emacs with GUI 
(and run `hexl-mode'). Is this a bug?


August

Re: Display corruption with binary files

[Eli Zaretskii]

> From: August Karlstrom <fusionfive@comhem.se>
> Date: Sat, 25 Nov 2006 10:42:39 GMT
> 
> One thing I've noticed with `hexl-find-file' though is that read-only 
> permission of a file is not respected (I'm able to edit a read-only 
> binary file). This is not the case if I open the file in Emacs with GUI 
> (and run `hexl-mode'). Is this a bug?

Yes, probably.  Please report this to the bug-reporting address.

Thanks.