bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 445 bytes --]

Hello,

I find it tedious when looking at unprintables in Emacs just to have to
decode them from their octal to hexadecimal form for further lookup.
I've therefore written a patch that allows Emacs to display such
unprintables in hexadecimal format instead.  I'm not sure about the
naming though and whether a hardcoded secondary format string is the way
to go.  Once that's resolved, I'll add documentation to the patch.

Vasilij Schneidermann

[-- Attachment #2: 0001-Add-customizable-to-display-unprintables-as-hex.patch --]
[-- Type: text/x-diff, Size: 2093 bytes --]

From fb5c6bc81b3f63ce6258af0f54b876bf13f566ea Mon Sep 17 00:00:00 2001
From: Vasilij Schneidermann <mail@vasilij.de>
Date: Sun, 28 May 2017 22:31:34 +0200
Subject: [PATCH] Add customizable to display unprintables as hex

* src/xdisp.c (get_next_display_element): Dispatch used format string
  for unprintables based on new display-unprintables-as-hex
  variable
  (display-unprintables-as-hex): New variable
---
 src/xdisp.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/src/xdisp.c b/src/xdisp.c
index ddb26b8def..b2b2f30594 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -7044,7 +7044,7 @@ get_next_display_element (struct it *it)
 	     translated too.
 
 	     Non-printable characters and raw-byte characters are also
-	     translated to octal form.  */
+	     translated to octal or hexadecimal form.  */
 	  if (((c < ' ' || c == 127) /* ASCII control chars.  */
 	       ? (it->area != TEXT_AREA
 		  /* In mode line, treat \n, \t like other crl chars.  */
@@ -7151,9 +7151,12 @@ get_next_display_element (struct it *it)
 		int len, i;
 
 		if (CHAR_BYTE8_P (c))
-		  /* Display \200 instead of \17777600.  */
+		  /* Display \200 or \x80 instead of \17777600.  */
 		  c = CHAR_TO_BYTE8 (c);
-		len = sprintf (str, "%03o", c + 0u);
+                const char *format_string = display_unprintables_as_hex
+					    ? "x%02x"
+					    : "%03o";
+		len = sprintf (str, format_string, c + 0u);
 
 		XSETINT (it->ctl_chars[0], escape_glyph);
 		for (i = 0; i < len; i++)
@@ -32209,6 +32212,14 @@ display table takes effect; in this case, Emacs does not consult
   /* Initialize to t, since we need to disable reordering until
      loadup.el successfully loads charprop.el.  */
   redisplay__inhibit_bidi = true;
+
+  DEFVAR_BOOL ("display-unprintables-as-hex",
+               display_unprintables_as_hex,
+    doc: /* Non-nil means display unprintables in hexadecimal format.
+The default is to use octal format (\200) whereas hexadecimal (\x80)
+may be more familar to users.  */);
+  display_unprintables_as_hex = false;
+
 }
 
 
-- 
2.13.0

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Eli Zaretskii]

> Date: Sun, 28 May 2017 22:42:50 +0200
> From: Vasilij Schneidermann <v.schneidermann@gmail.com>
> 
> I find it tedious when looking at unprintables in Emacs just to have to
> decode them from their octal to hexadecimal form for further lookup.
> I've therefore written a patch that allows Emacs to display such
> unprintables in hexadecimal format instead.

Please describe the use case where this happens.  We already have
quite a few related features, and I wonder whether you could simply
use one of them.  But perhaps I don't understand the situation where
you needed this.

Thanks.

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Vasilij Schneidermann]

> Please describe the use case where this happens.

OK, I'll go into a bit more detail on my envisioned use case.  Suppose
you open a buffer just to discover that some of the bytes aren't decoded
properly.  The culprits are \344, \366 and \374.  You suspect it's
latin-1, so you open a Wikipedia page on it and find a table of its
character mapping, however it's using hexadecimal instead of octal
formatting.  Generally it seems that the world has moved on and prefers
hexadecimal over octal formatting, except for `chmod` and Emacs.  You
sigh and convert the octal to hexadecimal.  Your hunch about this being
a latin-1 issue turns out to be true.  Why can't Emacs make this easier?

> We already have quite a few related features, and I wonder whether
> you could simply use one of them.

The only one I'm aware of you could use instead is to manipulate the
display table.  This allows for the same effect, but is problematic as
there are other modes manipulating it, such as whitespace-mode.  You'd
therefore need to manipulate `standard-display-table' and
`buffer-display-table'  with an appropriate hook to cover all cases.

I consider the patch to be a cleaner solution.

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Robert Cochran]

Vasilij Schneidermann <v.schneidermann@gmail.com> writes:

> [...] Generally it seems that the world has moved on and prefers
> hexadecimal over octal formatting, except for `chmod` and Emacs.  You
> sigh and convert the octal to hexadecimal. [...] Why can't Emacs make
> this easier?

Agreed. This makes Emacs really look like the odd man out. Personally,
chmod gets a pass because the octal very cleanly maps to Unix
permissions. But it looks awkward when Emacs does it, especially when I
believe it to have no reason to do so.

>> We already have quite a few related features, and I wonder whether
>> you could simply use one of them.
>
> The only one I'm aware of you could use instead is to manipulate the
> display table.  This allows for the same effect, but is problematic as
> there are other modes manipulating it, such as whitespace-mode.  You'd
> therefore need to manipulate `standard-display-table' and
> `buffer-display-table'  with an appropriate hook to cover all cases.
>
> I consider the patch to be a cleaner solution.

Not as well versed on these features that Eli is referring to, so I'm
less helpful here. However, what is wrong with adding what is clearly an
option, so it can be changed in one place and be done with it?

-- 
~Robert Cochran

GPG Fingerprint - E778 2DD4 FEA6 6A68 6F26  AD2D E5C3 EB36 4886 8871

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Eli Zaretskii]

> From: Robert Cochran <robert+Emacs@cochranmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  27122@debbugs.gnu.org
> Date: Mon, 29 May 2017 11:46:03 -0700
> 
> what is wrong with adding what is clearly an option, so it can be
> changed in one place and be done with it?

Complexity and maintainability.  The code which handles special
characters is already almost 140 lines of terse C, controlled by 2
obscure variables (ever heard of printable-chars, for example?) and 2
special faces.  And that's excluding the glyphless-char-display stuff,
which adds even more.  It's almost impossible to tell in advance how
will some specific codepoint be displayed if the terminal cannot
handle it "normally".  At least I need to read the involved code every
time I have to answer such questions.

I'd rather not add to this any more complexity, unless it's really
needed.  That is why I'd like to understand the details of the use
case: to see whether we already have a reasonable solution for it.

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Vasilij Schneidermann]

> Complexity and maintainability.  The code which handles special
> characters is already almost 140 lines of terse C, controlled by 2
> obscure variables (ever heard of printable-chars, for example?) and 2
> special faces.  And that's excluding the glyphless-char-display stuff,
> which adds even more.  It's almost impossible to tell in advance how
> will some specific codepoint be displayed if the terminal cannot
> handle it "normally".  At least I need to read the involved code every
> time I have to answer such questions.

While this is unfortunate, I find it more instructive to look at it in
terms of how much complexity is added (an extra conditional and a
customizable), not how much lurks in the existing code.  If you
constantly went by that argument, you would have to refrain from any
kind of Emacs hacking given its impenetrable code base.

The easiest way to solve this without adding any extra complexity would
be simply replacing the format string by the hex one.  I wouldn't want
to be involved in the ensuing discussion though...

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Robert Cochran]

Vasilij Schneidermann <v.schneidermann@gmail.com> writes:

> While this is unfortunate, I find it more instructive to look at it in
> terms of how much complexity is added (an extra conditional and a
> customizable), not how much lurks in the existing code.  If you
> constantly went by that argument, you would have to refrain from any
> kind of Emacs hacking given its impenetrable code base.

Agree with the sentiment, which is probably a better stated version of
what I tried to say earlier. The way I wrote it eariler glossed over the
details rather heavily. Apologies for that.

> The easiest way to solve this without adding any extra complexity would
> be simply replacing the format string by the hex one.  I wouldn't want
> to be involved in the ensuing discussion though...

IMO, this is the best solution. But the size that the ensuing discussion
would inenvitably grow to puts me off of this path as well. I was
initially thinking that the current behavior was more long-standing (10+
years), but it was actually introduced in its current form in 2010, in
d419e1d94 "Fix handling of 8-bit characters in a display table.". This
makes me more convinced that it should have been hex all along, given
the relative youth of the behavior, but no better time than the present
to change it, right?

--
~Robert Cochran

GPG Fingerprint - E778 2DD4 FEA6 6A68 6F26  AD2D E5C3 EB36 4886 8871

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[npostavs]

Robert Cochran <robert+Emacs@cochranmail.com> writes:

> IMO, this is the best solution. But the size that the ensuing discussion
> would inenvitably grow to puts me off of this path as well. I was
> initially thinking that the current behavior was more long-standing (10+
> years), but it was actually introduced in its current form in 2010, in
> d419e1d94 "Fix handling of 8-bit characters in a display table.".

While I'm also okay with changing this to hex display without an option,
I don't think that commit introduced the octal behaviour, it just
changed the implementation.  I think these these removed lines used to
do it:

-		    /* Insert three more glyphs into IT->ctl_chars for
-		       the octal display of the character.  */
-		    g = ((str[i] >> 6) & 7) + '0';
-		    XSETINT (it->ctl_chars[i * 4 + 1], g);
-		    g = ((str[i] >> 3) & 7) + '0';
-		    XSETINT (it->ctl_chars[i * 4 + 2], g);
-		    g = (str[i] & 7) + '0';
-		    XSETINT (it->ctl_chars[i * 4 + 3], g);

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Robert Cochran]

npostavs@users.sourceforge.net writes:

> While I'm also okay with changing this to hex display without an option,
> I don't think that commit introduced the octal behaviour, it just
> changed the implementation.  I think these these removed lines used to
> do it:
>
> -		    /* Insert three more glyphs into IT->ctl_chars for
> -		       the octal display of the character.  */
> -		    g = ((str[i] >> 6) & 7) + '0';
> -		    XSETINT (it->ctl_chars[i * 4 + 1], g);
> -		    g = ((str[i] >> 3) & 7) + '0';
> -		    XSETINT (it->ctl_chars[i * 4 + 2], g);
> -		    g = (str[i] & 7) + '0';
> -		    XSETINT (it->ctl_chars[i * 4 + 3], g);

Indeed. I wasn't looking closely enough. I mentally skipped over that
part because it was below the new sprintf. My bad, thanks for catching
that.

-- 
~Robert Cochran

GPG Fingerprint - E778 2DD4 FEA6 6A68 6F26  AD2D E5C3 EB36 4886 8871

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Eli Zaretskii]

> Date: Mon, 29 May 2017 08:28:59 +0200
> From: Vasilij Schneidermann <v.schneidermann@gmail.com>
> Cc: 27122@debbugs.gnu.org
> 
> OK, I'll go into a bit more detail on my envisioned use case.  Suppose
> you open a buffer just to discover that some of the bytes aren't decoded
> properly.  The culprits are \344, \366 and \374.  You suspect it's
> latin-1, so you open a Wikipedia page on it and find a table of its
> character mapping, however it's using hexadecimal instead of octal
> formatting.  Generally it seems that the world has moved on and prefers
> hexadecimal over octal formatting, except for `chmod` and Emacs.  You
> sigh and convert the octal to hexadecimal.  Your hunch about this being
> a latin-1 issue turns out to be true.  Why can't Emacs make this easier?

Does "C-x =" or "C-u C-x =" fit the bill in this case?  If the former
doesn't, how about adding the hex byte translation to what it shows?
IOW, what if instead of

  Char: \344 (4194276, #o17777744, #x3fffe4, raw-byte) point=...

"C-x =" would display this:

  Char: \344, #xe4 (4194276, #o17777744, #x3fffe4, raw-byte) point=...

> > We already have quite a few related features, and I wonder whether
> > you could simply use one of them.
> 
> The only one I'm aware of you could use instead is to manipulate the
> display table.

We also have the glyphless-char-display-control (and the underlying
char-table), although it will not help in your case.  Which is why I
asked for more details.

Also, some characters, hard-coded in the display engine, are handled
specially.  E.g., try "C-x 8 RET a0 RET" or "C-x 8 RET 2011 RET".

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Vasilij Schneidermann]

> Does "C-x =" or "C-u C-x =" fit the bill in this case?  If the former
> doesn't, how about adding the hex byte translation to what it shows?

This is the crutch I'm currently using.  It isn't practical for more
than a few characters.  If I considered it sufficient, I wouldn't be
handing in a patch.  Is it really necessary to explain the ergonomics of
viewing all characters in a buffer at once vs. inspecting them one by
one to?

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Eli Zaretskii]

> Date: Mon, 29 May 2017 22:15:58 +0200
> From: Vasilij Schneidermann <v.schneidermann@gmail.com>
> Cc: 27122@debbugs.gnu.org
> 
> > Does "C-x =" or "C-u C-x =" fit the bill in this case?  If the former
> > doesn't, how about adding the hex byte translation to what it shows?
> 
> This is the crutch I'm currently using.  It isn't practical for more
> than a few characters.  If I considered it sufficient, I wouldn't be
> handing in a patch.

Well, you didn't mention this at all, so I couldn't know.

OK, please submit the patch with the necessary documentation, and
please add a test for the original and the new behavior.

Also, I think the option should be called display-raw-bytes-as-hex,
since "unprintables", though more accurate, is IMO too technical for
most users, and the default behavior limits this display to raw bytes.

Finally, I think it would be good to make this option a defcustom, so
please add the necessary glue to cus-start.el.

Btw, is \xNN what people are used to in this kind of display?  Emacs
generally uses #xNN elsewhere, but if \xNN is more familiar, I don't
mind.

Thanks.

P.S.  A minor nit: we use tabs and spaces to indent C code, not just
spaces.

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 588 bytes --]

> OK, please submit the patch with the necessary documentation, and
> please add a test for the original and the new behavior.

Where and how should this feature be tested?  I've found
test/manual/redisplay-testsuite.el for manual testing of xdisp.c
features, but also test/src/textprop.el for automated testing which
suggests test/src/xdisp.el might work out.  The only issue with
automatic testing here is that all ERT tests I've seen run in batch mode
which prevents `emacs --batch --eval "(princ (char-to-string ?\200))"`
from doing the right thing.

I've fixed the remaining points.

[-- Attachment #2: 0001-Add-customizable-to-display-raw-bytes-as-hex.patch --]
[-- Type: text/x-diff, Size: 3783 bytes --]

From 5a6584d517736eed4a3a067bb7139bc4db6dd797 Mon Sep 17 00:00:00 2001
From: Vasilij Schneidermann <mail@vasilij.de>
Date: Sun, 28 May 2017 22:31:34 +0200
Subject: [PATCH] Add customizable to display raw bytes as hex

* src/xdisp.c (get_next_display_element): Dispatch used format string
  for unprintables based on new display-raw-bytes-as-hex
  variable
  (display-raw-bytes-as-hex): New variable
* doc/emacs/display.texi: Document the new variable
---
 doc/emacs/display.texi |  6 ++++++
 etc/NEWS               |  4 ++++
 lisp/cus-start.el      |  1 +
 src/xdisp.c            | 16 +++++++++++++---
 4 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/doc/emacs/display.texi b/doc/emacs/display.texi
index d07913cefb..edcb575722 100644
--- a/doc/emacs/display.texi
+++ b/doc/emacs/display.texi
@@ -1756,3 +1756,9 @@ Display Custom
 in text that is hard to read.  Call the function
 @code{tty-suppress-bold-inverse-default-colors} with a non-@code{nil}
 argument to suppress the effect of bold-face in this case.
+
+@vindex display-raw-bytes-as-hex
+  Raw bytes are displayed in octal format by default, for example a
+byte with a decimal value of 128 is displayed as @code{\200}.  To
+change display to the hexadecimal format of @code{\x80}, set the
+variable @code{display-raw-bytes-as-hex} to @code{t}.
diff --git a/etc/NEWS b/etc/NEWS
index 60066b7c9f..ee941624d8 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -363,6 +363,10 @@ large integers from being displayed as characters.
 ** Two new commands for finding the source code of Emacs Lisp
 libraries: 'find-library-other-window' and 'find-library-other-frame'.
 
++++
+** The new variable 'display-raw-bytes-as-hex' allows to change the
+display of raw bytes from octal to hex.
+
 \f
 * Editing Changes in Emacs 26.1
 
diff --git a/lisp/cus-start.el b/lisp/cus-start.el
index 4253d40b75..744fe7f69e 100644
--- a/lisp/cus-start.el
+++ b/lisp/cus-start.el
@@ -583,6 +583,7 @@ minibuffer-prompt-properties--setter
 		       (const :tag "Fit (t)" :value t)
 		       (const :tag "Grow only" :value grow-only))
 	      "25.1")
+	     (display-raw-bytes-as-hex display boolean "26.1")
 	     ;; xfaces.c
 	     (scalable-fonts-allowed display boolean "22.1")
 	     ;; xfns.c
diff --git a/src/xdisp.c b/src/xdisp.c
index ddb26b8def..10124d6f3d 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -7044,7 +7044,7 @@ get_next_display_element (struct it *it)
 	     translated too.
 
 	     Non-printable characters and raw-byte characters are also
-	     translated to octal form.  */
+	     translated to octal or hexadecimal form.  */
 	  if (((c < ' ' || c == 127) /* ASCII control chars.  */
 	       ? (it->area != TEXT_AREA
 		  /* In mode line, treat \n, \t like other crl chars.  */
@@ -7151,9 +7151,12 @@ get_next_display_element (struct it *it)
 		int len, i;
 
 		if (CHAR_BYTE8_P (c))
-		  /* Display \200 instead of \17777600.  */
+		  /* Display \200 or \x80 instead of \17777600.  */
 		  c = CHAR_TO_BYTE8 (c);
-		len = sprintf (str, "%03o", c + 0u);
+		const char *format_string = display_raw_bytes_as_hex
+					    ? "x%02x"
+					    : "%03o";
+		len = sprintf (str, format_string, c + 0u);
 
 		XSETINT (it->ctl_chars[0], escape_glyph);
 		for (i = 0; i < len; i++)
@@ -32209,6 +32212,13 @@ display table takes effect; in this case, Emacs does not consult
   /* Initialize to t, since we need to disable reordering until
      loadup.el successfully loads charprop.el.  */
   redisplay__inhibit_bidi = true;
+
+  DEFVAR_BOOL ("display-raw-bytes-as-hex", display_raw_bytes_as_hex,
+    doc: /* Non-nil means display raw bytes in hexadecimal format.
+The default is to use octal format (\200) whereas hexadecimal (\x80)
+may be more familar to users.  */);
+  display_raw_bytes_as_hex = false;
+
 }
 
 
-- 
2.13.0

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Eli Zaretskii]

> Date: Tue, 30 May 2017 10:16:28 +0200
> From: Vasilij Schneidermann <v.schneidermann@gmail.com>
> Cc: 27122@debbugs.gnu.org
> 
> > OK, please submit the patch with the necessary documentation, and
> > please add a test for the original and the new behavior.
> 
> Where and how should this feature be tested?  I've found
> test/manual/redisplay-testsuite.el for manual testing of xdisp.c
> features, but also test/src/textprop.el for automated testing which
> suggests test/src/xdisp.el might work out.

Yes, I think test/manual/redisplay-testsuite.el is where the
additional tests should go.

> I've fixed the remaining points.

> * src/xdisp.c (get_next_display_element): Dispatch used format string
>   for unprintables based on new display-raw-bytes-as-hex
>   variable
>   (display-raw-bytes-as-hex): New variable
> * doc/emacs/display.texi: Document the new variable

Please end each sentence in the commit log message with a period, and
please mention the bug number there.  Please also add a short log
entry for etc/NEWS.

Thanks.

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 527 bytes --]

> Yes, I think test/manual/redisplay-testsuite.el is where the
> additional tests should go.

OK, I've added a test there that allows you to toggle between both
representations.

> Please end each sentence in the commit log message with a period, and
> please mention the bug number there.  Please also add a short log
> entry for etc/NEWS.

Fixed.  I'm not sure what you mean by the NEWS entry though, the patch
has four lines in etc/NEWS describing the new feature.  Are you possibly
referring to a different format or file?

[-- Attachment #2: 0001-Add-customizable-to-display-raw-bytes-as-hex.patch --]
[-- Type: text/x-diff, Size: 5648 bytes --]

From 7c66dff5dc35e663dd3d39e2b30205924fa262c7 Mon Sep 17 00:00:00 2001
From: Vasilij Schneidermann <mail@vasilij.de>
Date: Sun, 28 May 2017 22:31:34 +0200
Subject: [PATCH] Add customizable to display raw bytes as hex

* src/xdisp.c (get_next_display_element): Dispatch used format string
  for unprintables based on new display-raw-bytes-as-hex
  variable.
  (display-raw-bytes-as-hex): New variable.
* doc/emacs/display.texi: Document the new variable.
---
 doc/emacs/display.texi             |  6 ++++++
 etc/NEWS                           |  4 ++++
 lisp/cus-start.el                  |  1 +
 src/xdisp.c                        | 16 +++++++++++++---
 test/manual/redisplay-testsuite.el | 27 ++++++++++++++++++++++++++-
 5 files changed, 50 insertions(+), 4 deletions(-)

diff --git a/doc/emacs/display.texi b/doc/emacs/display.texi
index d07913cefb..edcb575722 100644
--- a/doc/emacs/display.texi
+++ b/doc/emacs/display.texi
@@ -1756,3 +1756,9 @@ Display Custom
 in text that is hard to read.  Call the function
 @code{tty-suppress-bold-inverse-default-colors} with a non-@code{nil}
 argument to suppress the effect of bold-face in this case.
+
+@vindex display-raw-bytes-as-hex
+  Raw bytes are displayed in octal format by default, for example a
+byte with a decimal value of 128 is displayed as @code{\200}.  To
+change display to the hexadecimal format of @code{\x80}, set the
+variable @code{display-raw-bytes-as-hex} to @code{t}.
diff --git a/etc/NEWS b/etc/NEWS
index 60066b7c9f..ee941624d8 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -363,6 +363,10 @@ large integers from being displayed as characters.
 ** Two new commands for finding the source code of Emacs Lisp
 libraries: 'find-library-other-window' and 'find-library-other-frame'.
 
++++
+** The new variable 'display-raw-bytes-as-hex' allows to change the
+display of raw bytes from octal to hex.
+
 \f
 * Editing Changes in Emacs 26.1
 
diff --git a/lisp/cus-start.el b/lisp/cus-start.el
index 4253d40b75..744fe7f69e 100644
--- a/lisp/cus-start.el
+++ b/lisp/cus-start.el
@@ -583,6 +583,7 @@ minibuffer-prompt-properties--setter
 		       (const :tag "Fit (t)" :value t)
 		       (const :tag "Grow only" :value grow-only))
 	      "25.1")
+	     (display-raw-bytes-as-hex display boolean "26.1")
 	     ;; xfaces.c
 	     (scalable-fonts-allowed display boolean "22.1")
 	     ;; xfns.c
diff --git a/src/xdisp.c b/src/xdisp.c
index ddb26b8def..10124d6f3d 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -7044,7 +7044,7 @@ get_next_display_element (struct it *it)
 	     translated too.
 
 	     Non-printable characters and raw-byte characters are also
-	     translated to octal form.  */
+	     translated to octal or hexadecimal form.  */
 	  if (((c < ' ' || c == 127) /* ASCII control chars.  */
 	       ? (it->area != TEXT_AREA
 		  /* In mode line, treat \n, \t like other crl chars.  */
@@ -7151,9 +7151,12 @@ get_next_display_element (struct it *it)
 		int len, i;
 
 		if (CHAR_BYTE8_P (c))
-		  /* Display \200 instead of \17777600.  */
+		  /* Display \200 or \x80 instead of \17777600.  */
 		  c = CHAR_TO_BYTE8 (c);
-		len = sprintf (str, "%03o", c + 0u);
+		const char *format_string = display_raw_bytes_as_hex
+					    ? "x%02x"
+					    : "%03o";
+		len = sprintf (str, format_string, c + 0u);
 
 		XSETINT (it->ctl_chars[0], escape_glyph);
 		for (i = 0; i < len; i++)
@@ -32209,6 +32212,13 @@ display table takes effect; in this case, Emacs does not consult
   /* Initialize to t, since we need to disable reordering until
      loadup.el successfully loads charprop.el.  */
   redisplay__inhibit_bidi = true;
+
+  DEFVAR_BOOL ("display-raw-bytes-as-hex", display_raw_bytes_as_hex,
+    doc: /* Non-nil means display raw bytes in hexadecimal format.
+The default is to use octal format (\200) whereas hexadecimal (\x80)
+may be more familar to users.  */);
+  display_raw_bytes_as_hex = false;
+
 }
 
 
diff --git a/test/manual/redisplay-testsuite.el b/test/manual/redisplay-testsuite.el
index 694d55ab1d..2175cbab1b 100644
--- a/test/manual/redisplay-testsuite.el
+++ b/test/manual/redisplay-testsuite.el
@@ -34,7 +34,8 @@ test-insert-overlay
     (setq overlay (make-overlay opoint (point)))
     (while props
       (overlay-put overlay (car props) (cadr props))
-      (setq props (cddr props)))))
+      (setq props (cddr props)))
+    overlay))
 
 (defun test-redisplay-1 ()
   (insert "Test 1: Displaying adjacent and overlapping overlays:\n\n")
@@ -293,6 +294,29 @@ test-redisplay-4
 
   (insert "\n"))
 
+(defvar test-redisplay-5-expected-overlay nil)
+(defvar test-redisplay-5-result-overlay nil)
+
+(defun test-redisplay-5-toggle (_event)
+  (interactive "e")
+  (setq display-raw-bytes-as-hex (not display-raw-bytes-as-hex))
+  (let ((label (if display-raw-bytes-as-hex "\\x80" "\\200")))
+    (overlay-put test-redisplay-5-expected-overlay 'display
+                 (propertize label 'face 'escape-glyph))))
+
+(defun test-redisplay-5 ()
+  (insert "Test 5: Display of raw bytes:\n\n")
+  (insert "  Expected: ")
+  (setq test-redisplay-5-expected-overlay
+        (test-insert-overlay " " 'display
+                             (propertize "\\200" 'face 'escape-glyph)))
+  (insert "\n    Result: ")
+  (setq test-redisplay-5-result-overlay
+        (test-insert-overlay " " 'display "\200"))
+  (insert "\n\n")
+  (insert-button "Toggle between octal and hex display"
+                 'action 'test-redisplay-5-toggle))
+
 
 (defun test-redisplay ()
   (interactive)
@@ -309,5 +333,6 @@ test-redisplay
     (test-redisplay-2)
     (test-redisplay-3)
     (test-redisplay-4)
+    (test-redisplay-5)
     (goto-char (point-min))))
 
-- 
2.13.0

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[Eli Zaretskii]

> Date: Thu, 1 Jun 2017 19:05:09 +0200
> From: Vasilij Schneidermann <v.schneidermann@gmail.com>
> Cc: 27122@debbugs.gnu.org
> 
> > Yes, I think test/manual/redisplay-testsuite.el is where the
> > additional tests should go.
> 
> OK, I've added a test there that allows you to toggle between both
> representations.
> 
> > Please end each sentence in the commit log message with a period, and
> > please mention the bug number there.  Please also add a short log
> > entry for etc/NEWS.
> 
> Fixed.  I'm not sure what you mean by the NEWS entry though, the patch
> has four lines in etc/NEWS describing the new feature.  Are you possibly
> referring to a different format or file?

No, I meant to mention the NEWS change in the commit log message.  I
added that myself and pushed the changes.

Thanks.

bug#27122: 26.0.50; [PATCH] Add customizable to display unprintables as hex

[npostavs]

Eli Zaretskii <eliz@gnu.org> writes:

> Btw, is \xNN what people are used to in this kind of display?  Emacs
> generally uses #xNN elsewhere, but if \xNN is more familiar, I don't
> mind.

Emacs uses \xNN and \NNN in string literals, so toggling between those
alternative makes sense.