bug#13691: 24.3.50; Problem with glyphs in unibyte buffers

bug#13691: 24.3.50; Problem with glyphs in unibyte buffers

[Eduardo Ochs]

[-- Attachment #1: Type: text/plain, Size: 3997 bytes --]

In Emacs24 if I run the code below to define glyphs for the characters
171 and 187 the effect is different on unibyte (i.e., "coding:
raw-text-unix") and non-unibyte buffers, as shown in the attached
screenshot.

My guess is that for some range of characteres that includes 171 and
187 - but not 15, see the screenshot - the glyph lookup is either
disabled in unibyte buffers or it is done using some range translation
that I wasn't able to figure out what is.

Here is the test code:

--snip--snip--

cat > /tmp/g <<'%%%'
(progn

(menu-bar-mode 0)
(tool-bar-mode 0)
(scroll-bar-mode -1)

(defun eepitch-set-glyph (pos &optional char face)
  (aset standard-display-table pos
	(if char (vector (make-glyph-code char face)))))

(if (not standard-display-table)
    (setq standard-display-table (make-display-table)))

(defface eepitch-star-face
  '((t (:foreground "red")))
  "Face used for the red star glyph (char 15).")

(defface eev-glyph-face-green
  '((t (:foreground "green")))
  "Face used for the green glyphs (`<<' and `>>', chars 171 and 187).")

(eepitch-set-glyph ?\^O ?* 'eepitch-star-face)
(eepitch-set-glyph 171 171 'eev-glyph-face-green)
(eepitch-set-glyph 187 187 'eev-glyph-face-green)

(insert "\n;; " 15 171 187)

)


;; Local Variables:
;; coding:            raw-text-unix
;; End:
%%%

emacs23          -fg bisque -bg black -fn 6x13 -Q /tmp/g
emacs24-from-git -fg bisque -bg black -fn 6x13 -Q /tmp/g

--snip--snip--

  Cheers, thanks in advance =),
    Eduardo Ochs
    eduardoochs@gmail.com
    http://angg.twu.net/
    http://angg.twu.net/#eev



P.S.: I use glyphs a lot in my personal files (see
<http://angg.twu.net/glyphs.html>), and a few glyphs - 3, actually -
in a package that I would like to include in Emacs in a few more
months or years: <http://angg.twu.net/#eev>... I tried to keep this
first bug report quite concise, but I may have tests ready for several
other related cases where the behaviors of Emacs24 were a bit
unexpected to me...

P.P.S.: the Emacs23 used for comparison was the standard one in Debian
Squeeze,

  GNU Emacs 23.2.1 (i486-pc-linux-gnu, GTK+ Version 2.20.1) of
    2013-01-08 on murphy, modified by Debian

and the extra info for my Emacs24 from the git repository (I ran it
with "-Q") follows below.




In GNU Emacs 24.3.50.2 (i686-pc-linux-gnu, GTK+ Version 2.20.1)
 of 2013-02-11 on varda
Windowing system distributor `The X.Org Foundation', version 11.0.10707000
System Description:	Debian GNU/Linux 6.0.6 (squeeze)

Important settings:
  value of $LANG: C
  locale-coding-system: nil
  default enable-multibyte-characters: t

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind inotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty emacs)

[-- Attachment #2: glyph-bug-screenshot.png --]
[-- Type: image/png, Size: 5769 bytes --]

bug#13691: 24.3.50; Problem with glyphs in unibyte buffers

[Andreas Schwab]

Eduardo Ochs <eduardoochs@gmail.com> writes:

> In Emacs24 if I run the code below to define glyphs for the characters
> 171 and 187 the effect is different on unibyte (i.e., "coding:
> raw-text-unix") and non-unibyte buffers, as shown in the attached
> screenshot.

Unibyte characters are in the eight-bit charset, so you need to set the
display of the corresponding character in that charset, eg. (make-char
'eight-bit 171).

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#13691: 24.3.50; Problem with glyphs in unibyte buffers

[Eduardo Ochs]

[-- Attachment #1: Type: text/plain, Size: 1862 bytes --]

Hi Andreas,

On Tue, Feb 12, 2013 at 7:02 AM, Andreas Schwab <schwab@linux-m68k.org> wrote:
>
> Unibyte characters are in the eight-bit charset, so you need to set the
> display of the corresponding character in that charset, eg. (make-char
> 'eight-bit 171).

Thanks, that worked!

Now for something slightly trickier... =|

Let's start by using several simple, low-level functions, one for
each range... a smarter function with `cond's can be left for
later. The code below makes the green guillemets work both in
unibyte and multibyte buffers,

  (defun eepitch-set-glyph (pos &optional char face)
    (aset standard-display-table pos
	  (if char (vector (make-glyph-code char face)))))

  (defun eepitch-set-glyph-8bit (pos &optional char face)
    (aset standard-display-table (make-char 'eight-bit pos)
	  (if char (vector (make-glyph-code char face)))))

  (if (not standard-display-table)
      (setq standard-display-table (make-display-table))
    )

  (defface eev-glyph-face-green '((t :foreground "green")) "")
  (eepitch-set-glyph      171 171 'eev-glyph-face-green)
  (eepitch-set-glyph-8bit 171 171 'eev-glyph-face-green)
  (eepitch-set-glyph      187 187 'eev-glyph-face-green)
  (eepitch-set-glyph-8bit 187 187 'eev-glyph-face-green)

but now suppose that we want the char 191 to be displayed as a
blue nabla (unicode: 8711). Running

  (defface eev-glyph-face-math  '((t :foreground "RoyalBlue2"
				     :background "gray20")) "")
  (eepitch-set-glyph      191 8711 'eev-glyph-face-math)
  (eepitch-set-glyph-8bit 191 8711 'eev-glyph-face-math)

_almost_ does the job, but look at the new screenshot - it seems
that the `char' argument in `eepitch-set-glyph-8bit' needs some
translation too... is that right? Which translation?

  Cheers, and thanks in advance again...
    Eduardo Ochs
    eduardoochs@gmail.com
    http://angg.twu.net/#eev

[-- Attachment #2: glyph-bug-screenshot-2.png --]
[-- Type: image/png, Size: 3436 bytes --]

bug#13691: 24.3.50; Problem with glyphs in unibyte buffers

[Stefan Kangas]

Hi Eduardo,

You sent the below description to the Emacs bug list 7 years ago.

The email you sent is quite long, and I'm not sure what exactly the
issue is, or how to try to reproduce it.

Can you reproduce the issue on a recent version of Emacs, such as the
recently released version 27.1?

If yes, could you please provide a minimal step-by-step recipe for
reproducing it, starting from "emacs -Q"?

If I don't hear back from you within a couple of weeks, I'll just
close this bug as unreproducible.

Thanks in advance.

Best regards,
Stefan Kangas

Eduardo Ochs <eduardoochs@gmail.com> writes:

> Hi Andreas,
>
> On Tue, Feb 12, 2013 at 7:02 AM, Andreas Schwab <schwab@linux-m68k.org> wrote:
>>
>> Unibyte characters are in the eight-bit charset, so you need to set the
>> display of the corresponding character in that charset, eg. (make-char
>> 'eight-bit 171).
>
> Thanks, that worked!
>
> Now for something slightly trickier... =|
>
> Let's start by using several simple, low-level functions, one for
> each range... a smarter function with `cond's can be left for
> later. The code below makes the green guillemets work both in
> unibyte and multibyte buffers,
>
>   (defun eepitch-set-glyph (pos &optional char face)
>     (aset standard-display-table pos
> 	  (if char (vector (make-glyph-code char face)))))
>
>   (defun eepitch-set-glyph-8bit (pos &optional char face)
>     (aset standard-display-table (make-char 'eight-bit pos)
> 	  (if char (vector (make-glyph-code char face)))))
>
>   (if (not standard-display-table)
>       (setq standard-display-table (make-display-table))
>     )
>
>   (defface eev-glyph-face-green '((t :foreground "green")) "")
>   (eepitch-set-glyph      171 171 'eev-glyph-face-green)
>   (eepitch-set-glyph-8bit 171 171 'eev-glyph-face-green)
>   (eepitch-set-glyph      187 187 'eev-glyph-face-green)
>   (eepitch-set-glyph-8bit 187 187 'eev-glyph-face-green)
>
> but now suppose that we want the char 191 to be displayed as a
> blue nabla (unicode: 8711). Running
>
>   (defface eev-glyph-face-math  '((t :foreground "RoyalBlue2"
> 				     :background "gray20")) "")
>   (eepitch-set-glyph      191 8711 'eev-glyph-face-math)
>   (eepitch-set-glyph-8bit 191 8711 'eev-glyph-face-math)
>
> _almost_ does the job, but look at the new screenshot - it seems
> that the `char' argument in `eepitch-set-glyph-8bit' needs some
> translation too... is that right? Which translation?
>
>   Cheers, and thanks in advance again...
>     Eduardo Ochs
>     eduardoochs@gmail.com
>     http://angg.twu.net/#eev

bug#13691: 24.3.50; Problem with glyphs in unibyte buffers

[Eduardo Ochs]

It seems that the bug is solved in Emacs26. I don't have Emacs27
or git Emacs in this machine, but I will compile them tomorrow
and check.

The shell script that I used to do the test is below (for the sake of
completeness). It creates two very similar files that only differ in
their "coding:" lines and then opens them with "emacs26 -Q". After
opening each of them you will have to run `M-x eval-buffer' and then
do a `C-x C-e' after the `(insert ...)' line close to the end.

Just a curiosity... is there a standard(-ish) way to send a
script like this to a shell buffer? I use eev all the time and in
eev this is trivial - see:

  http://angg.twu.net/eev-intros/find-eepitch-intro.html#1

but sometimes I need to send scripts and tests like this to
people that don't use eev...

  Cheers, thanks, etc,
    Eduardo Ochs
    http://angg.twu.net/emacsconf2019.html


Here is the script...




cd /tmp/
rm -fv /tmp/glyph-bug*

cat > glyph-bug-raw-text <<'%%%%%%%%'
(defun eepitch-set-glyph (pos &optional char face)
  (aset standard-display-table pos
        (if char (vector (make-glyph-code char face)))))

(defun eepitch-set-glyph-8bit (pos &optional char face)
  (aset standard-display-table (make-char 'eight-bit pos)
        (if char (vector (make-glyph-code char face)))))

(if (not standard-display-table)
    (setq standard-display-table (make-display-table))
  )

(defface eev-glyph-face-green '((t :foreground "green")) "")
(eepitch-set-glyph      171 171 'eev-glyph-face-green)
(eepitch-set-glyph-8bit 171 171 'eev-glyph-face-green)
(eepitch-set-glyph      187 187 'eev-glyph-face-green)
(eepitch-set-glyph-8bit 187 187 'eev-glyph-face-green)

(defface eev-glyph-face-math  '((t :foreground "RoyalBlue2"
                                   :background "gray20")) "")
(eepitch-set-glyph      191 8711 'eev-glyph-face-math)
(eepitch-set-glyph-8bit 191 8711 'eev-glyph-face-math)

;; After doing an M-x eval-buffer,
;; run these tests "by hand" with C-x C-e:
;; (emacs-version)
;; (make-char 'eight-bit 191)
;; (insert "\n;; " 171 187 191)

;; Local Variables:
;; coding:  raw-text-unix
;; mode:    emacs-lisp
;; End:
%%%%%%%%



cat > glyph-bug-utf-8 <<'%%%%%%%%'
(defun eepitch-set-glyph (pos &optional char face)
  (aset standard-display-table pos
        (if char (vector (make-glyph-code char face)))))

(defun eepitch-set-glyph-8bit (pos &optional char face)
  (aset standard-display-table (make-char 'eight-bit pos)
        (if char (vector (make-glyph-code char face)))))

(if (not standard-display-table)
    (setq standard-display-table (make-display-table))
  )

(defface eev-glyph-face-green '((t :foreground "green")) "")
(eepitch-set-glyph      171 171 'eev-glyph-face-green)
(eepitch-set-glyph-8bit 171 171 'eev-glyph-face-green)
(eepitch-set-glyph      187 187 'eev-glyph-face-green)
(eepitch-set-glyph-8bit 187 187 'eev-glyph-face-green)

(defface eev-glyph-face-math  '((t :foreground "RoyalBlue2"
                                   :background "gray20")) "")
(eepitch-set-glyph      191 8711 'eev-glyph-face-math)
(eepitch-set-glyph-8bit 191 8711 'eev-glyph-face-math)

;; After doing an M-x eval-buffer,
;; run these tests "by hand" with C-x C-e:
;; (emacs-version)
;; (make-char 'eight-bit 191)
;; (insert "\n;; " 171 187 191)

;; Local Variables:
;; coding:  utf-8-unix
;; mode:    emacs-lisp
;; End:
%%%%%%%%


~/bigsrc/emacs26/src/emacs -Q -bg black -fg bisque /tmp/glyph-bug-raw-text
~/bigsrc/emacs26/src/emacs -Q -bg black -fg bisque /tmp/glyph-bug-utf-8
/usr/bin/emacs -Q -bg black -fg bisque /tmp/glyph-bug-raw-text
/usr/bin/emacs -Q -bg black -fg bisque /tmp/glyph-bug-utf-8

On Sun, 30 Aug 2020 at 23:32, Stefan Kangas <stefan@marxist.se> wrote:
>
> Hi Eduardo,
>
> You sent the below description to the Emacs bug list 7 years ago.
>
> The email you sent is quite long, and I'm not sure what exactly the
> issue is, or how to try to reproduce it.
>
> Can you reproduce the issue on a recent version of Emacs, such as the
> recently released version 27.1?
>
> If yes, could you please provide a minimal step-by-step recipe for
> reproducing it, starting from "emacs -Q"?
>
> If I don't hear back from you within a couple of weeks, I'll just
> close this bug as unreproducible.
>
> Thanks in advance.
>
> Best regards,
> Stefan Kangas
>
> Eduardo Ochs <eduardoochs@gmail.com> writes:
>
> > Hi Andreas,
> >
> > On Tue, Feb 12, 2013 at 7:02 AM, Andreas Schwab <schwab@linux-m68k.org> wrote:
> >>
> >> Unibyte characters are in the eight-bit charset, so you need to set the
> >> display of the corresponding character in that charset, eg. (make-char
> >> 'eight-bit 171).
> >
> > Thanks, that worked!
> >
> > Now for something slightly trickier... =|
> >
> > Let's start by using several simple, low-level functions, one for
> > each range... a smarter function with `cond's can be left for
> > later. The code below makes the green guillemets work both in
> > unibyte and multibyte buffers,
> >
> >   (defun eepitch-set-glyph (pos &optional char face)
> >     (aset standard-display-table pos
> >         (if char (vector (make-glyph-code char face)))))
> >
> >   (defun eepitch-set-glyph-8bit (pos &optional char face)
> >     (aset standard-display-table (make-char 'eight-bit pos)
> >         (if char (vector (make-glyph-code char face)))))
> >
> >   (if (not standard-display-table)
> >       (setq standard-display-table (make-display-table))
> >     )
> >
> >   (defface eev-glyph-face-green '((t :foreground "green")) "")
> >   (eepitch-set-glyph      171 171 'eev-glyph-face-green)
> >   (eepitch-set-glyph-8bit 171 171 'eev-glyph-face-green)
> >   (eepitch-set-glyph      187 187 'eev-glyph-face-green)
> >   (eepitch-set-glyph-8bit 187 187 'eev-glyph-face-green)
> >
> > but now suppose that we want the char 191 to be displayed as a
> > blue nabla (unicode: 8711). Running
> >
> >   (defface eev-glyph-face-math  '((t :foreground "RoyalBlue2"
> >                                    :background "gray20")) "")
> >   (eepitch-set-glyph      191 8711 'eev-glyph-face-math)
> >   (eepitch-set-glyph-8bit 191 8711 'eev-glyph-face-math)
> >
> > _almost_ does the job, but look at the new screenshot - it seems
> > that the `char' argument in `eepitch-set-glyph-8bit' needs some
> > translation too... is that right? Which translation?
> >
> >   Cheers, and thanks in advance again...
> >     Eduardo Ochs
> >     eduardoochs@gmail.com
> >     http://angg.twu.net/#eev

bug#13691: 24.3.50; Problem with glyphs in unibyte buffers

[Stefan Kangas]

Eduardo Ochs <eduardoochs@gmail.com> writes:

> It seems that the bug is solved in Emacs26. I don't have Emacs27
> or git Emacs in this machine, but I will compile them tomorrow
> and check.
>
> The shell script that I used to do the test is below (for the sake of
> completeness). It creates two very similar files that only differ in
> their "coding:" lines and then opens them with "emacs26 -Q". After
> opening each of them you will have to run `M-x eval-buffer' and then
> do a `C-x C-e' after the `(insert ...)' line close to the end.

Thanks, please report back with your findings.  (Chances are the bug is
fixed if it works on Emacs 26, but it's always good to know for sure.)

> Just a curiosity... is there a standard(-ish) way to send a
> script like this to a shell buffer? I use eev all the time and in
> eev this is trivial - see:
>
>   http://angg.twu.net/eev-intros/find-eepitch-intro.html#1
>
> but sometimes I need to send scripts and tests like this to
> people that don't use eev...

I'm not aware of anything like that, sorry.

bug#13691: 24.3.50; Problem with glyphs in unibyte buffers

[Stefan Kangas]

Stefan Kangas <stefan@marxist.se> writes:

> Eduardo Ochs <eduardoochs@gmail.com> writes:
>
>> It seems that the bug is solved in Emacs26. I don't have Emacs27
>> or git Emacs in this machine, but I will compile them tomorrow
>> and check.
>>
>> The shell script that I used to do the test is below (for the sake of
>> completeness). It creates two very similar files that only differ in
>> their "coding:" lines and then opens them with "emacs26 -Q". After
>> opening each of them you will have to run `M-x eval-buffer' and then
>> do a `C-x C-e' after the `(insert ...)' line close to the end.
>
> Thanks, please report back with your findings.  (Chances are the bug is
> fixed if it works on Emacs 26, but it's always good to know for sure.)

That was 12 weeks ago.  Did you have time to test if the bug is fixed on
Emacs 27?

If I don't hear back from you within a couple of weeks, I'll just assume
this has been fixed and close this bug.  Thanks in advance.

bug#13691: 24.3.50; Problem with glyphs in unibyte buffers

[Stefan Kangas]

Stefan Kangas <stefan@marxist.se> writes:

> Stefan Kangas <stefan@marxist.se> writes:
>
>> Eduardo Ochs <eduardoochs@gmail.com> writes:
>>
>>> It seems that the bug is solved in Emacs26. I don't have Emacs27
>>> or git Emacs in this machine, but I will compile them tomorrow
>>> and check.
>>>
>>> The shell script that I used to do the test is below (for the sake of
>>> completeness). It creates two very similar files that only differ in
>>> their "coding:" lines and then opens them with "emacs26 -Q". After
>>> opening each of them you will have to run `M-x eval-buffer' and then
>>> do a `C-x C-e' after the `(insert ...)' line close to the end.
>>
>> Thanks, please report back with your findings.  (Chances are the bug is
>> fixed if it works on Emacs 26, but it's always good to know for sure.)
>
> That was 12 weeks ago.  Did you have time to test if the bug is fixed on
> Emacs 27?
>
> If I don't hear back from you within a couple of weeks, I'll just assume
> this has been fixed and close this bug.  Thanks in advance.

No reply within 5 weeks, so I'm assuming that this has been fixed and
I'm closing this bug.

If this is still an issue, please reply to this email (use "Reply to
all" in your email client) and we can reopen the bug report.