bug#44672: 28.0.50; 'ç' key on BR-ABNT2 keyboard translates to strange symbols

bug#44672: 28.0.50; 'ç' key on BR-ABNT2 keyboard translates to strange symbols

[YohananDiamond]

[-- Attachment #1: Type: text/plain, Size: 6649 bytes --]

Hello! I use emacs with evil-mode, a minor mode package that adds vim-ish
keybindings for navigation and editing text. I have a BR-ABNT2 keyboard
and noticed my workflow would get more efficient if I bound the ç key to
what the : key would normally do. And so I did. But I noticed that, if I
enter a terminal frame and the first key to be typed is ç, strange
characters appear on the screen (usually a combination of Ã, § and
\247). This is probably an encoding-related problem but I don't know how
to proceed.

Note: I've tested the bug on a fresh emacsclient (emacs --daemon -Q &&
emacsclient -t) and a standalone emacs terminal frame (emacs -Q -nw) and
it happens in both, even without loading any personal init.el file.

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.21,
cairo version 1.17.3)
 of 2020-07-31 built on core
Repository revision: fbe363f5e18fb4975de168191f1ee9b686e48d9f
Repository branch: master
System Description: Arch Linux

Recent messages:
Cleaning up the recentf list...done (6 removed)
Unable to load color "brightpurple"
Unable to load color "brightred" [2 times]
Unable to load color "brightcyan"
Unable to load color "brightpurple" [2 times]
Unable to load color "brightred" [2 times]
Quit
Quit
Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --mandir=/usr/share/man --with-gameuser=:games
 --with-sound=alsa --with-modules --without-gconf --without-gsettings
 --with-x-toolkit=gtk3 --without-xaw3d --without-compress-install
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -flto'
 CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GLIB NOTIFY INOTIFY ACL
GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON
PDUMPER LCMS2

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Text

Minor modes in effect:
  recentf-mode: t
  global-hl-line-mode: t
  show-paren-mode: t
  electric-pair-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  global-evil-surround-mode: t
  evil-surround-mode: t
  evil-commentary-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  evil-mode: t
  evil-local-mode: t
  xclip-mode: t
  which-key-mode: t
  ivy-rich-mode: t
  ivy-mode: t
  global-magit-file-mode: t
  magit-auto-revert-mode: t
  global-git-commit-mode: t
  async-bytecomp-package-mode: t
  shell-dirtrack-mode: t
  global-auto-complete-mode: t
  rainbow-delimiters-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  global-visual-line-mode: t
  visual-line-mode: t
  transient-mark-mode: t

Load-path shadows:
/usr/share/emacs/site-lisp/xscheme hides
/usr/share/emacs/28.0.50/lisp/progmodes/xscheme

Features:
(shadow sort mail-extr emacsbug sendmail mule-util recentf tree-widget
wid-edit main base16-theme conf-general ace-window avy hl-line paren
elec-pair display-line-numbers conf-kb conf-fmodes cc-styles cc-align
cc-engine cc-vars cc-defs conf-evil evil-magit evil-surround
evil-commentary evil-commentary-integration evil evil-keybindings
evil-integration undo-tree evil-maps evil-commands reveal flyspell
ispell evil-jumps evil-command-window evil-types evil-search evil-ex
evil-macros evil-repeat evil-states evil-core evil-common windmove
thingatpt rect evil-digraphs evil-vars conf-packages xclip origami
origami-parsers rx cl which-key ivy-rich counsel xdg xref project
compile swiper ivy delsel ivy-faces ivy-overlay colir color
magit-submodule magit-obsolete magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log which-func imenu
magit-diff smerge-mode diff diff-mode magit-core magit-autorevert
autorevert filenotify magit-margin magit-transient magit-process
magit-mode git-commit transient format-spec magit-git magit-section
magit-utils crm log-edit message rmc puny dired dired-loaddefs rfc822
mml mml-sec epa derived epg epg-config gnus-util rmail rmail-loaddefs
text-property-search time-date mm-decode mm-bodies mm-encode mail-parse
rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev
mail-utils gmm-utils mailheader pcvs-util add-log with-editor
async-bytecomp advice async shell pcomplete comint ansi-color ring
server auto-complete-config auto-complete edmacro kmacro popup
rainbow-delimiters visual-regexp auto-package-update easy-mmode bind-map
core-rifle core-misc pcase core-style core-packages f dash s cl-extra
help-mode use-package-ensure use-package-core core-init finder-inf info
package easymenu browse-url url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
url-vars seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting font-render-setting cairo move-toolbar gtk
x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 357204 37428)
 (symbols 48 28422 2)
 (strings 32 114042 2952)
 (string-bytes 1 3307905)
 (vectors 16 40702)
 (vector-slots 8 448939 14850)
 (floats 8 203 510)
 (intervals 56 317 13)
 (buffers 992 11))

[-- Attachment #2: Type: text/html, Size: 7107 bytes --]

bug#44672: 28.0.50; 'ç' key on BR-ABNT2 keyboard translates to strange symbols

[Lars Ingebrigtsen]

YohananDiamond <mitonanan12@gmail.com> writes:

> Hello! I use emacs with evil-mode, a minor mode package that adds vim-ish
> keybindings for navigation and editing text. I have a BR-ABNT2 keyboard
> and noticed my workflow would get more efficient if I bound the ç key to
> what the : key would normally do. And so I did. But I noticed that, if I
> enter a terminal frame and the first key to be typed is ç, strange
> characters appear on the screen (usually a combination of Ã, § and
> \247). This is probably an encoding-related problem but I don't know how
> to proceed.
>
> Note: I've tested the bug on a fresh emacsclient (emacs --daemon -Q &&
> emacsclient -t) and a standalone emacs terminal frame (emacs -Q -nw) and
> it happens in both, even without loading any personal init.el file.

Do you have a step-by-step recipe for how to reproduce this bug,
starting from "emacs -Q"?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44672: 28.0.50; 'ç' key on BR-ABNT2 keyboard translates to strange symbols

[Lars Ingebrigtsen]

(Please keep the debbugs address in the Cc headers.)

YohananDiamond <mitonanan12@gmail.com> writes:

> Sure:
>
> - launch emacs on a terminal via "emacs -Q -nw"
> - insert the "ç" character on the terminal's standard input

I'm now sure what you mean by the latter.

> Remember that this is tested on a BR-ABNT2 keyboard layout, so it might be
> possible that you can't type it. Pasting the character seems to have the same
> effect, though, and some characters output similar gibberish: ä, ß, ð, đ, ŋ.

I started

emacs -Q -nw

and then pasted a "ç" character into the *scratch* buffer, and it looked
OK to me.

Did you mean something else?

bug#44672: 28.0.50; 'ç' key on BR-ABNT2 keyboard translates to strange symbols

[Lars Ingebrigtsen]

(Please keep the debbugs address in the Cc headers.)

YohananDiamond <mitonanan12@gmail.com> writes:

> That was supposed to be it. Hmm...

So pasting (with the mouse) a "ç" character into your "emacs -Q -nw"
window looks odd?  Do you have a screenshot?

> Are you using the same version as I? (GNU Emacs 28.0.50)

Yes.

bug#44672: 28.0.50; 'ç' key on BR-ABNT2 keyboard translates to strange symbols

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> So pasting (with the mouse) a "ç" character into your "emacs -Q -nw"
> window looks odd?  Do you have a screenshot?

More information was requested, but no response was given within a
month, so I'm closing this bug report.  If the problem still exists,
please respond to this email and we'll reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44672: 28.0.50; 'ç' key on BR-ABNT2 keyboard translates to strange symbols

[Yohanan]

[-- Attachment #1.1: Type: text/plain, Size: 1822 bytes --]

Oh, I'm sorry. I actually sent a reply, not sure why it didn't go through.
If this message gets through, I think it is because I accidentally removed
the debuggs address from the Cc headers, as you seem to have stated on
previous posts.
[image: image.png]
After pressing any other key (or even unfocusing the terminal window,
another character shows up: `\247`:
[image: image.png]
I have found similar "ç to Ã" issues on old webpages with bad encoding,
which makes me wonder if this is an issue with emacs understanding which
encoding I'm using.
Try out the following HTML page:
    <meta charset="ISO-8859-1">
    ç
On my machine, opening a HTML file with this code shows:
    ç

I've also noticed something else: I have a LANG environment variable that
has its value set to `en_US.UTF-8`. After I unset it, doing the same thing
(pasting ç onto the terminal while emacs is open) yielded a dash (`-`)
instead of the `Ã` character:
[image: image.png]
I also experimented with the LC_ALL environment variable, and the result of
pasting the `ç` character onto a `LANG=en_US.UTF8 LC_ALL=en_US emacs -Q
-nw` emacs instance is a `ç` (yay!) but followed by what seems to be a
space character:
[image: image.png]

On Sat, Jul 31, 2021 at 9:49 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
> > So pasting (with the mouse) a "ç" character into your "emacs -Q -nw"
> > window looks odd?  Do you have a screenshot?
>
> More information was requested, but no response was given within a
> month, so I'm closing this bug report.  If the problem still exists,
> please respond to this email and we'll reopen the bug report.
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no
>

[-- Attachment #1.2: Type: text/html, Size: 2716 bytes --]

[-- Attachment #2: image.png --]
[-- Type: image/png, Size: 21108 bytes --]

[-- Attachment #3: image.png --]
[-- Type: image/png, Size: 21211 bytes --]

[-- Attachment #4: image.png --]
[-- Type: image/png, Size: 22846 bytes --]

[-- Attachment #5: image.png --]
[-- Type: image/png, Size: 23025 bytes --]

bug#44672: 28.0.50; 'ç' key on BR-ABNT2 keyboard translates to strange symbols

[Lars Ingebrigtsen]

Yohanan <mitonanan12@gmail.com> writes:

> Oh, I'm sorry. I actually sent a reply, not sure why it didn't go through.

OK; reopening this bug report, then...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44672: 28.0.50; 'ç' key on BR-ABNT2 keyboard translates to strange symbols

[Lars Ingebrigtsen]

Yohanan <mitonanan12@gmail.com> writes:

> Try out the following HTML page:
>     <meta charset="ISO-8859-1">
>     ç
> On my machine, opening a HTML file with this code shows:
>     ç

Have you examined the file and confirmed that it's not really encoded in
utf-8?  It sounds unlikely that it's not, because that's exactly the
result you'd get if that's the case.

> I also experimented with the LC_ALL environment variable, and the result of
> pasting the `ç` character onto a `LANG=en_US.UTF8 LC_ALL=en_US emacs -Q
> -nw` emacs instance is a `ç` (yay!) but followed by what seems to be a space
> character:

As for the other issues, it kinda sounds like your terminal is confused
as to what input methods and language environments it's running in.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44672: 28.0.50; 'ç' key on BR-ABNT2 keyboard translates to strange symbols

[Yohanan]

[-- Attachment #1: Type: text/plain, Size: 1705 bytes --]

> Have you examined the file and confirmed that it's not really encoded in
> utf-8?  It sounds unlikely that it's not, because that's exactly the
result
> you'd get if that's the case.

I actually meant "open with the web browser", not on emacs. I didn't
specify it
properly so my bad here :P

> As for the other issues, it kinda sounds like your terminal is confused
> as to what input methods and language environments it's running in.

Oh. I was using `st` and after reading this I tested it in `alacritty` and
`xterm`. It works on these, so it's probably an issue with my `st`.

Since it's probably not a bug with emacs, I think we can close it here.
Thanks
for the help!

On Wed, Aug 4, 2021 at 3:36 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Yohanan <mitonanan12@gmail.com> writes:
>
> > Try out the following HTML page:
> >     <meta charset="ISO-8859-1">
> >     ç
> > On my machine, opening a HTML file with this code shows:
> >     ç
>
> Have you examined the file and confirmed that it's not really encoded in
> utf-8?  It sounds unlikely that it's not, because that's exactly the
> result you'd get if that's the case.
>
> > I also experimented with the LC_ALL environment variable, and the result
> of
> > pasting the `ç` character onto a `LANG=en_US.UTF8 LC_ALL=en_US emacs -Q
> > -nw` emacs instance is a `ç` (yay!) but followed by what seems to be a
> space
> > character:
>
> As for the other issues, it kinda sounds like your terminal is confused
> as to what input methods and language environments it's running in.
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no
>

[-- Attachment #2: Type: text/html, Size: 2370 bytes --]

bug#44672: 28.0.50; 'ç' key on BR-ABNT2 keyboard translates to strange symbols

[Lars Ingebrigtsen]

Yohanan <mitonanan12@gmail.com> writes:

> Since it's probably not a bug with emacs, I think we can close it here. Thanks
> for the help!

No problem; closing the bug report, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no