bug#19924: 24.4; incremental search for octal character

bug#19924: 24.4; incremental search for octal character

[vose]

[-- Attachment #1: Type: text/plain, Size: 7820 bytes --]


This bug -- PROBLEM 2 -- is being reported on behalf of D. Paddy
(I was unable to help solve the issue).

D. Paddy writes:

TWO (2) problems; for both I follow the GNU Emacs manual from
"https://www.gnu.org/software/emacs/manual/emacs.html"

PROBLEM 1: The following is quoted from page 91:

"To search for non-ASCII characters, use one of the following methods:
• Type C-q, followed by a non-graphic character or a sequence of octal
digits."

In my case I typed C-q, followed by 213 and then return (the enter key).
Rather than searching, the octal character \213 is inserted into the
file.

PROBLEM 2: Vose conjectured that the manual was simply mistaken, and
should have said something closer to: To search for an octal character,
type C-s C-q, followed by a sequence of octal digits.

In my case I typed C-s C-q, followed by 213 and then return (the enter
key).  Rather than searching, the message "Failing I-search: \213" was
displayed (see attached screen shot).

Vose suggested: M-x search-forward return C-q 213 return return
but that likewise fails.

Vose suggested: M-x find-file-literally, but the same problem occurs.

If this is a feature of emacs or of the documentation, then please
respond to the following plea for charity: exactly what buttons of the
keyboard do I press to successfully search for octal 213 ? And if
instead there is no problem because octal 213 is not really in the file
after all -- dispite what the screenshot seems to indicate -- and so
therefore searching for it *should fail*, then how do I change my .emacs
file so that emacs will stop lying to me about file contents?  As noted
above, the same problem occurs with M-x find-file-literally, i.e., the
buffer seems to contain octal 213 (same screen shot) but searching for
it fails.

Much obliged.

D. Paddy

P.S. The above was all using emacs -Q


In GNU Emacs 24.4.1 (x86_64-pc-linux-gnu)
 of 2015-02-09 on blackbox
 Windowing system distributor `The X.Org Foundation', version
 11.0.11502000
 Configured using:
  `configure --prefix=/usr --build=x86_64-pc-linux-gnu
   --host=x86_64-pc-linux-gnu --mandir=/usr/share/man
    --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc
     --localstatedir=/var/lib --disable-dependency-tracking
      --disable-silent-rules --libdir=/usr/lib64
 --program-suffix=-emacs-24
  --infodir=/usr/share/info/emacs-24 --localstatedir=/var
   --enable-locallisppath=/etc/emacs:/usr/share/emacs/site-lisp
    --with-gameuser=games --without-compress-install
     --with-file-notification=inotify --enable-acl --without-dbus
      --without-gnutls --without-gpm --without-hesiod --without-kerberos
       --without-kerberos5 --without-xml2 --without-selinux
 --without-wide-int
  --with-zlib --with-sound=no --with-x --without-ns --without-gconf
   --without-gsettings --without-toolkit-scroll-bars --with-gif
    --with-jpeg --with-png --with-rsvg --with-tiff --with-xpm
     --with-imagemagick --without-xft --without-libotf
 --without-m17n-flt
  --with-x-toolkit=no GENTOO_PACKAGE=app-editors/emacs-24.4-r1
   'CFLAGS=-O2 -pipe -march=native' CPPFLAGS= 'LDFLAGS=-Wl,-O1
    -Wl,--as-needed''

Important settings:
  value of $LANG: en_US.utf8
    locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  shell-dirtrack-mode: t
    tooltip-mode: t
      electric-indent-mode: t
        mouse-wheel-mode: t
          menu-bar-mode: t
            file-name-shadow-mode: t
              global-font-lock-mode: t
                blink-cursor-mode: t
                  auto-composition-mode: t
                    auto-encryption-mode: t
                      auto-compression-mode: t
                        line-number-mode: t
                          transient-mark-mode: t

Recent input:
<help-echo> m p e g <return> <help-echo> C-x b <return>
C-x C-s C-x k <return> C-x b <return> C-x k <return>
C-x b <return> C-x k <return> C-x b <return> C-x b
<return> C-q 2 7 3 C-g C-g C-g C-g C-s C-q 2 7 3 <return>
C-g C-g C-g C-_ C-s C-q 2 7 2 <return> C-g C-g C-g
C-g C-s C-q 2 7 2 C-s C-x k <return> C-x C-f v i <tab>
d <tab> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> D o <tab> w n <tab> v <backspace> f <tab>
7 <tab> v i <tab> <return> y C-g C-x C-f <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> D o <tab> w <tab> f e <tab> 7 <tab> q .
<tab> <tab> <return> C-s C-q 2 4 4 C-s C-g C-g C-g
C-s C-q 2 4 4 <return> C-g C-g C-s C-q \ 2 4 4 C-s
M-< <help-echo> C-s C-q 2 1 3 <return> C-g <help-echo>
<down-mouse-5> <mouse-5> <down-mouse-5> <mouse-5> <down-mouse-5>
<mouse-5> <down-mouse-5> <mouse-5> <down-mouse-5> <mouse-5>
<double-down-mouse-5> <double-mouse-5> <down-mouse-4>
<mouse-4> <double-down-mouse-4> <double-mouse-4> <down-mouse-4>
<mouse-4> <double-down-mouse-4> <double-mouse-4> <triple-down-mouse-4>
<triple-mouse-4> <triple-down-mouse-4> <triple-mouse-4>
<triple-down-mouse-4> <triple-mouse-4> C-a C-n C-k
C-a C-n C-k C-k C-k C-k C-k C-_ C-_ C-_ C-_ C-_ C-_
C-_ C-_ C-_ C-_ C-_ C-_ C-_ C-_ C-_ C-_ C-_ C-_ C-_
C-_ C-_ C-_ C-_ C-a C-l C-n C-l <help-echo> C-a C-s
C-q 2 1 3 <return> <help-echo> <help-echo> M-< M-<
C-x k <return> <help-echo> <menu-bar> <help-menu> <emacs-tutorial>
C-s b u g s <backspace> C-s C-s M-< <menu-bar> <help-menu>
<emacs-tutorial> <menu-bar> <help-menu> <send-emac
s-bug-report>

Recent messages:
Making completion list...
Quit [3 times]
Mark saved where search started
Mark set
byte-code: Beginning of buffer [2 times]
Undo! [6 times]
user-error: No further undo information [17 times]
Mark set [2 times]
Preparing tutorial ...
Mark saved where search started

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils tutorial hexl eldoc help-fns mule-util help-mode
easymenu ispell misearch multi-isearch tex-mode compile shell pcomplete
comint ansi-color ring latexenc site-gentoo time-date tooltip electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
inotify dynamic-setting x multi-tty emacs)

Memory information:
((conses 16 92178 45039)
 (symbols 48 19342 12)
 (miscs 40 94 665)
 (strings 32 14527 24688)
 (string-bytes 1 17114196)
 (vectors 16 16163)
 (vector-slots 8 1058778 186848)
 (floats 8 71 1128)
 (intervals 56 1097 969)
 (buffers 960 23)
 (heap 1024 70663 70634))
 


[-- Attachment #2: emacs bug (screenshot) --]
[-- Type: application/x-xcf, Size: 45824 bytes --]

bug#19924: 24.4; incremental search for octal character

[Drew Adams]

> "To search for non-ASCII characters, use one of the following methods:
> • Type C-q, followed by a non-graphic character or a sequence of octal
> digits."
> 
> In my case I typed C-q, followed by 213 and then return (the enter key).
> Rather than searching, the octal character \213 is inserted into the
> file.

Where did you read to hit RET?  Try just using `C-q 2 1 3' followed by
hitting `C-s' again.

> PROBLEM 2: Vose conjectured that the manual was simply mistaken, and
> should have said something closer to: To search for an octal character,
> type C-s C-q, followed by a sequence of octal digits.

Yes, that's what was meant: hit `C-q' while searching (i.e., after
using `C-s').

> In my case I typed C-s C-q, followed by 213 and then return (the enter
> key).  Rather than searching, the message "Failing I-search: \213" was
> displayed (see attached screen shot).

Again, don't hit RET.  Just use `C-s C-q 2 1 3 C-s'.

The doc should perhaps make it clear that what it describes doing is
to be done while searching, i.e., after using `C-s'.

bug#19924: 24.4; incremental search for octal character

[Andreas Schwab]

Drew Adams <drew.adams@oracle.com> writes:

> Again, don't hit RET.

Why not?  It's the canonical way to terminate a quoted-insert.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#19924: 24.4; incremental search for octal character

[Drew Adams]

> > Again, don't hit RET.
> 
> Why not?  It's the canonical way to terminate a quoted-insert.

Uh, because it exits Isearch?  It doesn't work (as expected)?

Am I misunderstanding something here?  If so, feel free to ignore,
and respond to the bug report in another way.

bug#19924: 24.4; incremental search for octal character

[Andreas Schwab]

Drew Adams <drew.adams@oracle.com> writes:

>> > Again, don't hit RET.
>> 
>> Why not?  It's the canonical way to terminate a quoted-insert.
>
> Uh, because it exits Isearch?

Does it?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#19924: 24.4; incremental search for octal character

[Drew Adams]

> >> > Again, don't hit RET.
> >> Why not?  It's the canonical way to terminate a quoted-insert.
> > Uh, because it exits Isearch?
> Does it?

No, apparently not, in this context - my bad.  My Emacs session
must have been in an altered state when it showed that behavior.

So presumably the only problem was the OP not understanding that
`C-q 213' (with or without `RET' after it) is done while searching,
i.e., after `C-s'.

So again, perhaps the doc should make sure that this is clear.

bug#19924: 24.4; incremental search for octal character

[Juri Linkov]

> So presumably the only problem was the OP not understanding that
> `C-q 213' (with or without `RET' after it) is done while searching,
> i.e., after `C-s'.

As the screenshot shows, the problem is that the search doesn't find
the character 213 in the `no-conversion' (binary) buffer, because
there is a mismatch between the coding of the buffer and the search.

bug#19924: 24.4; incremental search for octal character

[Drew Adams]

[-- Attachment #1: Type: text/plain, Size: 1628 bytes --]

Forwarding - The OP (Michael) left off the bug list as recipient.

> -----Original Message-----
> From: vose@eecs.utk.edu [mailto:vose@eecs.utk.edu]
> Sent: Thursday, March 05, 2015 2:33 PM
> To: schwab@linux-m68k.org; Drew Adams
> Cc: vose@eecs.utk.edu
> Subject: RE: bug#19924: 24.4; incremental search for octal character
> 
> I appologize for not being able to understand the emails youall passed
> my way.  Here is the situation:
> 
> 1) I bring up emacs : emacs -Q
> 
> 2) I open DPaddy's file : Control x f ~/Downloads/feb17/emacs-broken
> 
> 3) I see \213 in the buffer (see screenshot-1.jpg).
> 
> 4) I type : `C-s C-q 213 C-q' (i.e., Control s Control q 213 Control s)
>    but the search fails (see screenshot-2.jpg)
> 
> 4) I type : `C-s C-q 213 ret' (i.e., Control s Control q 213 return)
>    and get the same behavior (same as screenshot-2.jpg).
> 
> 6) I type `M-x hexl-mode ret' (i.e., Meta x hexl-mode return) and see
>    octal 213 (i.e., hex 8B) is in the buffer (see screenshot-3.jpg).
> 
> I anticipate that the following is an explanation of some kind:
> 
>  "As the screenshot shows, the problem is that the search doesn't find
>   the character 213 in the `no-conversion' (binary) buffer, because
>   there is a mismatch between the coding of the buffer and the search."
> 
> But I don't know what the above means.  In particular, I don't know how
> to search for octal 213.  Given my current (lack of) understanding, it
> seems that emacs is broken, or searching in octal is not possible.
> 
> Please give a(nother) clue...
> 
> Much obliged,
> 
> Michael
> 

[-- Attachment #2: screenshot-1.jpg --]
[-- Type: image/jpeg, Size: 118691 bytes --]

[-- Attachment #3: screenshot-2.jpg --]
[-- Type: image/jpeg, Size: 132254 bytes --]

[-- Attachment #4: screenshot-3.jpg --]
[-- Type: image/jpeg, Size: 105226 bytes --]

bug#19924: 24.4; incremental search for octal character

[Eli Zaretskii]

> Date: Thu, 5 Mar 2015 14:54:55 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: vose@eecs.utk.edu
> 
> > I anticipate that the following is an explanation of some kind:
> > 
> >  "As the screenshot shows, the problem is that the search doesn't find
> >   the character 213 in the `no-conversion' (binary) buffer, because
> >   there is a mismatch between the coding of the buffer and the search."
> > 
> > But I don't know what the above means.  In particular, I don't know how
> > to search for octal 213.  Given my current (lack of) understanding, it
> > seems that emacs is broken, or searching in octal is not possible.

It means that we have a missing feature: we don't have any reasonable
way of typing unibyte characters at Isearch's prompt.  We need to
provide one.  There was such a kludgey feature in the past, but it
conflicted with a much more useful possibility of inserting Unicode
codepoints with C-q, and so the kludge was deleted in one of the
previous versions.  We need to restore it, at least for when Isearch
was initiated from a unibyte buffer.

Another possibility would be to have an input method for that.

(The manual isn't wrong, strictly speaking: it says "non-ASCII
characters", not "raw bytes in unibyte buffers".)

bug#19924: 24.4; incremental search for octal character

[Drew Adams]

> It means that we have a missing feature: we don't have any reasonable
> way of typing unibyte characters at Isearch's prompt.  We need to
> provide one.  There was such a kludgey feature in the past, but it
> conflicted with a much more useful possibility of inserting Unicode
> codepoints with C-q, and so the kludge was deleted in one of the
> previous versions.  We need to restore it, at least for when Isearch
> was initiated from a unibyte buffer.

Huh? AFAIK, since Emacs 24.4 you can use `C-x 8 RET' to append a
Unicode char to the search string.

I don't have a recent dev build (since Oct. 2014), but in 24.4 that
certainly works.  (Dunno about using a unibyte buffer, though, so
maybe I'm missing your point.  In `emacs -Q' (GUI, not terminal),
it works fine for me.

If you have the Unicode character in the kill ring then you can
also use `M-y'.  And you can also use `M-e' followed by `C-x 8 RET',
of course.

bug#19924: 24.4; incremental search for octal character

[Dani Moncayo]

> I don't have a recent dev build (since Oct. 2014),

??  There are newer dev builds at

  https://sourceforge.net/projects/emacs-bin/files/snapshots/


-- 
Dani Moncayo

bug#19924: 24.4; incremental search for octal character

[Drew Adams]

> > I don't have a recent dev build (since Oct. 2014),
> 
> ??  There are newer dev builds at
> 
>   https://sourceforge.net/projects/emacs-bin/files/snapshots/

Hi Dani,

Yes, thanks.  My bad - I wasn't clear.  I still use a build from
2014/10/20 because later builds break my use of thumbnail frames.

It's still in my TODO list to narrow down the problem and file a
proper bug report about the regression.  I don't see the problem
from emacs -Q plus just my thumbnail code - there must be
something else in my setup that is causing the problem.  Maybe
I'll get some time this weekend to look into it.

bug#19924: 24.4; incremental search for octal character

[Eli Zaretskii]

> Date: Fri, 6 Mar 2015 08:27:33 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 19924@debbugs.gnu.org, vose@eecs.utk.edu
> 
> > It means that we have a missing feature: we don't have any reasonable
> > way of typing unibyte characters at Isearch's prompt.  We need to
> > provide one.  There was such a kludgey feature in the past, but it
> > conflicted with a much more useful possibility of inserting Unicode
> > codepoints with C-q, and so the kludge was deleted in one of the
> > previous versions.  We need to restore it, at least for when Isearch
> > was initiated from a unibyte buffer.
> 
> Huh? AFAIK, since Emacs 24.4 you can use `C-x 8 RET' to append a
> Unicode char to the search string.

Raw bytes are not Unicode characters.  Try searching in the file the
OP attached to the original bug report, after you visit that file with
find-file-literally, and you will see the problem.  Emacs interprets
what you type after C-q or "C-x RET 8" as a Unicode codepoint, and
thus doesn't find raw bytes it has in unibyte buffers.

> If you have the Unicode character in the kill ring then you can
> also use `M-y'.  And you can also use `M-e' followed by `C-x 8 RET',
> of course.

All true, but not really relevant to the OP's problem.

bug#19924: 24.4; incremental search for octal character

[Drew Adams]

> > > It means that we have a missing feature: we don't have any reasonable
> > > way of typing unibyte characters at Isearch's prompt.  We need to
> > > provide one.
> >
> > Huh? AFAIK, since Emacs 24.4 you can use `C-x 8 RET' to append a
> > Unicode char to the search string.
> 
> Raw bytes are not Unicode characters.

Sorry, I misread "unibyte characters" as "Unicode characters" in
what you wrote.  Please ignore.

bug#19924: 24.4; incremental search for octal character

[Lars Ingebrigtsen]

vose@eecs.utk.edu writes:

> TWO (2) problems; for both I follow the GNU Emacs manual from
> "https://www.gnu.org/software/emacs/manual/emacs.html"
>
> PROBLEM 1: The following is quoted from page 91:
>
> "To search for non-ASCII characters, use one of the following methods:
> • Type C-q, followed by a non-graphic character or a sequence of octal
> digits."
>
> In my case I typed C-q, followed by 213 and then return (the enter key).
> Rather than searching, the octal character \213 is inserted into the
> file.
>
> PROBLEM 2: Vose conjectured that the manual was simply mistaken, and
> should have said something closer to: To search for an octal character,
> type C-s C-q, followed by a sequence of octal digits.

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

Yes, that's what the manual means.  The section starts with "In addition
to characters described in the previous subsections, some of the other
characters you type during incremental search have special effects." and
all the keys described are during incremental search.

But I've clarified this in the Emacs 29 manual

> In my case I typed C-s C-q, followed by 213 and then return (the enter
> key).  Rather than searching, the message "Failing I-search: \213" was
> displayed (see attached screen shot).

Yes -- you probably didn't have that byte in the buffer?  For
instance, having this character in the buffer:

®

And then C-s C-q 2 5 6 RET works fine for me (at least in Emacs 29).

So it seems like things are working as designed here, and I'm therefore
closing this bug report.  If I misunderstood what the problem was about,
please respond to the debbugs address and we'll reopen.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no