bug#47534: Subject: 28.0.50; Regexp lower case pattern matches upper case

bug#47534: Subject: 28.0.50; Regexp lower case pattern matches upper case

[dalanicolai]

[-- Attachment #1.1: Type: text/plain, Size: 3938 bytes --]

When using the regexp builder and trying to match only lower case
letters (and spaces), Emacs also includes upper case matches. See attached
image (the regexp builder syntax in the image is set to string, but it
happens with other syntaxes too)
[image: image.png]

Also, I am unable to find in the manual any information about the option of
setting different syntaxes in the regexp builder (also the option is not
mentioned in the regexp-builder docstring). So I would additionally like to
report this as a documentation bug. (I am reporting it here, because maybe
I did not search clever enough, but anyway then it is really hard to find
for me.
Let me know if you would like me to report this separately.)

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.25,
cairo version 1.16.0)
 of 2021-02-18 built on daniel-fedora
Repository revision: 185121da6978553d538d37d6d0e67dc52e13311f
Repository branch: feature/native-comp
Windowing system distributor 'The X.Org Foundation', version 11.0.12010000
System Description: Fedora 34 (Workstation Edition Prerelease)

Configured using:
 'configure --with-nativecomp'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP
NOTIFY INOTIFY PDUMPER PNG RSVG SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
X11 XDBE XIM XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=none
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(mailalias mailclient browse-url url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util url-parse
url-vars mailcap help-mode pp shadow sort mail-extr emacsbug message rmc
puny dired dired-loaddefs rfc822 mml easymenu mml-sec epa derived epg
epg-config gnus-util rmail rmail-loaddefs auth-source cl-seq eieio
eieio-core eieio-loaddefs password-cache json map cl-macs
text-property-search seq byte-opt gv bytecomp byte-compile cconv
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr
mail-utils time-date subr-x re-builder rx thingatpt cl-loaddefs cl-lib
iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face pcase macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo move-toolbar gtk x-toolkit x multi-tty make-network-process
nativecomp emacs)

Memory information:
((conses 16 94092 14675)
 (symbols 48 7774 2)
 (strings 32 25567 2178)
 (string-bytes 1 959120)
 (vectors 16 17517)
 (vector-slots 8 362281 18788)
 (floats 8 34 184)
 (intervals 56 230 0)
 (buffers 992 11))

[-- Attachment #1.2: Type: text/html, Size: 4396 bytes --]

[-- Attachment #2: image.png --]
[-- Type: image/png, Size: 199734 bytes --]

bug#47534: Subject: 28.0.50; Regexp lower case pattern matches upper case

[Basil L. Contovounesios]

dalanicolai <dalanicolai@gmail.com> writes:

> When using the regexp builder and trying to match only lower case
> letters (and spaces), Emacs also includes upper case matches.

I think this is the effect of the user option case-fold-search, which
defaults to non-nil (this is the case across most search-related parts
of Emacs, including Isearch).  You can toggle it in re-builder
specifically with C-c C-c (reb-toggle-case), or across all buffers by
customising case-fold-search to be nil.

> Also, I am unable to find in the manual any information about the
> option of setting different syntaxes in the regexp builder

I think most of the documentation for re-builder.el is in its commentary
at the start of the file; see M-x find-library RET re-builder RET.

I haven't used re-builder much, but from M-x customize-group RET
re-builder RET I see there is a user option reb-re-syntax for
controlling the default syntax.  C-h m in the re-builder buffer further
reveals that the command C-c C-i (reb-change-syntax) can modify
reb-re-syntax on the fly.

> (also the option is not mentioned in the regexp-builder docstring). So
> I would additionally like to report this as a documentation bug.

Are you referring to the docstring of the re-builder command, or
something else?  And is it a listing of key bindings you would like to
see, or something else?  Would you like to provide a patch with
suggestions for improvements?

Thanks,

-- 
Basil

bug#47534: Subject: 28.0.50; Regexp lower case pattern matches upper case

[dalanicolai]

[-- Attachment #1: Type: text/plain, Size: 2101 bytes --]

Ah thanks. I thought I used case sensitive replace-regexp before, but
probably I am mistaking then. Section "34.3 Regular Expressions"  of the
Emacs manual states that "[a-z]" or "[:lower:]" should match only lower
case, while it is not mentioned at all that it will not work by default
(case-fold-search as well as reb-toggle-case and reb-change-syntax are not
mentioned at all in that section as far as I can find). Indeed I was
referring to the docstring of re-builder. Anyway, I am happy to provide a
patch.



On Thu, 1 Apr 2021 at 00:25, Basil L. Contovounesios <contovob@tcd.ie>
wrote:

> dalanicolai <dalanicolai@gmail.com> writes:
>
> > When using the regexp builder and trying to match only lower case
> > letters (and spaces), Emacs also includes upper case matches.
>
> I think this is the effect of the user option case-fold-search, which
> defaults to non-nil (this is the case across most search-related parts
> of Emacs, including Isearch).  You can toggle it in re-builder
> specifically with C-c C-c (reb-toggle-case), or across all buffers by
> customising case-fold-search to be nil.
>
> > Also, I am unable to find in the manual any information about the
> > option of setting different syntaxes in the regexp builder
>
> I think most of the documentation for re-builder.el is in its commentary
> at the start of the file; see M-x find-library RET re-builder RET.
>
> I haven't used re-builder much, but from M-x customize-group RET
> re-builder RET I see there is a user option reb-re-syntax for
> controlling the default syntax.  C-h m in the re-builder buffer further
> reveals that the command C-c C-i (reb-change-syntax) can modify
> reb-re-syntax on the fly.
>
> > (also the option is not mentioned in the regexp-builder docstring). So
> > I would additionally like to report this as a documentation bug.
>
> Are you referring to the docstring of the re-builder command, or
> something else?  And is it a listing of key bindings you would like to
> see, or something else?  Would you like to provide a patch with
> suggestions for improvements?
>
> Thanks,
>
> --
> Basil
>

[-- Attachment #2: Type: text/html, Size: 2655 bytes --]

bug#47534: Subject: 28.0.50; Regexp lower case pattern matches upper case

[dalanicolai]

[-- Attachment #1.1: Type: text/plain, Size: 2322 bytes --]

Here is a small patch for the re-builder docstring. I hope I did things
correctly...

On Thu, 1 Apr 2021 at 10:12, dalanicolai <dalanicolai@gmail.com> wrote:

> Ah thanks. I thought I used case sensitive replace-regexp before, but
> probably I am mistaking then. Section "34.3 Regular Expressions"  of the
> Emacs manual states that "[a-z]" or "[:lower:]" should match only lower
> case, while it is not mentioned at all that it will not work by default
> (case-fold-search as well as reb-toggle-case and reb-change-syntax are not
> mentioned at all in that section as far as I can find). Indeed I was
> referring to the docstring of re-builder. Anyway, I am happy to provide a
> patch.
>
>
>
> On Thu, 1 Apr 2021 at 00:25, Basil L. Contovounesios <contovob@tcd.ie>
> wrote:
>
>> dalanicolai <dalanicolai@gmail.com> writes:
>>
>> > When using the regexp builder and trying to match only lower case
>> > letters (and spaces), Emacs also includes upper case matches.
>>
>> I think this is the effect of the user option case-fold-search, which
>> defaults to non-nil (this is the case across most search-related parts
>> of Emacs, including Isearch).  You can toggle it in re-builder
>> specifically with C-c C-c (reb-toggle-case), or across all buffers by
>> customising case-fold-search to be nil.
>>
>> > Also, I am unable to find in the manual any information about the
>> > option of setting different syntaxes in the regexp builder
>>
>> I think most of the documentation for re-builder.el is in its commentary
>> at the start of the file; see M-x find-library RET re-builder RET.
>>
>> I haven't used re-builder much, but from M-x customize-group RET
>> re-builder RET I see there is a user option reb-re-syntax for
>> controlling the default syntax.  C-h m in the re-builder buffer further
>> reveals that the command C-c C-i (reb-change-syntax) can modify
>> reb-re-syntax on the fly.
>>
>> > (also the option is not mentioned in the regexp-builder docstring). So
>> > I would additionally like to report this as a documentation bug.
>>
>> Are you referring to the docstring of the re-builder command, or
>> something else?  And is it a listing of key bindings you would like to
>> see, or something else?  Would you like to provide a patch with
>> suggestions for improvements?
>>
>> Thanks,
>>
>> --
>> Basil
>>
>

[-- Attachment #1.2: Type: text/html, Size: 3115 bytes --]

[-- Attachment #2: re-builder-improve-docstring --]
[-- Type: application/octet-stream, Size: 846 bytes --]

diff --git a/lisp/emacs-lisp/re-builder.el b/lisp/emacs-lisp/re-builder.el
index 455fcac701..2bbee18afd 100644
--- a/lisp/emacs-lisp/re-builder.el
+++ b/lisp/emacs-lisp/re-builder.el
@@ -341,7 +341,12 @@ the regexp builder.  It displays a buffer named \"*RE-Builder*\"
 in another window, initially containing an empty regexp.
 
 As you edit the regexp in the \"*RE-Builder*\" buffer, the
-matching parts of the target buffer will be highlighted."
+matching parts of the target buffer will be highlighted.
+
+Case-sensitivity can be toggled with \\[reb-toggle-case].  The
+regexp builder supports three different forms of input which can
+be set with \\[reb-change-syntax]. More options and details
+are provided in the initial comment of`re-builder.el'."
   (interactive)
   (if (and (string= (buffer-name) reb-buffer)
 	   (reb-mode-buffer-p))

bug#47534: Subject: 28.0.50; Regexp lower case pattern matches upper case

[dalanicolai]

[-- Attachment #1.1: Type: text/plain, Size: 2494 bytes --]

And here is a small patch for the docs

On Thu, 1 Apr 2021 at 12:27, dalanicolai <dalanicolai@gmail.com> wrote:

> Here is a small patch for the re-builder docstring. I hope I did things
> correctly...
>
> On Thu, 1 Apr 2021 at 10:12, dalanicolai <dalanicolai@gmail.com> wrote:
>
>> Ah thanks. I thought I used case sensitive replace-regexp before, but
>> probably I am mistaking then. Section "34.3 Regular Expressions"  of the
>> Emacs manual states that "[a-z]" or "[:lower:]" should match only lower
>> case, while it is not mentioned at all that it will not work by default
>> (case-fold-search as well as reb-toggle-case and reb-change-syntax are not
>> mentioned at all in that section as far as I can find). Indeed I was
>> referring to the docstring of re-builder. Anyway, I am happy to provide a
>> patch.
>>
>>
>>
>> On Thu, 1 Apr 2021 at 00:25, Basil L. Contovounesios <contovob@tcd.ie>
>> wrote:
>>
>>> dalanicolai <dalanicolai@gmail.com> writes:
>>>
>>> > When using the regexp builder and trying to match only lower case
>>> > letters (and spaces), Emacs also includes upper case matches.
>>>
>>> I think this is the effect of the user option case-fold-search, which
>>> defaults to non-nil (this is the case across most search-related parts
>>> of Emacs, including Isearch).  You can toggle it in re-builder
>>> specifically with C-c C-c (reb-toggle-case), or across all buffers by
>>> customising case-fold-search to be nil.
>>>
>>> > Also, I am unable to find in the manual any information about the
>>> > option of setting different syntaxes in the regexp builder
>>>
>>> I think most of the documentation for re-builder.el is in its commentary
>>> at the start of the file; see M-x find-library RET re-builder RET.
>>>
>>> I haven't used re-builder much, but from M-x customize-group RET
>>> re-builder RET I see there is a user option reb-re-syntax for
>>> controlling the default syntax.  C-h m in the re-builder buffer further
>>> reveals that the command C-c C-i (reb-change-syntax) can modify
>>> reb-re-syntax on the fly.
>>>
>>> > (also the option is not mentioned in the regexp-builder docstring). So
>>> > I would additionally like to report this as a documentation bug.
>>>
>>> Are you referring to the docstring of the re-builder command, or
>>> something else?  And is it a listing of key bindings you would like to
>>> see, or something else?  Would you like to provide a patch with
>>> suggestions for improvements?
>>>
>>> Thanks,
>>>
>>> --
>>> Basil
>>>
>>

[-- Attachment #1.2: Type: text/html, Size: 3529 bytes --]

[-- Attachment #2: improve-regexp-doc-add-note-case-sensitive --]
[-- Type: application/octet-stream, Size: 1211 bytes --]

diff --git a/doc/lispref/searching.texi b/doc/lispref/searching.texi
index 16a8e56e90..05b81f0cd1 100644
--- a/doc/lispref/searching.texi
+++ b/doc/lispref/searching.texi
@@ -239,7 +239,7 @@ This variable is used by passing it as an argument to the function
   A @dfn{regular expression}, or @dfn{regexp} for short, is a pattern that
 denotes a (possibly infinite) set of strings.  Searching for matches for
 a regexp is a very powerful operation.  This section explains how to write
-regexps; the following section says how to search for them.
+regexps; the following section says how to search for them. By default Emacs
 
 @findex re-builder
 @cindex regular expressions, developing
@@ -251,6 +251,10 @@ matches in the target buffer are highlighted.  Each parenthesized
 sub-expression of the regexp is shown in a distinct face, which makes
 it easier to verify even very complex regexps.
 
+Note that by default Emacs search ignores case (@xref{Searching and Case}).
+To enable case-sensitive regexp search and match, set @code{case-fold-search}
+to @code{nil}.
+
 @menu
 * Syntax of Regexps::       Rules for writing regular expressions.
 * Regexp Example::          Illustrates regular expression syntax.

bug#47534: Subject: 28.0.50; Regexp lower case pattern matches upper case

[dalanicolai]

[-- Attachment #1: Type: text/plain, Size: 2713 bytes --]

Of course any feedback on the code (well... text)  or my workflow is very
welcome.

On Thu, 1 Apr 2021 at 12:27, dalanicolai <dalanicolai@gmail.com> wrote:

> And here is a small patch for the docs
>
> On Thu, 1 Apr 2021 at 12:27, dalanicolai <dalanicolai@gmail.com> wrote:
>
>> Here is a small patch for the re-builder docstring. I hope I did things
>> correctly...
>>
>> On Thu, 1 Apr 2021 at 10:12, dalanicolai <dalanicolai@gmail.com> wrote:
>>
>>> Ah thanks. I thought I used case sensitive replace-regexp before, but
>>> probably I am mistaking then. Section "34.3 Regular Expressions"  of the
>>> Emacs manual states that "[a-z]" or "[:lower:]" should match only lower
>>> case, while it is not mentioned at all that it will not work by default
>>> (case-fold-search as well as reb-toggle-case and reb-change-syntax are not
>>> mentioned at all in that section as far as I can find). Indeed I was
>>> referring to the docstring of re-builder. Anyway, I am happy to provide a
>>> patch.
>>>
>>>
>>>
>>> On Thu, 1 Apr 2021 at 00:25, Basil L. Contovounesios <contovob@tcd.ie>
>>> wrote:
>>>
>>>> dalanicolai <dalanicolai@gmail.com> writes:
>>>>
>>>> > When using the regexp builder and trying to match only lower case
>>>> > letters (and spaces), Emacs also includes upper case matches.
>>>>
>>>> I think this is the effect of the user option case-fold-search, which
>>>> defaults to non-nil (this is the case across most search-related parts
>>>> of Emacs, including Isearch).  You can toggle it in re-builder
>>>> specifically with C-c C-c (reb-toggle-case), or across all buffers by
>>>> customising case-fold-search to be nil.
>>>>
>>>> > Also, I am unable to find in the manual any information about the
>>>> > option of setting different syntaxes in the regexp builder
>>>>
>>>> I think most of the documentation for re-builder.el is in its commentary
>>>> at the start of the file; see M-x find-library RET re-builder RET.
>>>>
>>>> I haven't used re-builder much, but from M-x customize-group RET
>>>> re-builder RET I see there is a user option reb-re-syntax for
>>>> controlling the default syntax.  C-h m in the re-builder buffer further
>>>> reveals that the command C-c C-i (reb-change-syntax) can modify
>>>> reb-re-syntax on the fly.
>>>>
>>>> > (also the option is not mentioned in the regexp-builder docstring). So
>>>> > I would additionally like to report this as a documentation bug.
>>>>
>>>> Are you referring to the docstring of the re-builder command, or
>>>> something else?  And is it a listing of key bindings you would like to
>>>> see, or something else?  Would you like to provide a patch with
>>>> suggestions for improvements?
>>>>
>>>> Thanks,
>>>>
>>>> --
>>>> Basil
>>>>
>>>

[-- Attachment #2: Type: text/html, Size: 3988 bytes --]

bug#47534: Subject: 28.0.50; Regexp lower case pattern matches upper case

[Lars Ingebrigtsen]

dalanicolai <dalanicolai@gmail.com> writes:

> And here is a small patch for the docs

Thanks; both patches applied to Emacs 28 (with some changes).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no