bug#8688: 23.2; search-whitespace-regexp not working as documented

bug#8688: 23.2; search-whitespace-regexp not working as documented

[Worik Stanton]

This bug report will be sent to the Free Software Foundation,
not to your local site managers!
Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your bug report will be posted to the bug-gnu-emacs@gnu.org mailing list,
and to the gnu.emacs.bug news group.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug.  If you can, give
a recipe starting from `emacs -Q':

I set search-whitespace-regexp to [ \t\r\n]+ in customise as suggested
in the documentation but when I do a isearch-forward-regexp the space
character only matches a literal space.

  '(search-whitespace-regexp "[ \\t\\r\\n]+"))

is the line from .emacs

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
     `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
/usr/share/emacs/23.2/etc/DEBUG.


In GNU Emacs 23.2.1 (i686-pc-linux-gnu, GTK+ Version 2.24.4)
  of 2011-04-04 on rothera, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11001000
configured using `configure  '--build' 'i686-linux-gnu' '--build' 
'i686-linux-gnu' '--prefix=/usr' '--sharedstatedir=/var/lib' 
'--libexecdir=/usr/lib' '--localstatedir=/var/lib' 
'--infodir=/usr/share/info' '--mandir=/usr/share/man' '--with-pop=yes' 
'--enable-locallisppath=/etc/emacs23:/etc/emacs:/usr/local/share/emacs/23.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/23.2/site-lisp:/usr/share/emacs/site-lisp:/usr/share/emacs/23.2/leim' 
'--with-x=yes' '--with-x-toolkit=gtk' '--with-toolkit-scroll-bars' 
'build_alias=i686-linux-gnu' 'CFLAGS=-DDEBIAN -g -O2' 'LDFLAGS=-g' 
'CPPFLAGS=''

Important settings:
   value of $LC_ALL: nil
   value of $LC_COLLATE: nil
   value of $LC_CTYPE: nil
   value of $LC_MESSAGES: nil
   value of $LC_MONETARY: nil
   value of $LC_NUMERIC: nil
   value of $LC_TIME: nil
   value of $LANG: en_NZ.UTF-8
   value of $XMODIFIERS: nil
   locale-coding-system: utf-8-unix
   default enable-multibyte-characters: t

Major mode: Help

Minor modes in effect:
   shell-dirtrack-mode: t
   tooltip-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   line-number-mode: t
   transient-mark-mode: (only . t)
   view-mode: t

Recent input:
C-s C-s C-s C-s <return> SPC s l C-x C-s C-h k C-M-S-s
C-x 0 <down> <down> <down> <down> <down> <down> <down>
<down> <down> <down> <down> <down> <down> <down> <down>
<right> <right> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <right> <return>
<down> <down> <down> <down> <down> <down> <down> <down>
<up> <right> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <right> <right>
<right> <right> <right> <right> C-SPC <right> <right>
<right> <right> <right> <right> <right> <right> <right>
<right> <right> <left> C-w <down> <down> <down> <down>
<down> C-g <down> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <return> <down-mouse-1>
<mouse-movement> <mouse-1> <backspace> <backspace>
<backspace> <backspace> C-y <help-echo> <down-mouse-1>
<help-echo> <down-mouse-1> <help-echo> <down-mouse-1>
q q C-M-S-s d e f i n e d . SPC + C-s <backspace> <backspace>
C-s C-s <backspace> <backspace> <backspace> <return>
C-h k C-M-S-s C-x 0 <help-echo> <down-mouse-1> <help-echo>
<mouse-2> C-x b D i <tab> t <tab> <return> <down> <down>
<down> f o o SPC <return> b a r <tab> SPC <tab> SPC
<up> <up> <up> C-M-S-s f o o SPC + b <return> C-x C-s
C-a C-k C-k C-k C-x C-s <help-echo> <help-echo> <help-echo>
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo>
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo>
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo>
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo>
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo>
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo>
<help-echo> <help-echo> <help-echo> <help-echo> <menu-bar>
<help-menu> <send-emacs-bug-report> C-g C-g C-x b <return>
<help-echo> <help-echo> <down-mouse-1> <mouse-movement>
<mouse-movement> <drag-mouse-1> C-w <help-echo> <help-echo>
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo>
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo>
<help-echo> <help-echo> <help-echo> <menu-bar> <help-menu>
<send-emacs-bug-report>

Recent messages:
Mark saved where search started
Type C-x 1 to delete the help window, C-M-v to scroll help.
Mark saved where search started
Saving file 
/home/worik/Documents/Research/Masters/Thesis/Dissertation.tex...
Wrote /home/worik/Documents/Research/Masters/Thesis/Dissertation.tex
Saving file 
/home/worik/Documents/Research/Masters/Thesis/Dissertation.tex...
Wrote /home/worik/Documents/Research/Masters/Thesis/Dissertation.tex
Sorting environment...
Removing duplicates... done
Quit [2 times]
byte-code: Buffer is read-only: #<buffer *Help*>

Load-path shadows:
/usr/share/emacs/23.2/site-lisp/auctex/multi-prompt hides 
/usr/share/emacs/site-lisp/auctex/multi-prompt
/usr/share/emacs/23.2/site-lisp/auctex/font-latex hides 
/usr/share/emacs/site-lisp/auctex/font-latex
/usr/share/emacs/23.2/site-lisp/auctex/context hides 
/usr/share/emacs/site-lisp/auctex/context
/usr/share/emacs/23.2/site-lisp/auctex/tex-fold hides 
/usr/share/emacs/site-lisp/auctex/tex-fold
/usr/share/emacs/23.2/site-lisp/auctex/texmathp hides 
/usr/share/emacs/site-lisp/auctex/texmathp
/usr/share/emacs/23.2/site-lisp/auctex/latex hides 
/usr/share/emacs/site-lisp/auctex/latex
/usr/share/emacs/23.2/site-lisp/auctex/tex-fptex hides 
/usr/share/emacs/site-lisp/auctex/tex-fptex
/usr/share/emacs/23.2/site-lisp/auctex/tex-buf hides 
/usr/share/emacs/site-lisp/auctex/tex-buf
/usr/share/emacs/23.2/site-lisp/auctex/tex-mik hides 
/usr/share/emacs/site-lisp/auctex/tex-mik
/usr/share/emacs/23.2/site-lisp/auctex/bib-cite hides 
/usr/share/emacs/site-lisp/auctex/bib-cite
/usr/share/emacs/23.2/site-lisp/auctex/tex-bar hides 
/usr/share/emacs/site-lisp/auctex/tex-bar
/usr/share/emacs/23.2/site-lisp/auctex/tex-info hides 
/usr/share/emacs/site-lisp/auctex/tex-info
/usr/share/emacs/23.2/site-lisp/auctex/context-en hides 
/usr/share/emacs/site-lisp/auctex/context-en
/usr/share/emacs/23.2/site-lisp/auctex/tex hides 
/usr/share/emacs/site-lisp/auctex/tex
/usr/share/emacs/23.2/site-lisp/auctex/tex-style hides 
/usr/share/emacs/site-lisp/auctex/tex-style
/usr/share/emacs/23.2/site-lisp/auctex/tex-jp hides 
/usr/share/emacs/site-lisp/auctex/tex-jp
/usr/share/emacs/23.2/site-lisp/auctex/toolbar-x hides 
/usr/share/emacs/site-lisp/auctex/toolbar-x
/usr/share/emacs/23.2/site-lisp/auctex/context-nl hides 
/usr/share/emacs/site-lisp/auctex/context-nl
/usr/share/emacs/23.2/site-lisp/auctex/tex-font hides 
/usr/share/emacs/site-lisp/auctex/tex-font
/usr/share/emacs23/site-lisp/ess/noweb-mode hides 
/usr/share/emacs/site-lisp/noweb-mode
/usr/share/emacs/23.2/site-lisp/debian-startup hides 
/usr/share/emacs/site-lisp/debian-startup
/usr/share/emacs23/site-lisp/dictionaries-common/flyspell hides 
/usr/share/emacs/23.2/lisp/textmodes/flyspell
/usr/share/emacs23/site-lisp/dictionaries-common/ispell hides 
/usr/share/emacs/23.2/lisp/textmodes/ispell
/usr/share/emacs/23.2/site-lisp/octave3.2-emacsen/octave-mod hides 
/usr/share/emacs/23.2/lisp/progmodes/octave-mod
/usr/share/emacs/23.2/site-lisp/octave3.2-emacsen/octave-inf hides 
/usr/share/emacs/23.2/lisp/progmodes/octave-inf

Features:
(shadow sort mail-extr message sendmail ecomplete rfc822 mml mml-sec
password-cache mm-decode mm-bodies mm-encode mailcap mail-parse rfc2231
rfc2047 rfc2045 qp ietf-drums mailabbrev nnheader gnus-util netrc
time-date mm-util mail-prsvr gmm-utils mailheader canlock sha1 hex-util
hashcash mail-utils emacsbug cus-edit cus-start cus-load wid-edit pp
dired texmathp log-edit pcvs-util add-log vc vc-dispatcher preview
prv-emacs byte-opt tex-bar tex-buf toolbar-x noutline outline font-latex
bytecomp byte-compile latex easy-mmode edmacro kmacro tex-style tex
latexenc ess-help newcomment multi-isearch help-mode view vc-rcs
ess-roxy advice help-fns advice-preload hideshow imenu saveplace
ess-toolbar ess-mouse mouseme thingatpt browse-url ess-menu ess-swv
ess-noweb noweb-font-lock-mode ess-bugs-l essd-els ess-sas-d ess-sas-l
ess-sas-a executable shell ess-arc-d ess-vst-d ess-xls-d ess-lsp-l
ess-sta-d ess-sta-l cc-vars cc-defs make-regexp ess-sp6-d ess-sp5-d
ess-sp3-d ess-r-d ess-r-args assoc ess-s-l ess-inf ess-utils comint ring
ess-mode noweb-mode ess ess-custom regexp-opt easymenu ess-compat
ess-site preview-latex tex-site auto-loads tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd font-setting tool-bar dnd fontset
image fringe lisp-mode register page menu-bar rfn-eshadow timer select
scroll-bar mldrag mouse jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev loaddefs button minibuffer faces cus-face files
text-properties overlay md5 base64 format env code-pages mule custom
widget hashtable-print-readable backquote make-network-process dbusbind
system-font-setting font-render-setting gtk x-toolkit x multi-tty emacs)

bug#8688: 23.2; search-whitespace-regexp not working as documented

[Glenn Morris]

Worik Stanton wrote:

> I set search-whitespace-regexp to [ \t\r\n]+ in customise as suggested
> in the documentation but when I do a isearch-forward-regexp the space
> character only matches a literal space.
>
>  '(search-whitespace-regexp "[ \\t\\r\\n]+"))
>
> is the line from .emacs

I think you're being confused by the way customize treats regexps (it
confuses me as well; I'm not sure it is helpful). There is extra
escaping in the setting of search-whitespace-regexp that is written to
your .emacs. It should be:

  '(search-whitespace-regexp "[ \t\r\n]+"))

The doc of search-whitespace-regexp says:

  In the Customization buffer, that is `[' followed by a space,
  a tab, a carriage return (control-M), a newline, and `]+'

If you literally use that input, does it work?

Oh, it's not possible to type a TAB, or ctrl-M, or newline, in a custom
field, since it moves you out of the editable field... This is even more
confusing.

bug#8688: 23.2; search-whitespace-regexp not working as documented

[Andreas Schwab]

Glenn Morris <rgm@gnu.org> writes:

> I think you're being confused by the way customize treats regexps (it
> confuses me as well; I'm not sure it is helpful).

They are just literal strings, without lisp-level quoting.

> Oh, it's not possible to type a TAB, or ctrl-M, or newline, in a custom
> field

C-q works fine.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#8688: 23.2; search-whitespace-regexp not working as documented

[Glenn Morris]

Andreas Schwab wrote:

> Glenn Morris <rgm@gnu.org> writes:
>
>> I think you're being confused by the way customize treats regexps (it
>> confuses me as well; I'm not sure it is helpful).
>
> They are just literal strings, without lisp-level quoting.

I know, but I find it confusing, and it seems others do too (eg this;
also bug#2666). It makes it difficult to write doc-strings for 'regexp
defcustoms, as we see in both of these cases.

>> Oh, it's not possible to type a TAB, or ctrl-M, or newline, in a custom
>> field
>
> C-q works fine.

Yes, but that's not what the doc says people should type, and IMO
"ctrl-q ctrl-j" is a lot more confusing than "\n", especially in an
interface that is supposed to make life easier.

bug#8688: 23.2; search-whitespace-regexp not working as documented

[Unknown]

On 18/05/11 19:15, Glenn Morris wrote:
> Worik Stanton wrote:
>
>> I set search-whitespace-regexp to [ \t\r\n]+ in customise as suggested
>> in the documentation but when I do a isearch-forward-regexp the space
>> character only matches a literal space.
>>
>>   '(search-whitespace-regexp "[ \\t\\r\\n]+"))
>>
>> is the line from .emacs
> I think you're being confused by the way customize treats regexps (it
> confuses me as well; I'm not sure it is helpful). There is extra
> escaping in the setting of search-whitespace-regexp that is written to
> your .emacs. It should be:
>
>    '(search-whitespace-regexp "[ \t\r\n]+"))
>
> The doc of search-whitespace-regexp says:
>
>    In the Customization buffer, that is `[' followed by a space,
>    a tab, a carriage return (control-M), a newline, and `]+'
>
> If you"literally use that input, does it work?
Yes.  I copied the string direct from the documentation.  Perhaps this 
is a bug in the documentation?

Also I uncovered another bug, I guess it could be described as 
aesthetic.  I changed the "custom-set-variables"  command to...

(custom-set-variables
   ;; custom-set-variables was added by Custom.
   ;; If you edit it by hand, you could mess it up, so be careful.
   ;; Your init file should contain only one such instance.
   ;; If there is more than one, they won't work right.
  '(delete-by-moving-to-trash t)
  '(save-place t nil (saveplace))
  '(search-whitespace-regexp "[ \t\r\n]+"))

And executed it.  (regex-search works as expected now).

But C-h v search-whitespace-regexp  says it is set to.,.

search-whitespace-regexp's value is "[     ^M\n]+"

I was about to report that the custom-set-variables command did not work...

Thanks for your help, all the serious problems are solved.

cheers
Worik

> Oh, it's not possible to type a TAB, or ctrl-M, or newline, in a custom
> field, since it moves you out of the editable field... This is even more
> confusing.
>

bug#8688: 23.2; search-whitespace-regexp not working as documented

[Stefan Monnier]

>>> I think you're being confused by the way customize treats regexps (it
>>> confuses me as well; I'm not sure it is helpful).
>> They are just literal strings, without lisp-level quoting.
> I know, but I find it confusing, and it seems others do too (eg this;
> also bug#2666). It makes it difficult to write doc-strings for 'regexp
> defcustoms, as we see in both of these cases.
>>> Oh, it's not possible to type a TAB, or ctrl-M, or newline, in a custom
>>> field
>> C-q works fine.
> Yes, but that's not what the doc says people should type, and IMO
> "ctrl-q ctrl-j" is a lot more confusing than "\n", especially in an
> interface that is supposed to make life easier.

Maybe our regexp language should accept \n and \t as escape sequences,
i.e. "\t" and "\\t" should match the same things.  I know it would be
redundant, but in cases such as regexp isearch or the above it can be handy.


        Stefan

bug#8688: 23.2; search-whitespace-regexp not working as documented

[Lars Ingebrigtsen]

Worik Stanton <worik.stanton@gmail.com> writes:

> And executed it.  (regex-search works as expected now).
>
> But C-h v search-whitespace-regexp  says it is set to...
>
> search-whitespace-regexp's value is "[     ^M\n]+"
>
> I was about to report that the custom-set-variables command did not work...
>
> Thanks for your help, all the serious problems are solved.

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

It seems like the problems in this bug report stem from the various ways
Emacs presents and represents strings, and Emacs has many ways of doing
that.  For instance:

(let ((print-escape-newlines t))
  (prin1 "[ \t\r\n]+" (current-buffer)))
"[ 	^M\n]+"

(let ((print-escape-control-characters t))
  (prin1 "[ \t\r\n]+" (current-buffer)))
"[ \11\15\12]+"

etc.

This is, unfortunately, just the way it is, so I don't think there's
anything actionable in this bug report, and I'm closing it.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no