bug#63828: 29.0.90; project-query-replace-regexp tries to find-and-replace in binary files

bug#63828: 29.0.90; project-query-replace-regexp tries to find-and-replace in binary files

[Spencer Baugh]

1. emacs -Q
2. Open a git repository which has some binary files committed, or any
other project that has binary files.
3. C-x p r foo bar
4. query-replace will enter the binary files

Perhaps it should not do this?  Perhaps fileloop-initialize-replace
should skip binary files?  Perhaps by some customization?



In GNU Emacs 29.0.90 (build 3, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.15.12, Xaw scroll bars) of 2023-05-17 built on
 igm-qws-u22796a
Repository revision: 4d08492296c2a6d2910f2b740c2d2508275458fc
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: CentOS Linux 7 (Core)

Configured using:
 'configure --with-x-toolkit=lucid --with-gif=ifavailable'

Configured features:
CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND
SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XINPUT2 XPM LUCID
ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: C/*l

Minor modes in effect:
  bug-reference-prog-mode: t
  jane-fe-minor-mode: t
  editorconfig-mode: t
  which-function-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  auto-revert-mode: t
  shell-dirtrack-mode: t
  server-mode: t
  windmove-mode: t
  savehist-mode: t
  save-place-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tab-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  context-menu-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  abbrev-mode: t

Features:
(mhtml-mode css-mode js c-ts-common sgml-mode facemenu htmlize ox-odt
rng-loc rng-uri rng-parse rng-match rng-pttrn nxml-parse nxml-ns
nxml-enc xmltok nxml-util ox-latex ox-icalendar org-agenda ox-html table
ox-ascii ox-publish ox rect smiley gnus-cite gnus-async gnus-bcklg
gnus-agent gnus-srvr gnus-score score-mode nnvirtual nntp gnus-ml
gnus-msg disp-table nndoc gnus-cache gnus-dup mm-archive url-http-ntlm
ntlm hmac-md5 hex-util md4 network-stream url-cache debbugs-gnu
debbugs-compat debbugs soap-client url-http url-gw nsm rng-xsd rng-dt
rng-util xsd-regexp hl-line display-line-numbers org-goto sql cal-move
textsec uni-scripts idna-mapping ucs-normalize uni-confusable
textsec-check etags fileloop cc-mode cc-fonts cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine cc-vars cc-defs cc-bytecomp shadow
mail-extr emacsbug dabbrev dos-w32 find-cmd tutorial man sort conf-mode
make-mode mule-util completion descr-text cus-start quail shortdoc
cal-iso org-datetree tabify org-capture org-element org-persist org-id
org-refile avl-tree generator oc-basic ol-eww eww xdg url-queue mm-url
ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect gnus-art mm-uu mml2015
mm-view mml-smime smime gnutls dig ol-docview doc-view jka-compr
image-mode exif ol-bibtex bibtex ol-bbdb ol-w3m ol-doi org-link-doi misc
pulse find-dired bug-reference magit-imenu git-rebase face-remap vc-bzr
vc-src vc-sccs vc-svn vc-cvs vc-rcs log-view vc-git cl-print dired-aux
help-fns radix-tree misearch multi-isearch vc-fe sh-script treesit
executable goto-addr cus-edit cus-load vc-hg vc-dir vc vc-dispatcher
tramp tramp-cache time-stamp tramp-loaddefs trampver tramp-integration
tramp-compat ls-lisp ffap jane-merlin merlin-imenu merlin-xref
merlin-cap merlin jane-async-merlin jane-completion grep jane-common
jane-fe-project xref jane-fe-menu ecaml_plugin linum view gopcaml
magit-bookmark bookmark image+ advice image-file image-converter
editorconfig editorconfig-core editorconfig-core-handle
editorconfig-fnmatch whitespace jane-auto-modes vba-mode markdown-mode
color jane jane-micro-features jane-diff unified-test-mode shell-file
core core-buffer core-error core-util ert pp ewoc debug backtrace
jane-sexp jane-ocaml jane-tuareg-theme tuareg tuareg-compat tuareg-opam
skeleton flymake-proc flymake warnings thingatpt smie caml-types
caml-help caml-emacs find-file compile jane-cr jane-align
jane-deprecated jane-smerge gnu-elpa-keyring-update jane-ocp-indent
ocp-indent cl jane-util ob-shell page-ext dired-x magit-extras project
magit-submodule magit-obsolete magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log which-func imenu
magit-diff smerge-mode diff diff-mode git-commit log-edit pcvs-util
add-log magit-core magit-autorevert autorevert filenotify magit-margin
magit-transient magit-process with-editor shell server magit-mode
transient edmacro kmacro magit-git magit-section magit-utils crm dash
cl-extra help-mode windmove org ob ob-tangle ob-ref ob-lob ob-table
ob-exp org-macro org-src ob-comint org-pcomplete pcomplete org-list
org-footnote org-faces org-entities noutline outline icons ob-emacs-lisp
ob-core ob-eval org-cycle org-table ol rx org-fold org-fold-core
org-keys oc org-loaddefs find-func cal-menu calendar cal-loaddefs
gnus-sum shr pixel-fill kinsoku url-file svg dom gnus-group gnus-undo
gnus-start gnus-dbus dbus xml gnus-cloud nnimap nnmail mail-source utf7
nnoo parse-time iso8601 gnus-spec gnus-int gnus-range message sendmail
yank-media puny dired dired-loaddefs rfc822 mml mml-sec epa epg rfc6068
epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047
rfc2045 ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus
nnheader gnus-util text-property-search time-date mail-utils range
mm-util mail-prsvr wid-edit org-version org-compat org-macs format-spec
gdb-mi bindat gud comint easy-mmode files-x derived ansi-osc ansi-color
ring vundo pcase cyberpunk-theme savehist saveplace project-autoloads
vundo-autoloads magit-autoloads xref-autoloads csv-mode-autoloads
magit-section-autoloads cyberpunk-theme-autoloads
url-http-ntlm-autoloads url-auth git-commit-autoloads
with-editor-autoloads finder-inf info dash-autoloads vc-hgcmd-autoloads
package browse-url url url-proxy url-privacy url-expand url-methods
url-history url-cookie generate-lisp-file url-domsuf url-util mailcap
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
password-cache json subr-x map byte-opt gv bytecomp byte-compile
url-vars cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo x-toolkit
xinput2 x multi-tty make-network-process emacs)

Memory information:
((conses 16 1368302 192959)
 (symbols 48 70838 33)
 (strings 32 300583 10905)
 (string-bytes 1 10472195)
 (vectors 16 121005)
 (vector-slots 8 2726004 176199)
 (floats 8 754 373)
 (intervals 56 70981 4481)
 (buffers 976 141)
 (heap 1024 768733 170632))

bug#63828: 29.0.90; project-query-replace-regexp tries to find-and-replace in binary files

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Date: Thu, 01 Jun 2023 16:16:03 -0400
> 
> 
> 1. emacs -Q
> 2. Open a git repository which has some binary files committed, or any
> other project that has binary files.
> 3. C-x p r foo bar
> 4. query-replace will enter the binary files
> 
> Perhaps it should not do this?  Perhaps fileloop-initialize-replace
> should skip binary files?  Perhaps by some customization?

Either user option or prefix argument, IMO.  And this should be
documented with the caveat that a VCS can decide that a file is binary
even when it is not.  For example, I've seen Git treating very large
parser.c files in tree-sitter grammar repositories as binary files for
some reason.  Another example is CVS on MS-Windows: best practice is
to checkout all the files with the -kb switch, so that the client
doesn't attempt to convert EOLs to the Windows CR-LF format (which
ruins some files).

bug#63828: 29.0.90; project-query-replace-regexp tries to find-and-replace in binary files

[Dmitry Gutov]

On 01/06/2023 23:16, Spencer Baugh wrote:
> 1. emacs -Q
> 2. Open a git repository which has some binary files committed, or any
> other project that has binary files.
> 3. C-x p r foo bar
> 4. query-replace will enter the binary files
> 
> Perhaps it should not do this?  Perhaps fileloop-initialize-replace
> should skip binary files?  Perhaps by some customization?

It would be useful if you compare to what project-find-regexp does in 
such project: does it produce the matches in said binary files?

If it behaves better (from your POV), one solution could be to 
pre-filter the list of project files using the same search, before 
passing them to project-query-replace-regexp. It can have benefits in 
performance as well.

(BTW JFYI, you can press 'r' inside the project-find-regexp output 
buffer instead of using 'project-query-replace-regexp'.)