Why don't `occur`, `search-forward*` and `string-match*` respect `isearch-filter-predicate`?

Why don't `occur`, `search-forward*` and `string-match*` respect `isearch-filter-predicate`?

[Gabriele Nicolardi]

[-- Attachment #1: Type: text/plain, Size: 2040 bytes --]

Hi,

if I consult the definition of |isearch-filter-predicate|, I find:

    Predicate to filter hits of Isearch and replace commands. Isearch hits
    that don’t satisfy the predicate will be skipped.

Over the years, I have found this option very useful as it allows me to 
perform targeted searches and replacements in certain parts of the 
|LaTeX| documents I edit for work.
For example, by creating custom predicates, I can ensure that I operate 
only on the mathematical part of the document, or only on 
cross-reference labels; I can ignore comments inserted by the authors, 
and so on.

Unfortunately, however, some functions such as |occur|, 
|search-forward-*|, |string-match*|, do not take this variable into account.

For some of them, I managed to create modified versions:

|(defun search-forward-ifp (STRING &optional BOUND NOERROR COUNT) 
"Modified version of `search-forward` that filters (skips) matches 
according to `isearch-filter-predicate'." (let ((POINT (point))) (catch 
'filtered (while (search-forward STRING BOUND NOERROR COUNT) (let ((B 
(match-beginning 0)) (E (match-end 0))) ;; 1 - If all points of the 
region matching the search from ;; the previous `search-forward` fit the 
criteria accepted ;; by the filter, then the loop stops (throw) and ;; 
returns the position `(point)`: (when (funcall isearch-filter-predicate 
B E) (throw 'filtered (point))))) ;; 2 - If the search is unsuccessful, 
or it doesn't fit ;; the criteria accepted by the filter, then return to 
;; the starting position and return `nil`. (goto-char POINT) nil))) |

However, I couldn’t modify, for instance, |occur| and 
|string-match(-p)|. Is there a reason why these functions don’t adopt 
this mechanism? Or is it simply a lack of interest in implementing this 
option?

(I understand that the “i” in [i]search stands for interactive, but I 
often find myself using functions that are not strictly interactive 
within functions that are.)

It make sense to open a feature request?

Best regards,

Gabriele Nicolardi

​

[-- Attachment #2: Type: text/html, Size: 9579 bytes --]

Re: Why don't `occur`, `search-forward*` and `string-match*` respect `isearch-filter-predicate`?

[Basil L. Contovounesios]

Gabriele Nicolardi [2024-02-26 19:02 +0100] wrote:

> if I consult the definition of isearch-filter-predicate, I find:
>
>     Predicate to filter hits of Isearch and replace commands. Isearch hits
>     that don’t satisfy the predicate will be skipped.
>    
> Over the years, I have found this option very useful as it allows me to perform
> targeted searches and replacements in certain parts of the LaTeX documents I
> edit for work.
> For example, by creating custom predicates, I can ensure that I operate only on
> the mathematical part of the document, or only on cross-reference labels; I can
> ignore comments inserted by the authors, and so on.
>
> Unfortunately, however, some functions such as occur, search-forward-*,
> string-match*, do not take this variable into account.
[...]
> However, I couldn’t modify, for instance, occur and string-match(-p). Is there
> a reason why these functions don’t adopt this mechanism? Or is it simply a lack
> of interest in implementing this option?

isearch-filter-predicate is a user option of the isearch library, which
provides users with convenient, incremental search functionality.

OTOH search-forward, re-search-forward, and string-match are low-level
Elisp matching primitives (even if the first two can be called
interactively): I don't think they should be concerned with the user
options that a user-facing incremental search feature provides.

IOW, Elisp libraries need a simple and reliable way to match strings and
regexps for many of their core functions, outside the influence of any
user-facing conveniences.  Existing code relies on the current contract
of string-match & co., and could break if expected matches were skipped
because of isearch-filter-predicate.  The fact that these primitives
obey case-fold-search alone already catches some people off guard.

> (I understand that the “i” in [i]search stands for interactive,

[ I suspect it stands for incremental.
  Or the selfish first person singular pronoun ;). ]

> It make sense to open a feature request?

I think it could make sense for occur to obey isearch-filter-predicate,
since it's a more closely related user-facing utility (occur is defined
in the same library as other commands which use
isearch-filter-predicate).  If you submit a feature request, others
could be in a better position than me to judge.

Thanks,
-- 
Basil

Re: Why don't `occur`, `search-forward*` and `string-match*` respect `isearch-filter-predicate`?

[Alfred M. Szmidt]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 201 bytes --]

   > (I understand that the “i” in [i]search stands for interactive,

   [ I suspect it stands for incremental.

Yup, is is "incremental" not "interactive", since you can continue the
search ...

Re: Why don't `occur`, `search-forward*` and `string-match*` respect `isearch-filter-predicate`?

[Gabriele Nicolardi]

[-- Attachment #1: Type: text/plain, Size: 3102 bytes --]

Il 28/02/24 15:06, Basil L. Contovounesios ha scritto:

> Gabriele Nicolardi [2024-02-26 19:02 +0100] wrote:
>
>> if I consult the definition of isearch-filter-predicate, I find:
>>
>>      Predicate to filter hits of Isearch and replace commands. Isearch hits
>>      that don’t satisfy the predicate will be skipped.
>>     
>> Over the years, I have found this option very useful as it allows me to perform
>> targeted searches and replacements in certain parts of the LaTeX documents I
>> edit for work.
>> For example, by creating custom predicates, I can ensure that I operate only on
>> the mathematical part of the document, or only on cross-reference labels; I can
>> ignore comments inserted by the authors, and so on.
>>
>> Unfortunately, however, some functions such as occur, search-forward-*,
>> string-match*, do not take this variable into account.
> [...]
>> However, I couldn’t modify, for instance, occur and string-match(-p). Is there
>> a reason why these functions don’t adopt this mechanism? Or is it simply a lack
>> of interest in implementing this option?
> isearch-filter-predicate is a user option of the isearch library, which
> provides users with convenient, incremental search functionality.
>
> OTOH search-forward, re-search-forward, and string-match are low-level
> Elisp matching primitives (even if the first two can be called
> interactively): I don't think they should be concerned with the user
> options that a user-facing incremental search feature provides.
>
> IOW, Elisp libraries need a simple and reliable way to match strings and
> regexps for many of their core functions, outside the influence of any
> user-facing conveniences.  Existing code relies on the current contract
> of string-match & co., and could break if expected matches were skipped
> because of isearch-filter-predicate.  The fact that these primitives
> obey case-fold-search alone already catches some people off guard.
>
>> (I understand that the “i” in [i]search stands for interactive,
> [ I suspect it stands for incremental.
>    Or the selfish first person singular pronoun ;). ]
>
>> It make sense to open a feature request?
> I think it could make sense for occur to obey isearch-filter-predicate,
> since it's a more closely related user-facing utility (occur is defined
> in the same library as other commands which use
> isearch-filter-predicate).  If you submit a feature request, others
> could be in a better position than me to judge.

Thanks.

I have already written modified versions of |search-forward*| and 
|search-backward*| that I have tested and regularly use. I suppose I 
could ask for a modified version of |match-string*| that adheres to 
|isearch-filter-predicate|. As for |occur|, it would be useful if this 
function adhered to |isearch-filter-predicate| by default. 
Alternatively, this behavior could be managed through a variable.

Now that I think about it, the same thing could apply to 
|search-forward*|, adding the possibility of adhering to 
|isearch-filter-predicate| through the setting of a variable.

Gabriele Nicolardi

> Thanks,

[-- Attachment #2: Type: text/html, Size: 4982 bytes --]

RE: [External] : Re: Why don't `occur`, `search-forward*` and `string-match*` respect `isearch-filter-predicate`?

[Drew Adams]

Dunno whether it makes sense to have such
a variable, to allow code to change the
behavior of calls to such functions (e.g.
by let-binding it around such code).  I
think it's maybe something to consider.

But if that were done, it should be a new
variable, not `isearch-filter-predicate'.

`isearch-filter-predicate' needs to be
separate, for use by Isearch, query-* etc.,
which are essentially for interactive use.

Their interactive use can even let users
change the predicate (thus filtering) on
the fly (e.g. by advising its value):

https://www.emacswiki.org/emacs/DynamicIsearchFiltering

Values for `isearch-filter-predicate'
shouldn't affect the use of the functions
you mention.  That would be wildly unfair
to such code. ;-)  And the new variable
shouldn't directly affect Isearch etc.

As Basil mentioned, however, `occur', as
such, is also interactive.  A case could
I think be made for it too to respect
`isearch-filter-predicate'.  (Maybe this
was already considered at some point?)

Of course nothing prevents code for which
it makes sense to bind the new variable
(assuming one) to the current value of
`isearch-filter-predicate'.

Similarly, nothing prevents some code/user
from binding `isearch-filter-predicate'
to the current value of the new variable.

IOW, they should be different, but their
values can easily be made the same for
some context/use.

In sum, a new, separate variable could be
used for "worker" functions not directly
related to interactive use.  Variable
`isearch-filter-predicate' shouldn't
affect such functions directly, IMO.

Re: [External] : Re: Why don't `occur`, `search-forward*` and `string-match*` respect `isearch-filter-predicate`?

[Gabriele Nicolardi]

[-- Attachment #1: Type: text/plain, Size: 2782 bytes --]

This is an example of what I do:

(defmacro with-ifp-predicates (PREDICATES &rest body)
   "Allows assigning a list of predicates to the
variable `isearch-filter-predicate'.

Below is an example of usage:

(with-ifp-predicates '(skip-maths skip-comments)
   (query-replace \"foo\" \"bar\" nil (point-min) (point-max)))"
   (declare (indent 1))
   `(let ((isearch-filter-predicate isearch-filter-predicate))
      (mapc (lambda (predicate)
                (add-function :before-while isearch-filter-predicate 
predicate))
              ,PREDICATES)
      ,@body))

I always use let-binding for operations that involve 
isearch-filter-predicate.

In some cases, I would like to have something like this:

(with-ifp-predicates '(predicate1 predicate2)
   (save-window-excursion
     (occur "foo")
     (query-replace \"foo\" \"bar\" nil (point-min) (point-max))))

with |occur| showing me only the strings that match the active predicates.

The idea of a new variable that can use the same values as 
|isearch-filter-predicate| works for me.

Il 28/02/24 17:53, Drew Adams ha scritto:
> Dunno whether it makes sense to have such
> a variable, to allow code to change the
> behavior of calls to such functions (e.g.
> by let-binding it around such code).  I
> think it's maybe something to consider.
>
> But if that were done, it should be a new
> variable, not `isearch-filter-predicate'.
>
> `isearch-filter-predicate' needs to be
> separate, for use by Isearch, query-* etc.,
> which are essentially for interactive use.
>
> Their interactive use can even let users
> change the predicate (thus filtering) on
> the fly (e.g. by advising its value):
>
> https://www.emacswiki.org/emacs/DynamicIsearchFiltering
>
> Values for `isearch-filter-predicate'
> shouldn't affect the use of the functions
> you mention.  That would be wildly unfair
> to such code. ;-)  And the new variable
> shouldn't directly affect Isearch etc.
>
> As Basil mentioned, however, `occur', as
> such, is also interactive.  A case could
> I think be made for it too to respect
> `isearch-filter-predicate'.  (Maybe this
> was already considered at some point?)
>
> Of course nothing prevents code for which
> it makes sense to bind the new variable
> (assuming one) to the current value of
> `isearch-filter-predicate'.
>
> Similarly, nothing prevents some code/user
> from binding `isearch-filter-predicate'
> to the current value of the new variable.
>
> IOW, they should be different, but their
> values can easily be made the same for
> some context/use.
>
> In sum, a new, separate variable could be
> used for "worker" functions not directly
> related to interactive use.  Variable
> `isearch-filter-predicate' shouldn't
> affect such functions directly, IMO.

[-- Attachment #2: Type: text/html, Size: 3450 bytes --]

Re: Why don't `occur`, `search-forward*` and `string-match*` respect `isearch-filter-predicate`?

[Juri Linkov]

>> It make sense to open a feature request?
>
> I think it could make sense for occur to obey isearch-filter-predicate,
> since it's a more closely related user-facing utility (occur is defined
> in the same library as other commands which use
> isearch-filter-predicate).  If you submit a feature request, others
> could be in a better position than me to judge.

Like query-replace obeys isearch-filter-predicate by using
the default isearch function, occur could do the same.

OTOH, this might be undesirable for users who want
to see all occurrences including those that are skipped
during Isearch.  So this should be optional.

Re: Why don't `occur`, `search-forward*` and `string-match*` respect `isearch-filter-predicate`?

[Gabriele Nicolardi]

This code I wrote seems to work:

;; see https://emacs.stackexchange.com/a/80612/15606

(let ((re-search-forward-copy (symbol-function 're-search-forward)))
   (defun re-search-forward-ifp (REGEXP &optional BOUND NOERROR COUNT)
     "Modified version of `search-forward-regexp' that
     filters (skips) matches according to `isearch-filter-predicate'."

     (let ((POINT (point)))
       (catch 'filtered
         (while (funcall re-search-forward-copy REGEXP BOUND NOERROR COUNT)
           (let ((B (match-beginning 0))
                 (E (match-end 0)))
             (when (funcall isearch-filter-predicate B E)
               (throw 'filtered (point)))))
         (goto-char POINT)
         nil))))
(defalias 'search-forward-regexp-ifp 're-search-forward-ifp)

(defun how-many-ifp (regexp &optional rstart rend interactive)
   "Modified version of `how-many' that filters (skips) matches according to
`isearch-filter-predicate'."
   (interactive
    (keep-lines-read-args "How many matches for regexp"))
   (unwind-protect
       (progn
         (advice-add 're-search-forward :override #'re-search-forward-ifp)
         (how-many regexp rstart rend interactive))
     (advice-remove 're-search-forward #'re-search-forward-ifp)))
(defalias 'count-matches-ifp 'how-many-ifp)

(defun occur-ifp (regexp &optional nlines region)
   "Modified version of `occur' that filters (skips) matches according to
`isearch-filter-predicate'."
   (interactive
    (nconc (occur-read-primary-args)
           (and (use-region-p) (list (region-bounds)))))
   (unwind-protect
       (progn
         (advice-add 're-search-forward :override #'re-search-forward-ifp)
         (occur regexp nlines region))
     (advice-remove 're-search-forward #'re-search-forward-ifp)))

These are modified versions of re-search-forward, how-many and occur 
functions that obey to isearch-filter-pradicate but I am not entirely 
certain if I have written the code rigorously.

Perhaps they could be a starting point.