Two perceived query language imbalances

Two perceived query language imbalances

[Alexander Adolf]

Dear notmuch developers,

Quoting from `man notmuch-search-terms`:

> from:<name-or-address> or from:/<regex>/
>        The from: prefix is used to match the name or address of the
>        sender of an email message.
> 
> to:<name-or-address>
>        The to: prefix is used to match the names or addresses of any
>        recipient of an email message, (whether To, Cc, or Bcc).

Sorely missing to:/regex/ here. I was banging my head against this the
other day as I was searching for messages I had sent to various folks at
a specific organisation, but just couldn't remember whether their
addresses were "organisation.tld", or "organisationgroup.tld", or
"organisationlabs.tld", or similar. The workaround was to search for a
specific person's name, open a message, figure the email address scheme,
and then issue another search with that stem. "to:/regex/" would have
saved my day.

Any technical reason for "from" having a regex search, but "to" not?

> id:<message-id> or mid:<message-id> or mid:/<regex>/
>        For id: and mid:, message ID values are the literal contents of
>        the Message-ID: header of email messages, but without the '<',
>        '>' delimiters.

Similar thing here: "id:" and "mid:" can be used interchangeably, except
for regex search. Adding "id:/regex/" would seem most useful to me.


The query language should be the last thing to be getting in my way, I
think.


Many thanks in advance and looking forward to your thoughts,

  --alex

Re: Two perceived query language imbalances

[David Bremner]

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

>
> Any technical reason for "from" having a regex search, but "to" not?

Yes, it is purely technical. The regex support relies on the existence
of a Xapian value slot for the field in question. It isn't clear what
the time / space impact of adding another value slot would be. It is
probably not unbearable, but someone needs to do the tests.

>> id:<message-id> or mid:<message-id> or mid:/<regex>/
>>        For id: and mid:, message ID values are the literal contents of
>>        the Message-ID: header of email messages, but without the '<',
>>        '>' delimiters.
>
> Similar thing here: "id:" and "mid:" can be used interchangeably, except
> for regex search. Adding "id:/regex/" would seem most useful to me.
>

This was intentional, to avoid breaking existing scripts / internals
that rely on treating id: as the primary key for the database. At least
that was the concensus when we added it. It seems like one answer would
be for you to just use mid: all the time, since it already has the
behaviour you like.

d

Re: Two perceived query language imbalances

[Alexander Adolf]

Hello David,

Many thanks for your swift response, and explanations.

David Bremner <david@tethera.net> writes:

> [...]
>> Any technical reason for "from" having a regex search, but "to" not?
>
> Yes, it is purely technical. The regex support relies on the existence
> of a Xapian value slot for the field in question. It isn't clear what
> the time / space impact of adding another value slot would be. It is
> probably not unbearable, but someone needs to do the tests.

Showing of my lack of knowledge about Xapian, and hence what a "value
slot" in that context is, it seems that for giving me the suggested, new
regex capability on the "to" field would require extending the db
schema, an would therefore and inevitably have a performance impact
which would need to be evaluated before committing to adding this
feature.

Understood, and fair enough.

Is there a "wish list" of sorts, and if so, could it perhaps seem
worthwhile adding this to the wish list?

>>> id:<message-id> or mid:<message-id> or mid:/<regex>/
>>>        For id: and mid:, message ID values are the literal contents of
>>>        the Message-ID: header of email messages, but without the '<',
>>>        '>' delimiters.
>>
>> Similar thing here: "id:" and "mid:" can be used interchangeably, except
>> for regex search. Adding "id:/regex/" would seem most useful to me.
>>
>
> This was intentional, to avoid breaking existing scripts / internals
> that rely on treating id: as the primary key for the database. At least
> that was the concensus when we added it. It seems like one answer would
> be for you to just use mid: all the time, since it already has the
> behaviour you like.
> [...]

I see, thanks for the insight. In that light, what would be your take on
featuring "id" less prominently in the documentation, so as to foster
the sole use of "mid" for all new script developments?

You could maybe even go as far as marking "id" as "deprecated and kept
for backwards compatibility with legacy scripts only"; but without
actually ever removing it, of course, since as you say it's needed for
internal purposes.


My intent on both points is not to give the impression anything were
broken, but to have the query language and its documentation cause as
little confusion as possible, even for the casual user (i.e. myself).


Many thanks and looking forward to your thoughts,

  --alexander

Re: Two perceived query language imbalances

[Michael J Gruber]

Alexander Adolf venit, vidit, dixit 2021-04-27 13:34:03:
> Hello David,
[...]
> >>> id:<message-id> or mid:<message-id> or mid:/<regex>/
> >>>        For id: and mid:, message ID values are the literal contents of
> >>>        the Message-ID: header of email messages, but without the '<',
> >>>        '>' delimiters.
> >>
> >> Similar thing here: "id:" and "mid:" can be used interchangeably, except
> >> for regex search. Adding "id:/regex/" would seem most useful to me.
> >>
> >
> > This was intentional, to avoid breaking existing scripts / internals
> > that rely on treating id: as the primary key for the database. At least
> > that was the concensus when we added it. It seems like one answer would
> > be for you to just use mid: all the time, since it already has the
> > behaviour you like.
> > [...]
> 
> I see, thanks for the insight. In that light, what would be your take on
> featuring "id" less prominently in the documentation, so as to foster
> the sole use of "mid" for all new script developments?
> 
> You could maybe even go as far as marking "id" as "deprecated and kept
> for backwards compatibility with legacy scripts only"; but without
> actually ever removing it, of course, since as you say it's needed for
> internal purposes.

In that case, I suggest that `notmuch search --output=messages` outputs
message ids in the form `mid:...`, as well. Users can discover
`--output=messages` from command completion and could learn the "right"
search prefix from the command output.

Regards
Michael

Re: Two perceived query language imbalances

[David Bremner]

Michael J Gruber <git@grubix.eu> writes:

>
> In that case, I suggest that `notmuch search --output=messages` outputs
> message ids in the form `mid:...`, as well. Users can discover
> `--output=messages` from command completion and could learn the "right"
> search prefix from the command output.
>

In fact that's the kind of thing where we did not want to change the
function of id:. There is some weird corner cases because /foo/ is
(probably) a valid message-id.

Re: Two perceived query language imbalances

[David Bremner]

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

> Is there a "wish list" of sorts, and if so, could it perhaps seem
> worthwhile adding this to the wish list?

Yes, there is a list at

     https://nmbug.notmuchmail.org/status/#Wish-list

Your message is now on that list.