Partial words on notmuch search?

Partial words on notmuch search?

[Andrei Popescu]

[-- Attachment #1: Type: text/plain, Size: 483 bytes --]

Hi everybody,

I'm using notmuch 0.10.2 from Debian backports with Stefano Zacchiroli's 
mutt integration (but I've also tried the command line interface).

After reading all the docs and a few experiments I don't see how to do 
searches on partial words. If I missed the fine manual mentioning that 
I'd be glad to RTFM, just please point me to the right FM :)

Thanks,
Andrei
-- 
If you can't explain it simply, you don't understand it well enough.
(Albert Einstein)

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: Partial words on notmuch search?

[mailinglists]

Andrei Popescu <andreimpopescu@gmail.com>
writes:

> After reading all the docs and a few experiments I don't see how to do 
> searches on partial words. If I missed the fine manual mentioning that 
> I'd be glad to RTFM, just please point me to the right FM :)

That's OK - I asked the same question once.

Use '*'.

For example, Obam* should match "Obama".

Re: Partial words on notmuch search?

[Andrei Popescu]

[-- Attachment #1: Type: text/plain, Size: 670 bytes --]

On Du, 15 ian 12, 17:07:36, mailinglists@nawaz.org wrote:
> 
> Use '*'.
> 
> For example, Obam* should match "Obama".

Hmm...

$ notmuch count 'Debian'
65888
$ notmuch count '*ebian'
3
$ notmuch count '?ebian'
3
$ notmuch count 'ebian'
3
$ notmuch count '0ebian'
0

As far as I can tell, anything that is not a number or a letter is just 
stripped.

This is also interesting:
$ notmuch count 'debian'
65888
$ notmuch count 'dEbian'
65888
$ notmuch count 'Debian'
65887

Where can I read more about this? (except the source :)

Thanks,
Andrei
-- 
If you can't explain it simply, you don't understand it well enough.
(Albert Einstein)

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: Partial words on notmuch search?

[David Bremner]

On Mon, 16 Jan 2012 22:21:03 +0200, Andrei Popescu <andreimpopescu@gmail.com> wrot> 
> $ notmuch count 'Debian'
> 65888
> $ notmuch count '*ebian'
> 3
> $ notmuch count '?ebian'
> 3
> $ notmuch count 'ebian'
> 3
> $ notmuch count '0ebian'

You have to delve into the mysteries of xapian query parsing
unfortunately.

The short answer is that "*" only works at the end of a word.

Re: Partial words on notmuch search?

[Andrei Popescu]

[-- Attachment #1: Type: text/plain, Size: 470 bytes --]

On Lu, 16 ian 12, 18:26:21, David Bremner wrote:
> 
> You have to delve into the mysteries of xapian query parsing
> unfortunately.

Aha, after a quick (web) search I found:
http://xapian.org/docs/queryparser.html

Am I correct to assume this is "the" reference I was looking for?

> The short answer is that "*" only works at the end of a word.

Thanks,
Andrei
-- 
If you can't explain it simply, you don't understand it well enough.
(Albert Einstein)

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: Partial words on notmuch search?

[Austin Clements]

Quoth Andrei Popescu on Jan 16 at 10:21 pm:
> This is also interesting:
> $ notmuch count 'debian'
> 65888
> $ notmuch count 'dEbian'
> 65888
> $ notmuch count 'Debian'
> 65887

The first two will match stemmed versions of "debian" such as
"debian's" and "debianed".  However, starting a term with a capital
letter suppresses stemming (because it suggests that it's a name,
which you wouldn't want to modify), so your last query matches only
the term "debian".  This is probably documented somewhere, though I
don't know where.

> Where can I read more about this? (except the source :)

Most of this is in the Xapian query syntax document you found.  Really
we ought to beef-up Notmuch's query syntax documentation.

Re: Partial words on notmuch search?

[Jani Nikula]

On Mon, 16 Jan 2012 21:34:31 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> Quoth Andrei Popescu on Jan 16 at 10:21 pm:
> > This is also interesting:
> > $ notmuch count 'debian'
> > 65888
> > $ notmuch count 'dEbian'
> > 65888
> > $ notmuch count 'Debian'
> > 65887
> 
> The first two will match stemmed versions of "debian" such as
> "debian's" and "debianed".  However, starting a term with a capital
> letter suppresses stemming (because it suggests that it's a name,
> which you wouldn't want to modify), so your last query matches only
> the term "debian".  This is probably documented somewhere, though I
> don't know where.

Interesting. Is this done when adding the terms to the database, or when
searching? I presume the latter. How much control does notmuch have over
this?

The assumption that one wouldn't want to have stemming for names is very
much language dependent. [1]

BR,
Jani.


[1] http://en.wikipedia.org/wiki/Finnish_noun_cases (the same works for
names as well as nouns)

Re: Partial words on notmuch search?

[Austin Clements]

Quoth Jani Nikula on Jan 17 at  7:43 pm:
> On Mon, 16 Jan 2012 21:34:31 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> > Quoth Andrei Popescu on Jan 16 at 10:21 pm:
> > > This is also interesting:
> > > $ notmuch count 'debian'
> > > 65888
> > > $ notmuch count 'dEbian'
> > > 65888
> > > $ notmuch count 'Debian'
> > > 65887
> > 
> > The first two will match stemmed versions of "debian" such as
> > "debian's" and "debianed".  However, starting a term with a capital
> > letter suppresses stemming (because it suggests that it's a name,
> > which you wouldn't want to modify), so your last query matches only
> > the term "debian".  This is probably documented somewhere, though I
> > don't know where.
> 
> Interesting. Is this done when adding the terms to the database, or when
> searching? I presume the latter. How much control does notmuch have over
> this?

This is getting a bit out of my depth, but I believe indexing is done
with both stemmed and unstemmed versions of all terms (if stemming is
enabled) so that search can use either.

For indexing, Notmuch can set the stemmer (or no stemmer).  Xapian
provides stemmers for a variety of languages:
  http://xapian.org/docs/apidoc/html/classXapian_1_1Stem.html#6c46cedf2047b159a7e4c9d4468242b1

For query parsing, Notmuch can set both the stemmer and a "stemming
strategy" that controls when it stems or doesn't stem terms:
  http://xapian.org/docs/apidoc/html/classXapian_1_1QueryParser.html#c7dc3b55b6083bd3ff98fc8b2726c8fd

Improving notmuch query documentation [was: Re: Partial words on notmuch search?]

[Andrei Popescu]

[-- Attachment #1: Type: text/plain, Size: 1173 bytes --]

On Lu, 16 ian 12, 21:34:31, Austin Clements wrote:
> Quoth Andrei Popescu on Jan 16 at 10:21 pm:
> > This is also interesting:
> > $ notmuch count 'debian'
> > 65888
> > $ notmuch count 'dEbian'
> > 65888
> > $ notmuch count 'Debian'
> > 65887
> 
> The first two will match stemmed versions of "debian" such as
> "debian's" and "debianed".  However, starting a term with a capital
> letter suppresses stemming (because it suggests that it's a name,
> which you wouldn't want to modify), so your last query matches only
> the term "debian".  This is probably documented somewhere, though I
> don't know where.

Stemming is mentioned in the Xapian docs, but I didn't understand the 
meaning until I read your explanation, thanks :)

> > Where can I read more about this? (except the source :)
> 
> Most of this is in the Xapian query syntax document you found.  Really
> we ought to beef-up Notmuch's query syntax documentation.

If I get around to write something myself where do you suggest I should 
start, the wiki or the manpage?

Thanks,
Andrei
-- 
If you can't explain it simply, you don't understand it well enough.
(Albert Einstein)

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: Improving notmuch query documentation [was: Re: Partial words on notmuch search?]

[Austin Clements]

Quoth Andrei Popescu on Jan 18 at 12:14 am:
> On Lu, 16 ian 12, 21:34:31, Austin Clements wrote:
> > Quoth Andrei Popescu on Jan 16 at 10:21 pm:
> > > Where can I read more about this? (except the source :)
> > 
> > Most of this is in the Xapian query syntax document you found.  Really
> > we ought to beef-up Notmuch's query syntax documentation.
> 
> If I get around to write something myself where do you suggest I should 
> start, the wiki or the manpage?

Probably expanding man/man7/notmuch-search-terms.7 would be the way to
go.

Re: Improving notmuch query documentation [was: Re: Partial words on notmuch search?]

[Mark Anderson]

On Tue, 17 Jan 2012 16:29:23 -0600, Austin Clements <amdragon@MIT.EDU> wrote:
> Quoth Andrei Popescu on Jan 18 at 12:14 am:
> > On Lu, 16 ian 12, 21:34:31, Austin Clements wrote:
> > > Quoth Andrei Popescu on Jan 16 at 10:21 pm:
> > > > Where can I read more about this? (except the source :)
> > > 
> > > Most of this is in the Xapian query syntax document you found.  Really
> > > we ought to beef-up Notmuch's query syntax documentation.
> > 
> > If I get around to write something myself where do you suggest I should 
> > start, the wiki or the manpage?
> 
> Probably expanding man/man7/notmuch-search-terms.7 would be the way to
> go.

I would appreciate it if the limitations of id: search were explained
there too.  I have some rules that I would love to make based on pattern
matching the message-id of the message, because I have a tool that
generates scads of email and I want to be able to delete a lot of it.

I think that id: is only matchable as an entire string, and a
confirmation of that would be nice to see.

For those who cringe when hearing the mention of _deletion_ of emails,
do you have a suggestion for how many copies of bugs in the bug database
I should store in my mail repository?  Note that IT only gives me a
couple gigabytes of home directory storage, and I don't have an SSD
Linux laptop, so the index does eventually slow down.

In other words, there are plenty of emails I love to forget ever having
received. :)


-Mark

Re: Improving notmuch query documentation [was: Re: Partial words on notmuch search?]

[Austin Clements]

Quoth Mark Anderson on Jan 20 at 12:08 pm:
> On Tue, 17 Jan 2012 16:29:23 -0600, Austin Clements <amdragon@MIT.EDU> wrote:
> > Quoth Andrei Popescu on Jan 18 at 12:14 am:
> > > On Lu, 16 ian 12, 21:34:31, Austin Clements wrote:
> > > > Quoth Andrei Popescu on Jan 16 at 10:21 pm:
> > > > > Where can I read more about this? (except the source :)
> > > > 
> > > > Most of this is in the Xapian query syntax document you found.  Really
> > > > we ought to beef-up Notmuch's query syntax documentation.
> > > 
> > > If I get around to write something myself where do you suggest I should 
> > > start, the wiki or the manpage?
> > 
> > Probably expanding man/man7/notmuch-search-terms.7 would be the way to
> > go.
> 
> I would appreciate it if the limitations of id: search were explained
> there too.  I have some rules that I would love to make based on pattern
> matching the message-id of the message, because I have a tool that
> generates scads of email and I want to be able to delete a lot of it.
> 
> I think that id: is only matchable as an entire string, and a
> confirmation of that would be nice to see.

This is true.  thread:, id:, tag:, and is: are "boolean prefixes",
which means they only match on entire strings.

What sorts of message-id patterns did you want to search for?

[RFC] http://notmuchmail.org/searching/ [was: Re: Improving notmuch query documentation]

[Andrei POPESCU]

[-- Attachment #1: Type: text/plain, Size: 766 bytes --]

On Ma, 17 ian 12, 17:29:23, Austin Clements wrote:
> Quoth Andrei Popescu on Jan 18 at 12:14 am:
> > 
> > If I get around to write something myself where do you suggest I should 
> > start, the wiki or the manpage?
> 
> Probably expanding man/man7/notmuch-search-terms.7 would be the way to
> go.

I went against your advice and created a new wiki page[1]. It is 
currently not linked from anywhere on purpose, but IMVHO it would be 
nice to have it linked from the home page.

[1] http://notmuchmail.org/searching/

Before that I would appreciate comments, corrections, etc. and 
especially something to put in the 'Synonyms' section.

Thanks,
Andrei
-- 
If you can't explain it simply, you don't understand it well enough.
(Albert Einstein)

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [RFC] http://notmuchmail.org/searching/ [was: Re: Improving notmuch query documentation]

[Austin Clements]

Quoth Andrei POPESCU on Mar 15 at 11:39 am:
> On Ma, 17 ian 12, 17:29:23, Austin Clements wrote:
> > Quoth Andrei Popescu on Jan 18 at 12:14 am:
> > > 
> > > If I get around to write something myself where do you suggest I should 
> > > start, the wiki or the manpage?
> > 
> > Probably expanding man/man7/notmuch-search-terms.7 would be the way to
> > go.
> 
> I went against your advice and created a new wiki page[1]. It is 
> currently not linked from anywhere on purpose, but IMVHO it would be 
> nice to have it linked from the home page.
> 
> [1] http://notmuchmail.org/searching/
> 
> Before that I would appreciate comments, corrections, etc. and 
> especially something to put in the 'Synonyms' section.

I think having two divergent documents covering the same thing is less
than ideal, but perhaps they could be merged in the near future.

A few comments:

The section on "Languages other than English" isn't quite correct.
Xapian has no idea what language is being used, so it will still stem
terms in other languages, but using English stemming rules.

Notmuch doesn't use synonyms.

It might be worth pointing out that "+term1" and "term1" are
equivalent.

"notmuch search -term2" doesn't actually work.  I've never looked in
to why, but I've found that Xapian ignores '-' at the beginning of a
query or a parenthesized expression.  "notmuch search term1 -term2"
will work.

In the brackets section, you'll need shell escaping for those queries
to work.  It might be worth pointing out the need for shell escaping
at the beginning.

XOR, NEAR, and ADJ were intentionally undocumented in
notmuch-search-terms because they may go away some day and we don't
want people thinking they can depend on them.

Re: [RFC] http://notmuchmail.org/searching/ [was: Re: Improving notmuch query documentation]

[Andrei POPESCU]

[-- Attachment #1: Type: text/plain, Size: 2336 bytes --]

On Jo, 15 mar 12, 17:11:08, Austin Clements wrote:
> 
> I think having two divergent documents covering the same thing is less
> than ideal, but perhaps they could be merged in the near future.

I want to have this page more or less complete and descriptive. Once 
this is done I should be able to rewrite it more like a reference.

Regarding 'notmuch help search-terms':

$ notmuch help search-terms | wc -l
88

IMHO that text is better suited for a manpage, the help should be just a 
(very short) reference to refresh ones memory. What do you think?
 
> A few comments:
> 
> The section on "Languages other than English" isn't quite correct.
> Xapian has no idea what language is being used, so it will still stem
> terms in other languages, but using English stemming rules.

Then I think it's safe to assume the results are very much dependent on 
the language, so if the language has some similarities to English Xapian 
might do some stemming.

> Notmuch doesn't use synonyms.

Thanks.

> It might be worth pointing out that "+term1" and "term1" are
> equivalent.

Yes.

> "notmuch search -term2" doesn't actually work.  I've never looked in
> to why, but I've found that Xapian ignores '-' at the beginning of a
> query or a parenthesized expression.

Not sure what you mean here. Does Xapian just ignore the '-' and 
searches as if it wasn't specified? I'm usually testing stuff with 
'notmuch count', but I get:

$ notmuch count -Debian
Unrecognized option: -Debian

With 'search' I get results, but right now I can't think of a query to 
test.

> "notmuch search term1 -term2" will work.

Does 'notmuch search -term1 term2' work?

> In the brackets section, you'll need shell escaping for those queries
> to work.  It might be worth pointing out the need for shell escaping
> at the beginning.

Right, anything other than brackets and '*'?

> XOR, NEAR, and ADJ were intentionally undocumented in
> notmuch-search-terms because they may go away some day and we don't
> want people thinking they can depend on them.

In such case I think it's better to state so.

I'll integrate all your comments (if somebody else doesn't beat me to 
it).

Kind regards,
Andrei
-- 
If you can't explain it simply, you don't understand it well enough.
(Albert Einstein)

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [RFC] http://notmuchmail.org/searching/ [was: Re: Improving notmuch query documentation]

[Austin Clements]

Quoth Andrei POPESCU on Mar 16 at  2:30 am:
> On Jo, 15 mar 12, 17:11:08, Austin Clements wrote:
> > 
> > I think having two divergent documents covering the same thing is less
> > than ideal, but perhaps they could be merged in the near future.
> 
> I want to have this page more or less complete and descriptive. Once 
> this is done I should be able to rewrite it more like a reference.
> 
> Regarding 'notmuch help search-terms':
> 
> $ notmuch help search-terms | wc -l
> 88
> 
> IMHO that text is better suited for a manpage, the help should be just a 
> (very short) reference to refresh ones memory. What do you think?

I'm not quite sure what you mean.  That text is the man page.  Though
it sounds like a great idea to have a quick syntax reference at the
top of the manpage so it's the first thing people see when they run
'notmuch help search-terms' (and they can still scroll down to get the
details if they want).

> > A few comments:
> > 
> > The section on "Languages other than English" isn't quite correct.
> > Xapian has no idea what language is being used, so it will still stem
> > terms in other languages, but using English stemming rules.
> 
> Then I think it's safe to assume the results are very much dependent on 
> the language, so if the language has some similarities to English Xapian 
> might do some stemming.

My point is that Xapian *will* do stemming, but using English stemming
rules, whether or not the language is English.  Hence it's inaccurate
to say that text in other languages will be unstemmed.  (Also, to be
fair to Xapian, it has stemmers for a whole bunch of languages; it's
notmuch that always configures it for English.)

> > Notmuch doesn't use synonyms.
> 
> Thanks.
> 
> > It might be worth pointing out that "+term1" and "term1" are
> > equivalent.
> 
> Yes.
> 
> > "notmuch search -term2" doesn't actually work.  I've never looked in
> > to why, but I've found that Xapian ignores '-' at the beginning of a
> > query or a parenthesized expression.
> 
> Not sure what you mean here. Does Xapian just ignore the '-' and 
> searches as if it wasn't specified? I'm usually testing stuff with 

I looked at this again and realized I was slightly wrong, so I had a
discussion with Olly and dug more in the code.  At a high level, a
query is a bunch of "probs" combined with boolean operators and the
actual rule is that a prob consisting solely of a single '-' term is a
syntax error.  And if a query has a syntax error, Xapian will re-parse
the entire query without any flags (which means no boolean operators,
love/hate, phrases, or wildcards).

One upshot of this rule is that a standalone negation like
'-tag:inbox' will actually search for messages *with* tag:inbox (just
like searching for 'tag:inbox'):

$ notmuch count -- -tag:inbox
650
$ notmuch count -- tag:inbox
650
$ notmuch count -- -tag:inbox x
9905

> 'notmuch count', but I get:
> 
> $ notmuch count -Debian
> Unrecognized option: -Debian

You need to tell count that it's not an option.  The standard getopt
syntax for this works:
  notmuch count -- -Debian

> With 'search' I get results, but right now I can't think of a query to 
> test.
> 
> > "notmuch search term1 -term2" will work.
> 
> Does 'notmuch search -term1 term2' work?

'notmuch search -- -term1 term2' works.

> > In the brackets section, you'll need shell escaping for those queries
> > to work.  It might be worth pointing out the need for shell escaping
> > at the beginning.
> 
> Right, anything other than brackets and '*'?

Also double quotes.  E.g.,
  notmuch search subject:"This may look like a phrase, but don't be fooled"
will search for messages with "this" in the subject and the words
"may", "look", "like", etc anywhere (and in any order).

I think that's it for Xapian metacharacters, but of course things in
the query terms themselves could need shell escaping.  Rather than try
to think about this, I generally put my whole query in single quotes
unless it's something obviously trivial that won't contain any shell
metacharacters.

> > XOR, NEAR, and ADJ were intentionally undocumented in
> > notmuch-search-terms because they may go away some day and we don't
> > want people thinking they can depend on them.
> 
> In such case I think it's better to state so.
> 
> I'll integrate all your comments (if somebody else doesn't beat me to 
> it).
> 
> Kind regards,
> Andrei

Re: [RFC] http://notmuchmail.org/searching/ [was: Re: Improving notmuch query documentation]

[Andrei POPESCU]

[-- Attachment #1: Type: text/plain, Size: 1732 bytes --]

On Jo, 15 mar 12, 22:11:24, Austin Clements wrote:
> Quoth Andrei POPESCU on Mar 16 at  2:30 am:
> > 
> > $ notmuch help search-terms | wc -l
> > 88
> > 
> > IMHO that text is better suited for a manpage, the help should be just a 
> > (very short) reference to refresh ones memory. What do you think?
> 
> I'm not quite sure what you mean.  That text is the man page.  Though
> it sounds like a great idea to have a quick syntax reference at the
> top of the manpage so it's the first thing people see when they run
> 'notmuch help search-terms' (and they can still scroll down to get the
> details if they want).

On Vi, 16 mar 12, 13:52:35, David Bremner wrote:
> On Fri, 16 Mar 2012 02:30:53 +0200, Andrei POPESCU <andreimpopescu@gmail.com> wrote:
> 
> I'm less worried about the length of the documentation than about
> fragmentation. So I think if something is reference material, it should
> go in the man pages, or at least ship with notmuch.

What I mean is that 'notmuch help search-terms' is too verbose. IMHO 
there should be very good reasons to have it longer than 20 lines or so. 
Instead it's the entire section 'SEARCH SYNTAX' from the manpage.

This opinion is based also on what I see around at other terminal 
applications. The '--help' is seldom longer than a few lines and just 
lists the available options and parameters (more like a refresher). The 
manpage then explains them in more detail.

As I see it, the manpage (specifically section 'SEARCH SYNTAX' needs to 
be expanded somewhat and 'help search-terms' shortened (a lot).

Does this make sense?

Kind regards,
Andrei
-- 
If you can't explain it simply, you don't understand it well enough.
(Albert Einstein)

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [RFC] http://notmuchmail.org/searching/ [was: Re: Improving notmuch query documentation]

[David Bremner]

On Sat, 17 Mar 2012 00:29:52 +0200, Andrei POPESCU <andreimpopescu@gmail.com> wrote:
> On Jo, 15 mar 12, 22:11:24, Austin Clements wrote:

> As I see it, the manpage (specifically section 'SEARCH SYNTAX' needs to 
> be expanded somewhat and 'help search-terms' shortened (a lot).
> 
> Does this make sense?
> 

I don't see that happening in the near future since all notmuch help
does is call man. As Austin said, you're welcome to improve the man
pages, but making them diverge from the help would require a complete
rewrite of all of the help.

d

Re: [RFC] http://notmuchmail.org/searching/ [was: Re: Improving notmuch query documentation]

[Austin Clements]

Quoth Andrei POPESCU on Mar 17 at 12:29 am:
> On Jo, 15 mar 12, 22:11:24, Austin Clements wrote:
> > Quoth Andrei POPESCU on Mar 16 at  2:30 am:
> > > 
> > > $ notmuch help search-terms | wc -l
> > > 88
> > > 
> > > IMHO that text is better suited for a manpage, the help should be just a 
> > > (very short) reference to refresh ones memory. What do you think?
> > 
> > I'm not quite sure what you mean.  That text is the man page.  Though
> > it sounds like a great idea to have a quick syntax reference at the
> > top of the manpage so it's the first thing people see when they run
> > 'notmuch help search-terms' (and they can still scroll down to get the
> > details if they want).
> 
> On Vi, 16 mar 12, 13:52:35, David Bremner wrote:
> > On Fri, 16 Mar 2012 02:30:53 +0200, Andrei POPESCU <andreimpopescu@gmail.com> wrote:
> > 
> > I'm less worried about the length of the documentation than about
> > fragmentation. So I think if something is reference material, it should
> > go in the man pages, or at least ship with notmuch.
> 
> What I mean is that 'notmuch help search-terms' is too verbose. IMHO 
> there should be very good reasons to have it longer than 20 lines or so. 
> Instead it's the entire section 'SEARCH SYNTAX' from the manpage.

It is, quite literally, the manpage.  notmuch execs man when you run
notmuch help.

This was an intentional change a few releases ago.  Previously, we did
have separate manpages and internal help documentation and it didn't
work very well since they were perpetually out of sync.  Hence the
general concern about documentation fragmentation.

> This opinion is based also on what I see around at other terminal 
> applications. The '--help' is seldom longer than a few lines and just 
> lists the available options and parameters (more like a refresher). The 
> manpage then explains them in more detail.

That's true of simple commands, but most commands with subcommands
follow a style like notmuch.  In fact, notmuch's approach was modeled
directly off of git, and most modern VCSs do similar things.

> As I see it, the manpage (specifically section 'SEARCH SYNTAX' needs to 
> be expanded somewhat and 'help search-terms' shortened (a lot).

What did you think of my suggestion that the first thing in man
search-terms be a short reference so that's what you see immediately
when you run notmuch help search-terms?  That seems to accomplish what
you want without fragmenting the documentation and seems like a good
way to write the documentation anyway.

Re: [RFC] http://notmuchmail.org/searching/ [was: Re: Improving notmuch query documentation]

[Andrei POPESCU]

[-- Attachment #1: Type: text/plain, Size: 1873 bytes --]

On Vi, 16 mar 12, 20:20:17, Austin Clements wrote:
> 
> It is, quite literally, the manpage.  notmuch execs man when you run
> notmuch help.

Aha.
 
> This was an intentional change a few releases ago.  Previously, we did
> have separate manpages and internal help documentation and it didn't
> work very well since they were perpetually out of sync.  Hence the
> general concern about documentation fragmentation.

Of course, I understand.

> > This opinion is based also on what I see around at other terminal 
> > applications. The '--help' is seldom longer than a few lines and just 
> > lists the available options and parameters (more like a refresher). The 
> > manpage then explains them in more detail.
> 
> That's true of simple commands, but most commands with subcommands
> follow a style like notmuch.  In fact, notmuch's approach was modeled
> directly off of git, and most modern VCSs do similar things.

'git help' and 'man git' are quite different on my system, but I get the 
point.

> > As I see it, the manpage (specifically section 'SEARCH SYNTAX' needs to 
> > be expanded somewhat and 'help search-terms' shortened (a lot).
> 
> What did you think of my suggestion that the first thing in man
> search-terms be a short reference so that's what you see immediately
> when you run notmuch help search-terms?  That seems to accomplish what
> you want without fragmenting the documentation and seems like a good
> way to write the documentation anyway.

I still have a question: where does the first part of
'notmuch help search-terms' come from? I can't find the corresponding 
section in the manpage? How about expanding that part just a bit and not 
show the text that comes from the manpage?

Kind regards,
Andrei
-- 
If you can't explain it simply, you don't understand it well enough.
(Albert Einstein)

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [RFC] http://notmuchmail.org/searching/ [was: Re: Improving notmuch query documentation]

[Austin Clements]

Quoth Andrei POPESCU on Mar 17 at  4:40 pm:
> > > As I see it, the manpage (specifically section 'SEARCH SYNTAX' needs to 
> > > be expanded somewhat and 'help search-terms' shortened (a lot).

I realized that I didn't have a 'SEARCH SYNTAX' section and went
digging into notmuch's code history.  Are you running from git or
0.11?

We completely restructured and unified the documentation several
months ago, but it looks like we did so just days after the 0.11
freeze, which means there hasn't been a stable release with the new
documentation yet (though one is imminent).

Re: [RFC] http://notmuchmail.org/searching/ [was: Re: Improving notmuch query documentation]

[Andrei POPESCU]

[-- Attachment #1: Type: text/plain, Size: 902 bytes --]

On Sb, 17 mar 12, 13:16:47, Austin Clements wrote:
> Quoth Andrei POPESCU on Mar 17 at  4:40 pm:
> > > > As I see it, the manpage (specifically section 'SEARCH SYNTAX' needs to 
> > > > be expanded somewhat and 'help search-terms' shortened (a lot).
> 
> I realized that I didn't have a 'SEARCH SYNTAX' section and went
> digging into notmuch's code history.  Are you running from git or
> 0.11?

0.11 (Debian unstable/backports), but I just saw 0.12~rc1 in 
experimental.
 
> We completely restructured and unified the documentation several
> months ago, but it looks like we did so just days after the 0.11
> freeze, which means there hasn't been a stable release with the new
> documentation yet (though one is imminent).

Guess I should be looking at git then.

Kind regards,
Andrei
-- 
If you can't explain it simply, you don't understand it well enough.
(Albert Einstein)

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [RFC] http://notmuchmail.org/searching/ [was: Re: Improving notmuch query documentation]

[David Bremner]

On Fri, 16 Mar 2012 02:30:53 +0200, Andrei POPESCU <andreimpopescu@gmail.com> wrote:

> 
> Regarding 'notmuch help search-terms':
> 
> $ notmuch help search-terms | wc -l
> 88
> 
> IMHO that text is better suited for a manpage, the help should be just a 
> (very short) reference to refresh ones memory. What do you think?
>  

I'm less worried about the length of the documentation than about
fragmentation. So I think if something is reference material, it should
go in the man pages, or at least ship with notmuch.

d