unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* unread message appear `folded'
@ 2013-08-05 18:00 Ramakrishnan Muthukrishnan
  2013-08-05 18:05 ` Ramakrishnan Muthukrishnan
  0 siblings, 1 reply; 4+ messages in thread
From: Ramakrishnan Muthukrishnan @ 2013-08-05 18:00 UTC (permalink / raw)
  To: notmuch

[-- Attachment #1: Type: text/plain, Size: 538 bytes --]

Hi,

Pardon me for using non-standard terms here because of lack of knowledge
of the language used in the email world.

I have this thread in which some messages are tagged unread. When the
thread is opened, I see that those unread messages are not shown in full
in the threaded view. Instead they appear `folded' with only the summary
appearing in the thread, as if they are read. 

Here is a screenshot: http://i.imgur.com/Eh8SKe6.png

I am attaching the messages in the thread shown in the screenshot as a
tar.bz2 file.

Ramakrishnan


[-- Attachment #2: messages in the thread --]
[-- Type: application/octet-stream, Size: 7805 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: unread message appear `folded'
  2013-08-05 18:00 unread message appear `folded' Ramakrishnan Muthukrishnan
@ 2013-08-05 18:05 ` Ramakrishnan Muthukrishnan
  2013-08-05 18:59   ` Austin Clements
  0 siblings, 1 reply; 4+ messages in thread
From: Ramakrishnan Muthukrishnan @ 2013-08-05 18:05 UTC (permalink / raw)
  To: notmuch

Ramakrishnan Muthukrishnan <ram@rkrishnan.org> writes:

> Hi,
>
> Pardon me for using non-standard terms here because of lack of knowledge
> of the language used in the email world.
>
> I have this thread in which some messages are tagged unread. When the
> thread is opened, I see that those unread messages are not shown in full
> in the threaded view. Instead they appear `folded' with only the summary
> appearing in the thread, as if they are read. 
>
> Here is a screenshot: http://i.imgur.com/Eh8SKe6.png
>
> I am attaching the messages in the thread shown in the screenshot as a
> tar.bz2 file.

I just forgot to mention that I see this only when I do a
notmuch-search with `*' and not when I search with a tag. 

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: unread message appear `folded'
  2013-08-05 18:05 ` Ramakrishnan Muthukrishnan
@ 2013-08-05 18:59   ` Austin Clements
  2021-12-26 12:51     ` David Bremner
  0 siblings, 1 reply; 4+ messages in thread
From: Austin Clements @ 2013-08-05 18:59 UTC (permalink / raw)
  To: Ramakrishnan Muthukrishnan; +Cc: notmuch

Quoth Ramakrishnan Muthukrishnan on Aug 05 at 11:35 pm:
> Ramakrishnan Muthukrishnan <ram@rkrishnan.org> writes:
> 
> > Hi,
> >
> > Pardon me for using non-standard terms here because of lack of knowledge
> > of the language used in the email world.
> >
> > I have this thread in which some messages are tagged unread. When the
> > thread is opened, I see that those unread messages are not shown in full
> > in the threaded view. Instead they appear `folded' with only the summary
> > appearing in the thread, as if they are read. 
> >
> > Here is a screenshot: http://i.imgur.com/Eh8SKe6.png
> >
> > I am attaching the messages in the thread shown in the screenshot as a
> > tar.bz2 file.
> 
> I just forgot to mention that I see this only when I do a
> notmuch-search with `*' and not when I search with a tag. 

'*' turns out to be the key.  When showing a thread,
notmuch-show-build-buffer constructs a new query from the thread ID
and the original search query of the form

  thread:X and (original query)

If the original query is '*', this is

  thread:X and (*)

But * isn't part of the Xapian query syntax.  Notmuch specially
handles queries that exactly match "*" before passing the query to
Xapian.  When * is embedded in a larger query, this special casing
doesn't apply.  In fact, Xapian parses this query as

  (Tmail AND (and:(pos=1) FILTER GX))

The "and" in the query turned into a plain search term (the "AND" is a
proper boolean operator, but is unrelated to the provided query).

This is a symptom of a general problem where we assume queries are
textually composable, when they are not.  We have the same problem at
least in notmuch-search-filter and in notmuch tag query optimization.
In this particular case, the * causes Xapian's query grammar to fail
to parse the query, which Xapian handles by re-parsing the entire
query will all query features disabled (which includes disabling
support for boolean operators).  Unfortunately, just handling * better
isn't really a solution because it's just one of many things that
violates query composability.

Some solutions I can see are:

1) Switch to a composable query syntax (which would include *).  This,
   obviously, requires a custom query parser, but is the most
   localized change I can think of and keeps queries as strings.

2) Never construct queries by pasting strings together.  This would
   require changes to both the libnotmuch and CLI interfaces and
   queries could no longer be strings, but in the words of Alan
   Perlis, the string is a stark data structure.  (In the case of
   show, I would actually love it if we could specify separate search
   and match queries because that would eliminate --entire-thread as
   well as the fallback in notmuch-show-build-buffer when search
   returns nothing, but I digress.)

3) Keep queries as plain strings, but switch to some hybrid syntax
   that lets us combine Xapian queries with composable operators
   parsed by notmuch.  When we need to combine queries, do it using
   the composable operators.  This actually may not be a bad way to
   transition to a full custom query parser; I think it would be
   relatively easy to take over parsing Xapian's boolean syntax, but
   leave the "prob" parsing to Xapian.

I lean towards 3 because it seems like the least disruptive and offers
a smooth transition to 1.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: unread message appear `folded'
  2013-08-05 18:59   ` Austin Clements
@ 2021-12-26 12:51     ` David Bremner
  0 siblings, 0 replies; 4+ messages in thread
From: David Bremner @ 2021-12-26 12:51 UTC (permalink / raw)
  To: Ramakrishnan Muthukrishnan; +Cc: notmuch

Austin Clements <amdragon@MIT.EDU> writes:

> Quoth Ramakrishnan Muthukrishnan on Aug 05 at 11:35 pm:
>> Ramakrishnan Muthukrishnan <ram@rkrishnan.org> writes:
>> 
>> > Hi,
>> >
>> > Pardon me for using non-standard terms here because of lack of knowledge
>> > of the language used in the email world.
>> >
>> > I have this thread in which some messages are tagged unread. When the
>> > thread is opened, I see that those unread messages are not shown in full
>> > in the threaded view. Instead they appear `folded' with only the summary
>> > appearing in the thread, as if they are read. 
>> >
>> > Here is a screenshot: http://i.imgur.com/Eh8SKe6.png
>> >
>> > I am attaching the messages in the thread shown in the screenshot as a
>> > tar.bz2 file.
>> 
>> I just forgot to mention that I see this only when I do a
>> notmuch-search with `*' and not when I search with a tag. 
>
> '*' turns out to be the key.  When showing a thread,
> notmuch-show-build-buffer constructs a new query from the thread ID
> and the original search query of the form
>
>   thread:X and (original query)
>
> If the original query is '*', this is
>
>   thread:X and (*)
>
> But * isn't part of the Xapian query syntax.  Notmuch specially
> handles queries that exactly match "*" before passing the query to
> Xapian.  When * is embedded in a larger query, this special casing
> doesn't apply.  In fact, Xapian parses this query as
>
>   (Tmail AND (and:(pos=1) FILTER GX))
>
> The "and" in the query turned into a plain search term (the "AND" is a
> proper boolean operator, but is unrelated to the provided query).
>
> This is a symptom of a general problem where we assume queries are
> textually composable, when they are not.  We have the same problem at
> least in notmuch-search-filter and in notmuch tag query optimization.
> In this particular case, the * causes Xapian's query grammar to fail
> to parse the query, which Xapian handles by re-parsing the entire
> query will all query features disabled (which includes disabling
> support for boolean operators).  Unfortunately, just handling * better
> isn't really a solution because it's just one of many things that
> violates query composability.
>
> Some solutions I can see are:
>
> 1) Switch to a composable query syntax (which would include *).  This,
>    obviously, requires a custom query parser, but is the most
>    localized change I can think of and keeps queries as strings.
>
> 2) Never construct queries by pasting strings together.  This would
>    require changes to both the libnotmuch and CLI interfaces and
>    queries could no longer be strings, but in the words of Alan
>    Perlis, the string is a stark data structure.  (In the case of
>    show, I would actually love it if we could specify separate search
>    and match queries because that would eliminate --entire-thread as
>    well as the fallback in notmuch-show-build-buffer when search
>    returns nothing, but I digress.)
>
> 3) Keep queries as plain strings, but switch to some hybrid syntax
>    that lets us combine Xapian queries with composable operators
>    parsed by notmuch.  When we need to combine queries, do it using
>    the composable operators.  This actually may not be a bad way to
>    transition to a full custom query parser; I think it would be
>    relatively easy to take over parsing Xapian's boolean syntax, but
>    leave the "prob" parsing to Xapian.
>
> I lean towards 3 because it seems like the least disruptive and offers
> a smooth transition to 1.

Update on this very old bug. As of notmuch 0.34 the notmuch CLI supports
both 1 and 3 via --query=sexp, if compiled with the (currently optional)
sfsexp library. My plan is that once sfsexp is sufficiently widely
available (whatever that turns out to mean) to make it a mandatory
dependency. At that point we can transition the emacs front end to using
notmuch-sexp-queries(7) to compose queries.

In principle we could start that work now by checking "notmuch config
get built_with.sexp_queries". I'm not sure if it's worth the extra
complexity, but it would make the transition smoother, and probably help
test the s-expression query parser.

d

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2021-12-26 12:51 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-08-05 18:00 unread message appear `folded' Ramakrishnan Muthukrishnan
2013-08-05 18:05 ` Ramakrishnan Muthukrishnan
2013-08-05 18:59   ` Austin Clements
2021-12-26 12:51     ` David Bremner

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).