A puzzle to solve: saved categories vs. tags

A puzzle to solve: saved categories vs. tags

[John Wiegley]

Ok, I have 147k of archived todos and notes now.  Some are tagged, some are
not.  Most have an ARCHIVE_CATEGORY property (ever since Carsten so kindly
implemented it).

My desire: To hit C-u C-c \ and have it prompt me for the "entity" its going
to search for.  The possible entities are TAGS, or a property name.  Then it
asks for the text string, as usual.  The result should make it possible for me
to see all entries that came from a specific category or group of categories.

john

Re: A puzzle to solve: saved categories vs. tags

[Bastien]

John Wiegley <johnw@newartisans.com> writes:

> My desire: To hit C-u C-c \ and have it prompt me for the "entity" its
> going to search for. The possible entities are TAGS, or a property
> name. Then it asks for the text string, as usual. The result should
> make it possible for me to see all entries that came from a specific
> category or group of categories.

For now C-u C-c \ does something different:

,----[ (info "(Org)Tag searches") ]
| `C-c \'
|      Create a sparse tree with all headlines matching a tags search.
|      With a `C-u' prefix argument, ignore headlines that are not a TODO
|      line.
`----

I'm not using C-u C-c \ that much, mostly because I use agenda views a
lot.  But your request to make the C-c \ query interactive -- whatever
key it is bound to -- sounds nice.

As you probable know, you can already perform complex searches like:

  C-c \ +Urgent-@Work+CATEGORY="code"/NEXT

This will search for headlines with the tag "Urgent", without the tag
"@Work", with category "code" and which TODO keyword is "NEXT".  Yes,
that's quite complex and I guess we sometime prefer not to care about
the syntax of the query, but rather be prompted for it.

If Carsten implements this, I think all the search facilities in Org
will need a bit of uniformization/clarification.  For now we have:

| key     | function             | knows about    | C-u              |
|---------+----------------------+----------------+------------------|
| C-c /   | org-occur            | regexp         |                  |
| C-c \   | org-tags-sparse-tree | tags, and more | restrict to TODO |
| C-c C-v | org-show-todo-tree   | todo keywords  | ask for keyword  |

A few ideas about this:

1. I tend to use C-c \ a lot than C-c / -- but I find the C-c / key much
   more convenient.  I guess it's far too late to switch, but still.

2. AFAIK org-tags-sparse-tree is more powerful than org-show-todo-tree.
   If we add the possibility to build interactive queries, it will
   definitely become *the* universal search interface for Org.

3. Being able to restrict to TODO entries when searching for tags is
   nice, we should keep it somehow.  Maybe org-show-todo-tree could
   do something similar by restricting the search to tags?

If I sum up, here are the search capacities that we could end up with:

1. Search for a regexp
2. Search for a complex query
3. Prompt interactively for a complex query
4. Show all tagged entries
5. Prompt for a specific tag
6. Prompt for a specific tag (restricting to TODO entries)
7. Show all TODO entries
8. Prompt for a specific TODO
9. Prompt for a specific TODO (restricting to tagged entries)

All this might need to be put in the search functions somehow:

 {0,1,2,3} would be the "universal" search functions  
 {4,5,6}   would be the tag search
 {7,8,9}    would be the TODO search

Well, sorry for such a long input, I hope it's still useful.

-- 
Bastien

Re: A puzzle to solve: saved categories vs. tags

[John Wiegley]

Bastien <bzg@altern.org> writes:

> 1. Search for a regexp
> 2. Search for a complex query
> 3. Prompt interactively for a complex query
> 4. Show all tagged entries
> 5. Prompt for a specific tag
> 6. Prompt for a specific tag (restricting to TODO entries)
> 7. Show all TODO entries
> 8. Prompt for a specific TODO
> 9. Prompt for a specific TODO (restricting to tagged entries)

I didn't see "Prompt for a specific property and value" in your list.  Is this
already possible using a complex query?

John

Re: A puzzle to solve: saved categories vs. tags

[Carsten Dominik]

On Oct 7, 2007, at 14:27, Bastien wrote:

> If Carsten implements this, I think all the search facilities in Org
> will need a bit of uniformization/clarification.

Admittedly the search functions are one of the areas that
demonstrate most clearly how Org-mode has grown and evolved, adding
little things.  Could use a complete re-write, but there is no
time for this.

>   For now we have:
>
> | key     | function             | knows about    | C-u              |
> |---------+----------------------+----------------+------------------|
> | C-c /   | org-occur            | regexp         |                  |
> | C-c \   | org-tags-sparse-tree | tags, and more | restrict to TODO |
> | C-c C-v | org-show-todo-tree   | todo keywords  | ask for keyword  |
>
> A few ideas about this:
>
> 1. I tend to use C-c \ a lot than C-c / -- but I find the C-c / key 
> much
>    more convenient.  I guess it's far too late to switch, but still.

I guess you use a French keyboard?  On mine both are equally easily 
accessed.

> 2. AFAIK org-tags-sparse-tree is more powerful than org-show-todo-tree.
>    If we add the possibility to build interactive queries, it will
>    definitely become *the* universal search interface for Org.

That is true, but newbies will start with simple things, and it
is good to have simple commands to start off.

Thanks for the list of search options, a good start.

- Carsten

Re: A puzzle to solve: saved categories vs. tags

[Carsten Dominik]

[-- Attachment #1: Type: text/plain, Size: 1029 bytes --]


On Oct 7, 2007, at 11:14, John Wiegley wrote:

> Ok, I have 147k of archived todos and notes now.  Some are tagged, 
> some are
> not.  Most have an ARCHIVE_CATEGORY property (ever since Carsten so 
> kindly
> implemented it).
>
> My desire: To hit C-u C-c \ and have it prompt me for the "entity" its 
> going
> to search for.  The possible entities are TAGS, or a property name.  
> Then it
> asks for the text string, as usual.  The result should make it 
> possible for me
> to see all entries that came from a specific category or group of 
> categories.

If I understand correctly, this will be just for typing
convenience?  So that you do not have to fully type the
property name?

Because you can of course do this search as Bastien described.

Also, should this result in a sparse tree, or in an agenda buffer?
I guess you mean a sparse tree since you refer to `C-c \'.


Here is a first attempt at such a function, comments are welcome!
Maybe we should bind this function to `C-c /', instead of org-occur?

- Carsten


[-- Attachment #2: org-sparse-tree-interactive.el --]
[-- Type: application/octet-stream, Size: 1608 bytes --]

(defun org-sparse-tree (&optional arg)
  "Create a sparse tree, prompt for the details.
This command can create sparse trees.  You first need to select the type
of match used to create the tree:

t      Show entries with a specific TODO keyword.
T      Show entries selected by a tags match.
p      Enter a property name and its value (both with completion on existing
       names/values) and show entries with that property.
r      Show entries matching a regular expression"
  (interactive "P")
  (let (ans kwd value)
    (message "Sparse tree: [r]egexp   [t]odo-kwd   [T]ag   [p]roperty")
    (setq ans (read-char-exclusive))
    (cond
     ((equal ans ?t)
      (org-show-todo-tree '(4)))
     ((equal ans ?T)
      (call-interactively 'org-tags-sparse-tree))
     ((member ans '(?p ?P))
      (setq kwd (completing-read "Property: " 
				 (mapcar 'list (org-buffer-property-keys t t))))
      (setq value (completing-read "Value: "
				   (mapcar 'list (org-property-values kwd))))
      (or (string-match "\\`{.*}\\'") (setq value (concat "\"" value "\"")))
      (org-tags-sparse-tree arg (concat kwd "=" value)))
     ((member ans '(?r ?R))
      (call-interactively 'org-occur))
     (t (error "No such sparse tree command \"%c\"" ans)))))

(defun org-property-values (key)
  "Return a list of all values of property KEY."
  (save-excursion
    (save-restriction
      (widen)
      (goto-char (point-min))
      (let ((re (concat "^[ \t]*:" key ":[ \t]*\\(\\S-.*\\)"))
	    values)
	(while (re-search-forward re nil t)
	  (add-to-list 'values (org-trim (match-string 1))))
	(delete "" values)))))

[-- Attachment #3: Type: text/plain, Size: 146 bytes --]



--
Carsten Dominik
Sterrenkundig Instituut "Anton Pannekoek"
Universiteit van Amsterdam
Kruislaan 403
NL-1098SJ Amsterdam
phone: +31 20 525 7477

[-- Attachment #4: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: A puzzle to solve: saved categories vs. tags

[Bastien]

John Wiegley <johnw@newartisans.com> writes:

> Bastien <bzg@altern.org> writes:
>
>> 1. Search for a regexp
>> 2. Search for a complex query
>> 3. Prompt interactively for a complex query
>> 4. Show all tagged entries
>> 5. Prompt for a specific tag
>> 6. Prompt for a specific tag (restricting to TODO entries)
>> 7. Show all TODO entries
>> 8. Prompt for a specific TODO
>> 9. Prompt for a specific TODO (restricting to tagged entries)
>
> I didn't see "Prompt for a specific property and value" in your list. 

This would be the interactive (maybe prefix) version of "search for a
complex query".

> Is this already possible using a complex query?

Re-quoting myself:

,----
| As you probable know, you can already perform complex searches like:
| 
|   C-c \ +Urgent-@Work+CATEGORY="code"/NEXT
| 
| This will search for headlines with the tag "Urgent", without the tag
| "@Work", with category "code" and which TODO keyword is "NEXT".  Yes,
| that's quite complex and I guess we sometime prefer not to care about
| the syntax of the query, but rather be prompted for it.
`----

Then to search for a specific pair of property-value:

C-c \ PROPERTY="value"

As for now "value" should be the exact string, not a regexp.

-- 
Bastien

Re: A puzzle to solve: saved categories vs. tags

[Carsten Dominik]

On Oct 8, 2007, at 12:54, Bastien wrote:

> Carsten Dominik <dominik@science.uva.nl> writes:
>
>> Here is a first attempt at such a function, comments are welcome!
>> Maybe we should bind this function to `C-c /', instead of org-occur?
>
> Yes, this binding would be okay, unless there are lots of people using
> org-occur out there.
>
> And `org-sparse-tree' should match both John request and what I called
> (in such a bombastic manner) a "universal" search interface.
>
> There is a problem with (org-buffer-property-keys t t). Here is the
> backtrace:


Arrrgh!

make that (org-buffer-property-keys t) for now, bug fix in next release.

- Carsten

Re: A puzzle to solve: saved categories vs. tags

[Bastien]

Carsten Dominik <dominik@science.uva.nl> writes:

> Here is a first attempt at such a function, comments are welcome!
> Maybe we should bind this function to `C-c /', instead of org-occur?

Yes, this binding would be okay, unless there are lots of people using
org-occur out there.

And `org-sparse-tree' should match both John request and what I called
(in such a bombastic manner) a "universal" search interface.

There is a problem with (org-buffer-property-keys t t). Here is the
backtrace:

,----
| Debugger entered--Lisp error: (wrong-type-argument symbolp ("TODO" "TAGS" "ALLTAGS" "DEADLINE" "SCHEDULED" "CLOCK" "PRIORITY" "Release" "Incompatible_ALL" "Elapsed_time" "Location" "LOCATION_ALL" "People_ALL" "COLUMNS" "LOCATION" "CATEGORY" "ARCHIVE"))
|   add-to-list(("TODO" "TAGS" "ALLTAGS" "DEADLINE" "SCHEDULED" "CLOCK" "PRIORITY" "Release" "Incompatible_ALL" "Elapsed_time" "Location" "LOCATION_ALL" "People_ALL" "COLUMNS" "LOCATION" "CATEGORY" "ARCHIVE") "CATEGORY")
|   org-buffer-property-keys(t t)
|   eval((org-buffer-property-keys t t))
|   eval-expression((org-buffer-property-keys t t) nil)
|   call-interactively(eval-expression)
`----

-- 
Bastien

Re: A puzzle to solve: saved categories vs. tags

[Carsten Dominik]

[-- Attachment #1: Type: text/plain, Size: 397 bytes --]


On Oct 8, 2007, at 14:39, Bastien wrote:

> Carsten Dominik <dominik@science.uva.nl> writes:
>
>> Arrrgh!
>>
>> make that (org-buffer-property-keys t) for now, bug fix in next
>> release.
>
> This patch is also needed:

Indeed, thanks a lot.  So to make this simpler, here is
the modified version that should work out of the box.
Sorry once again for the mess and the bandwidth waste.

- Carsten

[-- Attachment #2: org-sparse-tree-interactive.el --]
[-- Type: application/octet-stream, Size: 1615 bytes --]

(defun org-sparse-tree (&optional arg)
  "Create a sparse tree, prompt for the details.
This command can create sparse trees.  You first need to select the type
of match used to create the tree:

t      Show entries with a specific TODO keyword.
T      Show entries selected by a tags match.
p      Enter a property name and its value (both with completion on existing
       names/values) and show entries with that property.
r      Show entries matching a regular expression"
  (interactive "P")
  (let (ans kwd value)
    (message "Sparse tree: [r]egexp   [t]odo-kwd   [T]ag   [p]roperty")
    (setq ans (read-char-exclusive))
    (cond
     ((equal ans ?t)
      (org-show-todo-tree '(4)))
     ((equal ans ?T)
      (call-interactively 'org-tags-sparse-tree))
     ((member ans '(?p ?P))
      (setq kwd (completing-read "Property: " 
				 (mapcar 'list (org-buffer-property-keys))))
      (setq value (completing-read "Value: "
				   (mapcar 'list (org-property-values kwd))))
      (unless (string-match "\\`{.*}\\'" value)
	(setq value (concat "\"" value "\"")))
      (org-tags-sparse-tree arg (concat kwd "=" value)))
     ((member ans '(?r ?R))
      (call-interactively 'org-occur))
     (t (error "No such sparse tree command \"%c\"" ans)))))

(defun org-property-values (key)
  "Return a list of all values of property KEY."
  (save-excursion
    (save-restriction
      (widen)
      (goto-char (point-min))
      (let ((re (concat "^[ \t]*:" key ":[ \t]*\\(\\S-.*\\)"))
	    values)
	(while (re-search-forward re nil t)
	  (add-to-list 'values (org-trim (match-string 1))))
	(delete "" values)))))

[-- Attachment #3: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: A puzzle to solve: saved categories vs. tags

[Carsten Dominik]

On Oct 7, 2007, at 14:27, Bastien wrote:

> If Carsten implements this, I think all the search facilities in Org
> will need a bit of uniformization/clarification.

Admittedly the search functions are one of the areas that
demonstrate most clearly how Org-mode has grown and evolved,
adding little things.

>   For now we have:
>
> | key     | function             | knows about    | C-u              |
> |---------+----------------------+----------------+------------------|
> | C-c /   | org-occur            | regexp         |                  |
> | C-c \   | org-tags-sparse-tree | tags, and more | restrict to TODO |
> | C-c C-v | org-show-todo-tree   | todo keywords  | ask for keyword  |
>
> A few ideas about this:
>
> 1. I tend to use C-c \ a lot than C-c / -- but I find the C-c / key 
> much
>    more convenient.  I guess it's far too late to switch, but still.

I guess you use a French keyboard?  On mine both are equally easily 
accessed.

> 2. AFAIK org-tags-sparse-tree is more powerful than org-show-todo-tree.
>    If we add the possibility to build interactive queries, it will
>    definitely become *the* universal search interface for Org.

That is true, but newbies will start with simple things, and it
is good to have simple commands to start off.

Thanks for the list of search options, a good start.

- Carsten



--
Carsten Dominik
Sterrenkundig Instituut "Anton Pannekoek"
Universiteit van Amsterdam
Kruislaan 403
NL-1098SJ Amsterdam
phone: +31 20 525 7477