inserting files within remember templates

inserting files within remember templates

[Adam Spiers]

I'm finally getting around to setting up remember properly for regular
use.  One thing I think I'll need is the ability to include the
contents of an external file in a remember template at the time of
instantiation.  Or if there was a % escape sequence for executing
arbitrary elisp, that would be even better, of course.  Is there
anything like that at the moment?  I couldn't see anything in the
docs.

Re: inserting files within remember templates

[Eddward DeVilla]

I haven't used remember, so I don't know the limitations, but could
C-x,C-i work?

Edd

On 11/5/07, Adam Spiers <orgmode@adamspiers.org> wrote:
> I'm finally getting around to setting up remember properly for regular
> use.  One thing I think I'll need is the ability to include the
> contents of an external file in a remember template at the time of
> instantiation.  Or if there was a % escape sequence for executing
> arbitrary elisp, that would be even better, of course.  Is there
> anything like that at the moment?  I couldn't see anything in the
> docs.
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>

Re: inserting files within remember templates

[Carsten Dominik]

Setting up remember mode runs a remember-mode-hook, you can use that.

I would be interested to know what kind of applications you have in mind
when you want to include a file into a template, or execute elisp code.
How about a few examples to persuade us that this is needed?

- Carsten

On  5Nov2007, at 7:17 PM, Adam Spiers wrote:

> I'm finally getting around to setting up remember properly for regular
> use.  One thing I think I'll need is the ability to include the
> contents of an external file in a remember template at the time of
> instantiation.  Or if there was a % escape sequence for executing
> arbitrary elisp, that would be even better, of course.  Is there
> anything like that at the moment?  I couldn't see anything in the
> docs.
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: inserting files within remember templates

[Adam Spiers]

On Tue, Nov 06, 2007 at 06:41:36AM +0100, Carsten Dominik wrote:
> Setting up remember mode runs a remember-mode-hook, you can use that.

That could be a workaround.

> I would be interested to know what kind of applications you have in mind
> when you want to include a file into a template, or execute elisp code.
> How about a few examples to persuade us that this is needed?

I want to integrate my use of mutt with org-mairix.el and remember.el.
Because mutt is an external application, I need some way of getting a
Message-Id which has been extracted from a message via a mutt helper
back into emacs.  Currently I do this by coding the helper to dump the
Message-Id into ~/.clip-mairix, and then the elisp code inserts the
contents of this file back into the org buffer.  However I would like
it to be inserted via a remember template, hence the request.  Calling
remember via emacsclient might possibly be an alternate way of
achieving the same thing, but I haven't thought about this yet.  If
one were to embed the text being passed from the mutt helper in the
CLI arguments to emacsclient, quoting might end up getting quite
hairy.  Allowing the remember template to extract the message id from
the environment via getenv might be a cleaner solution in that case.

That was the main use case, but I am sure people could think of
others.  Here is another one which is a bit silly but demonstrates the
point (and would be more conveniently invoked from within emacs than
via emacsclient): I have just started maintaining a journal of my
laptop's battery life to see whether it can be extended much by doing
full discharges more regularly.  So every day or two I want to log the
output of commands like:

  grep 'last full' /proc/acpi/battery/BAT0/info

This could easily be accomplished if remember templates allowed syntax
such as

,------
| * %T
| %(shell-command-to-string "grep 'last full' /proc/acpi/battery/BAT0/info")
`------

Re: inserting files within remember templates

[Adam Spiers]

On Tue, Nov 06, 2007 at 04:36:47PM +0000, Adam Spiers wrote:

[snip]

> This could easily be accomplished if remember templates allowed syntax
> such as
> 
> ,------
> | * %T
> | %(shell-command-to-string "grep 'last full' /proc/acpi/battery/BAT0/info")
> `------

OK, it turns out that this was easy to implement, and I think the
patch is small enough that it could be accepted even though I haven't
got around to sending back the copyright assignment form yet (sorry -
this *will* happen at some point!)

Diff against 5.13i:

--- a/org.el	Wed Oct 31 09:46:35 2007 +0000
+++ b/org.el	Tue Nov 06 22:30:13 2007 +0000
@@ -12806,6 +12806,17 @@ to be run from that hook to fucntion pro
 	  (replace-match
 	   (or (eval (intern (concat "v-" (match-string 1)))) "")
 	   t t))
+        ;; %() embedded elisp
+	(goto-char (point-min))
+        (while (re-search-forward "%\\((.+)\\)" nil t)
+          (goto-char (match-beginning 0))
+          (let ((template-start (point)))
+            (forward-char 1)
+            (condition-case error
+                (let ((result (eval (read (current-buffer)))))
+                  (delete-region template-start (point))
+                  (insert result))
+              (error (message "Error `%s' in remember template" error)))))
 	;; From the property list
 	(when plist-p
 	  (goto-char (point-min))


There might need to be some debate about how to handle read errors in
the case of invalid syntax.  Or perhaps some people already have
`%(...)' within their remember templates for some really unusual
reason, and don't want it interpreted?  Though in the latter case, one
could argue that it might make sense to require `%' always to be
escaped as `%%' if used literally, to be on the safe side.

Re: inserting files within remember templates

[Bastien]

Adam Spiers <orgmode@adamspiers.org> writes:

> OK, it turns out that this was easy to implement, and I think the
> patch is small enough that it could be accepted even though I haven't
> got around to sending back the copyright assignment form yet (sorry -
> this *will* happen at some point!)

This patch is nice, thanks!

Just a small caveat: If Carsten accepts it (or implements this function)
then we should make clear for beginners that the elisp code will only be
executed in the remember buffer, not in the buffer where `org-remember'
was invoked.

> There might need to be some debate about how to handle read errors in
> the case of invalid syntax.  Or perhaps some people already have
> `%(...)' within their remember templates for some really unusual
> reason, and don't want it interpreted?

Maybe %^(...) is clearer then.  In both %^{...} and %^(...) the ^ char
would mean that some action is undertaken.

-- 
Bastien

Re: inserting files within remember templates

[Carsten Dominik]

On  7Nov2007, at 3:47 AM, Bastien wrote:

> Adam Spiers <orgmode@adamspiers.org> writes:
>
>> OK, it turns out that this was easy to implement, and I think the
>> patch is small enough that it could be accepted even though I haven't
>> got around to sending back the copyright assignment form yet (sorry -
>> this *will* happen at some point!)
>
> This patch is nice, thanks!
>
> Just a small caveat: If Carsten accepts it (or implements this  
> function)
> then we should make clear for beginners that the elisp code will  
> only be
> executed in the remember buffer, not in the buffer where `org- 
> remember'
> was invoked.
>
>> There might need to be some debate about how to handle read errors in
>> the case of invalid syntax.  Or perhaps some people already have
>> `%(...)' within their remember templates for some really unusual
>> reason, and don't want it interpreted?
>
> Maybe %^(...) is clearer then.  In both %^{...} and %^(...) the ^ char
> would mean that some action is undertaken.

Hmmm, no, ^ means interactive.....

- Carsten

Re: inserting files within remember templates

[Carsten Dominik]

On  6Nov2007, at 11:39 PM, Adam Spiers wrote:

> On Tue, Nov 06, 2007 at 04:36:47PM +0000, Adam Spiers wrote:
>
> [snip]
>
>> This could easily be accomplished if remember templates allowed  
>> syntax
>> such as
>>
>> ,------
>> | * %T
>> | %(shell-command-to-string "grep 'last full' /proc/acpi/battery/ 
>> BAT0/info")
>> `------
>
> OK, it turns out that this was easy to implement, and I think the
> patch is small enough that it could be accepted even though I haven't
> got around to sending back the copyright assignment form yet (sorry -
> this *will* happen at some point!)

I'll take this patch, thanks.

- Carsten

>
> Diff against 5.13i:
>
> --- a/org.el	Wed Oct 31 09:46:35 2007 +0000
> +++ b/org.el	Tue Nov 06 22:30:13 2007 +0000
> @@ -12806,6 +12806,17 @@ to be run from that hook to fucntion pro
>  	  (replace-match
>  	   (or (eval (intern (concat "v-" (match-string 1)))) "")
>  	   t t))
> +        ;; %() embedded elisp
> +	(goto-char (point-min))
> +        (while (re-search-forward "%\\((.+)\\)" nil t)
> +          (goto-char (match-beginning 0))
> +          (let ((template-start (point)))
> +            (forward-char 1)
> +            (condition-case error
> +                (let ((result (eval (read (current-buffer)))))
> +                  (delete-region template-start (point))
> +                  (insert result))
> +              (error (message "Error `%s' in remember template"  
> error)))))
>  	;; From the property list
>  	(when plist-p
>  	  (goto-char (point-min))
>
>
> There might need to be some debate about how to handle read errors in
> the case of invalid syntax.  Or perhaps some people already have
> `%(...)' within their remember templates for some really unusual
> reason, and don't want it interpreted?  Though in the latter case, one
> could argue that it might make sense to require `%' always to be
> escaped as `%%' if used literally, to be on the safe side.
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: inserting files within remember templates

[Adam Spiers]

On Wed, Nov 07, 2007 at 09:42:55AM +0100, Carsten Dominik wrote:
> On 6 Nov 2007, at 11:39 PM, Adam Spiers wrote:
> > On Tue, Nov 06, 2007 at 04:36:47PM +0000, Adam Spiers wrote:
> > > This could easily be accomplished if remember templates allowed  
> > > syntax such as
> > > 
> > > ,------
> > > | * %T
> > > | %(shell-command-to-string "grep 'last full' /proc/acpi/battery/BAT0/info")
> > > `------
> >
> > OK, it turns out that this was easy to implement, and I think the
> > patch is small enough that it could be accepted even though I haven't
> > got around to sending back the copyright assignment form yet (sorry -
> > this *will* happen at some point!)
> 
> I'll take this patch, thanks.

[snipped]

> >There might need to be some debate about how to handle read errors in
> >the case of invalid syntax.  Or perhaps some people already have
> >`%(...)' within their remember templates for some really unusual
> >reason, and don't want it interpreted?  Though in the latter case, one
> >could argue that it might make sense to require `%' always to be
> >escaped as `%%' if used literally, to be on the safe side.

Here's a better version.  Differences:

  - Supports insertion of the contents of another file via %[/path/to/file]

  - Embeds any errors within the template, making it clearer that an
    error occurred, and also exactly which bit of the template caused
    it.  This also lets the user manually correct the error and finish
    their remembering workflow before having to go back and fix the
    template, which is much more in keeping with the "remember this
    quick before I forget!" spirit of remember itself.

  - Patches texinfo file.

  - Respects Carsten's apparent preference for having indent-tabs-mode
    set (though this may make the below patch's indentation look
    strange due to the hard tabs).

diff -r 09b1470ac170 org.el
--- a/org.el	Wed Oct 31 09:46:35 2007 +0000
+++ b/org.el	Sat Nov 24 14:39:51 2007 +0000
@@ -12806,6 +12806,30 @@ to be run from that hook to fucntion pro
 	  (replace-match
 	   (or (eval (intern (concat "v-" (match-string 1)))) "")
 	   t t))
+	;; %[] Insert contents of a file.
+	(goto-char (point-min))
+	(while (re-search-forward "%\\[\\(.+\\)\\]" nil t)
+	  (let ((start (match-beginning 0))
+		(end (match-end 0))
+		(filename (expand-file-name (match-string 1))))
+	    (goto-char start)
+	    (delete-region start end)
+	    (condition-case error
+		(insert-file-contents filename)
+	      (error (insert (format "%%![Couldn't insert %s: %s]"
+				     filename error))))))
+	;; %() embedded elisp
+	(goto-char (point-min))
+	(while (re-search-forward "%\\((.+)\\)" nil t)
+	  (goto-char (match-beginning 0))
+	  (let ((template-start (point)))
+	    (forward-char 1)
+	    (let ((result
+		   (condition-case error
+		       (eval (read (current-buffer)))
+		     (error (format "%%![Error: %s]" error)))))
+	      (delete-region template-start (point))
+	      (insert result))))
 	;; From the property list
 	(when plist-p
 	  (goto-char (point-min))
diff -r 09b1470ac170 org.texi
--- a/org.texi	Wed Oct 31 09:46:35 2007 +0000
+++ b/org.texi	Sat Nov 24 14:39:51 2007 +0000
@@ -4378,6 +4378,8 @@ insertion of content:
 %^g         @r{prompt for tags, with completion on tags in target file.}
 %^G         @r{prompt for tags, with completion all tags in all agenda files.}
 %:keyword   @r{specific information for certain link types, see below}
+%[pathname] @r{insert the contents of the file given by @code{pathname}}
+%(sexp)     @r{evaluate elisp @code{(sexp)} and replace with the result}
 @end example
 
 @noindent

Re: inserting files within remember templates

[Carsten Dominik]

[-- Attachment #1.1: Type: text/plain, Size: 4346 bytes --]

Nice!

Thanks you very much, Adam...

- Carsten

On 11/24/07, Adam Spiers <orgmode@adamspiers.org> wrote:
>
> On Wed, Nov 07, 2007 at 09:42:55AM +0100, Carsten Dominik wrote:
> > On 6 Nov 2007, at 11:39 PM, Adam Spiers wrote:
> > > On Tue, Nov 06, 2007 at 04:36:47PM +0000, Adam Spiers wrote:
> > > > This could easily be accomplished if remember templates allowed
> > > > syntax such as
> > > >
> > > > ,------
> > > > | * %T
> > > > | %(shell-command-to-string "grep 'last full'
> /proc/acpi/battery/BAT0/info")
> > > > `------
> > >
> > > OK, it turns out that this was easy to implement, and I think the
> > > patch is small enough that it could be accepted even though I haven't
> > > got around to sending back the copyright assignment form yet (sorry -
> > > this *will* happen at some point!)
> >
> > I'll take this patch, thanks.
>
> [snipped]
>
> > >There might need to be some debate about how to handle read errors in
> > >the case of invalid syntax.  Or perhaps some people already have
> > >`%(...)' within their remember templates for some really unusual
> > >reason, and don't want it interpreted?  Though in the latter case, one
> > >could argue that it might make sense to require `%' always to be
> > >escaped as `%%' if used literally, to be on the safe side.
>
> Here's a better version.  Differences:
>
>   - Supports insertion of the contents of another file via
> %[/path/to/file]
>
>   - Embeds any errors within the template, making it clearer that an
>     error occurred, and also exactly which bit of the template caused
>     it.  This also lets the user manually correct the error and finish
>     their remembering workflow before having to go back and fix the
>     template, which is much more in keeping with the "remember this
>     quick before I forget!" spirit of remember itself.
>
>   - Patches texinfo file.
>
>   - Respects Carsten's apparent preference for having indent-tabs-mode
>     set (though this may make the below patch's indentation look
>     strange due to the hard tabs).
>
> diff -r 09b1470ac170 org.el
> --- a/org.el    Wed Oct 31 09:46:35 2007 +0000
> +++ b/org.el    Sat Nov 24 14:39:51 2007 +0000
> @@ -12806,6 +12806,30 @@ to be run from that hook to fucntion pro
>           (replace-match
>            (or (eval (intern (concat "v-" (match-string 1)))) "")
>            t t))
> +       ;; %[] Insert contents of a file.
> +       (goto-char (point-min))
> +       (while (re-search-forward "%\\[\\(.+\\)\\]" nil t)
> +         (let ((start (match-beginning 0))
> +               (end (match-end 0))
> +               (filename (expand-file-name (match-string 1))))
> +           (goto-char start)
> +           (delete-region start end)
> +           (condition-case error
> +               (insert-file-contents filename)
> +             (error (insert (format "%%![Couldn't insert %s: %s]"
> +                                    filename error))))))
> +       ;; %() embedded elisp
> +       (goto-char (point-min))
> +       (while (re-search-forward "%\\((.+)\\)" nil t)
> +         (goto-char (match-beginning 0))
> +         (let ((template-start (point)))
> +           (forward-char 1)
> +           (let ((result
> +                  (condition-case error
> +                      (eval (read (current-buffer)))
> +                    (error (format "%%![Error: %s]" error)))))
> +             (delete-region template-start (point))
> +             (insert result))))
>         ;; From the property list
>         (when plist-p
>           (goto-char (point-min))
> diff -r 09b1470ac170 org.texi
> --- a/org.texi  Wed Oct 31 09:46:35 2007 +0000
> +++ b/org.texi  Sat Nov 24 14:39:51 2007 +0000
> @@ -4378,6 +4378,8 @@ insertion of content:
> %^g         @r{prompt for tags, with completion on tags in target file.}
> %^G         @r{prompt for tags, with completion all tags in all agenda
> files.}
> %:keyword   @r{specific information for certain link types, see below}
> +%[pathname] @r{insert the contents of the file given by @code{pathname}}
> +%(sexp)     @r{evaluate elisp @code{(sexp)} and replace with the result}
> @end example
>
> @noindent
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>

[-- Attachment #1.2: Type: text/html, Size: 7332 bytes --]

[-- Attachment #2: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: inserting files within remember templates

[Adam Spiers]

On Sat, Nov 24, 2007 at 02:52:27PM +0000, Adam Spiers wrote:
> On Wed, Nov 07, 2007 at 09:42:55AM +0100, Carsten Dominik wrote:
> > On 6 Nov 2007, at 11:39 PM, Adam Spiers wrote:
> > > On Tue, Nov 06, 2007 at 04:36:47PM +0000, Adam Spiers wrote:
> > > > This could easily be accomplished if remember templates allowed  
> > > > syntax such as
> > > > 
> > > > ,------
> > > > | * %T
> > > > | %(shell-command-to-string "grep 'last full' /proc/acpi/battery/BAT0/info")
> > > > `------
> > >
> > > OK, it turns out that this was easy to implement, and I think the
> > > patch is small enough that it could be accepted even though I haven't
> > > got around to sending back the copyright assignment form yet (sorry -
> > > this *will* happen at some point!)
> > 
> > I'll take this patch, thanks.

[snipped]

> Here's a better version.  Differences:

[snipped]

>   - Patches texinfo file.

[snipped]

> diff -r 09b1470ac170 org.texi
> --- a/org.texi	Wed Oct 31 09:46:35 2007 +0000
> +++ b/org.texi	Sat Nov 24 14:39:51 2007 +0000
> @@ -4378,6 +4378,8 @@ insertion of content:
>  %^g         @r{prompt for tags, with completion on tags in target file.}
>  %^G         @r{prompt for tags, with completion all tags in all agenda files.}
>  %:keyword   @r{specific information for certain link types, see below}
> +%[pathname] @r{insert the contents of the file given by @code{pathname}}
> +%(sexp)     @r{evaluate elisp @code{(sexp)} and replace with the result}
>  @end example
>  
>  @noindent

I just noticed that we both forgot to patch the defcustom with some of
the recent changes!  This should fix it:

diff -r 742454cf3979 org.el
--- a/org.el	Wed Dec 05 09:48:23 2007 +0000
+++ b/org.el	Sun Dec 09 12:18:26 2007 +0000
@@ -1436,7 +1436,7 @@ a first line starting with a star, to ac
 a first line starting with a star, to act as the org-mode headline.
 Furthermore, the following %-escapes will be replaced with content:
 
-  %^{prompt}  prompt the user for a string and replace this sequence with it.
+  %^{prompt}  prompt the user for a string and replace this sequence with it
   %t          time stamp, date only
   %T          time stamp with date and time
   %u, %U      like the above, but inactive time stamps
@@ -1447,6 +1447,12 @@ Furthermore, the following %-escapes wil
   %i          initial content, the region when remember is called with C-u.
               If %i is indented, the entire inserted text will be indented
               as well.
+  %c          content of the clipboard, or current kill ring head
+  %^g         prompt for tags, with completion on tags in target file
+  %^G         prompt for tags, with completion all tags in all agenda files
+  %:keyword   specific information for certain link types, see below
+  %[pathname] insert the contents of the file given by `pathname'
+  %(sexp)     evaluate elisp `(sexp)' and replace with the result
 
   %?          After completing the template, position cursor here.
 

Re: inserting files within remember templates

[Bastien]

Hi Adam,

Adam Spiers <orgmode@adamspiers.org> writes:

> Currently I do this by coding the helper to dump the Message-Id into
> ~/.clip-mairix, and then the elisp code inserts the contents of this
> file back into the org buffer.  However I would like it to be inserted
> via a remember template, hence the request.

I've seen your patch in the other post and it looks fine. But FWIW here
was my first reaction: why don't you try to implement a new link type
for message/mail buffers?  Then you just need to access the source of
the message, link to it, use relevant properties and insert them thru
the template.  :message-id would be one of the available properties.

This might look more heavy than inserting elisp code directly in the
template, but I just wanted to mention this might also be a way to do
it.  
 
-- 
Bastien

Re: inserting files within remember templates

[Adam Spiers]

On Wed, Nov 07, 2007 at 02:37:54AM +0000, Bastien wrote:
> Adam Spiers <orgmode@adamspiers.org> writes:
> > Currently I do this by coding the helper to dump the Message-Id into
> > ~/.clip-mairix, and then the elisp code inserts the contents of this
> > file back into the org buffer.  However I would like it to be inserted
> > via a remember template, hence the request.
> 
> I've seen your patch in the other post and it looks fine. But FWIW here
> was my first reaction: why don't you try to implement a new link type
> for message/mail buffers?

Because I use mutt for mail and hence need to use some primitive form
of IPC:

  http://article.gmane.org/gmane.emacs.orgmode/4250

> Then you just need to access the source of
> the message, link to it, use relevant properties and insert them thru
> the template.  :message-id would be one of the available properties.

org-mairix.el already does exactly that for gnus integration.

Re: inserting files within remember templates

[Bastien]

Adam Spiers <orgmode@adamspiers.org> writes:

> On Wed, Nov 07, 2007 at 02:37:54AM +0000, Bastien wrote:
>> I've seen your patch in the other post and it looks fine. But FWIW here
>> was my first reaction: why don't you try to implement a new link type
>> for message/mail buffers?
>
> Because I use mutt for mail and hence need to use some primitive form
> of IPC:
>
>   http://article.gmane.org/gmane.emacs.orgmode/4250

Yes:

> Currently I do this by coding the helper to dump the Message-Id into
> ~/.clip-mairix, and then the elisp code inserts the contents of this
> file back into the org buffer.  However I would like it to be inserted
> via a remember template, hence the request.

I thought your mutt helper could copy the whole message to ~/.clip-msg,
then you wouldn't need some elisp code to insert ~/.clip-msg but rather
a org-message.el that would let you get the message-id as a link prop
from the template...

But having %(...) is more straightforward.

-- 
Bastien

Re: inserting files within remember templates

[Adam Spiers]

On Wed, Nov 07, 2007 at 12:46:45PM +0000, Bastien wrote:
> > Currently I do this by coding the helper to dump the Message-Id into
> > ~/.clip-mairix, and then the elisp code inserts the contents of this
> > file back into the org buffer.  However I would like it to be inserted
> > via a remember template, hence the request.
> 
> I thought your mutt helper could copy the whole message to ~/.clip-msg,
> then you wouldn't need some elisp code to insert ~/.clip-msg but rather
> a org-message.el that would let you get the message-id as a link prop
> from the template...

Ah, I see!

> But having %(...) is more straightforward.

Possibly.  I suppose it depends on the relative merits of parsing the
mail via the mutt helper (which is Perl in my case) vs. doing it with
elisp.  Maybe I should change the helper to store an elisp form
representing a property list of the mail's metadata via the temporary
file rather than a preformatted mairix link - that way other
mutt/org/remember users have more flexibility in their remember
templates.  But then, when and how would emacs parse that plist as a
replacement for the normal `org-store-link-props' invocation?

Re: inserting files within remember templates

[Bastien]

Adam Spiers <orgmode@adamspiers.org> writes:

> I suppose it depends on the relative merits of parsing the mail via
> the mutt helper (which is Perl in my case) vs. doing it with elisp.

In fact, I was not thinking of "parsing" the message.  I imagined that
the mutt helper would just tell where the message file is, then Emacs
would do the job of creating a link by visiting this file, storing all
relevant information into the properties of this link, those properties
being immediately reusable by a template.

> But then, when and how would emacs parse that plist as a
> replacement for the normal `org-store-link-props' invocation?

This is where a new org-message.el is required, so that you can create
links from a message-mode buffer then reuse this link and its properties
in a template.

But again, I don't know if this approach is more efficient/convenient.
It's just because I implemented new link types recently so my mind is a
bit (too much?) into it.

Best,

-- 
Bastien

Re: inserting files within remember templates

[Adam Spiers]

On Wed, Nov 07, 2007 at 02:39:07PM +0000, Bastien wrote:
> Adam Spiers <orgmode@adamspiers.org> writes:
> 
> > I suppose it depends on the relative merits of parsing the mail via
> > the mutt helper (which is Perl in my case) vs. doing it with elisp.
> 
> In fact, I was not thinking of "parsing" the message.  I imagined that
> the mutt helper would just tell where the message file is, then Emacs
> would do the job of creating a link by visiting this file

That's going to be costly for mails with very large attachments.

> storing all relevant information into the properties of this link
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
By "relevant information" presumably you mean the mail's subject,
sender, recipients, message-id and so on?  In which case the mail
(header, at least) has to be parsed at some point, no?

> those properties being immediately reusable by a template.

Yes, that's certainly desirable.

Re: inserting files within remember templates

[Bastien]

Adam Spiers <orgmode@adamspiers.org> writes:

>> In fact, I was not thinking of "parsing" the message.  I imagined that
>> the mutt helper would just tell where the message file is, then Emacs
>> would do the job of creating a link by visiting this file
>
> That's going to be costly for mails with very large attachments.

Email headers would be enough.

>> storing all relevant information into the properties of this link
>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> By "relevant information" presumably you mean the mail's subject,
> sender, recipients, message-id and so on?  

Yes.

> In which case the mail (header, at least) has to be parsed at some
> point, no?

From the message buffer.  I guess message-mode provides facilities to
get the message-id, the author, etc.

-- 
Bastien

Using org-remember to include stored link?

[Georg C. F. Greve]

[-- Attachment #1.1: Type: text/plain, Size: 402 bytes --]

Hi all,

is it possible to call org-remember to insert a previously stored link?

Regards,
Georg

-- 
Georg C. F. Greve                                 <greve@fsfeurope.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Join the Fellowship and protect your freedom!     (http://www.fsfe.org)
What everyone should know about DRM                   (http://DRM.info)

[-- Attachment #1.2: Type: application/pgp-signature, Size: 306 bytes --]

[-- Attachment #2: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Using org-remember to include stored link?

[Bastien]

"Georg C. F. Greve" <greve@fsfeurope.org> writes:

> is it possible to call org-remember to insert a previously stored
> link?

What do you mean?  

Being able to define an org-remember template that will prompt for a
link from the stored link?  Something like a %l to insert the last
stored link and %L to prompt for a link in the last stored links?

Or maybe you mean: calling org-remember from any buffer and make it
believe it's called from where the previously link was stored?

The first idea looks good.  But I'm not sure about what you have in
mind. 

-- 
Bastien

Re: Using org-remember to include stored link?

[Georg C. F. Greve]

[-- Attachment #1.1: Type: text/plain, Size: 926 bytes --]

On Sun, 14 Oct 2007 03:33:08 +0100
Bastien <bzg@altern.org> wrote: 

 >> is it possible to call org-remember to insert a previously stored
 >> link?

 b> What do you mean?  

To call org-remember to insert the last link that was added to
org-stored-links, ideally with the possibility to also supply a default
template to be used.


 b> Being able to define an org-remember template that will prompt for a
 b> link from the stored link?  Something like a %l to insert the last
 b> stored link and %L to prompt for a link in the last stored links?

Yes. This sounds like what I had in mind.

Regards,
Georg

-- 
Georg C. F. Greve                                 <greve@fsfeurope.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Join the Fellowship and protect your freedom!     (http://www.fsfe.org)
What everyone should know about DRM                   (http://DRM.info)

[-- Attachment #1.2: Type: application/pgp-signature, Size: 306 bytes --]

[-- Attachment #2: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Using org-remember to include stored link?

[Carsten Dominik]

On Oct 13, 2007, at 16:52, Georg C. F. Greve wrote:

> Hi all,
>
> is it possible to call org-remember to insert a previously stored link?

The remember buffer is in org-mode, you can just call C-c C-l to insert
any previously stored link...

- Carsten

Re: Using org-remember to include stored link?

[Georg C. F. Greve]

[-- Attachment #1.1: Type: text/plain, Size: 2800 bytes --]

On Sun, 14 Oct 2007 08:14:10 +0200
Carsten Dominik <dominik@science.uva.nl> wrote: 

 >> is it possible to call org-remember to insert a previously stored
 >> link?

 cd> The remember buffer is in org-mode, you can just call C-c C-l to
 cd> insert any previously stored link...

True. Although that means I would need to delete the link that is
inserted by the template, and then insert the one I want. I'd prefer
this to happen with fewer keystrokes. If I could call org-mode with a
customized template to insert the stored link without being prompted for
the other templates, that would be great.

Ideally for what I have in mind the templates would not be defined per
file. That dependency really is not useful in my case.

Allow me to explain.

I insert three different kinds of items into my org files:

 * NEXT  actions for things that should be done

 * QUEUE items for things that should be done in the future, but depend
         on completion of another NEXT action to become NEXT actions.

 * WAITING items where I wait for input / response from others.

Much of this is linking to emails, of course. So the normal insertion is
a NEXT action with link to an email, or a thread of emails. But
sometimes they will be QUEUE items.

For some mails I send, I want to insert WAITING items.

So I would like to be able to define three templates: "NEXT" "QUEUE"
"WAITING" that could be inserted into either file, for which I would
like to be prompted.

So org-remember-templates would ideally be a list of templates only,
identified by template names, and a second variable org-remember-files
would specify the files with their prompts and their default template,
as well as the default place (heading) to store the item.

Then it would be possible to call org-remember without arguments with
the same behaviour as today, but it would be possible to specify
different templates as arguments, overriding the default to make sure
that when it gets called from special places or in a special way, it
does not have to use the default template only.

In combination with something that inserts the last stored link into the
template automatically, I could then have the fastest way of turning
outgoing mail into WAITING items in the correct file.

See http://www.emacswiki.org/cgi-bin/wiki/org-mairix.el function

 org-mairix-message-send-and-exit-with-link

that sends a message and stores a org-mairix link in the process.

Regards,
Georg

-- 
Georg C. F. Greve                                 <greve@fsfeurope.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Join the Fellowship and protect your freedom!     (http://www.fsfe.org)
What everyone should know about DRM                   (http://DRM.info)

[-- Attachment #1.2: Type: application/pgp-signature, Size: 306 bytes --]

[-- Attachment #2: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Using org-remember to include stored link?

[Bastien]

"Georg C. F. Greve" <greve@fsfeurope.org> writes:

> Although that means I would need to delete the link that is inserted
> by the template, and then insert the one I want. I'd prefer this to
> happen with fewer keystrokes.

If there is no %a or %A in your template, there is no link in it.  

Then with a correct position for %?, inserting the last link is just
*one* keystroke away, right? 

-- 
Bastien

Re: Using org-remember to include stored link?

[Georg C. F. Greve]

[-- Attachment #1.1: Type: text/plain, Size: 882 bytes --]

On Sun, 14 Oct 2007 15:03:24 +0100
Bastien <bzg@altern.org> wrote: 

 b> If there is no %a or %A in your template, there is no link in it.  

True, although it does not solve the issue of NEXT/QUEUE/WAITING.


 b> Then with a correct position for %?, inserting the last link is just
 b> *one* keystroke away, right?

Well, it would be "C-l C-l" "RET" "RET" at the very least.

Since I practically already told it that I want this link inserted as
WAITING item by calling the proper sending function, that is three more
keystrokes that I'd like. ;)

Regards,
Georg

-- 
Georg C. F. Greve                                 <greve@fsfeurope.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Join the Fellowship and protect your freedom!     (http://www.fsfe.org)
What everyone should know about DRM                   (http://DRM.info)

[-- Attachment #1.2: Type: application/pgp-signature, Size: 306 bytes --]

[-- Attachment #2: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Using org-remember to include stored link?

[Carsten Dominik]

On Oct 14, 2007, at 14:05, Georg C. F. Greve wrote:

> On Sun, 14 Oct 2007 08:14:10 +0200
> Carsten Dominik <dominik@science.uva.nl> wrote:
>
>>> is it possible to call org-remember to insert a previously stored
>>> link?
>
>  cd> The remember buffer is in org-mode, you can just call C-c C-l to
>  cd> insert any previously stored link...
>
> True. Although that means I would need to delete the link that is
> inserted by the template, and then insert the one I want. I'd prefer
> this to happen with fewer keystrokes. If I could call org-mode with a
> customized template to insert the stored link without being prompted 
> for
> the other templates, that would be great.
>
> Ideally for what I have in mind the templates would not be defined per
> file. That dependency really is not useful in my case.

You can use nil for the file name and still provide a headline,
I believe.  I see now that the customization interface does not
allow this - I will change this.
>
> In combination with something that inserts the last stored link into 
> the
> template automatically, I could then have the fastest way of turning
> outgoing mail into WAITING items in the correct file.
>
> See http://www.emacswiki.org/cgi-bin/wiki/org-mairix.el function
>
>  org-mairix-message-send-and-exit-with-link

For this specific application, you might consider a different route:

- Define a prefix argument for this function.  When given, org-remember
will be called directly by the function, with a custom template into 
which
you can insert the link.  If you use `let' to define a local value
for org-remember-templates, that template will be used for this call.

So I mean that you end your org-mairix-send-and-exit-with-link
with something like this (totally untested, only a draft):

(when arg

   (let ((org-remember-templates
           '((?w (concat "* WAITING %?\n  " (format "[[%s][%s]]" link 
desc))
                 nil ; no file name
                 "WAITING" ; the headline
                 ))))
        (org-remember)))

Since we are only defining a single template, you will not even be 
prompted
for the template but directly placed into the remember buffer.


- Carsten

Re: Using org-remember to include stored link?

[Georg C. F. Greve]

[-- Attachment #1.1.1: Type: text/plain, Size: 981 bytes --]

On Sun, 14 Oct 2007 17:33:44 +0200
Carsten Dominik <dominik@science.uva.nl> wrote: 

 cd> For this specific application, you might consider a different route:

Thanks for your input. I had some look at this route, and ended up with
a function that inserts the remember item (defined in calling function)
and inserts it into the org-remember-templates before calling
org-remember.

I would like to do this in a way that it only temporarily modifies the
org-remember-templates, but at the moment, this function does that
permanently.

My Lisp knowledge is not really good enough to figure this one out
without help, I am afraid. I tried to address that issue by using
copy-sequence and copy-alist to work on local copies of the
org-remember-templates, but that does not seem to solve the issue, which
is why I am attaching the function now in its first implementation.

Any input you might be able to provide would be greatly appreciated.

Regards,
Georg



[-- Attachment #1.1.2: Type: application/emacs-lisp, Size: 1037 bytes --]

[-- Attachment #1.1.3: Type: text/plain, Size: 299 bytes --]


-- 
Georg C. F. Greve                                 <greve@fsfeurope.org>
Free Software Foundation Europe	                 (http://fsfeurope.org)
Join the Fellowship and protect your freedom!     (http://www.fsfe.org)
What everyone should know about DRM                   (http://DRM.info)

[-- Attachment #1.2: Type: application/pgp-signature, Size: 306 bytes --]

[-- Attachment #2: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Using org-remember to include stored link?

[Bastien]

Hi Georg,

"Georg C. F. Greve" <greve@fsfeurope.org> writes:

> I would like to do this in a way that it only temporarily modifies the
> org-remember-templates, but at the moment, this function does that
> permanently.

See my comment below.

> (defun org-mairix-sent-message-remember ()
>   "Function to be called by org-mairix-message-send-and-exit-with-link
> via hook to store a link to a sent message by calling remember.
>
> It works by first inserting the 'org-mairix' link provided by
> org-mairix-message-send-and-exit-with-link for '%a' in the
> org-mairix-message-sent-remember-template string and then
> iterating through the org-remember-templates, replacing all the
> standard items by the org-mairix-message-sent-remember-template
> before calling org-remember."
>   (let* ((templates) (templ)
> 	 (org-remember-templates org-remember-templates)

Why do you need to copy the global value of `org-remember-templates'?
Can't you just define it *locally*?

(let* ((org-remember-templates 
         '((?w org-mairix-message-sent-remember-template 
               nil ; no file name
               "WAITING" ; the headline))))
   ...)

This shouldn't modify the global set of templates.

-- 
Bastien

Re: inserting files within remember templates

[Adam Spiers]

Caveat: the following was written using a sleep-deprived brain %-)

Georg, judging by your mail

  http://article.gmane.org/gmane.emacs.orgmode/3814/

I think you and I are trying to achieve something similar with our
org/mairix integration efforts.  I would also find your suggestion of
`org-remember-files' useful as a way of choosing the destination file
orthogonally from the choice of remember template.  Alternatively if
remember templates supported prefix keys in the same way that
org-agenda-custom-commands now does, it would be possible to have the
prefix key for choosing the template, and the sub-keymap key for
choosing the destination file, or vice-versa, though I think a
separate variable associating key shortcuts with destination files
would be cleaner.  Would be good to get Carsten's thoughts on the
relevant merits of these two approaches.

The main difference between our needs is that while I use mutt as the
mail client, IIRC you are using gnus.  Using a MUA external to emacs
has its own set of challenges, as previously discussed with Bastien:

  http://thread.gmane.org/gmane.emacs.orgmode/4217/focus=4289

I have written a simple Perl script which when a mail is piped into it
(e.g. from a mutt macro), will extract properties from that mail and
dump an elisp form into a temporary file which emacs can then
evaluate, e.g.

  (org-store-link-props
    :type "mairix"
    :link "mairix:m:m3abqlrjf2.fsf@cerebro.fsfeurope.org"
    :from "\"Georg C. F. Greve\" <greve@fsfeurope.org>"
    :subject "Re: [Orgmode] Using org-remember to include stored link?"
    :subjectquery "s:Using,org,remember,to,include,stored,link"
    :message-id "<m3abqlrjf2.fsf@cerebro.fsfeurope.org>"
    :message-id-query "m:m3abqlrjf2.fsf@cerebro.fsfeurope.org")

I chose to parse the mail and extract properties once at the time the
mutt macro was invoked, mainly because I knew I could implement it
quicker in Perl than in elisp :-)  However I am now wondering if that
was a design mistake...

The end goal is that by invoking `org-remember' and then a single
keystroke to select the right remember template, something like:

* TODO [#B] todo description provided by prompting user
  [[mairix:m:m3abqlrjf2.fsf@cerebro.fsfeurope.org][mail from Georg C. F. Greve: Re: [Orgmode] Using org-remember to include stored link?]]

will get inserted at the top of a particular TODO.org file, according
to the current value of `org-email-link-description-format'.

However, at this point I am somewhat stuck, since invoking
`org-store-link-props' by merely evaluating the elisp in the file is
not sufficient to:

  (a) push the link onto the `org-stored-links' list, and
  (b) figure out the link description using `org-email-link-description'.

In normal usage, `org-store-link' takes care of both of these, but
what if instead, you want to store a link non-interactively for later
use in a remember template, and you already know what type of link you
want to store?

In this case, it seems that there are two issues with
`org-store-link', presumably due to it having been designed to support
interactive usage only.  Firstly, the type of link is automagically
determined by iterating over `org-store-link-functions', which in this
case is not what I want (since I want to enforce storage of a mairix
link).  Secondly, it is automatically invoked from 'org-remember' via
`org-remember-annotation', which would presumably push a link to the
current buffer ahead of the stored mairix link in the org-stored-links
list, meaning that %a gets the wrong link substituted.

It looks like Georg faced the same problem (a) when writing
`org-mairix-sent-message-remember' as he had to copy the following
code out of `org-store-link':

     (setq org-stored-links
           (cons (list link desc) org-stored-links))

One solution to (a) might be to factor out the code at the end of
`org-store-link' into a separate helper function which could then be
reused by the elisp in the temporary file?  But what about (b) and the
question of how to get the remember template to include the stored
link?

On Thu, Nov 08, 2007 at 01:09:28PM +0000, Bastien wrote:
> Hi Georg,
> 
> "Georg C. F. Greve" <greve@fsfeurope.org> writes:
> 
> > I would like to do this in a way that it only temporarily modifies the
> > org-remember-templates, but at the moment, this function does that
> > permanently.
> 
> See my comment below.
> 
> > (defun org-mairix-sent-message-remember ()
> >   "Function to be called by org-mairix-message-send-and-exit-with-link
> > via hook to store a link to a sent message by calling remember.
> >
> > It works by first inserting the 'org-mairix' link provided by
> > org-mairix-message-send-and-exit-with-link for '%a' in the
> > org-mairix-message-sent-remember-template string and then
> > iterating through the org-remember-templates, replacing all the
> > standard items by the org-mairix-message-sent-remember-template
> > before calling org-remember."
> >   (let* ((templates) (templ)
> > 	 (org-remember-templates org-remember-templates)
> 
> Why do you need to copy the global value of `org-remember-templates'?
> Can't you just define it *locally*?
> 
> (let* ((org-remember-templates 
>          '((?w org-mairix-message-sent-remember-template 
>                nil ; no file name
>                "WAITING" ; the headline))))
>    ...)
> 
> This shouldn't modify the global set of templates.

[Aside: I haven't thought too hard about how to handle remembering of
messages just *sent*, since AFAIK mutt does not have a hook for
executing macros just after sending mail and saving a copy locally,
but regardless of whether we're trying to remember a link to a mail
just sent or one previous received, it seems to be the same problem.]

Locally overriding the global value of the `org-remember-templates'
list by iterating over it with %a substitutions sounds a bit hackish
to me.  If we can fix the above issues I mention, I believe it would
no longer be necessary to do that.

Finally, as Bastien suggested, I could instead have had the mutt macro
dump the mail header into a file which emacs could then parse, perhaps
by using `message-fetch-mail' in the same way that org-mairix.el
already does.  But I suspect that the issues I mention above with
`org-store-link' will still cause problems.

I hope that all made sense.  I can't think 100% straight right now %-)