advice: how to export a list without exporting all entries

advice: how to export a list without exporting all entries

[Eric S Fraga]

Hi,

I chair a particular committee at work and am responsible for keeping
track of any actions that arise from our meetings.  I use org for this
(obviously ;-).  So far, so good.

I would like to move to a system in which all the actions are numbered
sequentially.  At present, they are numbered sequentially within a list
for each meeting.  I would like to have a single list which grows over
time. However, when I distribute the minutes of the latest meeting, I
would like to only have those actions which are not yet complete listed
in the document I circulate.  The complication is that I want those
actions that have actually been done still in the list but not exported.

Is there any way to /comment/ out individual list items (whether bullet
or enumerated) on export?  I export typically to latex but this need not
be a constraint.  Simply putting [ ] versus [X] boxes on the items is not
satisfactory as the list would be very long if all items were included in
the export.

Is there some hook that I can intercept that would enable this?  Can I
encapsulate individual list items into latex macros with the status of
the [ ] or [X] boxes?  I am more than happy to write latex code as
required!  Or even, at a push, elisp code...

Thanks,
eric

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.91.g38c6b)

Re: advice: how to export a list without exporting all entries

[Nicolas]

Hello,

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> I would like to move to a system in which all the actions are numbered
> sequentially.  At present, they are numbered sequentially within a list
> for each meeting.  I would like to have a single list which grows over
> time. However, when I distribute the minutes of the latest meeting, I
> would like to only have those actions which are not yet complete listed
> in the document I circulate.  The complication is that I want those
> actions that have actually been done still in the list but not exported.
>
> Is there any way to /comment/ out individual list items (whether bullet
> or enumerated) on export?  I export typically to latex but this need not
> be a constraint.  Simply putting [ ] versus [X] boxes on the items is not
> satisfactory as the list would be very long if all items were included in
> the export.
>
> Is there some hook that I can intercept that would enable this?  Can I
> encapsulate individual list items into latex macros with the status of
> the [ ] or [X] boxes?  I am more than happy to write latex code as
> required!  Or even, at a push, elisp code...

I'm not sure to fully understand what you want, but couldn't you
delete-matching-lines toggled check-boxes in a copy of the original
buffer, and export that?

Regards,

-- 
Nicolas

Re: advice: how to export a list without exporting all entries

[Eric S Fraga]

Nicolas <n.goaziou@gmail.com> writes:

> Hello,
>
> Eric S Fraga <e.fraga@ucl.ac.uk> writes:
>
>> I would like to move to a system in which all the actions are numbered
>> sequentially.  At present, they are numbered sequentially within a list
>> for each meeting.  I would like to have a single list which grows over
>> time. However, when I distribute the minutes of the latest meeting, I
>> would like to only have those actions which are not yet complete listed
>> in the document I circulate.  The complication is that I want those
>> actions that have actually been done still in the list but not exported.
>>
>> Is there any way to /comment/ out individual list items (whether bullet
>> or enumerated) on export?  I export typically to latex but this need not
>> be a constraint.  Simply putting [ ] versus [X] boxes on the items is not
>> satisfactory as the list would be very long if all items were included in
>> the export.
>>
>> Is there some hook that I can intercept that would enable this?  Can I
>> encapsulate individual list items into latex macros with the status of
>> the [ ] or [X] boxes?  I am more than happy to write latex code as
>> required!  Or even, at a push, elisp code...
>
> I'm not sure to fully understand what you want, but couldn't you
> delete-matching-lines toggled check-boxes in a copy of the original
> buffer, and export that?
>
> Regards,

Thanks for the suggestion.  I think you did understand me and, yes, that
would work, but only *if* each list entry were a single line.
Unfortunately, I tend to fill my list paragraphs so that each item in
the list is often several lines and, in fact, often several paragraphs
(especially when it concerns minutes of a meeting and resulting
actions).  delete-matching-lines would delete the first line of a list
entry only.

I need to be able to "delete" whole list entries automatically based on
their status, whether in a copy or during export.

Thanks again,
eric

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.113.g9010a.dirty)

Re: advice: how to export a list without exporting all entries

[Nicolas]

Hello,

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Nicolas <n.goaziou@gmail.com> writes:

>> Eric S Fraga <e.fraga@ucl.ac.uk> writes:

>>> I would like to move to a system in which all the actions are numbered
>>> sequentially.  At present, they are numbered sequentially within a list
>>> for each meeting.  I would like to have a single list which grows over
>>> time. However, when I distribute the minutes of the latest meeting, I
>>> would like to only have those actions which are not yet complete listed
>>> in the document I circulate.  The complication is that I want those
>>> actions that have actually been done still in the list but not exported.

>>> Is there any way to /comment/ out individual list items (whether bullet
>>> or enumerated) on export?  I export typically to latex but this need not
>>> be a constraint.  Simply putting [ ] versus [X] boxes on the items is not
>>> satisfactory as the list would be very long if all items were included in
>>> the export.

>>> Is there some hook that I can intercept that would enable this?  Can I
>>> encapsulate individual list items into latex macros with the status of
>>> the [ ] or [X] boxes?  I am more than happy to write latex code as
>>> required!  Or even, at a push, elisp code...

>> I'm not sure to fully understand what you want, but couldn't you
>> delete-matching-lines toggled check-boxes in a copy of the original
>> buffer, and export that?
>>
>> Regards,
>
> Thanks for the suggestion.  I think you did understand me and, yes, that
> would work, but only *if* each list entry were a single line.
> Unfortunately, I tend to fill my list paragraphs so that each item in
> the list is often several lines and, in fact, often several paragraphs
> (especially when it concerns minutes of a meeting and resulting
> actions).  delete-matching-lines would delete the first line of a list
> entry only.
>
> I need to be able to "delete" whole list entries automatically based on
> their status, whether in a copy or during export.

Maybe the following functions might help you. Their docstring is explicit.

#+begin_src emacs-lisp
(defun esf-list-remove-if (predicate struct)
  "Remove all items satisfying PREDICATE in STRUCT and in buffer.
PREDICATE is a function called with item position as argument.
The modified STRUCT is returned."
  (let ((rev-struct (reverse struct))
        res e)
    (while rev-struct
      (setq e (pop rev-struct))
      (let ((item (car e)))
        (if (funcall predicate item)
            (delete-region item (nth 6 e))
          (push e res))))
    res))

(defun esf-clear-toggled-checkboxes ()
  "Remove toggled check-boxes from list at point.
Move point at the end of the list."
  (interactive)
  (if (not (org-at-item-p))
      (error "Not at a list item")
    (let* ((struct (org-list-struct))
           (end (copy-marker (org-list-get-bottom-point struct))))
      (esf-list-remove-if (lambda (e)
                            (equal "[X]" (org-list-get-checkbox e struct)))
                          struct)
      (goto-char end))))
#+end_src

Then, remove toggled checkboxes in an hook called just before list processing:

#+begin_src emacs-lisp
(add-hook 'org-export-preprocess-after-tree-selection-hook
          (lambda ()
            (goto-char (point-min))
            (while (org-list-search-forward (org-item-beginning-re) nil t)
              (esf-clear-toggled-checkboxes))))
#+end_src

Note that this solution doesn't handle nested lists (i.e. checkboxes in
lists inside a drawer itself in a list).

HTH,

Regards,

-- 
Nicolas

Re: Re: advice: how to export a list without exporting all entries

[Eric S Fraga]

Nicolas <n.goaziou@gmail.com> writes:

> Hello,

Hi!

> Eric S Fraga <e.fraga@ucl.ac.uk> writes:

[...]

>> I need to be able to "delete" whole list entries automatically based on
>> their status, whether in a copy or during export.
>
> Maybe the following functions might help you. Their docstring is
> explicit.

Thanks for the code.  This looks like it would do what I originally
thought I wanted but Carsten, as always, has come to the rescue and
provided exactly what I needed (as opposed to what I asked for ;-).  See
his recent email on the list about =org-new-numbered-action=!

Thanks again,
eric
-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.122.g2f72.dirty)

Re: advice: how to export a list without exporting all entries

[Nicolas]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Thanks for the code.  This looks like it would do what I originally
> thought I wanted but Carsten, as always, has come to the rescue and
> provided exactly what I needed (as opposed to what I asked for ;-).  See
> his recent email on the list about =org-new-numbered-action=!

Ah, I didn't follow closely this thread. I wish I could read minds like
Carsten!

Regards,

-- 
Nicolas

Re: advice: how to export a list without exporting all entries

[Carsten Dominik]

On 21.3.2011, at 10:16, Eric S Fraga wrote:

> Hi,
> 
> I chair a particular committee at work and am responsible for keeping
> track of any actions that arise from our meetings.  I use org for this
> (obviously ;-).  So far, so good.
> 
> I would like to move to a system in which all the actions are numbered
> sequentially.  At present, they are numbered sequentially within a list
> for each meeting.  I would like to have a single list which grows over
> time. However, when I distribute the minutes of the latest meeting, I
> would like to only have those actions which are not yet complete listed
> in the document I circulate.  The complication is that I want those
> actions that have actually been done still in the list but not exported.
> 
> Is there any way to /comment/ out individual list items (whether bullet
> or enumerated) on export?  I export typically to latex but this need not
> be a constraint.  Simply putting [ ] versus [X] boxes on the items is not
> satisfactory as the list would be very long if all items were included in
> the export.
> 
> Is there some hook that I can intercept that would enable this?  Can I
> encapsulate individual list items into latex macros with the status of
> the [ ] or [X] boxes?  I am more than happy to write latex code as
> required!  Or even, at a push, elisp code...
> 
> Thanks,
> eric

Hi Eric,

I guess you could use one of the export hooks to clean up your
exported file when you are done.  However, the problem will always
be that if you take a list environment, the numbering of the list
items will change when you take out items.  An I am sure you know
from experience how nice it is to be able to talk about the number
of an action item, in order to get everyone on the same page
quickly.  I am afraid it will be hard to achieve this with the
path you are thinking about.

Defining and tracking action items through a series of meetings where
I have to do the notes is a frequent tasks for me.  Here is my workflow
for this - maybe you can pick and choose some stuff for yourself.
In particular, I use outline nodes for tasks, in order to keep access
to the machinery Org gives me for tracking stuff.

1. I use a special Org file for each meeting series

2. In the file, I use the following setup (at the end of the file, for
   the file variables):


     * COMMENT SETUP
  
     #+TODO: TODO | DELG CNCL DOME
     #+EXPORT_EXCLUDE_TAGS: noexport closed
  
     # Local Variables:
     # org-todo-state-tags-triggers: ((done ("closed" . t)))
     # End:

   Your tags and TODO states may be different, of cause.
   What is important here is that I define that some tags mark a
   tree to be excluded from export.  Also, I set a local value for
   =org-todo-state-tags-triggers= which arranges for action items to
   receive a "closed" tag when they are marked done.  Since this tag
   also excludes stuff from exporting, done items will automatically
   not be exported.

3. I am not using an ordered list for the action items.  Instead, I
   give them explicit numbers, in the sequence in which I am defining
   the actions.  I am using some custom code (see end of this mail) to
   create these actions with unique numbers.  Running
   `M-x org-maction-new-action' (this is `C-c n' in my setup) will
   insert a new action that looks like this:

       ** TODO Action #2:                                 :#2:

   The action starts with "Action #2", and it is tagged by "#2" to that
   I can tag other stuff relating to this action and search for it.
   Once I mark this action as DONE, it will automatically become

       ** DONE Action #2: Do this and that          :#2:Peter:closed:

   In this way it will be excluded from export, unless you change
   EXPORT_EXCLUDE_TAGS (remove the "closed" there) temporarily to
   produce an export with all the DONE stuff included.

4. My workflow is then as follows:

   a. For each meeting I start a new top-level section in the document
      for the notes.  In the notes I create new actions and leave them
      there for the time being.

   b. I also have a top-level section "ACTIONS" in the file.  This
      section contains all the actions from previous meetings, each
      action identified by its unique number.  I have this section
      right after the meeting notes sections, so that I can easily
      export the latest meeting notes together with the action list,
      simply by selection the two sequential trees.

   c. During the meeting, I create new actions inside the meeting
      notes.  I also tag each action with names of people who are
      responsible for it, so that I can later easily make lists of
      actions for individual people, to send around reminders.

   d. After the meeting I export the current meeting notes and the
      ACTIONS section (as a list of old actions) and send it around.
      At this moment, I have the new actions only in the meeting
      notes, and not in the general actions list, to highlight
      them as new.  Depending on your preferences, you could also do
      (e) before sending out the notes.

   e. When that is done, I *copy* the new actions from the last
      meeting to the ACTIONS section.  I organize this section, change
      the sequence of actions, create subsections with related actions
      together etc.  For trivial task lists, the list just stays simple
      and flat, but from complex consortia I do a lot of structuring
      and ordering.

      I then deactivate the actions in the meeting notes I just sent
      around by marking the entire new notes tree with an ARCHIVE tag.
      Then, TODO lists and other Agenda views will get entries from
      the ACTIONS tree, not from the meeting notes.  Instead of
      copying, you could also *move* (refile) the actions to the
      ACTIONS tree, but I like to keep them there (deactivated), so
      that I have a copy of the notes exactly as sent around, in the
      file.  The idea is that the meeting notes are fixed, and all
      procedural stuff with the actions will be added and tracked
      in the ACTIONS tree.

   f. Between meetings I use the ACTIONS tree to generate reminders
      for people in the list, check DONE tasks off etc.  Before the
      next meeting, I create an agenda view to show me which actions
      have been closed during the last period.  I add this list to the
      agenda for the next meeting to inform everyone about the stuff
      that is closed and off the list.  I also attach an export of the
      current ACTIONS tree to the agenda, for reference during the
      following meeting.  The closed items will not be on that tree
      anymore, because the export does not use items with the
      "closed" tag.  The nice thing (and I think this was Eric's
      intent as well) is that the members of the meeting always have
      an uncluttered list of actions, while you as the responsible
      person still have all the old tags and the history at your
      finger tips.

This has worked really well for me, both for meetings that happen
locally, and for meetings with international groups who meet via
telephone/video conferencing.

Below you can find the (trivial) custom code I am using to create
new actions with unique identifying numbers.

With kind regards

- Carsten


Appendix: The meeting actions code 
----------------------------------

  (defgroup org-mactions nil
    "Meeting action items in Org mode."
    :tag "Org MActions"
    :group 'org)
  
  (defcustom org-maction-todo-keyword "TODO"
    "Default TODO keyword for new meeting actions."
    :group 'org-maction
    :type 'string)
  
  (defcustom org-maction-tag-prefix "#"
    "Prefix for numbered action tags action tags"
    :group 'org-maction
    :type 'string)
  
  (defcustom org-maction-action-format "Action #%d: "
    "Default structure of the headling of a new action.
  %d will become the number of the action."
    :group 'org-maction
    :type 'string)
  
  (defun org-maction-new-number ()
    "One plut the current highest action number."
    (let ((re (concat "\\`" (regexp-quote org-maction-tag-prefix)
                      "\\([0-9]+\\)\\'")))
      (1+ (apply 'max 0
                 (mapcar
                  (lambda (e)
                    (if (string-match re (car e))
                        (string-to-number (match-string 1 (car e)))
                      0))
                  (org-get-buffer-tags))))))
  
  (defun org-maction-new-action ()
    "Insert a new meeting action."
    (interactive)
    (let* ((num (org-maction-new-number))
           (tag (concat org-maction-tag-prefix (number-to-string num))))
      (org-insert-todo-heading nil 'force)
      (unless (eql (char-before) ?\ ) (insert " "))
      (insert (format org-maction-action-format num))
      (org-toggle-tag tag 'on)
      (if (= (point-max) (point-at-bol))
          (save-excursion (goto-char (point-at-eol)) (insert "\n")))
      (unless (eql (char-before) ?\ ) (insert " "))))

Re: advice: how to export a list without exporting all entries

[Eric S Fraga]

Carsten Dominik <carsten.dominik@gmail.com> writes:

[...]

>
> Hi Eric,
>
> I guess you could use one of the export hooks to clean up your
> exported file when you are done.  However, the problem will always
> be that if you take a list environment, the numbering of the list
> items will change when you take out items.  An I am sure you know
> from experience how nice it is to be able to talk about the number
> of an action item, in order to get everyone on the same page
> quickly.  I am afraid it will be hard to achieve this with the
> path you are thinking about.

Yes, I am coming to that conclusion as well!

> Defining and tracking action items through a series of meetings where
> I have to do the notes is a frequent tasks for me.  Here is my workflow
> for this - maybe you can pick and choose some stuff for yourself.
> In particular, I use outline nodes for tasks, in order to keep access
> to the machinery Org gives me for tracking stuff.

Thanks for the *very* detailed workflow.  Fantastic and I think it would
work well with my mode of operation.  I guess I was too stuck in the
/list/ mode and didn't even think of using outline nodes.  I don't mind
numbering them manually as we're talking about tens of actions over
several years (typically <10 per meeting with 3 meetings per year for
each individual "committee").  But, in any case, your code looks like
even this bit is easily automated.

What will work particularly well in my workflow will be the
todo-state-tag trigger part which I had not even realised could be done
(silly of me, of course ;-).  Tying the =closed= tag to the export
exclusion is so obvious in hindsight.

Again, many thanks for this and for your code as well.  I'll be
implementing it like right now!

eric

PS - your whole workflow description might be very useful on Worg...

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.113.g9010a.dirty)

Re: advice: how to export a list without exporting all entries

[Eric S Fraga]

Carsten,

one question about your workflow.  I've actually implemented much of
what you have suggested but I am having one problem:

Carsten Dominik <carsten.dominik@gmail.com> writes:

[...]

> 3. I am not using an ordered list for the action items.  Instead, I
>    give them explicit numbers, in the sequence in which I am defining
>    the actions.  I am using some custom code (see end of this mail) to
>    create these actions with unique numbers.  Running
>    `M-x org-maction-new-action' (this is `C-c n' in my setup) will
>    insert a new action that looks like this:
>
>        ** TODO Action #2:                                 :#2:

This is fine except that #x tags don't work in latex export!  Obviously,
you must not export to latex.  The # is not escaped when it is in a tag
although it is within the headline.  My immediate easy solution is to
change the tag prefix (A) but I wonder if this is a bug in the latex
export?

In any case, I'm liking this solution to my minutes+actions problem.

Thanks again,
eric

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.113.g9010a.dirty)

Re: advice: how to export a list without exporting all entries

[Carsten Dominik]

On 24.3.2011, at 21:45, Eric S Fraga wrote:

> Carsten,
> 
> one question about your workflow.  I've actually implemented much of
> what you have suggested but I am having one problem:
> 
> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
> [...]
> 
>> 3. I am not using an ordered list for the action items.  Instead, I
>>   give them explicit numbers, in the sequence in which I am defining
>>   the actions.  I am using some custom code (see end of this mail) to
>>   create these actions with unique numbers.  Running
>>   `M-x org-maction-new-action' (this is `C-c n' in my setup) will
>>   insert a new action that looks like this:
>> 
>>       ** TODO Action #2:                                 :#2:
> 
> This is fine except that #x tags don't work in latex export!  Obviously,
> you must not export to latex.  The # is not escaped when it is in a tag
> although it is within the headline.  My immediate easy solution is to
> change the tag prefix (A) but I wonder if this is a bug in the latex
> export?

Yes, apparently this is a problem in LaTeX export.

> 
> In any case, I'm liking this solution to my minutes+actions problem.

This is good to hear!

- Carsten

Re: advice: how to export a list without exporting all entries

[Carsten Dominik]

On 24.3.2011, at 21:45, Eric S Fraga wrote:

> Carsten,
> 
> one question about your workflow.  I've actually implemented much of
> what you have suggested but I am having one problem:
> 
> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
> [...]
> 
>> 3. I am not using an ordered list for the action items.  Instead, I
>>   give them explicit numbers, in the sequence in which I am defining
>>   the actions.  I am using some custom code (see end of this mail) to
>>   create these actions with unique numbers.  Running
>>   `M-x org-maction-new-action' (this is `C-c n' in my setup) will
>>   insert a new action that looks like this:
>> 
>>       ** TODO Action #2:                                 :#2:
> 
> This is fine except that #x tags don't work in latex export!

I have fixed the LaTeX exporter to take care of this issue.

- Carsten

Re: advice: how to export a list without exporting all entries

[Eric S Fraga]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> On 24.3.2011, at 21:45, Eric S Fraga wrote:

[...]

>> This is fine except that #x tags don't work in latex export!
>
> I have fixed the LaTeX exporter to take care of this issue.
>
> - Carsten

Excellent.  Thanks!

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.116.g35a2e)