Some suggestions - mostly for non (X)Emacs updating of org mode files.

Some suggestions - mostly for non (X)Emacs updating of org mode files.

[Tim O'Callaghan]

Not sure how useful any of this would be to other people, but here is
some stuff that i have been thinking about. 

The great thing about org files is that they are ASCII and therefore
very portable. The only problem with org files is that you need
(X)Emacs to do anything good with them!

This is not quite true, but it is in the case i am talking about. I
regularly copy my org files to the SDCard in my mobile phone, so i can
read them on the move. I have a text editing application on the phone,
so i can also update and create files on the move.

My problem is that the editor is not (X)Emacs, and so cannot
intelligently update the org information. The ideas below are not all
about this use case, but its what started me thinking about them.

And so on to the ideas.

* org-save-hooks
  I have an Org file, and to export useful lists from it that i can
  use offline i have to go into agenda mode and export the
  information.
  
  What i think would be cool is if i could just do an export at save
  time. What i think would be cooler, would be that i could specify
  these exports in a #+VAR with TAG search criteria. 

  Something like #+AUTOEXPORT TYPE filename <search criteria>
  e.g.:
  #+AUTOEXPORT ASCII work_stuff.txt +WORK-HOME
  #+AUTOEXPORT ICAL  home_appointments.ical +HOME-WORK

* Fast Update mode - for minimal editing in another editor
  This is where the **** in the outline is prefixed with a number or
  character, and processed at load time.

  1** TODO This top level task is done archive it
  +** TODO move this one TODO stage further

  Then something in my org config like: 
  (("1"  my-mark-done-and-archive) ("+" org-cycle))

* embedded file link - for tables, possibly images where supported.
  This is something i was thinking about for linking external tables
  into a document. To have the table as a separate document, possibly
  a CSV one converted to tbl mode automagicaly. The basic idea being
  that i can use something other than emacs to update the data in the
  tables and see the updates in my org document.

* an option to export CSV using quotes and commas.

* The ability to change the command prefix from ctrl-c.
  I have migrated from the pinkie killing ctrl to the more finger
  friendly alt and escape keys for most of my (X)Emacs usage.

* Agenda Collections.
  Essentially the ability to define org-agenda-files from a #+
  file link. This allows me to separate out work and home for
  example. When i open my work file, it agendas my work org
  files. When i open my home org file it could agenda my home and work
  projects, if i set the links up.
  e.g:
  #+COLLECTION file://blah.org

What do people think? and also does anyone have a non (X)Emacs
strategy for dealing with or updating org-mode?

cheers,

Tim.

Re: Some suggestions - mostly for non (X)Emacs updating of org mode files.

[Nic]

Tim O'Callaghan <timo@dspsrv.com> writes:

> What do people think? and also does anyone have a non (X)Emacs
> strategy for dealing with or updating org-mode?

I like your ideas.


The reason I'm working on microformat exports (eg: org-export-as-xoxo)
is because I want to be able to share org files more widely and to use
org-mode as a general authoring tool.

Imagine this: an org file sitting underneath a FUSE file
system... every time the org file is updated the FUSE file system
calls emacs to export the data to microformats. The FUSE fs presents
the org file and the microformat.

Any webserver can then serve the microformat up to end users.


Nic

Re: Some suggestions - mostly for non (X)Emacs updating of org mode files.

[Carsten Dominik]

Hi Tim,


On Apr 25, 2006, at 16:27, Tim O'Callaghan wrote:
>
> * org-save-hooks
>   I have an Org file, and to export useful lists from it that i can
>   use offline i have to go into agenda mode and export the
>   information.
>
>   What i think would be cool is if i could just do an export at save
>   time. What i think would be cooler, would be that i could specify
>   these exports in a #+VAR with TAG search criteria.
>
>   Something like #+AUTOEXPORT TYPE filename <search criteria>
>   e.g.:
>   #+AUTOEXPORT ASCII work_stuff.txt +WORK-HOME
>   #+AUTOEXPORT ICAL  home_appointments.ical +HOME-WORK

Arranging for export at save time is not hard, this can be done with 
hooks and is not a problem.  However, I do not understand exactly what 
you mean with the search options.

Do you mean to produce an agenda buffer with those search options and 
then export the agenda buffer itself?

Or do you mean to construct another Org-mode document containing only 
the trees with these tags and export that one?

>
> * Fast Update mode - for minimal editing in another editor
>   This is where the **** in the outline is prefixed with a number or
>   character, and processed at load time.
>
>   1** TODO This top level task is done archive it
>   +** TODO move this one TODO stage further
>
>   Then something in my org config like:
>   (("1"  my-mark-done-and-archive) ("+" org-cycle))

At first I did not like the look of this, but after some thinking, this 
might not be a bad idea, and pretty useful, too.  Mind that this will 
only work if you export the full Org-mode file.  If you make a 
selection first (e.g. in the agenda buffer) and export that, the 
connection between the selected lines and the originial Org-mode file 
are broken, and it is not possible to link the autoupdate information 
back in a safe way.

Then I don't really thing you would have to be able to customize this, 
as there are only very few operations for which this makes sense:

A****  Archive
T**** Mark TODO
D**** Mark DONE
N**** Cycle TODO to the next state

Can't really think of anything else.

>
> * embedded file link - for tables, possibly images where supported.
>   This is something i was thinking about for linking external tables
>   into a document. To have the table as a separate document, possibly
>   a CSV one converted to tbl mode automagicaly. The basic idea being
>   that i can use something other than emacs to update the data in the
>   tables and see the updates in my org document.

THis is very hard and really transforms Org-mode files into something 
which is no longer plain text, so I am a bit worried here.  What is 
wrong about using the proper file link, and then both editing and 
looking at the external table/file in the proper application?

>
> * an option to export CSV using quotes and commas.

For tables, I take it?

>
> * The ability to change the command prefix from ctrl-c.
>   I have migrated from the pinkie killing ctrl to the more finger
>   friendly alt and escape keys for most of my (X)Emacs usage.

This is very hard to do.  The keymap of Org-mode i extremely full, on 
many systems ALT and META is actually the same etc.  Your best bet for 
this is to write a mode hook that makes your own key bindings.  Just 
copy the entire define-key org-mode-map section and then hack it your 
way, wrap a function around it and call this function in org-mode-hook 
or org-load-hook.

>
> * Agenda Collections.
>   Essentially the ability to define org-agenda-files from a #+
>   file link. This allows me to separate out work and home for
>   example. When i open my work file, it agendas my work org
>   files. When i open my home org file it could agenda my home and work
>   projects, if i set the links up.
>   e.g:
>   #+COLLECTION file://blah.org

I can see very much the use of this.  The only problem I see is that 
org-agenda is a *global* command, that is not always called from an 
Org-mode buffer.  Lets say you are loading several org-mode files, with 
several different COLLECTION lines.  Then, if you are in a non-org-mode 
file, which collection should be used?

In principle I would think that what you want can be done using File 
Variables already now, for example in the first line of your document:

   -*- mode: org; org-agenda-files: "~/blah.agendafiles"; -*-

But of course it would be more consistent to drive this form a #+ line.

- Carsten

Re: Some suggestions - mostly for non (X)Emacs updating of org mode files.

[Tim O'Callaghan]

On Thu, Apr 27, 2006 at 01:23:10PM +0200, Carsten Dominik wrote:
> Hi Tim,
> 
Hi ;)

First, my use case is mostly for mobile phones and PDAs. I have one
big org file that i use for almost everything; other than some
personal and work related development projects. I have an editor on my
phone that will allow me to perform searches on tags and update an org
file. Other people using smart phones and PDAs can probably do the
same, but may not have the range of input options i have with a full
keyboard. They would possibly have T-9 predictive texting and/or just
numbers.

But this is mostly my itch that i want you to help scratch :)

> 
> On Apr 25, 2006, at 16:27, Tim O'Callaghan wrote:
> >
> >* org-save-hooks
> >  I have an Org file, and to export useful lists from it that i can
> >  use offline i have to go into agenda mode and export the
> >  information.
> >
> >  What i think would be cool is if i could just do an export at save
> >  time. What i think would be cooler, would be that i could specify
> >  these exports in a #+VAR with TAG search criteria.
> >
> >  Something like #+AUTOEXPORT TYPE filename <search criteria>
> >  e.g.:
> >  #+AUTOEXPORT ASCII work_stuff.txt +WORK-HOME
> >  #+AUTOEXPORT ICAL  home_appointments.ical +HOME-WORK
> 
> Arranging for export at save time is not hard, this can be done with 
> hooks and is not a problem.  However, I do not understand exactly 
> what you mean with the search options.
> 
> Do you mean to produce an agenda buffer with those search options and 
> then export the agenda buffer itself?
> 

Yes and/or sparse trees generated by tag/searching. 

> Or do you mean to construct another Org-mode document containing only 
> the trees with these tags and export that one?
> 

How about both? My reasoning for this is using tags for Getting Things
Done contexts. I tag something as NEXT, CALL, BUY, ATHOME, and i can
search for those tags. It would be easier for me, and many others i
suspect, to auto generate a file containing just those next actions
for a context from my huge org file. This would be more useful to
other as an agenda view export. But it might be as simple to export a
sparse tree into an org like file instead. I can see both may be used.

> >
> >* Fast Update mode - for minimal editing in another editor
> >  This is where the **** in the outline is prefixed with a number or
> >  character, and processed at load time.
> >
> >  1** TODO This top level task is done archive it
> >  +** TODO move this one TODO stage further
> >
> >  Then something in my org config like:
> >  (("1"  my-mark-done-and-archive) ("+" org-cycle))
> 
> At first I did not like the look of this, but after some thinking, 
> this might not be a bad idea, and pretty useful, too.  Mind that this 
> will only work if you export the full Org-mode file.  If you make a 
> selection first (e.g. in the agenda buffer) and export that, the 
> connection between the selected lines and the originial Org-mode file 
> are broken, and it is not possible to link the autoupdate information 
> back in a safe way.
> 

You have mechanisms for linking back into an org file. You can already
link to a heading using searches and radio references. You could add a
file reference at the end of the line like:
* TODO This is a Top level task that i want to perform at home :HOME:<file:work.org::ThisTopLevelTask>. 
Or just autogenerate that link.

As i use one org file, it might be as simple as mapping the headings
back. If there is any ambiguity you could ask the user to select the
right heading. This would probably be best done in agenda mode.

> Then I don't really thing you would have to be able to customize 
> this, as there are only very few operations for which this makes 
> sense:
> 
> A****  Archive
> T**** Mark TODO
> D**** Mark DONE
> N**** Cycle TODO to the next state
> 
> Can't really think of anything else.
> 

I prefer configurable, because then people can use numbers. This is
the idea that the editor may have limited UI. I'm using a j2me based
editor called JPE at the moment:
http://my-communicator.com/s80/software/applications.php?fldAuto=556&faq=2

But other people may be using something like this: 
http://www.getjar.com/products/3960/TextEditor

Or this which i'm currently playing with: 
http://www.bermin.net/index.html

As for other things, it depends on what you want emacs to be able to
do with an externally changed org mode file. For me this is about
using org mode in an intelligent way with my mobile phone/pda. I can
imagine wanting to write functions like:

>* move this huge piece of text and tables down a level
<* move this huge piece of text and tables up a level
M* ask to recategorise this heading when i open org mode
+* remind me about this when i open org mode so i can brain dump on it
   in a real editor.
D* ask me to schedule this as an event when i open org mode.
O* open my mail client to send an email to this email address i just got
C* search bbdb for the contact details of the phone no on this line.
c* search ldap for the contact details of this name
B* open a web browser to this link i wanted to check out when i got back to my machine
R* remind me to look at TheseSearchTags headings when i get back to my machine.

> >
> >* embedded file link - for tables, possibly images where supported.
> >  This is something i was thinking about for linking external tables
> >  into a document. To have the table as a separate document, 
> >possibly
> >  a CSV one converted to tbl mode automagicaly. The basic idea being
> >  that i can use something other than emacs to update the data in 
> >the
> >  tables and see the updates in my org document.
> 
> THis is very hard and really transforms Org-mode files into something 
> which is no longer plain text, so I am a bit worried here.  What is 
> wrong about using the proper file link, and then both editing and 
> looking at the external table/file in the proper application?
> 

Nothing. I guess i just prefer the emacs for all of my heavy
lifting. Again this is based on the idea that changes could be made to
a table by a different app, and still work with org mode. An example
app might be:
http://mixa.dev.juga.ru/microcalc/microcalc.htm

It might not be feasible for CSV files, but displaying linked images
should be feasible.

With regards to file linking a BUG report: w32-shell-execute needs to
be mswindows-shell-execute in org-file-apps-defaults-windowsnt for
XEmacs 21.4.19

That implementation of links does not allow me to link to an external
file for editing. Say i have a to <file:~/.bashrc> it tries to find a
mime type and execute a program for it, which then fails. How about an
<edit:> link, to just open the file directly? Or if the mime operation
fails, just default to opening the file?

Also you probably want to perform an (expand-file-name) on local file
names under windows. As under win2k that will convert the relative
path to a usable windows one.

> >
> >* an option to export CSV using quotes and commas.
> 
> For tables, I take it?
> 

Yep.

> >
> >* The ability to change the command prefix from ctrl-c.
> >  I have migrated from the pinkie killing ctrl to the more finger
> >  friendly alt and escape keys for most of my (X)Emacs usage.
> 
> This is very hard to do.  The keymap of Org-mode i extremely full, on 
> many systems ALT and META is actually the same etc.  Your best bet 
> for this is to write a mode hook that makes your own key bindings.  
> Just copy the entire define-key org-mode-map section and then hack it 
> your way, wrap a function around it and call this function in 
> org-mode-hook or org-load-hook.
> 

It does look big alright. I was thinking of just creating a
'prefix-key' variable, and tagging it on to all of the keymaps.

Also I should note a windows BUG: ALT-TAB switches applications under
windows, and so cannot be used for completion. I was thinking it might
be worth hijacking the abbrev mode key Alt-/, as i use that all the
time for completion.

> >
> >* Agenda Collections.
> >  Essentially the ability to define org-agenda-files from a #+
> >  file link. This allows me to separate out work and home for
> >  example. When i open my work file, it agendas my work org
> >  files. When i open my home org file it could agenda my home and 
> >work
> >  projects, if i set the links up.
> >  e.g:
> >  #+COLLECTION file://blah.org
> 
> I can see very much the use of this.  The only problem I see is that 
> org-agenda is a *global* command, that is not always called from an 
> Org-mode buffer.  Lets say you are loading several org-mode files, 
> with several different COLLECTION lines.  Then, if you are in a 
> non-org-mode file, which collection should be used?
> 

Good point. I guess it would be an agenda thing. As the user I would
have to specify WORK.ORG as current collection designator, and agenda
would then build the from the linked files. 

> In principle I would think that what you want can be done using File 
> Variables already now, for example in the first line of your 
> document:
> 
>   -*- mode: org; org-agenda-files: "~/blah.agendafiles"; -*-
> 
> But of course it would be more consistent to drive this form a #+ 
> line.
> 

Never occurred to me to do it of that way. I'll use that for the moment.

Tim.

Re: Some suggestions - mostly for non (X)Emacs updating of org mode files.

[Carsten Dominik]

Tim, there were a few things in your email which I previously 
overlooked, sorry.

> With regards to file linking a BUG report: w32-shell-execute needs to
> be mswindows-shell-execute in org-file-apps-defaults-windowsnt for
> XEmacs 21.4.19

Does anyone know what the systematics on this are?  Which Emacs 
versions use which nomenclature?

> That implementation of links does not allow me to link to an external
> file for editing. Say i have a to <file:~/.bashrc> it tries to find a
> mime type and execute a program for it, which then fails. How about an
> <edit:> link, to just open the file directly? Or if the mime operation
> fails, just default to opening the file?

I don't think this is needed.  For one, you can set up org-file-app for 
specific files to use Emacs.  But more importantly, if you use mouse-3 
to follow a file link, or equivalently if you follow the link with a 
prefix argument like `C-u C-c C-o' or `C-u RET' (the latter if you have 
turned on org-return-follows-link) then the file will always be opened 
in Emacs.

Maybe I need to make org-file-apps to accept regular expressions, not 
just extensions for identifying file types....

>
> Also you probably want to perform an (expand-file-name) on local file
> names under windows. As under win2k that will convert the relative
> path to a usable windows one.

In what situation?  When following a file link?

Thanks

- Carsten

Re: Some suggestions - mostly for non (X)Emacs updating of org mode files.

[Tim O'Callaghan]

On Tue, May 16, 2006 at 09:51:18AM +0200, Carsten Dominik wrote:
> 
> Tim, there were a few things in your email which I previously 
> overlooked, sorry.
> 
> >With regards to file linking a BUG report: w32-shell-execute needs 
> >to
> >be mswindows-shell-execute in org-file-apps-defaults-windowsnt for
> >XEmacs 21.4.19
> 
> Does anyone know what the systematics on this are?  Which Emacs 
> versions use which nomenclature?
> 

As far as i know XEmacs uses mswindows-* and Gnu Emacs uses w32-*, but
the parameters seem to be the same.

> >That implementation of links does not allow me to link to an 
> >external
> >file for editing. Say i have a to <file:~/.bashrc> it tries to find 
> >a
> >mime type and execute a program for it, which then fails. How about 
> >an
> ><edit:> link, to just open the file directly? Or if the mime 
> >operation
> >fails, just default to opening the file?
> 
> I don't think this is needed.  For one, you can set up org-file-app 
> for specific files to use Emacs.  But more importantly, if you use 
> mouse-3 to follow a file link, or equivalently if you follow the link 
> with a prefix argument like `C-u C-c C-o' or `C-u RET' (the latter if 
> you have turned on org-return-follows-link) then the file will always 
> be opened in Emacs.
> 

I'll try those out. As for using a mouse-3 I prefer using the
keyboard, its quicker than using a mouse. I would prefer that under
windows, it is the other way around. So that any file that is not
specifically mentioned in the org-file-apps is opened by default, and
C-RET performs the system mime operation.

> Maybe I need to make org-file-apps to accept regular expressions, not 
> just extensions for identifying file types....
> 
> >
> >Also you probably want to perform an (expand-file-name) on local 
> >file
> >names under windows. As under win2k that will convert the relative
> >path to a usable windows one.
> 
> In what situation?  When following a file link?
> 

Under XEmacs ms windows the link file:~/.bashrc means nothing. I've
come across this before as i use my .xemacs config on a few different
platforms. If you use (expand-file-name "~/.bashrc") on windows it
might return "C:\\home\ dir\\tim\\.bashrc" and on linux ("/home\
dir/tim/.bashrc"). When combined with calling an external process is
better, especially since it also shell-quotes the string.

IIRC This works for Gnu Emacs and XEmacs.

Could i also suggest that when opening a link that ends in a / or \
that it invokes dired mode on that directory?

Tim.

Re: Some suggestions - mostly for non (X)Emacs updating of org mode files.

[Carsten Dominik]

On May 16, 2006, at 10:44, Tim O'Callaghan wrote:
> As far as i know XEmacs uses mswindows-* and Gnu Emacs uses w32-*, but
> the parameters seem to be the same.

OK, I'll check how to fix this.

> I'll try those out. As for using a mouse-3 I prefer using the
> keyboard, its quicker than using a mouse. I would prefer that under
> windows, it is the other way around. So that any file that is not
> specifically mentioned in the org-file-apps is opened by default, and
> C-RET performs the system mime operation.

Hmm.  I guess there could be an option to reverse this, yes.

> Under XEmacs ms windows the link file:~/.bashrc means nothing. I've
> come across this before as i use my .xemacs config on a few different
> platforms. If you use (expand-file-name "~/.bashrc") on windows it
> might return "C:\\home\ dir\\tim\\.bashrc" and on linux ("/home\
> dir/tim/.bashrc"). When combined with calling an external process is
> better, especially since it also shell-quotes the string.
>
> IIRC This works for Gnu Emacs and XEmacs.

I do expand-file-name on the file path before opening it.  Can you give 
me an example on when this fils to work correctly?

>
> Could i also suggest that when opening a link that ends in a / or \
> that it invokes dired mode on that directory?

Add an entry '(directory . emacs) to org-file-apps.  Thi indicates that 
directories should be opened in Emacs, and that automatically means 
dired.  I just see that this feature is not properly documented - will 
fix this.

Thanks.

Re: Some suggestions - mostly for non (X)Emacs updating of org mode files.

[Tim O'Callaghan]

> >IIRC This works for Gnu Emacs and XEmacs.
> 
> I do expand-file-name on the file path before opening it.  Can you 
> give me an example on when this fils to work correctly?
> 

I checked, and its its not the expand file name problem i thought it
was. Once i patched the w32-type problem i was having i saw it Its a
remember mode thing. 

If i use remember mode on a line that has nothing on it, then remember
mode tags :: on the end of the file. When you try and open the file
the :: expands to !! on the end of the filename. 

[[file:bashrc::][file:basrc::]] does not open bashrc, rather it tries
to open bashrc!!

Tim.

Re: Some suggestions - mostly for non (X)Emacs updating of org mode files.

[Carsten Dominik]

On May 16, 2006, at 12:24, Tim O'Callaghan wrote:

>>> IIRC This works for Gnu Emacs and XEmacs.
>>
>> I do expand-file-name on the file path before opening it.  Can you
>> give me an example on when this fils to work correctly?
>>
>
> I checked, and its its not the expand file name problem i thought it
> was. Once i patched the w32-type problem i was having i saw it Its a
> remember mode thing.
>
> If i use remember mode on a line that has nothing on it, then remember
> mode tags :: on the end of the file. When you try and open the file
> the :: expands to !! on the end of the filename.
>
> [[file:bashrc::][file:basrc::]] does not open bashrc, rather it tries
> to open bashrc!!

This is fixed now.  Using `C-c l' in an empty line in a file no longer 
adds "::context" to the file link.

- Carsten