Re: Google calendar to org mode script and a feature request for agenda

Re: Google calendar to org mode script and a feature request for agenda

[Torsten Wagner]

[-- Attachment #1: Type: text/plain, Size: 1953 bytes --]

Just by reading and without the possibility to test it. Why you multiply the hour with 36 instead of 3600 to get seconds?

Grettings
 Torsten 

Eric S Fraga <ucecesf@ucl.ac.uk> wrote:

>On Wed, 7 Jul 2010 17:38:45 -0300, Daniel Martins <danielemc@gmail.com> wrote:
>> 
>> Eric,
>> 
>> Your awk seems to get timed appts in GMT
>> 
>> How can I adapt it to GMT-3
>> 
>> Daniel
>
>Okay!  I think I've got this working for any time zone (as well as
>adding some more functionality -- read the prologue in the script for
>info).  
>
>Attached is the awk script.  I use this from within a shell script (on
>Linux) that essentially does this:
>
>--8<---------------cut here---------------start------------->8---
>ICS=basic.ics
>ORG=googlecalendar.org
>AWK=ical2org.awk
>
># get the Google calendar
>wget  http://www.google.com/calendar/ical/[...]/basic.ics
>
># convert the ical entries to org format, adjusting for the
># time zone information
>
># this next command yields hours from UTC, + or -, times 100
># Note: this does not cater for those people living in time zones
># that are not aligned with discrete hours (e.g. Newfoundland)...
>timezone=$(date +%z | sed 's/^\([+-]\)0/\1/')
>
># convert this to seconds for use in the awk script
>seconds=$(($timezone*36))
>
># and now process the ics file with appropriate time zone
>awk -f $AWK --assign SECONDS=$seconds < $ICS > $ORG
>--8<---------------cut here---------------end--------------->8---
>
>Please test this all out and let me know if it works.  If the date and
>sed commands work, you should be adjusting the times by -3*3600=-10800
>seconds.  This seems to be working for me with BST (aka GMT+1).
>
>eric
>
>-- 
>Eric S Fraga
>GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D
>
>_______________________________________________
>Emacs-orgmode mailing list
>Please use `Reply All' to send replies to the list.
>Emacs-orgmode@gnu.org
>http://lists.gnu.org/mailman/listinfo/emacs-orgmode

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Google calendar to org mode script and a feature request for agenda

[Nick Dokos]

Torsten Wagner  <torsten.wagner@gmail.com> wrote:

> Just by reading and without the possibility to test it. Why you multiply the hour with 36 instead of 3600 to get seconds?
> 
> > ...
> ># this next command yields hours from UTC, + or -, times 100
                                                      ^^^^^^^^^
There's your factor of 100 (although I, too, have not tested it.)

> ># Note: this does not cater for those people living in time zones
> ># that are not aligned with discrete hours (e.g. Newfoundland)...
> >timezone=$(date +%z | sed 's/^\([+-]\)0/\1/')
> >

Cheers,
Nick

Re: Google calendar to org mode script and a feature request for agenda

[Eric S Fraga]

On Fri, 09 Jul 2010 09:57:48 +0900, Torsten Wagner <torsten.wagner@gmail.com> wrote:
> 
> Just by reading and without the possibility to test it. Why you multiply the hour with 36 instead of 3600 to get seconds?

Because the value returned by the %z format for the date command is in
"hundred hours" form so that, for instance, GMT+1 gives +0100...
Furthermore, note that I remove the leading 0 or else the bash
arithmetic evaluation treats the number as octal.

The comment is a little terse!  Sorry about that.
-- 
Eric S Fraga
GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D

Re: [google] updated awk script for google to org conversion

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 598 bytes --]

I've updated the awk script for converting Google calendar iCal export to org:

- the time zone calculation is now down completely within the awk
  script, not requiring any external calculations.

- fixed handling of continuation lines as Google seems to have
  changed, since last night, the format of the export iCal entries.

Again, comments, questions and criticisms all more than welcome!

Please note that the shell command required to use this is now much simpler:

  awk -f thisawkscript < icsfile > orgfile

as there is no longer any need to define the SECONDS variable.

Enjoy! :)

eric

[-- Attachment #2: ical2org.awk --]
[-- Type: text/plain, Size: 6961 bytes --]

# awk script for converting an iCal formatted file to a sequence of org-mode headings.
# this may not work in general but seems to work for day and timed events from Google's
# calendar, which is really all I need right now...
#
# usage:
#   awk -f THISFILE < icalinputfile.ics > orgmodeentries.org
#
# Note: change org meta information generated below for author and
# email entries!
#
# Known bugs:
# - not so much a bug as a possible assumption: date entries with no time
#   specified are assumed to be independent of the time zone.
#
# Eric S Fraga
# 20100629 - initial version
# 20100708 - added end times to timed events
#          - adjust times according to time zone information
#          - fixed incorrect transfer for entries with ":" embedded within the text
#          - added support for multi-line summary entries (which become headlines)
# 20100709 - incorporated time zone identification
#          - fixed processing of continuation lines as Google seems to
#            have changed, in the last day, the number of spaces at
#            the start of the line for each continuation...
#          - remove backslashes used to protect commas in iCal text entries
#
# Last change: 2010.07.09 10:42:54
#----------------------------------------------------------------------------------

# a function to take the iCal formatted date+time, convert it into an
# internal form (seconds since time 0), and adjust according to the
# local time zone (specified by +-seconds calculated in the BEGIN
# section)

function datetimestamp(input)
{
    # convert the iCal Date+Time entry to a format that mktime can understand
    datespec = gensub("([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])T([0-9][0-9])([0-9][0-9])([0-9][0-9]).*[\r]*", "\\1 \\2 \\3 \\4 \\5 \\6", "g", input);
    # print "date spec : " datespec; convert this date+time into
    # seconds from the beginning of time and include adjustment for
    # time zone, as determined in the BEGIN section below.  For time
    # zone adjustment, I have not tested edge effects, specifically
    # what happens when UTC time is a different day to local time and
    # especially when an event with a duration crosses midnight in UTC
    # time.  It should work but...
    timestamp = mktime(datespec) + seconds;
    # print "adjusted    : " timestamp
    # print "Time stamp  : " strftime("%Y-%m-%d %H:%M", timestamp);
    return timestamp;
}

BEGIN {
    # use a colon to separate the type of data line from the actual contents
    FS = ":";
    
    # determine the number of seconds to use for adjusting for time
    # zone difference from UTC.  This is used in the function
    # datetimestamp above.  The time zone information returned by
    # strftime() is in hours * 100 so we multiply by 36 to get
    # seconds.  This does not work for time zones that are not an
    # integral multiple of hours (e.g. Newfoundland)
    seconds = gensub("([+-])0", "\\1", "", strftime("%z")) * 36;
    
    date = "";
    entry = ""
    first = 1;			# true until an event has been found
    headline = ""
    icalentry = ""  # the full entry for inspection
    id = ""
    indescription = 0;
    time2given = 0;
    
    print "#+TITLE:     Main Google calendar entries"
    print "#+AUTHOR:    Eric S Fraga"
    print "#+EMAIL:     e.fraga@ucl.ac.uk"
    print "#+DESCRIPTION: converted using the ical2org awk script"
    print "#+CATEGORY: google"
    print " "
}

# continuation lines (at least from Google) start with two spaces
# if the continuation is after a description or a summary, append the entry
# to the respective variable

/^[ ]+/ { 
    if (indescription) {
	entry = entry gensub("\r", "", "g", gensub("^[ ]+", "", "", $0));
    } else if (insummary) {
	summary = summary gensub("\r", "", "g", gensub("^[ ]+", "", "", $0))
    }
    icalentry = icalentry "\n" $0
}

/^BEGIN:VEVENT/ {
    # start of an event.  if this is the first, output the preamble from the iCal file
    if (first) {
	print "* COMMENT original iCal preamble"
	print gensub("\r", "", "g", icalentry)
	icalentry = ""
    }
    first = false;
}
# any line that starts at the left with a non-space character is a new data field

/^[A-Z]/ {
    # we ignore DTSTAMP lines as they change every time you download
    # the iCal format file which leads to a change in the converted
    # org file as I output the original input.  This change, which is
    # really content free, makes a revision control system update the
    # repository and confuses.
    if (! index("DTSTAMP", $1)) icalentry = icalentry "\n" $0
    # this line terminates the collection of description and summary entries
    indescription = 0;
    insummary = 0;
}

# this type of entry represents a day entry, not timed, with date stamp YYYYMMDD

/^DTSTART;VALUE=DATE/ {
    date = gensub("([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9]).*[\r]", "\\1-\\2-\\3", "g", $2)
    # print date
}

# this represents a timed entry with date and time stamp YYYYMMDDTHHMMSS
# we ignore the seconds

/^DTSTART:/ {
    # print $0
    date = strftime("%Y-%m-%d %H:%M", datetimestamp($2));
    # print date;
}

# and the same for the end date; here we extract only the time and append this to the 
# date+time found by the DTSTART entry.  We assume that entry was there, of course.
# should probably add some error checking here!  In time...

/^DTEND:/ {
    # print $0
    time2 = strftime("%H:%M", datetimestamp($2));
    date = date "-" time2;
}

# The description will the contents of the entry in org-mode.
# this line may be continued.

/^DESCRIPTION/ { 
    $1 = "";
    entry = entry "\n" gensub("\r", "", "g", $0);
    indescription = 1;
}

# the summary will be the org heading

/^SUMMARY/ { 
    $1 = "";
    summary = gensub("\r", "", "g", $0);
    insummary = 1;
}

# the unique ID will be stored as a property of the entry

/^UID/ { 
    $1 = "";
    id = gensub("\r", "", "g", $0);
}

# when we reach the end of the event line, we output everything we
# have collected so far, creating a top level org headline with the
# date/time stamp, unique ID property and the contents, if any

/^END:VEVENT/ {
    # translate \n sequences to actual newlines and unprotect commas (,)
    print "* " gensub("\\\\,", ",", "g", gensub("\\\\n", " ", "g", summary))
    print "  :PROPERTIES:"
    print "  :ID:       " id
    print "  :END:"
    print "  <" date ">"
    # for the entry, convert all embedded "\n" strings to actual newlines
    print ""
    # translate \n sequences to actual newlines and unprotect commas (,)
    print gensub("\\\\,", ",", "g", gensub("\\\\n", "\n", "g", entry));
    print "** COMMENT original iCal entry"
    print gensub("\r", "", "g", icalentry)
    summary = ""
    date = ""
    entry = ""
    icalentry = ""
    indescription = 0
}

# Local Variables:
# time-stamp-line-limit: 1000
# time-stamp-format: "%04y.%02m.%02d %02H:%02M:%02S"
# time-stamp-active: t
# time-stamp-start: "Last change:[ \t]+"
# time-stamp-end: "$"
# End:

[-- Attachment #3: Type: text/plain, Size: 75 bytes --]

-- 
Eric S Fraga
GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D

[-- Attachment #4: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [google] updated awk script for google to org conversion

[Daniel Martins]

Eric,

It is working (up to the moment) perfectly!!

I am really grateful to you!

Daniel

2010/7/9 Eric S Fraga <ucecesf@ucl.ac.uk>:
> I've updated the awk script for converting Google calendar iCal export to org:
>
> - the time zone calculation is now down completely within the awk
>  script, not requiring any external calculations.
>
> - fixed handling of continuation lines as Google seems to have
>  changed, since last night, the format of the export iCal entries.
>
> Again, comments, questions and criticisms all more than welcome!
>
> Please note that the shell command required to use this is now much simpler:
>
>  awk -f thisawkscript < icsfile > orgfile
>
> as there is no longer any need to define the SECONDS variable.
>
> Enjoy! :)
>
> eric
>
> --
> Eric S Fraga
> GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>
>

Re: [google] updated awk script for google to org conversion

[Sven Bretfeld]

Hi Eric

Thank you very much for the script and the idea. I'm a fan of MobileOrg
for Android, but the diary functions are not convenient to deal with at
the moment. So I adopted your method for date handling. Some
observations:

1. wget doesn't overwrite an older basic.ics laying in the same folder.
   Instead it names the files of further downloads to basic.ics.1,
   basic.ics.2 and so on. Therefore I added an additional line (rm $ICS)
   to your shell-script (not the awk-script, but the one with the
   automatized procedure you sent in a previous posting):

--8<---------------cut here---------------start------------->8---
#!/bin/bash

ICS=/home/sven/bin/google2org/basic.ics
ORG=/home/sven/aktuell/myconf/googlecalendar.org
AWK=/home/sven/bin/google2org/ical2org.awk

cd /home/sven/bin/google2org/
wget http://www.google.com/calendar/ical/[...]/basic.ics

timezone=$(date +%z | sed 's/^\([+-]\)0/\1/')

seconds=$(($timezone*36))

awk -f $AWK < $ICS > $ORG

rm $ICS
--8<---------------cut here---------------end--------------->8---

2. Running the above script with cron, makes Emacs continually ask if
   googlecalendar.org should really be edited, because the file had
   changed on disk. So I put the whole thing into an Emacs function:

--8<---------------cut here---------------start------------->8---
(defun my-org-to-google ()
  (interactive)
  (save-excursion
  (find-file "~/aktuell/myconf/mygtd.org")
  (org-export-icalendar-this-file))
)

(defun my-google-to-org ()
  (interactive)
  (save-excursion
    (kill-buffer "googlecalendar.org")
    (shell-command
     (format
      "sh /home/sven/bin/google2org/importgoogle.sh")))
)

(if (string-equal system-name "kamaloka")
    (run-at-time "3:00am" nil 'my-org-to-google)
    (run-at-time nil 3600 'my-google-to-org)
)
--8<---------------cut here---------------end--------------->8---

Every night a new ical-export is created from my main org-file at
3:00am, waiting to be uploaded in the morning (do you know if it's
possible to automatically import this to GoogleCL?). Every hour the new
entries of GoogleCL are imported to googlecalendar.org (which I include
my agenda-files, of course). To avoid problems, this happens only on one
of my PCs (kamaloka).

Greetings,

Sven

Re: [google] updated awk script for google to org conversion

[Rémi Vanicat]

"Sven Bretfeld" <sven.bretfeld@gmx.ch> writes:


[...]

>
> 2. Running the above script with cron, makes Emacs continually ask if
>    googlecalendar.org should really be edited, because the file had
>    changed on disk. So I put the whole thing into an Emacs function:
>

you could use auto-revert-mode:

  Toggle reverting buffer when file on disk changes.

  With arg, turn Auto Revert mode on if and only if arg is positive.
  This is a minor mode that affects only the current buffer.
  Use `global-auto-revert-mode' to automatically revert all buffers.
  Use `auto-revert-tail-mode' if you know that the file will only grow
  without being changed in the part that is already in the buffer.


[...]


-- 
Rémi Vanicat

Re: [google] updated awk script for google to org conversion

[Michael Steiner]

On 2010-07-12 05:22, Rémi Vanicat wrote:
> "Sven Bretfeld"<sven.bretfeld@gmx.ch>  writes:
>
>
> [...]
>
>>
>> 2. Running the above script with cron, makes Emacs continually ask if
>>     googlecalendar.org should really be edited, because the file had
>>     changed on disk. So I put the whole thing into an Emacs function:
>>
>
> you could use auto-revert-mode:

That's what i do: i slightly changed ical2org.awk by replacing line 77

  print "#+TITLE:     Main Google calendar entries"

with

  print "#+TITLE:     Main Google calendar entries -*- mode: 
auto-revert; mode: org; -*-"

which does make it automatic exactly for this files.  Works fairly well. 
In addition as i only do one-way sync i also turn the files (via 
file-permissions) read-only so the buffer is read-only and not 
accidentally modifled ..

-michael-

BTW: Sven, if you use option -O <outputfilename> you don't have the 
problem .[0-9] files; besides it's easier to manage multiple calendars ...

Re: Re: [google] updated awk script for google to org conversion

[Sven Bretfeld]

Michael Steiner <michisteiner@verizon.net> writes:

> On 2010-07-12 05:22, Rémi Vanicat wrote:
>> "Sven Bretfeld"<sven.bretfeld@gmx.ch>  writes:
>> you could use auto-revert-mode:
>
> That's what i do: i slightly changed ical2org.awk by replacing line 77
>
>  print "#+TITLE:     Main Google calendar entries"
>
> with
>
>  print "#+TITLE:     Main Google calendar entries -*- mode:
> auto-revert; mode: org; -*-"

Thank you very much. I will try that.

Greetings, 

Sven

Re: [google] updated awk script for google to org conversion

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

On 12 Jul 2010 09:49:29 +0200, "Sven Bretfeld" <sven.bretfeld@gmx.ch> wrote:
> 
> Hi Eric
> 
> Thank you very much for the script and the idea. I'm a fan of MobileOrg
> for Android, but the diary functions are not convenient to deal with at
> the moment. So I adopted your method for date handling. Some
> observations:
> 
> 1. wget doesn't overwrite an older basic.ics laying in the same folder.
>    Instead it names the files of further downloads to basic.ics.1,
>    basic.ics.2 and so on. Therefore I added an additional line (rm $ICS)
>    to your shell-script (not the awk-script, but the one with the
>    automatized procedure you sent in a previous posting):

Yes, you are completely correct.  I trimmed down my own shell script a
bit too much.  I do have a "rm ${ICS}" command just before the
wget...  Sorry about that.

> 2. Running the above script with cron, makes Emacs continually ask if
>    googlecalendar.org should really be edited, because the file had
>    changed on disk. So I put the whole thing into an Emacs function:

I think somebody else has already suggested auto-revert-mode, which is
what I use so I don't see this happening at all.

Anyway, hope it continues to work for you and please do give any other
feedback you may have.


[-- Attachment #2: Type: text/plain, Size: 75 bytes --]

-- 
Eric S Fraga
GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D

[-- Attachment #3: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [google] updated awk script for google to org conversion

[Marcelo de Moraes Serpa]

Very cool. Someone should setup a tutorial on Google<->org sync. As of
now, the information is spread around and outdated.

Thanks!

Marcelo.

On Mon, Jul 12, 2010 at 12:50 PM, Eric S Fraga <ucecesf@ucl.ac.uk> wrote:
> On 12 Jul 2010 09:49:29 +0200, "Sven Bretfeld" <sven.bretfeld@gmx.ch> wrote:
>>
>> Hi Eric
>>
>> Thank you very much for the script and the idea. I'm a fan of MobileOrg
>> for Android, but the diary functions are not convenient to deal with at
>> the moment. So I adopted your method for date handling. Some
>> observations:
>>
>> 1. wget doesn't overwrite an older basic.ics laying in the same folder.
>>    Instead it names the files of further downloads to basic.ics.1,
>>    basic.ics.2 and so on. Therefore I added an additional line (rm $ICS)
>>    to your shell-script (not the awk-script, but the one with the
>>    automatized procedure you sent in a previous posting):
>
> Yes, you are completely correct.  I trimmed down my own shell script a
> bit too much.  I do have a "rm ${ICS}" command just before the
> wget...  Sorry about that.
>
>> 2. Running the above script with cron, makes Emacs continually ask if
>>    googlecalendar.org should really be edited, because the file had
>>    changed on disk. So I put the whole thing into an Emacs function:
>
> I think somebody else has already suggested auto-revert-mode, which is
> what I use so I don't see this happening at all.
>
> Anyway, hope it continues to work for you and please do give any other
> feedback you may have.
>
>
> --
> Eric S Fraga
> GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>
>

Google calendar to org mode script and a feature request for agenda

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 2495 bytes --]

Hello,

I finally got myself an Android phone (and am loving it, especially
with an "all you can eat" data plan :-).  I have tried mobileorg for
Android but it's obviously early days and it's actually not really
what I want.

However, I am very happy with the Google syncing provided by the
phone, both for email and for the calendar.  This has caused me to
revisit the possibility of a smoother integration between Google's
calendar and org-mode.

First, I have created an awk script (yes, I'm dating myself: I do use
awk in preference to perl et al.) which allows me to convert the iCal
export from Google calendar to a sequence of org headlines (see
attached).  It's a preliminary version and seems to translate what I
need: simple day and timed events, mostly those that I will tend to do
on the phone as opposed to when I'm at my computer.  I'm not trying
for a comprehensive translation program here...  but I'm putting it on
the list in case anybody finds it useful.

I use this script as follows:

   wget [address specified by google for my calendar]  --> basic.ics
   awk -f ical2org.awk < basic.ics >> googlecalendar.org

this appends any entries in my specific google calendar to the given
org file.  When I go the other way (creating an ics file from org), I
import the org items into a different calendar on google.  When
entries have been synced, I typically delete the original entries
placed in google to avoid them being downloaded over and over again
[1].

* A feature request: time prompt for insert diary agenda function

Anyway, my increased use of google's calendar, has highlighted a
short-coming (?) of the agenda view (or more strictly speaking, the
iCal exporter): entries in which the time of appointment, say, is on
the headline but the date is on the following line, say, get converted
to "day" events as opposed to day+time events.  I.e. something like

   * 11am meeting with colleages
     <2010-06-30 Wed>

does not get exported as a timed event.  Obviously, the easy solution
is to put the time in the date stamp.  However, I like using the
"insert diary" function in the agenda view for defining appointments
and this doesn't allow the time to be specified other than in a
headline.  Would it be possible to enhance the insert diary function
to prompt for a time (and while we're at it, tags as well)?

Thanks,
eric


Footnotes: 
[1]  it may be possible to use the unique ID for each event to avoid creating duplicates... something to look at in the future.


[-- Attachment #2: ical2org.awk --]
[-- Type: text/plain, Size: 2428 bytes --]

# awk script for converting an iCal formatted file to a sequence of org-mode headings.
# this may not work in general but seems to work for day and timed events from Google's
# calendar, which is really all I need right now...
#
# Eric S Fraga, 20100629

BEGIN {
  # use a colon to separate the type of data line from the actual contents
  FS = ":";
  entry = ""
  headline = ""
  id = ""
  indescription = 0;
}

# any line that starts at the left with a non-space character is a new data field

/^[A-Z]/ {indescription = 0;}

# this type of entry represents a day entry, not timed, with date stamp YYYYMMDD
/^DTSTART;VALUE=DATE/ {
  date = gensub("([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9]).*[\r]", "\\1-\\2-\\3", "g", $2)
  #print date
}

# this represents a timed entry with date and time stamp YYYYMMDDTHHMMSS
# we ignore the seconds

/^DTSTART:/ {
  #print $0
  date = gensub("([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])T([0-9][0-9])([0-9][0-9]).*[\r]", "\\1-\\2-\\3 \\4:\\5", "g", $2);
  # print date;
}

# The description will the contents of the entry in org-mode.
# this line may be continued and as I do not know if other lines may be continued,
# we only process continuation lines that come immediately after this one

/^DESCRIPTION/ { 
  entry = gensub("\r", "", "g", $2);
  indescription = 1;
}

# continuation lines (at least from Google) start with two spaces
# if the continuation is after a description, append the entry

/^  / { 
  # print "** continuation line: " $0
  if (indescription) {
    entry = entry gensub("\r", "", "g", $0);
  }
}

# the summary will be the org heading

/^SUMMARY/ { headline = gensub("(.*)[\r]", "\\1", "g", $2); }

# the unique ID will be stored as a property of the entry

/^UID/ { id = gensub("(.*)[\r]", "\\1", "g", $2); }

# when we reach the end of the event line, we output everything we
# have collected so far, creating a top level org headline with the
# date/time stamp, unique ID property and the contents, if any

/^END:VEVENT/ {
  print "* " headline
  print "  :PROPERTIES:"
  print "  :ID:       " id
  print "  :END:"
  print "  <" date ">"
  # for the entry, convert all embedded "\n" strings to actual newlines
  print ""
  print gensub("\\\\n", "\n", "g", entry); # need 4 backslash to get one in the pattern!
  headline = ""
  date = ""
  entry = ""
  indescription = 0
}

END {
    print strftime("* COMMENT ical2org finished at [%Y-%m-%d %H:%M]");
}

[-- Attachment #3: Type: text/plain, Size: 75 bytes --]

-- 
Eric S Fraga
GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D

[-- Attachment #4: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Google calendar to org mode script and a feature request for agenda

[Stephen Eglen]

>
> does not get exported as a timed event.  Obviously, the easy solution
> is to put the time in the date stamp.  However, I like using the
> "insert diary" function in the agenda view for defining appointments
> and this doesn't allow the time to be specified other than in a
> headline.  Would it be possible to enhance the insert diary function
> to prompt for a time (and while we're at it, tags as well)?

hi Eric, 
I had a similar request, and we worked out a solution to this a while
ago.  In the agenda, try 'i d', and then type something like
'11:00-11:45 Seminar with Bob', and the code should parse out the
timestamp as long as you have the following variable set to t:

(setq org-agenda-insert-diary-extract-time t)

Stephen

Re: Google calendar to org mode script and a feature request for agenda

[Daniel Martins]

Eric,

Your awk seems to get timed appts in GMT

How can I adapt it to GMT-3

Daniel

2010/6/29 Eric S Fraga <ucecesf@ucl.ac.uk>:
> Hello,
>
> I finally got myself an Android phone (and am loving it, especially
> with an "all you can eat" data plan :-).  I have tried mobileorg for
> Android but it's obviously early days and it's actually not really
> what I want.
>
> However, I am very happy with the Google syncing provided by the
> phone, both for email and for the calendar.  This has caused me to
> revisit the possibility of a smoother integration between Google's
> calendar and org-mode.
>
> First, I have created an awk script (yes, I'm dating myself: I do use
> awk in preference to perl et al.) which allows me to convert the iCal
> export from Google calendar to a sequence of org headlines (see
> attached).  It's a preliminary version and seems to translate what I
> need: simple day and timed events, mostly those that I will tend to do
> on the phone as opposed to when I'm at my computer.  I'm not trying
> for a comprehensive translation program here...  but I'm putting it on
> the list in case anybody finds it useful.
>
> I use this script as follows:
>
>   wget [address specified by google for my calendar]  --> basic.ics
>   awk -f ical2org.awk < basic.ics >> googlecalendar.org
>
> this appends any entries in my specific google calendar to the given
> org file.  When I go the other way (creating an ics file from org), I
> import the org items into a different calendar on google.  When
> entries have been synced, I typically delete the original entries
> placed in google to avoid them being downloaded over and over again
> [1].
>
> * A feature request: time prompt for insert diary agenda function
>
> Anyway, my increased use of google's calendar, has highlighted a
> short-coming (?) of the agenda view (or more strictly speaking, the
> iCal exporter): entries in which the time of appointment, say, is on
> the headline but the date is on the following line, say, get converted
> to "day" events as opposed to day+time events.  I.e. something like
>
>   * 11am meeting with colleages
>     <2010-06-30 Wed>
>
> does not get exported as a timed event.  Obviously, the easy solution
> is to put the time in the date stamp.  However, I like using the
> "insert diary" function in the agenda view for defining appointments
> and this doesn't allow the time to be specified other than in a
> headline.  Would it be possible to enhance the insert diary function
> to prompt for a time (and while we're at it, tags as well)?
>
> Thanks,
> eric
>
>
> Footnotes:
> [1]  it may be possible to use the unique ID for each event to avoid creating duplicates... something to look at in the future.
>
>
> --
> Eric S Fraga
> GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>
>

Re: Google calendar to org mode script and a feature request for agenda

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 784 bytes --]

On Wed, 7 Jul 2010 17:38:45 -0300, Daniel Martins <danielemc@gmail.com> wrote:
> 
> Eric,
> 
> Your awk seems to get timed appts in GMT
> 
> How can I adapt it to GMT-3
> 
> Daniel

Daniel,

yes, thanks for pointing this out. The script indeed does not
translate the times to the local time zone.  Google outputs (in the
basic.ics file) times in GMT.  However, it also includes the time zone
information but my awk script ignores that information.  The script
also ignores the end time for timed events.

Obviously time for a v2 of this script!

let me have a think about this and I'll see if I can do something a
bit more intelligent (at least for Linux systems where possibly
compatible time zone information can be found in /usr/share/zoninfo/*)
and fix up the end times as well.


[-- Attachment #2: Type: text/plain, Size: 75 bytes --]

-- 
Eric S Fraga
GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D

[-- Attachment #3: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Google calendar to org mode script and a feature request for agenda

[Daniel Martins]

OK Eric

Thank you!

Linux solution is fine!

I think we share the same "addiction" at moment.

Linux-Emacs-OrgMode-Android  and due to Android the need of Google
calendar syncing!

I tried using

gcaldaemon (there is a .deb file in

http://blog.philippheckel.com/2008/09/30/gcaldaemon-deb-package-for-ubuntu-kubuntu/

)

The org -> ics -> google calendar  works with some minor problem, but
I am not satisfied.

A simpler solution seems to be better

Daniel

PS When I saw GoogleCL I had some hope for a "final solution" of this problem

2010/7/7 Eric S Fraga <ucecesf@ucl.ac.uk>:
> On Wed, 7 Jul 2010 17:38:45 -0300, Daniel Martins <danielemc@gmail.com> wrote:
>>
>> Eric,
>>
>> Your awk seems to get timed appts in GMT
>>
>> How can I adapt it to GMT-3
>>
>> Daniel
>
> Daniel,
>
> yes, thanks for pointing this out. The script indeed does not
> translate the times to the local time zone.  Google outputs (in the
> basic.ics file) times in GMT.  However, it also includes the time zone
> information but my awk script ignores that information.  The script
> also ignores the end time for timed events.
>
> Obviously time for a v2 of this script!
>
> let me have a think about this and I'll see if I can do something a
> bit more intelligent (at least for Linux systems where possibly
> compatible time zone information can be found in /usr/share/zoninfo/*)
> and fix up the end times as well.
>
>
> --
> Eric S Fraga
> GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D
>
>

Re: Google calendar to org mode script and a feature request for agenda

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 1435 bytes --]

On Wed, 7 Jul 2010 17:38:45 -0300, Daniel Martins <danielemc@gmail.com> wrote:
> 
> Eric,
> 
> Your awk seems to get timed appts in GMT
> 
> How can I adapt it to GMT-3
> 
> Daniel

Okay!  I think I've got this working for any time zone (as well as
adding some more functionality -- read the prologue in the script for
info).  

Attached is the awk script.  I use this from within a shell script (on
Linux) that essentially does this:

--8<---------------cut here---------------start------------->8---
ICS=basic.ics
ORG=googlecalendar.org
AWK=ical2org.awk

# get the Google calendar
wget  http://www.google.com/calendar/ical/[...]/basic.ics

# convert the ical entries to org format, adjusting for the
# time zone information

# this next command yields hours from UTC, + or -, times 100
# Note: this does not cater for those people living in time zones
# that are not aligned with discrete hours (e.g. Newfoundland)...
timezone=$(date +%z | sed 's/^\([+-]\)0/\1/')

# convert this to seconds for use in the awk script
seconds=$(($timezone*36))

# and now process the ics file with appropriate time zone
awk -f $AWK --assign SECONDS=$seconds < $ICS > $ORG
--8<---------------cut here---------------end--------------->8---

Please test this all out and let me know if it works.  If the date and
sed commands work, you should be adjusting the times by -3*3600=-10800
seconds.  This seems to be working for me with BST (aka GMT+1).

eric

[-- Attachment #2: ical2org.awk --]
[-- Type: text/plain, Size: 5496 bytes --]

# awk script for converting an iCal formatted file to a sequence of org-mode headings.
# this may not work in general but seems to work for day and timed events from Google's
# calendar, which is really all I need right now...
#
# usage:
#   awk -f THISFILE --assign SECONDS=SSSS < icalinputfile.ics > orgmodeentries.org
#
# where "SSSS" is the number of seconds to adjust time to take into
# account the local time zone relative to UTC (e.g. GMT+1 == 3600,
# GMT-1 == -3600).  I have not tested edge effects, specifically what
# happens when UTC time is a different day to local time and
# especially when an event with a duration crosses midnight in UTC
# time.  It should work but...
#
# Note: change org meta information generated below for author and
# email entries!
#
# Known bugs:
# - if the iCal entry has an event reminder, the description field in
#   the VALARM entry for the event will supercede the description field
#   of the originating entry.
# - not so much a bug as a possible assumption: date entries with no time
#   specified are assumed to be independent of the time zone.
#
# Eric S Fraga
# 20100629 - initial version
# 20100708 - added end times to timed events
#          - adjust times according to time zone information
#          - fixed incorrect transfer for entries with ":" embedded within the text
#          - added support for multi-line summary entries (which become headlines)
#----------------------------------------------------------------------------------

# a function to take the iCal formatted date+time, converted it into
# an internal form (seconds since time 0), and adjust according to the
# local time zone (specified by +-SECONDS on the argument to awk)

function datetimestamp(input)
{
    # convert the iCal Date+Time entry to a format that mktime can understand
    datespec = gensub("([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])T([0-9][0-9])([0-9][0-9])([0-9][0-9]).*[\r]*", "\\1 \\2 \\3 \\4 \\5 \\6", "g", input);
    # print "date spec   : " datespec;
    # convert this date+time into seconds from the beginning of time
    timestamp = mktime(datespec);
    # print "time stamp  : " timestamp;
    # and adjust for the time zone, number of seconds from GMT/UTC.
    timestamp += SECONDS;
    # print "adjusted    : " timestamp
    # print "Time stamp  : " strftime("%Y-%m-%d %H:%M", timestamp);
    return timestamp;
}

BEGIN {
    # use a colon to separate the type of data line from the actual contents
    FS = ":";
    date = "";
    time2given = 0;
    entry = ""
    icalentry = ""  # the full entry for inspection
    headline = ""
    id = ""
    indescription = 0;
    print "#+TITLE:     Main Google calendar entries"
    print "#+AUTHOR:    Eric S Fraga"
    print "#+EMAIL:     e.fraga@ucl.ac.uk"
    print "#+DESCRIPTION: converted using the ical2org awk script"
    print "#+CATEGORY: google"
    print " "
}

# any line that starts at the left with a non-space character is a new data field

/^[A-Z]/ {
    if (! index("DTSTAMP", $1)) icalentry = icalentry "\n" $0
    # this line terminates the collection of description and summary entries
    indescription = 0;
    insummary = 0;
}

# this type of entry represents a day entry, not timed, with date stamp YYYYMMDD

/^DTSTART;VALUE=DATE/ {
    date = gensub("([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9]).*[\r]", "\\1-\\2-\\3", "g", $2)
    # print date
}

# this represents a timed entry with date and time stamp YYYYMMDDTHHMMSS
# we ignore the seconds

/^DTSTART:/ {
    # print $0
    date = strftime("%Y-%m-%d %H:%M", datetimestamp($2));
    # print date;
}

# and the same for the end date; here we extract only the time and append this to the 
# date+time found by the DTSTART entry.  We assume that entry was there, of course.
# should probably add some error checking here!  In time...

/^DTEND:/ {
    # print $0
    time2 = strftime("%H:%M", datetimestamp($2));
    date = date "-" time2;
}

# The description will the contents of the entry in org-mode.
# this line may be continued.

/^DESCRIPTION/ { 
    $1 = "";
    entry = gensub("\r", "", "g", $0);
    indescription = 1;
}

# continuation lines (at least from Google) start with two spaces
# if the continuation is after a description or a summary, append the entry
# to the respective variable

/^  / { 
    # print "** continuation line: " $0
    if (indescription) {
	entry = entry gensub("\r", "", "g", $0);
    } else if (insummary) {
	summary = summary gensub("\r", "", "g", $0)
    }
    icalentry = icalentry "\n" $0
}

# the summary will be the org heading

/^SUMMARY/ { 
    $1 = "";
    summary = gensub("\r", "", "g", $0);
    insummary = 1;
}

# the unique ID will be stored as a property of the entry

/^UID/ { 
    $1 = "";
    id = gensub("\r", "", "g", $0);
}

# when we reach the end of the event line, we output everything we
# have collected so far, creating a top level org headline with the
# date/time stamp, unique ID property and the contents, if any

/^END:VEVENT/ {
    print "* " gensub ("\\\\n", " ", "g", summary)
    print "  :PROPERTIES:"
    print "  :ID:       " id
    print "  :END:"
    print "  <" date ">"
    # for the entry, convert all embedded "\n" strings to actual newlines
    print ""
    print gensub("\\\\n", "\n", "g", entry); # need 4 backslash to get one in the pattern!
    print "** COMMENT original iCal entry"
    print gensub("\r", "", "g", icalentry)
    summary = ""
    date = ""
    entry = ""
    icalentry = ""
    indescription = 0
}

[-- Attachment #3: Type: text/plain, Size: 75 bytes --]

-- 
Eric S Fraga
GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D

[-- Attachment #4: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode