org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

I have written a package 'org-caldav' which can sync items to a remote
calendar server using the CalDAV protocol. The main purpose of this
package is to make better use of Org in combination with Android-based
mobile devices (yes, there is mobileOrg, but I have several problems
with it; that's a topic for another day, though).

I think org-caldav is now "good enough" to allow some testing by
adventurous people. I put the code up on github here

https://github.com/dengste/org-caldav

I've tested the code with my Owncloud server and also with Google
Calendar. Please see the Readme on how to set things up. Most
importantly, please use a new, dedicated calendar for testing this
package; do *not* use your precious main calendar. Also, while
org-caldav should only access the calendar you provide through
`org-caldav-calendar-id', you should have a backup of all your calendar
data, just in case.

The initial sync can take some time, depending on the number of
events. Especially the CalDAV interface to Google can be quite slow at
times. If the initial sync aborts for any reason, just run
`org-caldav-sync' again. Note that after the initial sync, only new
events will be transferred, so things should become much snappier.

A few further notes regarding the code:

I am relying mainly on three other packages to do the real heavy
lifting, which is also why org-caldav is actually pretty small:

- url-dav.el for talking to WebDAV servers (CalDAV is essentially WebDAV
  with a few extensions). Unfortunately, the current version of url-dav
  that is shipping with Emacs is broken for practically all CalDAV
  servers (see bug #11916), which is why I provide a patched version of
  url-dav.el in the above repository. You have to make sure that this
  patched version is loaded instead of the one in Emacs proper. I am
  hoping that this will be fixed in Emacs 24.2.

- org-icalendar.el is doing all the conversion from org to icalendar
  events which are needed for CalDAV. This has the advantage that you
  can use all the variables from that package to control how things
  should be exported. The only exception is the actual org files being
  searched for time stamps, which is determined through
  `org-caldav-files'.

- icalendar.el for parsing events from CalDAV. I took some code from
  that package to generate org items from the calendar events.

The syncing between org and the external calendar is not yet really
finished. I would like to have proper two-way-syncing so that you can
freely change events on any side, but this is rather difficult to do and
I'd first like to get some feedback on the current code. Also, I have
a hunch that some Org experts around here might have some good ideas on
how to properly implement this.

Here's what the code can currently do:

- All items in `org-caldav-files' with active time stamps are synced to
  the calendar.

- If you change an item in one of those files in orgmode, this change
  will also get synced to the external calendar (to be exact: this will
  generate a new event and the old one will be deleted).

- If you create a *new* event in the calendar through Android or any
  other client (like browser), this event will land in the
  `org-caldav-inbox' file as an org item.

However:

- If you *change* an item directly on the CalDAV server, this will *not*
  get synced back to Org. What happens exactly depends on the server
  (whether it renames the event or not). Just try it out. The item might
  even get deleted after the next sync - you have been warned.

- Likewise, if you *change* and item in `org-caldav-inbox', this will
  not get synced back to the calendar.

OK, this has gotten way too long already. Please try it out and let me
know how it goes.

Cheers,
David

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

David Engster writes:
> I have written a package 'org-caldav' which can sync items to a remote
> calendar server using the CalDAV protocol. The main purpose of this
> package is to make better use of Org in combination with Android-based
> mobile devices (yes, there is mobileOrg, but I have several problems
> with it; that's a topic for another day, though).
>
> I think org-caldav is now "good enough" to allow some testing by
> adventurous people. I put the code up on github here
>
> https://github.com/dengste/org-caldav

I must admit I'm a tiny bit baffled that no one seems to be interested
in this. Anyway, no hard feelings ( ;-) ), but could please someone with
the necessary permissions put a link to the above on

http://orgmode.org/worg/org-tutorials/org-google-sync.html

Thanks,
David

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Mehul Sanghvi]

On Fri, Jul 20, 2012 at 12:36 PM, David Engster <deng@randomsample.de> wrote:
> David Engster writes:
>> I have written a package 'org-caldav' which can sync items to a remote
>> calendar server using the CalDAV protocol. The main purpose of this
>> package is to make better use of Org in combination with Android-based
>> mobile devices (yes, there is mobileOrg, but I have several problems
>> with it; that's a topic for another day, though).
>>
>> I think org-caldav is now "good enough" to allow some testing by
>> adventurous people. I put the code up on github here
>>
>> https://github.com/dengste/org-caldav
>
> I must admit I'm a tiny bit baffled that no one seems to be interested
> in this. Anyway, no hard feelings ( ;-) ), but could please someone with
> the necessary permissions put a link to the above on
>
> http://orgmode.org/worg/org-tutorials/org-google-sync.html
>
> Thanks,
> David
>


Actually I'm very interested in this, just haven't had time to try it
out.  Hopefully
while on vacation next week :)


-- 
Mehul N. Sanghvi
email: mehul.sanghvi@gmail.com

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Jason F. McBrayer]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

 

On 20.07.2012 12:36, David Engster wrote: 

> I must admit I'm a
tiny bit baffled that no one seems to be interested in this. Anyway, no
hard feelings ( ;-) ), but could please someone with the necessary
permissions put a link to the above on

Honestly, I'm interested, but I
haven't had time to look at it. I'm planning on setting up davical on my
home server, but I don't have the round tuits for it yet, and I didn't
really want to test this until I have davical up and running. 

--

+-----------------------------------------------------------+
| Jason
F. McBrayer jmcbray@carcosa.net |
| If someone conquers a thousand times
a thousand others in |
| battle, and someone else conquers himself, the
latter one |
| is the greatest of all conquerors. --- The Dhammapada |
 

[-- Attachment #2: Type: text/html, Size: 1121 bytes --]

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Nick Dokos]

David Engster <deng@randomsample.de> wrote:

> David Engster writes:
> > I have written a package 'org-caldav' which can sync items to a remote
> > calendar server using the CalDAV protocol. The main purpose of this
> > package is to make better use of Org in combination with Android-based
> > mobile devices (yes, there is mobileOrg, but I have several problems
> > with it; that's a topic for another day, though).
> >
> > I think org-caldav is now "good enough" to allow some testing by
> > adventurous people. I put the code up on github here
> >
> > https://github.com/dengste/org-caldav
> 
> I must admit I'm a tiny bit baffled that no one seems to be interested
> in this. Anyway, no hard feelings ( ;-) ), but could please someone with
> the necessary permissions put a link to the above on
> 
> http://orgmode.org/worg/org-tutorials/org-google-sync.html
> 

Added a section for org-caldav as requested (the original email, pretty
much verbatim) - let me know if something needs fixing.

BTW, you too (that's a collective you :-) ) can do things like this on
Worg - see the section "Maintenance of Worg" on the main Worg page:
http://orgmode.org/worg.

Nick

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Suvayu Ali]

On Fri, Jul 20, 2012 at 06:36:40PM +0200, David Engster wrote:
> David Engster writes:
> > I have written a package 'org-caldav' which can sync items to a remote
> > calendar server using the CalDAV protocol. The main purpose of this
> > package is to make better use of Org in combination with Android-based
> > mobile devices (yes, there is mobileOrg, but I have several problems
> > with it; that's a topic for another day, though).
> >
> > I think org-caldav is now "good enough" to allow some testing by
> > adventurous people. I put the code up on github here
> >
> > https://github.com/dengste/org-caldav
> 
> I must admit I'm a tiny bit baffled that no one seems to be interested
> in this. Anyway, no hard feelings ( ;-) ), but could please someone with
> the necessary permissions put a link to the above on
> 
> http://orgmode.org/worg/org-tutorials/org-google-sync.html
> 

I am very interested, just extremely short of time. I think I will have
some time to have a look at this in about a week from now.

> Thanks,
> David
> 

:)

-- 
Suvayu

Open source is the future. It sets us free.

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Torsten Wagner]

Hey David,

herewith I claim interest but currently lack of time to test it...
just to keep you  motivated  ;)

Guess many are interested actually, this was on the mailing list
several times already.
Any chance to merge with mobileOrg? Would be good to have both ways
and you might save some time doing redundant stuff in both projects.

Torsten


On 15 July 2012 04:12, David Engster <deng@randomsample.de> wrote:
> I have written a package 'org-caldav' which can sync items to a remote
> calendar server using the CalDAV protocol. The main purpose of this
> package is to make better use of Org in combination with Android-based
> mobile devices (yes, there is mobileOrg, but I have several problems
> with it; that's a topic for another day, though).
>
> I think org-caldav is now "good enough" to allow some testing by
> adventurous people. I put the code up on github here
>
> https://github.com/dengste/org-caldav
>
> I've tested the code with my Owncloud server and also with Google
> Calendar. Please see the Readme on how to set things up. Most
> importantly, please use a new, dedicated calendar for testing this
> package; do *not* use your precious main calendar. Also, while
> org-caldav should only access the calendar you provide through
> `org-caldav-calendar-id', you should have a backup of all your calendar
> data, just in case.
>
> The initial sync can take some time, depending on the number of
> events. Especially the CalDAV interface to Google can be quite slow at
> times. If the initial sync aborts for any reason, just run
> `org-caldav-sync' again. Note that after the initial sync, only new
> events will be transferred, so things should become much snappier.
>
> A few further notes regarding the code:
>
> I am relying mainly on three other packages to do the real heavy
> lifting, which is also why org-caldav is actually pretty small:
>
> - url-dav.el for talking to WebDAV servers (CalDAV is essentially WebDAV
>   with a few extensions). Unfortunately, the current version of url-dav
>   that is shipping with Emacs is broken for practically all CalDAV
>   servers (see bug #11916), which is why I provide a patched version of
>   url-dav.el in the above repository. You have to make sure that this
>   patched version is loaded instead of the one in Emacs proper. I am
>   hoping that this will be fixed in Emacs 24.2.
>
> - org-icalendar.el is doing all the conversion from org to icalendar
>   events which are needed for CalDAV. This has the advantage that you
>   can use all the variables from that package to control how things
>   should be exported. The only exception is the actual org files being
>   searched for time stamps, which is determined through
>   `org-caldav-files'.
>
> - icalendar.el for parsing events from CalDAV. I took some code from
>   that package to generate org items from the calendar events.
>
> The syncing between org and the external calendar is not yet really
> finished. I would like to have proper two-way-syncing so that you can
> freely change events on any side, but this is rather difficult to do and
> I'd first like to get some feedback on the current code. Also, I have
> a hunch that some Org experts around here might have some good ideas on
> how to properly implement this.
>
> Here's what the code can currently do:
>
> - All items in `org-caldav-files' with active time stamps are synced to
>   the calendar.
>
> - If you change an item in one of those files in orgmode, this change
>   will also get synced to the external calendar (to be exact: this will
>   generate a new event and the old one will be deleted).
>
> - If you create a *new* event in the calendar through Android or any
>   other client (like browser), this event will land in the
>   `org-caldav-inbox' file as an org item.
>
> However:
>
> - If you *change* an item directly on the CalDAV server, this will *not*
>   get synced back to Org. What happens exactly depends on the server
>   (whether it renames the event or not). Just try it out. The item might
>   even get deleted after the next sync - you have been warned.
>
> - Likewise, if you *change* and item in `org-caldav-inbox', this will
>   not get synced back to the calendar.
>
> OK, this has gotten way too long already. Please try it out and let me
> know how it goes.
>
> Cheers,
> David
>

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Torsten Wagner writes:
> Guess many are interested actually, this was on the mailing list
> several times already.
> Any chance to merge with mobileOrg? Would be good to have both ways
> and you might save some time doing redundant stuff in both projects.

Well, there's really no redundancy since mobileOrg is written in
Java. Also, mobileOrg actually can already sync with the phone's
calendar, but this still has several problems which made it unusable for
me. Also, AFAIK it's only a one-way sync (Org->Calendar but not the
other way round).

-David

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Robert Eckl]

[-- Attachment #1: Type: text/plain, Size: 1497 bytes --]

First, thanks for the great work, it’s very interesting, I found this
yesterday.

 

I’ve done first steps with org-caldav:

 

Acutally, an appointment must not have any text outside the Properties

 

--------

* TerminTest                                                      :Tag:

:PROPERTIES:

:LOCATION: Büro

:END:

<2012-07-28 Sat 20:00-21:00>

--------

 

Will be synced while

 

--------

* TerminTest                                                      :Tag:

:PROPERTIES:

:LOCATION: Büro

:END:

<2012-07-28 Sat 20:00-21:00>

Somewhat

--------

Not. 

 

Combined with MobileOrg this behavior is bothering. 

 

If an appointment is scheduled, it will not be synced if there is no active
timestamp without the keyword SCHEDULED:

By changing the state  Scheduled / Not Scheduled of an appointment it is
possible, that the appointment not will be synced even though the rules
mentioned above are fulfilled.

Example:

* TODO Probe Irgendwo                                            :atag:

  <2012-07-26 Thu>

  :LOGBOOK:

  - State "CANCELLED"  from "TODO"       [2012-07-16 Mo 23:24] \\

    irgendwas

  - State "DONE"       from "TODO"       [2012-07-11 Mi 00:09]

  - State "DONE"       from "TODO"       [2012-07-03 Di 21:21]

  :END:

  :PROPERTIES:

  :ID:       automatic-ID

  :LAST_REPEAT: [2012-07-16 Mo 23:24]

  :LOCATION: Veranstaltungsort

  :END:

 

Cu,

Robert


[-- Attachment #2: Type: text/html, Size: 4554 bytes --]

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Aurélien Aptel]

On Fri, Jul 20, 2012 at 6:36 PM, David Engster <deng@randomsample.de> wrote:
> I must admit I'm a tiny bit baffled that no one seems to be interested
> in this. Anyway, no hard feelings ( ;-) ), but could please someone with

I'm interested in this too for my own project Org-sync [1], even if
calendars are not *directly* related to it.

1: http://orgmode.org/worg/org-contrib/gsoc2012/student-projects/org-sync/index.html

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Robert Eckl writes:
> Acutally, an appointment must not have any text outside the Properties

[...]

I think there's a bug in org-icalendar with regards to handling
tags. I'm getting an icalendar event like this:

BEGIN:VEVENT
UID:f90ec513423879f1e09c7c10fc0e15a9-orgmodecaldav
DTSTART:20120728T200000
DTEND:20120728T210000
SUMMARY:TerminTest                                                      :Tag:
DESCRIPTION: <2012-07-28 Sat 20:00-21:00>\n\nSomewhat
LOCATION: Büro
CATEGORIES:Tag,Termine
END:VEVENT

Putting this event generates an error from Google, but I think it's due
to the :Tag: in the SUMMARY line; the DESCRIPTION looks OK to me.

> Combined with MobileOrg this behavior is bothering.
>
> If an appointment is scheduled, it will not be synced if there is no active
> timestamp without the keyword SCHEDULED:

Again, this looks like a problem with org-icalendar. I'm not very
familiar with that code, but I'll look into it.

Thanks for testing,
David

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

David Engster writes:
> Robert Eckl writes:
>> Acutally, an appointment must not have any text outside the Properties
>
> [...]
>
> I think there's a bug in org-icalendar with regards to handling
> tags. I'm getting an icalendar event like this:
>
> BEGIN:VEVENT
> UID:f90ec513423879f1e09c7c10fc0e15a9-orgmodecaldav
> DTSTART:20120728T200000
> DTEND:20120728T210000
> SUMMARY:TerminTest                                                      :Tag:

Actually, that happened because when I copy&pasted your example the tag
wasn't recognized as such but just simple text (it didn't have the
proper face). This is why it ended up in the SUMMARY line as normal text
above. However, this shouldn't happen when you manually enter tags in
your Org files.

The first way to debug issues like the ones you reported is to create an
Org file with some item that is not synced, then run
org-export-icalendar-this-file and look at the resulting .ics file.

-David

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Torsten Wagner]

Hey David,

found a little bit time to test it.
First of all great work and one more step to get org-mode on-the-go
I set-up very quickly a ownCloud server since I am interested in
moving away from google stuff.
So some problems might simply be a misconfigured ownCloud server...

Sync went ok however, I notice I have to log-out from the ownCloud
webmask... again maybe a ownCloud server setting
The location was not synced yet.

Furthermore, I wonder how to use it finally. I can see the new entries
in the from-calendar.org file. Can I move them to appropiate places in
my working file?
What would happen to a sync in that case?
I guess my working scheme would be like this

Create appointments in my org-file...
sync
on-the-go make changes or add new entries to the calendar
sync
move all from the file from-calendar.org to the right places in the
work org-file.
sync

Is this the intended way of usage?

Also, you said not to add the from-calendar.org file but why not using
the following scheme?
Read all appointments from from-calendar.org and sync them with the
calendar (they should be there already) mark all double entries in the
calendar to be still not in the original work file e.g. the title
could be [WIP] Title....
That would remind people to move them out of from-calendar.org to the
right places. As soon as they removed them from from-calendar.org they
could be marked by using the title  [org-file] Titel. That would help
people to tell them that they will finally find entries later in the
work file. A tag would esp. help if there will be a support for
multiple org-files to sync.
As for multiple org-files the org-agenda mechanism might be helpful.
It already enables to use several org-files to create an agenda. Maybe
syncing from there is easier?!


Thanks again for the great work

Torsten


On 15 July 2012 04:12, David Engster <deng@randomsample.de> wrote:
> I have written a package 'org-caldav' which can sync items to a remote
> calendar server using the CalDAV protocol. The main purpose of this
> package is to make better use of Org in combination with Android-based
> mobile devices (yes, there is mobileOrg, but I have several problems
> with it; that's a topic for another day, though).
>
> I think org-caldav is now "good enough" to allow some testing by
> adventurous people. I put the code up on github here
>
> https://github.com/dengste/org-caldav
>
> I've tested the code with my Owncloud server and also with Google
> Calendar. Please see the Readme on how to set things up. Most
> importantly, please use a new, dedicated calendar for testing this
> package; do *not* use your precious main calendar. Also, while
> org-caldav should only access the calendar you provide through
> `org-caldav-calendar-id', you should have a backup of all your calendar
> data, just in case.
>
> The initial sync can take some time, depending on the number of
> events. Especially the CalDAV interface to Google can be quite slow at
> times. If the initial sync aborts for any reason, just run
> `org-caldav-sync' again. Note that after the initial sync, only new
> events will be transferred, so things should become much snappier.
>
> A few further notes regarding the code:
>
> I am relying mainly on three other packages to do the real heavy
> lifting, which is also why org-caldav is actually pretty small:
>
> - url-dav.el for talking to WebDAV servers (CalDAV is essentially WebDAV
>   with a few extensions). Unfortunately, the current version of url-dav
>   that is shipping with Emacs is broken for practically all CalDAV
>   servers (see bug #11916), which is why I provide a patched version of
>   url-dav.el in the above repository. You have to make sure that this
>   patched version is loaded instead of the one in Emacs proper. I am
>   hoping that this will be fixed in Emacs 24.2.
>
> - org-icalendar.el is doing all the conversion from org to icalendar
>   events which are needed for CalDAV. This has the advantage that you
>   can use all the variables from that package to control how things
>   should be exported. The only exception is the actual org files being
>   searched for time stamps, which is determined through
>   `org-caldav-files'.
>
> - icalendar.el for parsing events from CalDAV. I took some code from
>   that package to generate org items from the calendar events.
>
> The syncing between org and the external calendar is not yet really
> finished. I would like to have proper two-way-syncing so that you can
> freely change events on any side, but this is rather difficult to do and
> I'd first like to get some feedback on the current code. Also, I have
> a hunch that some Org experts around here might have some good ideas on
> how to properly implement this.
>
> Here's what the code can currently do:
>
> - All items in `org-caldav-files' with active time stamps are synced to
>   the calendar.
>
> - If you change an item in one of those files in orgmode, this change
>   will also get synced to the external calendar (to be exact: this will
>   generate a new event and the old one will be deleted).
>
> - If you create a *new* event in the calendar through Android or any
>   other client (like browser), this event will land in the
>   `org-caldav-inbox' file as an org item.
>
> However:
>
> - If you *change* an item directly on the CalDAV server, this will *not*
>   get synced back to Org. What happens exactly depends on the server
>   (whether it renames the event or not). Just try it out. The item might
>   even get deleted after the next sync - you have been warned.
>
> - Likewise, if you *change* and item in `org-caldav-inbox', this will
>   not get synced back to the calendar.
>
> OK, this has gotten way too long already. Please try it out and let me
> know how it goes.
>
> Cheers,
> David
>

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Torsten Wagner writes:
> Sync went ok however, I notice I have to log-out from the ownCloud
> webmask... again maybe a ownCloud server setting

Yes, Owncloud can be a bit finicky here. From my experience, you should
not be fooled by its 4.x version number; it often behaves more like a
0.4.

> The location was not synced yet.

Yes, I will add that eventually.

> Furthermore, I wonder how to use it finally. I can see the new entries
> in the from-calendar.org file. Can I move them to appropiate places in
> my working file?

No.

> What would happen to a sync in that case?

You would get a double entry.

> I guess my working scheme would be like this
>
> Create appointments in my org-file...
> sync
> on-the-go make changes or add new entries to the calendar
> sync
> move all from the file from-calendar.org to the right places in the
> work org-file.
> sync
>
> Is this the intended way of usage?

That's what I'm aiming for. But it requires to do have a proper 2-way
sync, which is not there yet.

> Also, you said not to add the from-calendar.org file but why not using
> the following scheme?
> Read all appointments from from-calendar.org and sync them with the
> calendar (they should be there already) mark all double entries in the
> calendar to be still not in the original work file e.g. the title
> could be [WIP] Title....
> That would remind people to move them out of from-calendar.org to the
> right places. As soon as they removed them from from-calendar.org they
> could be marked by using the title  [org-file] Titel. That would help
> people to tell them that they will finally find entries later in the
> work file. A tag would esp. help if there will be a support for
> multiple org-files to sync.
> As for multiple org-files the org-agenda mechanism might be helpful.
> It already enables to use several org-files to create an agenda. Maybe
> syncing from there is easier?!

These are good suggestions. The problem is, as already mentioned, to
implement proper 2-way sync from CalDAV to Org. For this, the 'etags'
from the CalDAV server have to be saved in the Org items as
PROPERTIES. I guess not all people will be thrilled to have their Org
items polluted by stuff like that, but it's the most straightforward and
reliable way to do it. Another problem is that there's no unique mapping
from iCalendar to Org items.

The good news is that I've now fixed the problems with url-dav and
xml.el in Emacs proper, meaning that org-caldav will work without any
additional files with current Emacs from bzr (which will become 24.2).

-David

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Luis Anaya]

David Engster <deng@randomsample.de> writes:

> Thanks for testing,

I would like to play with this, but I've been busy finalizing the
exporter code. It is in the queue of things to check out.  I was
wondering about it earlier on today. 

I'm following your repo on github. 


-- 
Luis Anaya
papo anaya aroba hot mail punto com
"Do not use 100 words if you can say it in 10" - Yamamoto Tsunetomo

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Bastien]

Hi David,

David Engster <deng@randomsample.de> writes:

> https://github.com/dengste/org-caldav

Thanks for this.  I tested it but can't make it work.

I use Emacs from after your patch to url-dav (>7/26/2012)
and I use this simple configuration:

(setq org-caldav-calendar-id "4ttssrunbsh9km06csbjkb22m4@group.calendar.google.com"
      org-caldav-url "https://www.google.com/calendar/dav"
      org-caldav-files '("~/org/rdv.org")
      org-caldav-inbox "~/org/inbox.org")

(The id is a true one for a public test calendar.)

Then I'm asked for a password.

I tried entering my Google password directly, then my username and my
password, then I gave up.

I guess I miss something obvious.  Thanks for your help!

-- 
 Bastien

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Bastien writes:
> I use Emacs from after your patch to url-dav (>7/26/2012)

Then I guess you patched org-caldav-sync to not test for
`url-dav-patched-version'? Anyway, I've now added support for the
Emacs-bzr version of url-dav.

> and I use this simple configuration:
>
> (setq org-caldav-calendar-id
> "4ttssrunbsh9km06csbjkb22m4@group.calendar.google.com"
>       org-caldav-url "https://www.google.com/calendar/dav"
>       org-caldav-files '("~/org/rdv.org")
>       org-caldav-inbox "~/org/inbox.org")
>
> (The id is a true one for a public test calendar.)
>
> Then I'm asked for a password.

It does not ask for a username? That is very strange. It should first
ask: "Username [for Google CalDAV]:". If it does not do that, maybe you
have that information in your .authinfo?

Please restart Emacs and evaluate

(url-retrieve-synchronously
  "https://www.google.com/calendar/dav/4ttssrunbsh9km06csbjkb22m4@group.calendar.google.com/events/")

Does that one ask you for a username?

-David

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Bastien]

David Engster <deng@randomsample.de> writes:

> Then I guess you patched org-caldav-sync to not test for
> `url-dav-patched-version'? Anyway, I've now added support for the
> Emacs-bzr version of url-dav.

Yes I did.

> It does not ask for a username? 

NO.

> That is very strange. It should first
> ask: "Username [for Google CalDAV]:". If it does not do that, maybe you
> have that information in your .authinfo?

I have this:

machine smtp.gmail.com login bastienguerry@gmail.com password xxxxxx

> Please restart Emacs and evaluate
>
> (url-retrieve-synchronously
>   "https://www.google.com/calendar/dav/4ttssrunbsh9km06csbjkb22m4@group.calendar.google.com/events/")
>
> Does that one ask you for a username?

No, just for a password.  I enter my password and I'm asked for a
password, indefinitely.

This is with GNU Emacs 24.1.50.1 (i686-pc-linux-gnu, GTK+ Version
2.24.10) of 2012-07-26 on myhost.

-- 
 Bastien

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Bastien writes:
> David Engster <deng@randomsample.de> writes:
>> That is very strange. It should first
>> ask: "Username [for Google CalDAV]:". If it does not do that, maybe you
>> have that information in your .authinfo?
>
> I have this:
>
> machine smtp.gmail.com login bastienguerry@gmail.com password xxxxxx

Since we're connecting to google.com this shouldn't matter, obviously,
thus I'm at a loss why it doesn't ask. I currently cannot test anything
reliably because I'm on vacation and there's no 3G around here, so I'm
afraid this has to wait a bit. Unless "someone" beats me to it. ;-)

-David

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Philipp Haselwarter]

Works with my davical server, as it seems!

But when I refile an entry from org-caldav-inbox to org-caldav-files it
gets downloaded into the inbox again the next time I sync. That's not
supposed to happen, is it?

It'd be neat if editing of synced events worked to a higher extent. Do
you plan on working toward this?

Busy times are announcing themselves and this is what I need to get
through them ;)
-- 
Philipp Haselwarter

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Stephen Eglen]

David Engster <deng <at> randomsample.de> writes:

> 
> Bastien writes:
> > David Engster <deng <at> randomsample.de> writes:
> >> That is very strange. It should first
> >> ask: "Username [for Google CalDAV]:". If it does not do that, maybe you
> >> have that information in your .authinfo?
> >
> > I have this:
> >
> > machine smtp.gmail.com login bastienguerry <at> gmail.com password xxxxxx
> 
> Since we're connecting to google.com this shouldn't matter, obviously,
> thus I'm at a loss why it doesn't ask. I currently cannot test anything
> reliably because I'm on vacation and there's no 3G around here, so I'm
> afraid this has to wait a bit. Unless "someone" beats me to it. 
> 
> -David
> 
> 

hi David, Bastien,

did either of you resolve this problem about org-caldav just asking for the
password, and not the username?  I am now trying org-caldav, and am seeing the
same problem.  Happy to debug further if you can tell me which functions to look
into.

Thanks, Stephen

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Stephen Eglen writes:
> David Engster <deng <at> randomsample.de> writes:
>> Bastien writes:
>> > David Engster <deng <at> randomsample.de> writes:
>> >> That is very strange. It should first
>> >> ask: "Username [for Google CalDAV]:". If it does not do that, maybe you
>> >> have that information in your .authinfo?
>> >
>> > I have this:
>> >
>> > machine smtp.gmail.com login bastienguerry <at> gmail.com password xxxxxx
>> 
>> Since we're connecting to google.com this shouldn't matter, obviously,
>> thus I'm at a loss why it doesn't ask. I currently cannot test anything
>> reliably because I'm on vacation and there's no 3G around here, so I'm
>> afraid this has to wait a bit. Unless "someone" beats me to it. 

> did either of you resolve this problem about org-caldav just asking for the
> password, and not the username?  I am now trying org-caldav, and am seeing the
> same problem.  Happy to debug further if you can tell me which functions to look
> into.

I'm at a loss why this happens, and I could not reproduce it with my
account. This problem really has nothing to do with org-caldav, since it
is delegating this part to the url package. As I've already written to
Bastien, please try

 (url-retrieve-synchronously
   "https://www.google.com/calendar/dav/YOUR-CALENDAR-ID@group.calendar.google.com/events/")

If this doesn't ask for your username, then there's your
problem. Depending on the Emacs version you're using, you might want to
set

(setq auth-source-debug t)

to see whether some .netrc file or similar is setting the username for
you (look into the *Messages* buffer).

-David

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Suvayu Ali]

Hi David,

On Mon, Dec 03, 2012 at 08:50:35PM +0100, David Engster wrote:
> 
> I'm at a loss why this happens, and I could not reproduce it with my
> account. This problem really has nothing to do with org-caldav, since it
> is delegating this part to the url package. As I've already written to
> Bastien, please try
> 
>  (url-retrieve-synchronously
>    "https://www.google.com/calendar/dav/YOUR-CALENDAR-ID@group.calendar.google.com/events/")
> 
> If this doesn't ask for your username, then there's your
> problem. Depending on the Emacs version you're using, you might want to
> set
> 
> (setq auth-source-debug t)
> 
> to see whether some .netrc file or similar is setting the username for
> you (look into the *Messages* buffer).
> 

I see the same issue, I get only a password prompt.  This is what I get
on trying your suggestion above.

Contacting host: www.google.com:443
auth-source-search: found 1 results (max 1) matching (:max 1 :host "www.google.com:443" :port "https")
auth-source-search: found 1 CACHED results matching (:max 1 :host "www.google.com:443" :port "https")
#<buffer  *http www.google.com:443*-771021>

I'm not sure from this if my netrc is being used.  My netrc also has the
password for my Google account, if it is being used should I even get a
password prompt?

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Stephen Eglen]

On Tue, Dec 04 2012, Suvayu Ali wrote:

> Hi David,
>
> On Mon, Dec 03, 2012 at 08:50:35PM +0100, David Engster wrote:
>> 
>> I'm at a loss why this happens, and I could not reproduce it with my
>> account. This problem really has nothing to do with org-caldav, since it
>> is delegating this part to the url package. As I've already written to
>> Bastien, please try
>> 
>>  (url-retrieve-synchronously
>>    "https://www.google.com/calendar/dav/YOUR-CALENDAR-ID@group.calendar.google.com/events/")
>> 
>> If this doesn't ask for your username, then there's your
>> problem. Depending on the Emacs version you're using, you might want to
>> set
>> 
>> (setq auth-source-debug t)
>> 
>> to see whether some .netrc file or similar is setting the username for
>> you (look into the *Messages* buffer).
>> 
>
> I see the same issue, I get only a password prompt.  This is what I get
> on trying your suggestion above.
>
> Contacting host: www.google.com:443
> auth-source-search: found 1 results (max 1) matching (:max 1 :host "www.google.com:443" :port "https")
> auth-source-search: found 1 CACHED results matching (:max 1 :host "www.google.com:443" :port "https")
> #<buffer  *http www.google.com:443*-771021>
>
> I'm not sure from this if my netrc is being used.  My netrc also has the
> password for my Google account, if it is being used should I even get a
> password prompt?
>
> Cheers,

Dear all,
I think it is a problem somewhere with my local setup; if I add the
following to .authinfo (chmod 600) then it uses the right username:

Machine www.google.com login my-gmail-login

I'm now about to see how to setup .authinfo.gpg so that I can also
include my password in that file.

As David has pointed out, try the call to url-retrieve-synchronously
first, as that needs to be working before org-caldav works.

Stephen

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Stephen Eglen]

> I think it is a problem somewhere with my local setup; if I add the
> following to .authinfo (chmod 600) then it uses the right username:
>
> Machine www.google.com login my-gmail-login
>
> I'm now about to see how to setup .authinfo.gpg so that I can also
> include my password in that file.

Hmmm. not having any luck here; it works with .authinfo, but not with
.authinfo.gpg in Emacs 24.1.

I have

(require 'auth-source)

but it doesn't seem to help.  Does url.el and friends in Emacs 24.1
use auth-source?

Stephen

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Stephen Eglen writes:
>> I think it is a problem somewhere with my local setup; if I add the
>> following to .authinfo (chmod 600) then it uses the right username:
>>
>> Machine www.google.com login my-gmail-login
>>
>> I'm now about to see how to setup .authinfo.gpg so that I can also
>> include my password in that file.
>
> Hmmm. not having any luck here; it works with .authinfo, but not with
> .authinfo.gpg in Emacs 24.1.

It's bug:

http://lists.gnu.org/archive/html/bug-gnu-emacs/2012-07/msg00674.html

Not sure if that fix is in 24.2, but surely the latest pretest for 24.3
should work (which is pretty stable).

I'm still wondering why it didn't work at the beginning. I mean, even if
you had your Google username set in some .netrc or .authinfo, the
password should still work?

-David

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[David Engster]

Suvayu Ali writes:
> On Mon, Dec 03, 2012 at 08:50:35PM +0100, David Engster wrote:
>> I'm at a loss why this happens, and I could not reproduce it with my
>> account. This problem really has nothing to do with org-caldav, since it
>> is delegating this part to the url package. As I've already written to
>> Bastien, please try
>> 
>>  (url-retrieve-synchronously
>>    "https://www.google.com/calendar/dav/YOUR-CALENDAR-ID@group.calendar.google.com/events/")
>> 
>> If this doesn't ask for your username, then there's your
>> problem. Depending on the Emacs version you're using, you might want to
>> set
>> 
>> (setq auth-source-debug t)
>> 
>> to see whether some .netrc file or similar is setting the username for
>> you (look into the *Messages* buffer).
>> 
>
> I see the same issue, I get only a password prompt.  This is what I get
> on trying your suggestion above.
>
> Contacting host: www.google.com:443
> auth-source-search: found 1 results (max 1) matching (:max 1 :host
> "www.google.com:443" :port "https")
> auth-source-search: found 1 CACHED results matching (:max 1 :host
> "www.google.com:443" :port "https")
> #<buffer  *http www.google.com:443*-771021>
>
> I'm not sure from this if my netrc is being used.

Yes. The mesage "found 1 results" means it found an entry in your .netrc
or .authinfo which matches.

>  My netrc also has the password for my Google account, if it is being
> used should I even get a password prompt?

The auth-source stuff can be a bit tricky. For https connections with
the URL package, you need to use the following

  machine www.google.com:443 port https login USERNAME password PASS

That's no typo: You have to use ":443" as well as "port https". See also

 (info "(auth)Help for users")

-David

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Stephen Eglen]

>
> It's bug:
>
> http://lists.gnu.org/archive/html/bug-gnu-emacs/2012-07/msg00674.html
>
> Not sure if that fix is in 24.2, but surely the latest pretest for 24.3
> should work (which is pretty stable).

Thanks David; indeed, its working fine now in the 24.2.90 pretest that
was recently released.


> I'm still wondering why it didn't work at the beginning. I mean, even if
> you had your Google username set in some .netrc or .authinfo, the
> password should still work?

The problem (I think) was that I had a misformatted .netrc with a
username in it that I thought was just for one machine, but must have
been for all machines!  Removing that .netrc file solved my problem,
thanks.

Stephen

Re: org-caldav: Sync Org with external calendars through CalDAV (Owncloud, Google, ...)

[Stephen Eglen]

Now that I've got org-caldav working (thanks David), here's a simple
helper function so that the files specified in org-agenda-files are
exported.  I hope it works for others too!

(defun org-caldav-set-files-from-org-agenda ()
  "Set `org-caldav-files' from the files specified in 'org-agenda-files'.
Ensure however that `org-caldav-inbox' is not included in the file list."
  (let ((files (org-agenda-files)))
    (when (member (file-truename org-caldav-inbox) files)
      (setq files (delete (file-truename org-caldav-inbox) files)))
    (setq org-caldav-files files)))

(org-caldav-set-files-from-org-agenda)

Stephen