Bug report for ox-icalendar: newlines should be CRLF

Bug report for ox-icalendar: newlines should be CRLF

[Stephen J. Eglen]

[-- Attachment #1: Type: text/plain, Size: 1549 bytes --]

Hello,

If I have a test file

----------------------------------------------------------------------

* test 1

<2023-01-05 Thu 19:00-19:20>

----------------------------------------------------------------------


and then convert to an .ics file using C-c C-e c f

I get the attached test.ics file, but copied here too:

BEGIN:VCALENDAR
VERSION:2.0
X-WR-CALNAME:test
PRODID//Stephen J. Eglen//Emacs with Org mode//EN
X-WR-TIMEZONE:GMT
X-WR-CALDESC:
CALSCALE:GREGORIAN
BEGIN:VEVENT ***
DTSTAMP:20230105T221431Z ***
UID:TS1-81DE7E9C-0385-49CE-8528-C2FD51372CB4 ***
DTSTART:20230105T190000Z ***
DTEND:20230105T192000Z ***
SUMMARY:test 1 ***
DESCRIPTION:<2023-01-05 Thu 19:00>–<2023-01-05 Thu 19:20> ***
CATEGORIES:test ***
END:VEVENT
END:VCALENDAR

The lines from BEGIN:VEVENT to CATEGORIES:test inclusive have CR (^M) as well
as LF (^J) as newline [in the text above, I've converted the ^M to ***
to see them eaer.  Is that intended?  Uploading the file to
https://icalendar.org/validator.html gives me the error

> Lines not delimited by CRLF sequence near line # 1
> Reference: RFC 5545 3.1. Content Lines

Nicolas (cc'ed) has confirmed that it seems to be a problem.  Before I
work slowly on a suggested patch, does anyone have an idea on the best
way forward?  Is it possible to encode the file as "DOS" ,using
something like set-buffer-file-coding-system?  SOme lines do however
have \r and \n see e.g. org-icalendar-fold-string which inserts both \r
and \n




Best wishes, Stephen



[-- Attachment #2: test.ics --]
[-- Type: application/octet-stream, Size: 404 bytes --]

BEGIN:VCALENDAR
VERSION:2.0
X-WR-CALNAME:test
PRODID:-//Stephen J. Eglen//Emacs with Org mode//EN
X-WR-TIMEZONE:GMT
X-WR-CALDESC:
CALSCALE:GREGORIAN
BEGIN:VEVENT
DTSTAMP:20230105T221431Z
UID:TS1-81DE7E9C-0385-49CE-8528-C2FD51372CB4
DTSTART:20230105T190000Z
DTEND:20230105T192000Z
SUMMARY:test 1
DESCRIPTION:<2023-01-05 Thu 19:00>–<2023-01-05 Thu 19:20>
CATEGORIES:test
END:VEVENT
END:VCALENDAR

Re: Bug report for ox-icalendar: newlines should be CRLF

[Detlef Steuer]

Hello,

just to confirm. The exporter is inconsistent in adding
which kind of EOL sequence.

Detlef

Am Fri, 06 Jan 2023 08:30:41 +0000
schrieb "Stephen J. Eglen" <stephen@eglen.org.uk>:

> Hello,
> 
> If I have a test file
> 
> ----------------------------------------------------------------------
> 
> * test 1
> 
> <2023-01-05 Thu 19:00-19:20>
> 
> ----------------------------------------------------------------------
> 
> 
> and then convert to an .ics file using C-c C-e c f
> 
> I get the attached test.ics file, but copied here too:
> 
> BEGIN:VCALENDAR
> VERSION:2.0
> X-WR-CALNAME:test
> PRODID//Stephen J. Eglen//Emacs with Org mode//EN
> X-WR-TIMEZONE:GMT
> X-WR-CALDESC:
> CALSCALE:GREGORIAN
> BEGIN:VEVENT ***
> DTSTAMP:20230105T221431Z ***
> UID:TS1-81DE7E9C-0385-49CE-8528-C2FD51372CB4 ***
> DTSTART:20230105T190000Z ***
> DTEND:20230105T192000Z ***
> SUMMARY:test 1 ***
> DESCRIPTION:<2023-01-05 Thu 19:00>–<2023-01-05 Thu 19:20> ***
> CATEGORIES:test ***
> END:VEVENT
> END:VCALENDAR
> 
> The lines from BEGIN:VEVENT to CATEGORIES:test inclusive have CR (^M)
> as well as LF (^J) as newline [in the text above, I've converted the
> ^M to *** to see them eaer.  Is that intended?  Uploading the file to
> https://icalendar.org/validator.html gives me the error
> 
> > Lines not delimited by CRLF sequence near line # 1
> > Reference: RFC 5545 3.1. Content Lines  
> 
> Nicolas (cc'ed) has confirmed that it seems to be a problem.  Before I
> work slowly on a suggested patch, does anyone have an idea on the best
> way forward?  Is it possible to encode the file as "DOS" ,using
> something like set-buffer-file-coding-system?  SOme lines do however
> have \r and \n see e.g. org-icalendar-fold-string which inserts both
> \r and \n
> 
> 
> 
> 
> Best wishes, Stephen
> 
> 

Re: Bug report for ox-icalendar: newlines should be CRLF

[Ihor Radchenko]

"Stephen J. Eglen" <stephen@eglen.org.uk> writes:

> The lines from BEGIN:VEVENT to CATEGORIES:test inclusive have CR (^M) as well
> as LF (^J) as newline [in the text above, I've converted the ^M to ***
> to see them eaer.  Is that intended?  Uploading the file to
> https://icalendar.org/validator.html gives me the error
>
>> Lines not delimited by CRLF sequence near line # 1
>> Reference: RFC 5545 3.1. Content Lines
>
> Nicolas (cc'ed) has confirmed that it seems to be a problem.  Before I
> work slowly on a suggested patch, does anyone have an idea on the best
> way forward?  Is it possible to encode the file as "DOS" ,using
> something like set-buffer-file-coding-system?  SOme lines do however
> have \r and \n see e.g. org-icalendar-fold-string which inserts both \r
> and \n

Sorry for the late reply.

Do I understand correctly that all the ical lines must use \r\n? [1]

If so, we can simply wrap the export output into
`org-icalendar-fold-string' in `org-icalendar-template'. Does it make
sense?

[1] https://icalendar.org/iCalendar-RFC-5545/3-1-content-lines.html


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Bug report for ox-icalendar: newlines should be CRLF

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> Sorry for the late reply.
>
> Do I understand correctly that all the ical lines must use \r\n? [1]

Following up here that I have pushed a fix for the EOL issue in
ox-icalendar:

https://list.orgmode.org/87355ikzwk.fsf@localhost/T/#m180c100587d3d88ab5787942271a546b51891996

Re: Bug report for ox-icalendar: newlines should be CRLF

[Stephen J. Eglen]

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> Sorry for the late reply.
>>
>> Do I understand correctly that all the ical lines must use \r\n? [1]
>
> Following up here that I have pushed a fix for the EOL issue in
> ox-icalendar:
>
> https://list.orgmode.org/87355ikzwk.fsf@localhost/T/#m180c100587d3d88ab5787942271a546b51891996

Thanks for copying me in on this.

There is a related issue about EOLs, not just \r\n -- each line should
be a maximum of 75 characters; this is handled by 
org-icalendar-fold-string 

I'll check out the revised code to see if this is still an issue.

Stephen

Re: Bug report for ox-icalendar: newlines should be CRLF

[Ihor Radchenko]

"Stephen J. Eglen" <stephen@eglen.org.uk> writes:

>> https://list.orgmode.org/87355ikzwk.fsf@localhost/T/#m180c100587d3d88ab5787942271a546b51891996
>
> Thanks for copying me in on this.
>
> There is a related issue about EOLs, not just \r\n -- each line should
> be a maximum of 75 characters; this is handled by 
> org-icalendar-fold-string 

May you please provide a link to the iCalendar spec document section
describing this requirement?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Bug report for ox-icalendar: newlines should be CRLF

[tomas]

[-- Attachment #1: Type: text/plain, Size: 742 bytes --]

On Fri, Apr 14, 2023 at 10:13:48AM +0000, Ihor Radchenko wrote:
> "Stephen J. Eglen" <stephen@eglen.org.uk> writes:
> 
> >> https://list.orgmode.org/87355ikzwk.fsf@localhost/T/#m180c100587d3d88ab5787942271a546b51891996
> >
> > Thanks for copying me in on this.
> >
> > There is a related issue about EOLs, not just \r\n -- each line should
> > be a maximum of 75 characters; this is handled by 
> > org-icalendar-fold-string 
> 
> May you please provide a link to the iCalendar spec document section
> describing this requirement?

It's in rfc5545 [1], referenced to from rfc7986 [2].

Cheers
[1] https://datatracker.ietf.org/doc/html/rfc5545#section-3.1
[2] https://datatracker.ietf.org/doc/html/rfc7986#section-1
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Bug report for ox-icalendar: newlines should be CRLF

[Jack Kamm]

<tomas@tuxteam.de> writes:

>> > There is a related issue about EOLs, not just \r\n -- each line should
>> > be a maximum of 75 characters; this is handled by 
>> > org-icalendar-fold-string 
>> 
>> May you please provide a link to the iCalendar spec document section
>> describing this requirement?
>
> It's in rfc5545 [1], referenced to from rfc7986 [2].

Since VEVENT and VTODO are wrapped in `org-icalendar-fold-string', I
think the only place this error may occur is in the preamble created by
`org-icalendar--vcalendar', e.g. in PRODID, X-WR-CALDESC, etc.

The fix is to move the call to `org-icalendar-fold-string' out of
org-icalendar--vevent/vtodo, and instead put it in
`org-icalendar--vcalendar'. Then the line folding will apply to the
whole VCALENDAR instead of just the VTODO and VEVENT. An earlier version
of my CRLF patch actually did something like this.

I can push such a fix over the weekend. Let me know if you'd rather I
send the patch for review here first.

Re: Bug report for ox-icalendar: newlines should be CRLF

[Jack Kamm]

Jack Kamm <jackkamm@gmail.com> writes:

> <tomas@tuxteam.de> writes:
>
>>> > There is a related issue about EOLs, not just \r\n -- each line should
>>> > be a maximum of 75 characters; this is handled by 
>>> > org-icalendar-fold-string 
>>> 
>>> May you please provide a link to the iCalendar spec document section
>>> describing this requirement?
>>
>> It's in rfc5545 [1], referenced to from rfc7986 [2].
>
> Since VEVENT and VTODO are wrapped in `org-icalendar-fold-string', I
> think the only place this error may occur is in the preamble created by
> `org-icalendar--vcalendar', e.g. in PRODID, X-WR-CALDESC, etc.
>
> The fix is to move the call to `org-icalendar-fold-string' out of
> org-icalendar--vevent/vtodo, and instead put it in
> `org-icalendar--vcalendar'. Then the line folding will apply to the
> whole VCALENDAR instead of just the VTODO and VEVENT. An earlier version
> of my CRLF patch actually did something like this.
>
> I can push such a fix over the weekend. Let me know if you'd rather I
> send the patch for review here first.

I've pushed this change now:
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f4446ce795c924a1e115e360d3674f6ad89be845

Re: Bug report for ox-icalendar: newlines should be CRLF

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

> I've pushed this change now:
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f4446ce795c924a1e115e360d3674f6ad89be845

Fixed.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Bug report for ox-icalendar: newlines should be CRLF

[Stephen J. Eglen]

> It's in rfc5545 [1], referenced to from rfc7986 [2].

yes, that's it.   There is also this validator that can check files
against the spec:  https://icalendar.org/validator.html

Stpehen