Inconsistent handling of multi-line properties

Inconsistent handling of multi-line properties

[Hanno Perrey]

Hej,

I have noticed that properties that stretch over multiple lines using 
the :value+: syntax are ignored by org-element-property and therefore 
also by e.g. org-export-get-node-property when exporting to ics via 
ox-icalendar.el (see example below). I was wondering now whether this is 
intentional and to be expected or a bug?

* heading with multi-line property
:PROPERTIES:
:LOCATION: Someplace
:LOCATION+: Some Street 5
:LOCATION+: 12345 Small Town
:END:

#+begin_src emacs-lisp
(org-entry-get (point) "LOCATION")
#+end_src

#+RESULTS:
: Someplace Some Street 5 12345 Small Town


#+begin_src emacs-lisp
(save-excursion
   (goto-char (point-min))
   (org-element-property :LOCATION (org-element-at-point)))
#+end_src

#+RESULTS:
: Someplace

#+begin_src emacs-lisp
(save-excursion
   (goto-char (point-min))
   (org-element-property :LOCATION+ (org-element-at-point)))
#+end_src

#+RESULTS:
: 12345 Small Town


Thanks and cheers,

Hanno

Re: Inconsistent handling of multi-line properties

[Ihor Radchenko]

Hanno Perrey <hanno@hoowl.se> writes:

> Hej,
>
> I have noticed that properties that stretch over multiple lines using 
> the :value+: syntax are ignored by org-element-property and therefore 
> also by e.g. org-export-get-node-property when exporting to ics via 
> ox-icalendar.el (see example below). I was wondering now whether this is 
> intentional and to be expected or a bug?

I proposed to change PROPERTY/PROPERTY+ handling in org-element (among
other things):
https://orgmode.org/list/87bl4p6n0m.fsf@localhost

However, that code still need some testing.

Best,
Ihor

Re: Inconsistent handling of multi-line properties

[Greg Sullivan]

[-- Attachment #1: Type: text/plain, Size: 1173 bytes --]

I would really like the ability to use the ":myProperty+: " to create
multi-line properties for the org-export-taskjuggler exporter.
However, the mentioned patch is quite large and mostly focused on headline
caching, as far as I can tell.
@Ihor or @Hanno, Is there a patch just to get the :myProperty+: syntax to
work properly with org-element-property?
Thanks much.
-- Greg

--
Greg Sullivan   email: gregs@sulliwood.org   cell: 617-417-4746
70 Pigeon Hill Street, Rockport, MA 01966


On Sat, Oct 2, 2021 at 11:19 AM Ihor Radchenko <yantar92@gmail.com> wrote:

> Hanno Perrey <hanno@hoowl.se> writes:
>
> > Hej,
> >
> > I have noticed that properties that stretch over multiple lines using
> > the :value+: syntax are ignored by org-element-property and therefore
> > also by e.g. org-export-get-node-property when exporting to ics via
> > ox-icalendar.el (see example below). I was wondering now whether this is
> > intentional and to be expected or a bug?
>
> I proposed to change PROPERTY/PROPERTY+ handling in org-element (among
> other things):
> https://orgmode.org/list/87bl4p6n0m.fsf@localhost
>
> However, that code still need some testing.
>
> Best,
> Ihor
>
>

[-- Attachment #2: Type: text/html, Size: 1969 bytes --]

Re: Inconsistent handling of multi-line properties

[Kaushal Modi]

On Sat, Oct 2, 2021 at 11:03 AM Hanno Perrey <hanno@hoowl.se> wrote:
>
> Hej,
>
> I have noticed that properties that stretch over multiple lines using
> the :value+: syntax are ignored by org-element-property and therefore
> also by e.g. org-export-get-node-property when exporting to ics via
> ox-icalendar.el (see example below). I was wondering now whether this is
> intentional and to be expected or a bug?

I use the :value+: syntax for the subtree properties regularly. For
exports, though, you need to prefix the properties with EXPORT_.

See the (org) Export Settings node in Org manual.

> When exporting sub-trees, special node properties can override the
above keywords.  These properties have an ‘EXPORT_’ prefix.  For
example, ‘DATE’ becomes, ‘EXPORT_DATE’ when used for a specific
sub-tree.  Except for ‘SETUPFILE’, all other keywords listed above have
an ‘EXPORT_’ equivalent.

Here's one of the pathogenic test cases of ox-hugo:

=====
** Custom front matter in multiple lines
:PROPERTIES:
:EXPORT_FILE_NAME: custom-front-matter-multiple-lines
:EXPORT_DATE: 2017-07-24
:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :foo bar
:EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :baz zoo
:EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :alpha 1
:EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :beta "two words"
:EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :gamma 10
:EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :empty_string ""
:END:
=====

All the HUGO_CUSTOM_FRONT_MATTER properties get collected as expected.

Here's another example:

=====
#+author:
#+options: toc:nil
* Heading
:PROPERTIES:
:EXPORT_AUTHOR: abc def
:EXPORT_AUTHOR+: ghi jkl
:EXPORT_AUTHOR+: kmo pqr
:END:
=====

C-c C-e C-s t A exports to:

=====
                       _________________________

                                HEADING

                        abc def ghi jkl kmo pqr
                       _________________________


=====

Re: Inconsistent handling of multi-line properties

[Kaushal Modi]

On Mon, Feb 28, 2022 at 3:34 PM Kaushal Modi <kaushal.modi@gmail.com> wrote:
>
> On Sat, Oct 2, 2021 at 11:03 AM Hanno Perrey <hanno@hoowl.se> wrote:
> >
> > Hej,
> >
> > I have noticed that properties that stretch over multiple lines using
> > the :value+: syntax are ignored by org-element-property and therefore
> > also by e.g. org-export-get-node-property when exporting to ics via
> > ox-icalendar.el (see example below). I was wondering now whether this is
> > intentional and to be expected or a bug?
>
> I use the :value+: syntax for the subtree properties regularly. For
> exports, though, you need to prefix the properties with EXPORT_.
>
> See the (org) Export Settings node in Org manual.
>
> > When exporting sub-trees, special node properties can override the
> above keywords.  These properties have an ‘EXPORT_’ prefix.  For
> example, ‘DATE’ becomes, ‘EXPORT_DATE’ when used for a specific
> sub-tree.  Except for ‘SETUPFILE’, all other keywords listed above have
> an ‘EXPORT_’ equivalent.

Sorry, ignore my email noise. I had never tried ox-icalendar before.
But I just tried it with your test snippet and it doesn't work as
expected even with the EXPORT_ prefix.

Also, I don't see any location info get exported to the .ics file
(with or without the EXPORT_ prefix). So I don't know what I should
expect to see in the .ics file.

Re: Inconsistent handling of multi-line properties

[Greg Sullivan]

[-- Attachment #1: Type: text/plain, Size: 4073 bytes --]

Thank you very much for that suggestion.
Unfortunately, that does not seem to work for org-export-taskjuggler.
An exported taskjuggler file has extremely simple syntax. For a report, for
example, there is the header, "textreport id name {"
then a series of "attribute_name", "value" pairs such as "formats html".
Some attributes take multi-line string values, which are designated
"-8<- \n line1 \n line2 \n ->8-" (but with actual newlines rather than
"\n"s).
The taskjuggler export uses a very simple approach: when an org headline is
tagged as a report, it cycles through the properties, and for each
property, simply emits the property name followed by the property value (
org-taskjuggler-valid-report-attributes contains names such as "formats'
and "center"):

   (org-taskjuggler--indent-string
>     (org-taskjuggler--build-attributes
>      report org-taskjuggler-valid-report-attributes))
>
where

> (defun org-taskjuggler--build-attributes (item attributes)
>   (mapconcat
>    (lambda (attribute)
>      (let ((value (org-element-property
>                    (intern (upcase (format ":%s" attribute)))
>                    item)))
>        (and value (format "%s %s\n" attribute value))))
>    (remq nil attributes) ""))
>


For an org source such as

** Reports
> :PROPERTIES:
> :REPORT_KIND: textreport
> :formats:  html
> :center: -8<-
> :center+:    [#Plan Plan] | [#Resource_Allocation Resource Allocation]
> :center+:    ----
> :center+:    === Plan ===
> :center+:    <[report id="plan"]>
> :center+:    ----
> :center+:    === Resource Allocation ===
> :center+:    <[report id="resourceGraph"]>
> :center+: ->8-
> :END:
>

unfortunately, (org-element-property :center) returns only "-8<-", whereas
I want it to return a 9-line string.

-- Greg

--
Greg Sullivan   email: gregs@sulliwood.org   cell: 617-417-4746
70 Pigeon Hill Street, Rockport, MA 01966


On Mon, Feb 28, 2022 at 3:37 PM Kaushal Modi <kaushal.modi@gmail.com> wrote:

> On Sat, Oct 2, 2021 at 11:03 AM Hanno Perrey <hanno@hoowl.se> wrote:
> >
> > Hej,
> >
> > I have noticed that properties that stretch over multiple lines using
> > the :value+: syntax are ignored by org-element-property and therefore
> > also by e.g. org-export-get-node-property when exporting to ics via
> > ox-icalendar.el (see example below). I was wondering now whether this is
> > intentional and to be expected or a bug?
>
> I use the :value+: syntax for the subtree properties regularly. For
> exports, though, you need to prefix the properties with EXPORT_.
>
> See the (org) Export Settings node in Org manual.
>
> > When exporting sub-trees, special node properties can override the
> above keywords.  These properties have an ‘EXPORT_’ prefix.  For
> example, ‘DATE’ becomes, ‘EXPORT_DATE’ when used for a specific
> sub-tree.  Except for ‘SETUPFILE’, all other keywords listed above have
> an ‘EXPORT_’ equivalent.
>
> Here's one of the pathogenic test cases of ox-hugo:
>
> =====
> ** Custom front matter in multiple lines
> :PROPERTIES:
> :EXPORT_FILE_NAME: custom-front-matter-multiple-lines
> :EXPORT_DATE: 2017-07-24
> :EXPORT_HUGO_CUSTOM_FRONT_MATTER: :foo bar
> :EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :baz zoo
> :EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :alpha 1
> :EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :beta "two words"
> :EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :gamma 10
> :EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :empty_string ""
> :END:
> =====
>
> All the HUGO_CUSTOM_FRONT_MATTER properties get collected as expected.
>
> Here's another example:
>
> =====
> #+author:
> #+options: toc:nil
> * Heading
> :PROPERTIES:
> :EXPORT_AUTHOR: abc def
> :EXPORT_AUTHOR+: ghi jkl
> :EXPORT_AUTHOR+: kmo pqr
> :END:
> =====
>
> C-c C-e C-s t A exports to:
>
> =====
>                        _________________________
>
>                                 HEADING
>
>                         abc def ghi jkl kmo pqr
>                        _________________________
>
>
> =====
>
>

[-- Attachment #2: Type: text/html, Size: 5877 bytes --]

Re: Inconsistent handling of multi-line properties

[Ihor Radchenko]

Greg Sullivan <gregs@sulliwood.org> writes:

> I would really like the ability to use the ":myProperty+: " to create
> multi-line properties for the org-export-taskjuggler exporter.
> However, the mentioned patch is quite large and mostly focused on headline
> caching, as far as I can tell.
> @Ihor or @Hanno, Is there a patch just to get the :myProperty+: syntax to
> work properly with org-element-property?

That patch has been merged. However, the part about merging
:myProperty+: into :myProperty: did not work as well as I thought
initially. Now, all the instances of :myProperty+: do get merged and can
be accessed via (org-element-property :MYPROPERTY+ element). However,
they are not merged into :myProperty: because it would interfere with
property inheritance.

Merging everything into :MYPROPERTY is possible, but it might break
third-party packages that do not expect element properties to be lists
(currently, most export backends assume element properties to be strings).
I am reluctant to work on this unless Nicolas displays a clear support of
such change.

An alternative could be modifying org-element-property or
org-export-get-node-property to support multiline properties.

Meanwhile, export backends might be changed to look into :PROPERTY+ in
addition to :PROPERTY. For org-export-taskjuggler, you may implement
this feature yourself. Note that org-export-taskjuggler is not a part of
Org core. It is in org-contrib and currently not maintained.

Best,
Ihor

Re: Inconsistent handling of multi-line properties

[Ihor Radchenko]

Hanno Perrey <hanno@hoowl.se> writes:

> Hej,
>
> I have noticed that properties that stretch over multiple lines using 
> the :value+: syntax are ignored by org-element-property and therefore 
> also by e.g. org-export-get-node-property when exporting to ics via 
> ox-icalendar.el (see example below). I was wondering now whether this is 
> intentional and to be expected or a bug?
>
> * heading with multi-line property
> :PROPERTIES:
> :LOCATION: Someplace
> :LOCATION+: Some Street 5
> :LOCATION+: 12345 Small Town
> :END:

Confirmed.

I am not sure if this should be fixed on org-export-get-node-property
level. Icalendar may want to concatenate the multi-line property
specially. The usual Org approach is merging such properties into a
single line.

I can see multiple solutions:
1. Change Org's behaviour globally and make org-element-property return
   a list for multi-line properties. This will likely break things, but
   I would be in favour, unless Nicolas disagrees.
2. Change org-export-get-node-property to behave like org-entry-get and
   concatenate multi-line properties into a single line
3. Change ox-icalendar to consider :LOCATION+ properties and merge them
   during export.

Best,
Ihor
   

Re: Inconsistent handling of multi-line properties

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 657 bytes --]

Ihor Radchenko <yantar92@gmail.com> writes:

>> I have noticed that properties that stretch over multiple lines using 
>> the :value+: syntax are ignored by org-element-property and therefore 
>> also by e.g. org-export-get-node-property when exporting to ics via 
>> ox-icalendar.el (see example below). I was wondering now whether this is 
>> intentional and to be expected or a bug?

> 3. Change ox-icalendar to consider :LOCATION+ properties and merge them
>    during export.

I went with this approach.
See the attached tentative patch.

CCing Jack - you expressed interest in ox-icalendar in the past.

P.S. We need a maintainer for ox-icalendar ;)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ox-icalendar-Add-support-for-multi-line-SUMMARY-LOCA.patch --]
[-- Type: text/x-patch, Size: 3730 bytes --]

From 84fb56dcbccc2c99be4fed172184c25cfe2a3395 Mon Sep 17 00:00:00 2001
Message-ID: <84fb56dcbccc2c99be4fed172184c25cfe2a3395.1705922024.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Mon, 22 Jan 2024 12:12:16 +0100
Subject: [PATCH] ox-icalendar: Add support for multi-line SUMMARY, LOCATION,
 and DESCRIPTION

* lisp/ox-icalendar.el (org-icalendar-entry): Use `org-entry-get' to
account for both PROP and PROP+ in SUMMARY, LOCATION, and DESCRIPTION
properties.  Use newline as accumulated value separator.
* etc/ORG-NEWS (iCalendar export now supports multiline =SUMMARY=,
=LOCATION=, and =DESCRIPTION= properties): Announce the breaking
change.

Reported-by: Hanno Perrey <hanno@hoowl.se>
Link: https://orgmode.org/list/87o821dv7o.fsf@localhost
---
 etc/ORG-NEWS         | 26 ++++++++++++++++++++++++++
 lisp/ox-icalendar.el | 19 +++++++++++--------
 2 files changed, 37 insertions(+), 8 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 847ddf614..dc6886318 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -13,6 +13,32 @@ Please send Org bug reports to mailto:emacs-orgmode@gnu.org.
 
 * Version 9.7 (not released yet)
 ** Important announcements and breaking changes
+*** iCalendar export now supports multiline =SUMMARY=, =LOCATION=, and =DESCRIPTION= properties
+
+Previously, it was not possible to specify multi-line location,
+summary, or description when exporting to iCalendar.
+
+In the following example, =LOCATION= was exported as "Someplace",
+ignoring the other lines.
+
+#+begin_src org
+,* heading with multi-line property
+:PROPERTIES:
+:LOCATION: Someplace
+:LOCATION+: Some Street 5
+:LOCATION+: 12345 Small Town
+:END:
+#+end_src
+
+Now, =SUMMARY+=, =LOCATION+=, and =DESCRIPTION+= properties can be
+used to create multi-line values.
+
+In the above example, =LOCATION= is now exported as
+
+: Someplace
+: Some Street 5
+: 12345 Small Town
+
 *** ~org-agenda-search-headline-for-time~ now ignores all the timestamp in headings
 
 Previously, ~org-agenda-search-headline-for-time~ made Org agenda
diff --git a/lisp/ox-icalendar.el b/lisp/ox-icalendar.el
index 3dd2c88d8..f6513563d 100644
--- a/lisp/ox-icalendar.el
+++ b/lisp/ox-icalendar.el
@@ -643,13 +643,15 @@ (defun org-icalendar-entry (entry contents info)
        (let ((todo-type (org-element-property :todo-type entry))
 	     (uid (or (org-element-property :ID entry) (org-id-new)))
 	     (summary (org-icalendar-cleanup-string
-		       (or (org-element-property :SUMMARY entry)
-			   (org-export-data
-			    (org-element-property :title entry) info))))
-	     (loc (org-icalendar-cleanup-string
-		   (org-export-get-node-property
-		    :LOCATION entry
-		    (org-property-inherit-p "LOCATION"))))
+		       (or
+                        (let ((org-property-separators '(("SUMMARY" . "\n"))))
+                          (org-entry-get entry "SUMMARY"))
+			(org-export-data
+			 (org-element-property :title entry) info))))
+	     (loc
+              (let ((org-property-separators '(("LOCATION" . "\n"))))
+                (org-icalendar-cleanup-string
+                 (org-entry-get entry "LOCATION" 'selective))))
 	     (class (org-icalendar-cleanup-string
 		     (org-export-get-node-property
 		      :CLASS entry
@@ -658,7 +660,8 @@ (defun org-icalendar-entry (entry contents info)
 	     ;; (headline) or contents (inlinetask).
 	     (desc
 	      (org-icalendar-cleanup-string
-	       (or (org-element-property :DESCRIPTION entry)
+	       (or (let ((org-property-separators '(("DESCRIPTION" . "\n"))))
+                     (org-entry-get entry "DESCRIPTION"))
 		   (let ((contents (org-export-data inside info)))
 		     (cond
 		      ((not (org-string-nw-p contents)) nil)
-- 
2.43.0


[-- Attachment #3: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Inconsistent handling of multi-line properties

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>>> I have noticed that properties that stretch over multiple lines using 
>>> the :value+: syntax are ignored by org-element-property and therefore 
>>> also by e.g. org-export-get-node-property when exporting to ics via 
>>> ox-icalendar.el (see example below). I was wondering now whether this is 
>>> intentional and to be expected or a bug?
>
>> 3. Change ox-icalendar to consider :LOCATION+ properties and merge them
>>    during export.
>
> I went with this approach.
> See the attached tentative patch.

Thanks Ihor -- this is a nice improvement for ox-icalendar.

My only suggestion would be to add this information and your small
location example to the manual.

> CCing Jack - you expressed interest in ox-icalendar in the past.
>
> P.S. We need a maintainer for ox-icalendar ;)

Sure, I'm happy to help with this :)

Please forward/CC any issues you'd like me to look at.  I'll setup an
email filter to try and catch iCalendar related subjects, but sometimes
I miss things on the list.

Re: Inconsistent handling of multi-line properties

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

>>> 3. Change ox-icalendar to consider :LOCATION+ properties and merge them
>>>    during export.
>>
>> I went with this approach.
>> See the attached tentative patch.
>
> Thanks Ihor -- this is a nice improvement for ox-icalendar.
>
> My only suggestion would be to add this information and your small
> location example to the manual.

Done.
Fixed, on main. 
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=8ec89c53c

>> CCing Jack - you expressed interest in ox-icalendar in the past.
>>
>> P.S. We need a maintainer for ox-icalendar ;)
>
> Sure, I'm happy to help with this :)

Thanks!
You can replace Maintainer: line in ox-icalendar.el with your name.
(Nicolas informed me that he no longer maintains Org mode libraries)

> Please forward/CC any issues you'd like me to look at.  I'll setup an
> email filter to try and catch iCalendar related subjects, but sometimes
> I miss things on the list.

Sure.
I checked my records and found one outstanding issue related to
ox-icalendar -
https://list.orgmode.org/orgmode/20211230225919.1a660666@hsu-hh.de/

The patch proposed by Detlef still has outstanding issues to address,
but the author does not have free time to work on this since the
beginning of last year (I did several follow-ups off list; latest in
September).

At this point, we should either cancel that patch or amend it ourselves
and apply.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Inconsistent handling of multi-line properties

[Jack Kamm]

Ihor Radchenko <yantar92@posteo.net> writes:

> Thanks!
> You can replace Maintainer: line in ox-icalendar.el with your name.

Done.

> Sure.
> I checked my records and found one outstanding issue related to
> ox-icalendar -
> https://list.orgmode.org/orgmode/20211230225919.1a660666@hsu-hh.de/
>
> The patch proposed by Detlef still has outstanding issues to address,
> but the author does not have free time to work on this since the
> beginning of last year (I did several follow-ups off list; latest in
> September).
>
> At this point, we should either cancel that patch or amend it ourselves
> and apply.

Thanks -- I will amend the patch, and send an update to that thread
before applying.