From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: hokomo Newsgroups: gmane.emacs.bugs Subject: bug#56241: [PATCH] icalendar doesn't correctly process arbitrary diary sexp entries Date: Mon, 27 Jun 2022 11:42:34 +0200 Message-ID: References: <21bff87f-b8dc-c7bc-41ab-97c7077a0535@airmail.cc> <87edza8rbu.fsf@gnus.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------x92lbi8WXdL97kq43XFKoufV" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="10907"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Cc: 56241@debbugs.gnu.org To: Lars Ingebrigtsen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Jun 27 14:32:57 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o5nvE-0002an-Kg for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 27 Jun 2022 14:32:56 +0200 Original-Received: from localhost ([::1]:51338 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o5nvD-0005Jf-I3 for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 27 Jun 2022 08:32:55 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:47150) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o5nuO-0005GT-Mj for bug-gnu-emacs@gnu.org; Mon, 27 Jun 2022 08:32:05 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:56343) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o5nuO-000591-Dd for bug-gnu-emacs@gnu.org; Mon, 27 Jun 2022 08:32:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1o5nuO-0007Ue-B0 for bug-gnu-emacs@gnu.org; Mon, 27 Jun 2022 08:32:04 -0400 X-Loop: help-debbugs@gnu.org Resent-From: hokomo Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 27 Jun 2022 12:32:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 56241 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 56241-submit@debbugs.gnu.org id=B56241.165633309227098 (code B ref 56241); Mon, 27 Jun 2022 12:32:04 +0000 Original-Received: (at 56241) by debbugs.gnu.org; 27 Jun 2022 12:31:32 +0000 Original-Received: from localhost ([127.0.0.1]:50223 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o5ntn-0006zN-Ox for submit@debbugs.gnu.org; Mon, 27 Jun 2022 08:31:31 -0400 Original-Received: from [37.120.193.124] (port=34734 helo=mail.cock.li) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o5lGU-00083E-Q3 for 56241@debbugs.gnu.org; Mon, 27 Jun 2022 05:42:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=airmail.cc; s=mail; t=1656322955; bh=VStskTBxz/jmr8YJe6Jxh63ZUdgk4DqmVbDVgrRDyZU=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=Jjzcp5RQb+AMigMzRzzXGO0r8BM1fodFtsCAQZ9jTMeqOoKuRuJPbrsfZV8aJXfe5 9I6uKwkEwvNMpeJio/Te1YhsVBfDabVLJGASoeZGQEwG5biiI8/4jAsbdnemdNiMcW qoIUQopuRpkO+8AqyOZ1MUA79A6JMQCpRwUai7b2QAiigXTnHAsVd0+kNLU6AY0Cex jCAzYD14VDF93Ls2+X2RglIAoYx9HsledGdtvt4tQEu5f6rMRhQCrkRSzhQMPD8EH8 FvhWPk0Q49cgXL2itwD/NFZ/WQPC0P0pC2Voy+cMFVCNCQRo3YWLzayZ+JtwSnLJHx GslOaKm8lpaHg== Content-Language: en-US In-Reply-To: <87edza8rbu.fsf@gnus.org> X-Mailman-Approved-At: Mon, 27 Jun 2022 08:31:26 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:235444 Archived-At: This is a multi-part message in MIME format. --------------x92lbi8WXdL97kq43XFKoufV Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit >> `icalendar-export-sexp-enumeration-days' is set to 366 to guarantee >> that the sexp event occurs at least once. It looks like there's a >> different bug (?) where, even if an entry is recognized as an >> arbitrary diary sexp, if it doesn't produce any events, the converter >> will go ahead with trying to interpret it in a different way and >> eventually fail. E.g., lowering the enumeration days to 0 gives: >> >> "Error in line 0 -- (error Could not parse date): ‘%%(my-float 7 0 1) >> First Sunday in July 2’" >> >> after exhausting all of the known entry types. Should I file this as a >> separate bug? > > No, we can work on this problem here in this bug report. > > Do you have a recipe to reproduce the problem, starting from "emacs -Q"? Yep, you can use the same testing script and just replace the last expression with: (let ((icalendar-export-sexp-enumeration-days 0)) (test-diary-sexp "%%(diary-float 7 0 1) First Sunday in July 1") (test-diary-sexp "%%(my-float 7 0 1) First Sunday in July 2")) which should give you the error I mentioned. I don't have a patch for this on hand but it would probably require restructuring the surrounding code a little bit. There's another bug that concerns sexps that contain more than a single closing parenthesis. Seems like the icalendar code uses a bunch of regexes to parse the sexp (see `icalendar--convert-sexp-to-ical'), rather than something like `read' or `read-from-string' (which is what diary does, as well as org-agenda). To reproduce (with the same testing code as before): (let ((icalendar-export-sexp-enumeration-days 366)) (test-diary-sexp "%%(= (calendar-day-of-week date) 0) Sunday 1") (test-diary-sexp "%%(= 0 (calendar-day-of-week date)) Sunday 2")) No matter whether the closing parentheses are bunched together or separated, parsing fails either way. I've attached a draft of a patch that modifies as little as possible and makes the above two cases work, but that's not enough as there's special handling of `and' forms. Furthermore, sexps that span multiple lines fail for the same reason, even though diary handles them just fine of course. Ideally the whole function should be rewritten to just use `read-from-string'. I don't understand why `and' forms are handled specially though, so I'm not sure how to proceed here. A last thing to note -- even though org-agenda properly handles multiline sexps and sexps with an arbitrary number of closing parentheses (because it does the parsing manually and uses `forward-sexp'), Org's parser itself does not properly handle multiline sexps (but does handle multiple closing parentheses). To confirm, use `org-element-at-point' at the beginning of these two diary sexps in an org-mode buffer: %%(let ((a 123) (b 456)) (+ a b)) %%(let ((a 123) (b 456)) (+ a b)) All of these bugs also propagate to ox-icalendar (which is how I got into exploring this whole thing) since it relies on the Org parse tree to pull out 'diary-sexp elements and on icalendar to export them into iCalendar format. Let me know if you want me to report any of these as separate bugs and what would be the best thing to do here. Regards, hokomo --------------x92lbi8WXdL97kq43XFKoufV Content-Type: text/x-patch; charset=UTF-8; name="0002-Draft-fix-nested-diary-sexps.patch" Content-Disposition: attachment; filename="0002-Draft-fix-nested-diary-sexps.patch" Content-Transfer-Encoding: base64 RnJvbSBlMzVlYTFmM2E5YzYyZDljMDdkMTBkOWVjODliZjg3OTA1ODc3NWMyIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBob2tvbW8gPGhva29tb0BhaXJtYWlsLmNjPgpEYXRl OiBTYXQsIDI1IEp1biAyMDIyIDEzOjQzOjI4ICswMjAwClN1YmplY3Q6IFtQQVRDSF0gRml4 IG5lc3RlZCBkaWFyeSBzZXhwcwoKLS0tCiBsaXNwL2NhbGVuZGFyL2ljYWxlbmRhci5lbCB8 IDggKysrKystLS0KIDEgZmlsZSBjaGFuZ2VkLCA1IGluc2VydGlvbnMoKyksIDMgZGVsZXRp b25zKC0pCgpkaWZmIC0tZ2l0IGEvbGlzcC9jYWxlbmRhci9pY2FsZW5kYXIuZWwgYi9saXNw L2NhbGVuZGFyL2ljYWxlbmRhci5lbAppbmRleCA0NmJjZjQ1NjZmLi4yYjZhZjc5ODEyIDEw MDY0NAotLS0gYS9saXNwL2NhbGVuZGFyL2ljYWxlbmRhci5lbAorKysgYi9saXNwL2NhbGVu ZGFyL2ljYWxlbmRhci5lbApAQCAtMTY0MSw5ICsxNjQxLDExIEBAIGljYWxlbmRhci0tY29u dmVydC1zZXhwLXRvLWljYWwKICAgICAgICAgICAgICAgICAgICAgICAgZW50cnktbWFpbikK ICAgICAgICAgIDs7IHJlZ3VsYXIgc2V4cCBlbnRyeQogICAgICAgICAgKGljYWxlbmRhci0t ZG1zZyAiZGlhcnktc2V4cCAlcyIgZW50cnktbWFpbikKLSAgICAgICAgIChsZXQgKChwMSAo c3Vic3RyaW5nIGVudHJ5LW1haW4gKG1hdGNoLWJlZ2lubmluZyAxKSAobWF0Y2gtZW5kIDEp KSkKLSAgICAgICAgICAgICAgIChwMiAoc3Vic3RyaW5nIGVudHJ5LW1haW4gKG1hdGNoLWJl Z2lubmluZyAyKSAobWF0Y2gtZW5kIDIpKSkKLSAgICAgICAgICAgICAgIChub3cgKG9yIHN0 YXJ0IChjdXJyZW50LXRpbWUpKSkpCisgICAgICAgICAobGV0KiAoKGVudHJ5LW1haW4gKHN1 YnN0cmluZyBlbnRyeS1tYWluIDIpKQorICAgICAgICAgICAgICAgIChyZXMgKHJlYWQtZnJv bS1zdHJpbmcgZW50cnktbWFpbikpCisgICAgICAgICAgICAgICAgKHAxIChwcmluMS10by1z dHJpbmcgKGNhciByZXMpKSkKKyAgICAgICAgICAgICAgICAocDIgKHN1YnN0cmluZyBlbnRy eS1tYWluIChjZHIgcmVzKSkpCisgICAgICAgICAgICAgICAgKG5vdyAob3Igc3RhcnQgKGN1 cnJlbnQtdGltZSkpKSkKICAgICAgICAgICAgKGRlbGV0ZSBuaWwKICAgICAgICAgICAgICAg ICAgICAobWFwY2FyCiAgICAgICAgICAgICAgICAgICAgIChsYW1iZGEgKG9mZnNldCkKLS0g CjIuMzUuMwoK --------------x92lbi8WXdL97kq43XFKoufV--