patch for org-capture.el to allow for no file extension on open-source, protocol

patch for org-capture.el to allow for no file extension on open-source, protocol

[Stacey Marshall]

[-- Attachment #1: Type: text/plain, Size: 3732 bytes --]

The issue was that the URL I was opening had the full filename as-is.  
No extension needed to be added or removed.  If no suffix is provided in 
the alist then the function failed.  The patch allows both online-suffix 
and working-suffix to not be required.


```
 From 213b7968199bebcf11d0bcf0bbc44d0843f19ecc Mon Sep 17 00:00:00 2001
 From: Stacey Marshall <stacey.marshall@gmail.com>
Date: Sat, 9 May 2020 21:31:04 +0100
Subject: [PATCH] org-capture.el: Allow for no file extension on 
open-source
  protocol

* lisp/org-capture.el (org-protocol-open-source): Remove requirement
to have :online-suffix and :working-suffix.

The problem here was that two assignments expected suffix's to be
provided in org-protocol-project-alist.  When they were missing an
error occurred.  With the changes source URLs that include the file
name as-is, such as those presented by openGrok, may be opened.

TINYCHANGE
---
  lisp/org-protocol.el | 26 ++++++++++++++++----------
  1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/lisp/org-protocol.el b/lisp/org-protocol.el
index 55a534d0d..35f58f490 100644
--- a/lisp/org-protocol.el
+++ b/lisp/org-protocol.el
@@ -191,7 +191,7 @@ Example:
            :working-suffix \".org\"
            :base-url \"https://orgmode.org/worg/\"
            :working-directory \"/home/user/org/Worg/\")
-         (\"http://localhost/org-notes/\"
+         (\"localhost org-notes/\"
            :online-suffix \".html\"
            :working-suffix \".org\"
            :base-url \"http://localhost/org/\"
@@ -202,12 +202,17 @@ Example:
            :working-directory \"~/site/content/post/\"
            :online-suffix \".html\"
            :working-suffix \".md\"
-          :rewrites ((\"\\(https://site.com/[0-9]+/[0-9]+/[0-9]+/\\)\" 
. \".md\")))))
-
-
-   The last line tells `org-protocol-open-source' to open
-   /home/user/org/index.php, if the URL cannot be mapped to an existing
-   file, and ends with either \"org\" or \"org/\".
+          :rewrites ((\"\\(https://site.com/[0-9]+/[0-9]+/[0-9]+/\\)\" 
. \".md\")))
+         (\"GNU emacs OpenGrok\"
+          :base-url 
\"https://opengrok.housegordon.com/source/xref/emacs/\"
+          :working-directory \"~/dev/gnu-emacs/\")))
+
+   The :rewrites line of \"localhost org-notes\" entry tells
+   `org-protocol-open-source' to open /home/user/org/index.php,
+   if the URL cannot be mapped to an existing file, and ends with
+   either \"org\" or \"org/\".  The \"GNU emacs OpenGrok\" entry
+   does not include any suffix properties, allowing local source
+   file to be opened as found by OpenGrok.

  Consider using the interactive functions `org-protocol-create' and
  `org-protocol-create-for-org' to help you filling this variable with 
valid contents."
@@ -545,11 +550,12 @@ The location for a browser's bookmark should look 
like this:
  		   ;; ending than strip-suffix here:
  		   (f1 (substring f 0 (string-match "\\([\\?#].*\\)?$" f)))
                     (start-pos (+ (string-match wsearch f1) (length 
base-url)))
-                   (end-pos (string-match
-			     (regexp-quote strip-suffix) f1))
+                   (end-pos (if strip-suffix
+			      (string-match (regexp-quote strip-suffix) f1)
+			      (length f1)))
  		   ;; We have to compare redirects without suffix below:
  		   (f2 (concat wdir (substring f1 start-pos end-pos)))
-                   (the-file (concat f2 add-suffix)))
+                   (the-file (if add-suffix (concat f2 add-suffix) 
f2)))

  	      ;; Note: the-file may still contain `%C3' et al here because 
browsers
  	      ;; tend to encode `&auml;' in URLs to `%25C3' - `%25' being 
`%'.
-- 
2.21.1 (Apple Git-122.3)

```


--
[Stacey](https://staceymarshall.wordpress.com/)

[-- Attachment #2: Type: text/html, Size: 4093 bytes --]

Re: patch for org-capture.el to allow for no file extension on open-source, protocol

[Nicolas Goaziou]

Hello,

"Stacey Marshall" <stacey.marshall@gmail.com> writes:

> The issue was that the URL I was opening had the full filename as-is.
> No extension needed to be added or removed.  If no suffix is provided
> in the alist then the function failed.  The patch allows both
> online-suffix and working-suffix to not be required.

Thank you. Could you add a test in "test-org-protocol.el" ?

Regards,

-- 
Nicolas Goaziou

Re: patch for org-capture.el to allow for no file extension on open-source, protocol

[Stacey Marshall]

[-- Attachment #1.1: Type: text/plain, Size: 5120 bytes --]

On 10 May 2020, at 13:57, Nicolas Goaziou wrote:

> Hello,
>
> "Stacey Marshall" <stacey.marshall@gmail.com> writes:
>
>> The issue was that the URL I was opening had the full filename as-is.
>> No extension needed to be added or removed.  If no suffix is provided
>> in the alist then the function failed.  The patch allows both
>> online-suffix and working-suffix to not be required.
>
> Thank you. Could you add a test in "test-org-protocol.el" ?
>
> Regards,
>
> -- 
> Nicolas Goaziou

Hi Nicolas,

Test patch below.  I've attached both patch files too.


```
 From 9263002dc97753e297f8571511839eea0b39252e Mon Sep 17 00:00:00 2001
 From: Stacey Marshall <stacey.marshall@gmail.com>
Date: Mon, 11 May 2020 00:58:40 +0100
Subject: [PATCH 2/2] test-org-protocol.el: Fix tests and add new test 
for
  as-is filename.

* testing/lisp/test-org-protocol.el 
(test-org-protocol/org-protocol-parse-parameters):
Fix issue of missing flag to indicate new paramenter testing.

* testing/lisp/test-org-protocol.el 
(test-org-protocol/org-protocol-open-source):
Fix test3 which was not working.  Add test4, for opening file name
as-is without suffix modification.

Tested using ert.

TINYCHANGE
---
  testing/lisp/test-org-protocol.el | 37 ++++++++++++++++++++++++++-----
  1 file changed, 32 insertions(+), 5 deletions(-)

diff --git a/testing/lisp/test-org-protocol.el 
b/testing/lisp/test-org-protocol.el
index 6ee368dcf..42ad5e1a3 100644
--- a/testing/lisp/test-org-protocol.el
+++ b/testing/lisp/test-org-protocol.el
@@ -40,7 +40,7 @@
  		      "url=https%3A%2F%2Forgmode.org%2Forg.html%23capture-protocol&"
  		      "title=The%20Org%20Manual&"
  		      "body=9.4.2%20capture%20protocol"))
-	 (data (org-protocol-parse-parameters url)))
+	 (data (org-protocol-parse-parameters url t)))
      (should (string= (plist-get data :template) "p"))
      (should (string= (plist-get data :url) 
"https://orgmode.org/org.html#capture-protocol"))
      (should (string= (plist-get data :title) "The Org Manual"))
@@ -138,6 +138,8 @@
    "Test org-protocol://open-source links."
    (let* ((temp-file-name1 (make-temp-file "org-protocol-test1"))
  	 (temp-file-name2 (make-temp-file "org-protocol-test2"))
+	 (temp-file-name3 (make-temp-file "org-protocol-test3" nil ".md"))
+	 (temp-file-name4 (make-temp-file "org-protocol-test4" nil ".c"))
  	 (org-protocol-project-alist
  	  `((test1
  	     :base-url "http://example.com/"
@@ -149,10 +151,13 @@
  	     :working-directory ,(file-name-directory temp-file-name2))
  	    (test3
  	     :base-url "https://blog-example.com/"
-	     :working-directory ,(file-name-directory temp-file-name2)
+	     :working-directory ,(file-name-directory temp-file-name3)
  	     :online-suffix ".html"
  	     :working-suffix ".md"
-	     :rewrites 
(("\\(https://blog-example.com/[0-9]+/[0-9]+/[0-9]+/\\)" . ".md")))))
+	     :rewrites 
(("\\(https://blog-example.com/[0-9]+/[0-9]+/[0-9]+/\\)" . ".md")))
+	    (test4
+	     :base-url "http://as-is.example.com/"
+	     :working-directory ,(file-name-directory temp-file-name4))))
  	 (test-cases
  	  (list
  	   ;; Old-style URLs
@@ -166,6 +171,16 @@
  		    (url-hexify-string
  		     (concat "http://another.example.com/" (file-name-nondirectory 
temp-file-name2) ".js")))
  	    temp-file-name2)
+	   (cons
+	    (concat "/some/directory/org-protocol:/open-source:/"
+	   	    (url-hexify-string
+	   	     (concat "https://blog-example.com/" (file-name-nondirectory 
(file-name-sans-extension temp-file-name3)) ".html")))
+	    temp-file-name3)
+	   (cons
+	    (concat "/some/directory/org-protocol:/open-source:/"
+	   	    (url-hexify-string
+	   	     (concat "http://as-is.example.com/" (file-name-nondirectory 
temp-file-name4))))
+	    temp-file-name4)
  	   ;; New-style URLs
  	   (cons
  	    (concat "/some/directory/org-protocol:/open-source?url="
@@ -176,7 +191,17 @@
  	    (concat "/some/directory/org-protocol:/open-source?url="
  		    (url-hexify-string
  		     (concat "http://another.example.com/" (file-name-nondirectory 
temp-file-name2) ".js")))
-	    temp-file-name2))))
+	    temp-file-name2)
+	   (cons
+	    (concat "/some/directory/org-protocol:/open-source?url="
+	   	    (url-hexify-string
+	   	     (concat "https://blog-example.com/" (file-name-nondirectory 
(file-name-sans-extension temp-file-name3)) ".html")))
+	    temp-file-name3)
+	   (cons
+	    (concat "/some/directory/org-protocol:/open-source?url="
+		    (url-hexify-string
+		     (concat "http://as-is.example.com/" (file-name-nondirectory 
temp-file-name4))))
+	    temp-file-name4))))
      (mapc (lambda (test-case)
  	    (should (string=
  		     (org-protocol-check-filename-for-protocol
@@ -185,7 +210,9 @@
  		     (cdr test-case))))
  	  test-cases)
      (delete-file temp-file-name1)
-    (delete-file temp-file-name2)))
+    (delete-file temp-file-name2)
+    (delete-file temp-file-name3)
+    (delete-file temp-file-name4)))

  (defun test-org-protocol/org-protocol-greedy-handler (fname)
    ;; fname should be a list of parsed items
-- 
2.21.1 (Apple Git-122.3)


```


--
[Stacey](https://staceymarshall.wordpress.com/)

[-- Attachment #1.2: Type: text/html, Size: 5931 bytes --]

[-- Attachment #2: 0001-org-capture.el-Allow-for-no-file-extension-on-open-s.patch --]
[-- Type: text/plain, Size: 3458 bytes --]

From 213b7968199bebcf11d0bcf0bbc44d0843f19ecc Mon Sep 17 00:00:00 2001
From: Stacey Marshall <stacey.marshall@gmail.com>
Date: Sat, 9 May 2020 21:31:04 +0100
Subject: [PATCH 1/2] org-capture.el: Allow for no file extension on
 open-source protocol

* lisp/org-capture.el (org-protocol-open-source): Remove requirement
to have :online-suffix and :working-suffix.

The problem here was that two assignments expected suffix's to be
provided in org-protocol-project-alist.  When they were missing an
error occurred.  With the changes source URLs that include the file
name as-is, such as those presented by openGrok, may be opened.

TINYCHANGE
---
 lisp/org-protocol.el | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/lisp/org-protocol.el b/lisp/org-protocol.el
index 55a534d0d..35f58f490 100644
--- a/lisp/org-protocol.el
+++ b/lisp/org-protocol.el
@@ -191,7 +191,7 @@ Example:
           :working-suffix \".org\"
           :base-url \"https://orgmode.org/worg/\"
           :working-directory \"/home/user/org/Worg/\")
-         (\"http://localhost/org-notes/\"
+         (\"localhost org-notes/\"
           :online-suffix \".html\"
           :working-suffix \".org\"
           :base-url \"http://localhost/org/\"
@@ -202,12 +202,17 @@ Example:
           :working-directory \"~/site/content/post/\"
           :online-suffix \".html\"
           :working-suffix \".md\"
-          :rewrites ((\"\\(https://site.com/[0-9]+/[0-9]+/[0-9]+/\\)\" . \".md\")))))
-
-
-   The last line tells `org-protocol-open-source' to open
-   /home/user/org/index.php, if the URL cannot be mapped to an existing
-   file, and ends with either \"org\" or \"org/\".
+          :rewrites ((\"\\(https://site.com/[0-9]+/[0-9]+/[0-9]+/\\)\" . \".md\")))
+         (\"GNU emacs OpenGrok\"
+          :base-url \"https://opengrok.housegordon.com/source/xref/emacs/\"
+          :working-directory \"~/dev/gnu-emacs/\")))
+
+   The :rewrites line of \"localhost org-notes\" entry tells
+   `org-protocol-open-source' to open /home/user/org/index.php,
+   if the URL cannot be mapped to an existing file, and ends with
+   either \"org\" or \"org/\".  The \"GNU emacs OpenGrok\" entry
+   does not include any suffix properties, allowing local source
+   file to be opened as found by OpenGrok.
 
 Consider using the interactive functions `org-protocol-create' and
 `org-protocol-create-for-org' to help you filling this variable with valid contents."
@@ -545,11 +550,12 @@ The location for a browser's bookmark should look like this:
 		   ;; ending than strip-suffix here:
 		   (f1 (substring f 0 (string-match "\\([\\?#].*\\)?$" f)))
                    (start-pos (+ (string-match wsearch f1) (length base-url)))
-                   (end-pos (string-match
-			     (regexp-quote strip-suffix) f1))
+                   (end-pos (if strip-suffix
+			      (string-match (regexp-quote strip-suffix) f1)
+			      (length f1)))
 		   ;; We have to compare redirects without suffix below:
 		   (f2 (concat wdir (substring f1 start-pos end-pos)))
-                   (the-file (concat f2 add-suffix)))
+                   (the-file (if add-suffix (concat f2 add-suffix) f2)))
 
 	      ;; Note: the-file may still contain `%C3' et al here because browsers
 	      ;; tend to encode `&auml;' in URLs to `%25C3' - `%25' being `%'.
-- 
2.21.1 (Apple Git-122.3)


[-- Attachment #3: 0002-test-org-protocol.el-Fix-tests-and-add-new-test-for-.patch --]
[-- Type: text/plain, Size: 4552 bytes --]

From 9263002dc97753e297f8571511839eea0b39252e Mon Sep 17 00:00:00 2001
From: Stacey Marshall <stacey.marshall@gmail.com>
Date: Mon, 11 May 2020 00:58:40 +0100
Subject: [PATCH 2/2] test-org-protocol.el: Fix tests and add new test for
 as-is filename.

* testing/lisp/test-org-protocol.el (test-org-protocol/org-protocol-parse-parameters):
Fix issue of missing flag to indicate new paramenter testing.

* testing/lisp/test-org-protocol.el (test-org-protocol/org-protocol-open-source):
Fix test3 which was not working.  Add test4, for opening file name
as-is without suffix modification.

Tested using ert.

TINYCHANGE
---
 testing/lisp/test-org-protocol.el | 37 ++++++++++++++++++++++++++-----
 1 file changed, 32 insertions(+), 5 deletions(-)

diff --git a/testing/lisp/test-org-protocol.el b/testing/lisp/test-org-protocol.el
index 6ee368dcf..42ad5e1a3 100644
--- a/testing/lisp/test-org-protocol.el
+++ b/testing/lisp/test-org-protocol.el
@@ -40,7 +40,7 @@
 		      "url=https%3A%2F%2Forgmode.org%2Forg.html%23capture-protocol&"
 		      "title=The%20Org%20Manual&"
 		      "body=9.4.2%20capture%20protocol"))
-	 (data (org-protocol-parse-parameters url)))
+	 (data (org-protocol-parse-parameters url t)))
     (should (string= (plist-get data :template) "p"))
     (should (string= (plist-get data :url) "https://orgmode.org/org.html#capture-protocol"))
     (should (string= (plist-get data :title) "The Org Manual"))
@@ -138,6 +138,8 @@
   "Test org-protocol://open-source links."
   (let* ((temp-file-name1 (make-temp-file "org-protocol-test1"))
 	 (temp-file-name2 (make-temp-file "org-protocol-test2"))
+	 (temp-file-name3 (make-temp-file "org-protocol-test3" nil ".md"))
+	 (temp-file-name4 (make-temp-file "org-protocol-test4" nil ".c"))
 	 (org-protocol-project-alist
 	  `((test1
 	     :base-url "http://example.com/"
@@ -149,10 +151,13 @@
 	     :working-directory ,(file-name-directory temp-file-name2))
 	    (test3
 	     :base-url "https://blog-example.com/"
-	     :working-directory ,(file-name-directory temp-file-name2)
+	     :working-directory ,(file-name-directory temp-file-name3)
 	     :online-suffix ".html"
 	     :working-suffix ".md"
-	     :rewrites (("\\(https://blog-example.com/[0-9]+/[0-9]+/[0-9]+/\\)" . ".md")))))
+	     :rewrites (("\\(https://blog-example.com/[0-9]+/[0-9]+/[0-9]+/\\)" . ".md")))
+	    (test4
+	     :base-url "http://as-is.example.com/"
+	     :working-directory ,(file-name-directory temp-file-name4))))
 	 (test-cases
 	  (list
 	   ;; Old-style URLs
@@ -166,6 +171,16 @@
 		    (url-hexify-string
 		     (concat "http://another.example.com/" (file-name-nondirectory temp-file-name2) ".js")))
 	    temp-file-name2)
+	   (cons
+	    (concat "/some/directory/org-protocol:/open-source:/"
+	   	    (url-hexify-string
+	   	     (concat "https://blog-example.com/" (file-name-nondirectory (file-name-sans-extension temp-file-name3)) ".html")))
+	    temp-file-name3)
+	   (cons
+	    (concat "/some/directory/org-protocol:/open-source:/"
+	   	    (url-hexify-string
+	   	     (concat "http://as-is.example.com/" (file-name-nondirectory temp-file-name4))))
+	    temp-file-name4)
 	   ;; New-style URLs
 	   (cons
 	    (concat "/some/directory/org-protocol:/open-source?url="
@@ -176,7 +191,17 @@
 	    (concat "/some/directory/org-protocol:/open-source?url="
 		    (url-hexify-string
 		     (concat "http://another.example.com/" (file-name-nondirectory temp-file-name2) ".js")))
-	    temp-file-name2))))
+	    temp-file-name2)
+	   (cons
+	    (concat "/some/directory/org-protocol:/open-source?url="
+	   	    (url-hexify-string
+	   	     (concat "https://blog-example.com/" (file-name-nondirectory (file-name-sans-extension temp-file-name3)) ".html")))
+	    temp-file-name3)
+	   (cons
+	    (concat "/some/directory/org-protocol:/open-source?url="
+		    (url-hexify-string
+		     (concat "http://as-is.example.com/" (file-name-nondirectory temp-file-name4))))
+	    temp-file-name4))))
     (mapc (lambda (test-case)
 	    (should (string=
 		     (org-protocol-check-filename-for-protocol
@@ -185,7 +210,9 @@
 		     (cdr test-case))))
 	  test-cases)
     (delete-file temp-file-name1)
-    (delete-file temp-file-name2)))
+    (delete-file temp-file-name2)
+    (delete-file temp-file-name3)
+    (delete-file temp-file-name4)))
 
 (defun test-org-protocol/org-protocol-greedy-handler (fname)
   ;; fname should be a list of parsed items
-- 
2.21.1 (Apple Git-122.3)

Re: patch for org-capture.el to allow for no file extension on open-source, protocol

[Nicolas Goaziou]

Hello,

"Stacey Marshall" <stacey.marshall@gmail.com> writes:

> Test patch below.  I've attached both patch files too.

Thank you! I applied the first patch, i.e., the actual feature.

However, I realize, a bit late, that you haven't signed FSF papers, and
both patches are above the TINYCHANGE limit. Would you want to sign them
so we can also apply the tests, too?

Regards,

-- 
Nicolas Goaziou

Re: patch for org-capture.el to allow for no file extension on open-source, protocol

[Nicolas Goaziou]

Completing myself,

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> However, I realize, a bit late, that you haven't signed FSF papers, and
> both patches are above the TINYCHANGE limit. Would you want to sign them
> so we can also apply the tests, too?

Also, would you mind adding a few words into ORG-NEWS to announce the change?

Thanks!

Re: patch for org-capture.el to allow for no file extension on open-source, protocol

[Stacey Marshall]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

Will do.  sorry didn't realize I was over the Tiny change limit.  I can
sign FSF papers if still needed.

Stace

On Tue, 12 May 2020 at 16:05, Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> Completing myself,
>
> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
> > However, I realize, a bit late, that you haven't signed FSF papers, and
> > both patches are above the TINYCHANGE limit. Would you want to sign them
> > so we can also apply the tests, too?
>
> Also, would you mind adding a few words into ORG-NEWS to announce the
> change?
>
> Thanks!
>
-- 
Sent from Gmail Mobile

[-- Attachment #2: Type: text/html, Size: 1117 bytes --]

Re: patch for org-capture.el to allow for no file extension on open-source, protocol

[Nicolas Goaziou]

Hello,

Stacey Marshall <stacey.marshall@gmail.com> writes:

> Will do.  sorry didn't realize I was over the Tiny change limit.  I can
> sign FSF papers if still needed.

It is still needed :)

Please let me know once the process is complete, or if you have question
regarding the process.

Thank you!

Regards,

-- 
Nicolas Goaziou

Re: patch for org-capture.el to allow for no file extension on open-source, protocol

[Stacey Marshall]

[-- Attachment #1: Type: text/plain, Size: 702 bytes --]

On 13 May 2020, at 9:02, Nicolas Goaziou wrote:

> Hello,
>
> Stacey Marshall <stacey.marshall@gmail.com> writes:
>
>> Will do.  sorry didn't realize I was over the Tiny change limit.  I 
>> can
>> sign FSF papers if still needed.
>
> It is still needed :)
>
> Please let me know once the process is complete, or if you have 
> question
> regarding the process.
>
> Thank you!
>
> Regards,
>
> -- 
> Nicolas Goaziou


Hi Nicolas,

Well that took much longer than I expected.  The signature process is 
finally complete.

If you would still like me to update ORG-NEWS then I shall have to brush 
up on my git and send a new patch...

Let me know....


--
[Stacey](https://staceymarshall.wordpress.com/)

[-- Attachment #2: Type: text/html, Size: 1256 bytes --]