* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
@ 2015-11-28 21:53 Alain Schneble
2015-11-28 23:42 ` Alain Schneble
0 siblings, 1 reply; 18+ messages in thread
From: Alain Schneble @ 2015-11-28 21:53 UTC (permalink / raw)
To: 22044
url-expand.el and url-parse.el seem to not conform to RFC 3986 "Uniform
Resource Identifier (URI): Generic Syntax" in some cases. But I assume
they should. Here is a list of issues found in url-expand-file-name
and url-generic-parse-url, respectively:
1. resolving relative "fragment-only" URIs against a given absolute
base URI (see RFC3986, section 5. Reference Resolution, and
especially 5.2.2. Transform References):
(url-expand-file-name "#s" "http://a/b/c/d;p?q")
=> "#s" but should be http://a/b/c/d;p?q#s"
(url-expand-file-name "#bar" "http://host")
=> "#bar" but should be "http://host#bar"
(url-expand-file-name "#bar" "http://host/")
=> "#bar" but should be "http://host/#bar"
(url-expand-file-name "#bar" "http://host/foo")
=> "#bar" but should be "http://host/foo#bar"
2. resolving relative "query-only" URIs against a given absolute base
URI (see RFC3986, same sections as mentioned in point 1.):
(url-expand-file-name "?y" "http://a/b/c/d;p?q")
=> "http://a/b/c/?y" but should be "http://a/b/c/d;p?y"
(url-expand-file-name "?y" "http://a/b/c/d")
=> "http://a/b/c/?y" but should be "http://a/b/c/d?y")
3. removing dot segments (see RFC3986, section 5.2.4. Remove Dot
Segments):
(url-expand-file-name "/./g" "http://a/b/c/d;p?q")
=> "http://a/./g" but should be "http://a/g"
(url-expand-file-name "/../g" "http://a/b/c/d;p?q")
=> "http://a/../g" but should be "http://a/g"
4. empty fragment information is lost after parsing URI:
(equal (url-generic-parse-url "#")
(url-parse-make-urlobj nil nil nil nil nil "" "" nil nil))
^
=> nil but should be t (fragment component is actually nil instead
of an empty string)
Same issue with URLs having a number sign (#) as suffix:
"/foo/bar#"
"/foo/bar/#"
"http://host#"
"http://host?#"
"http://host?query#"
"http://host/#"
"http://host/?#"
"http://host/?query#"
"http://host/foo#"
"http://host/foo?#"
"http://host/foo?query#"
... and so forth
With the current behavior, the inverse function url-recreate-url
won't be able to reconstruct exactly the same URI. For example:
(url-recreate-url (url-generic-parse-url "#"))
=> "" but should be "#"
Alain
In GNU Emacs 24.4.1 (i686-pc-mingw32)
of 2014-10-24 on LEG570
Windowing system distributor `Microsoft Corp.', version 6.3.9600
Configured using:
`configure --prefix=/c/usr'
Important settings:
value of $LANG: DES
locale-coding-system: cp1252
^ permalink raw reply [flat|nested] 18+ messages in thread* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-11-28 21:53 bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 Alain Schneble @ 2015-11-28 23:42 ` Alain Schneble 2015-12-25 18:01 ` Lars Ingebrigtsen 2015-12-25 19:58 ` Lars Ingebrigtsen 0 siblings, 2 replies; 18+ messages in thread From: Alain Schneble @ 2015-11-28 23:42 UTC (permalink / raw) To: 22044 [-- Attachment #1: Type: text/plain, Size: 179 bytes --] Tags: patch To address these issues and make the behavior consistent with RFC 3984, I propose the following changes to url-parse.el and url-expand.el, see attached patch. Alain [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: patch --] [-- Type: text/x-patch, Size: 29649 bytes --] From 5b6e6db3c31be8ed63f36b9f82e66610c86f6cb0 Mon Sep 17 00:00:00 2001 From: Alain Nicolas Schneble <a.s@realize.ch> Date: Fri, 27 Nov 2015 15:54:12 +0100 Subject: [PATCH] Make relative URL parsing and resolution consistent with RFC 3986 (bug#22044) * test/automated/url-parse-tests.el: add tests covering url-generic-parse-url. * test/automated/url-expand-tests.el: add tests covering url-expand-file-name. * test/automated/url-parse.el (url-generic-parse-url): keep empty fragment information in URL-struct. * lisp/url/url-parse.el (url-path-and-query): do not artificially turn empty path and query into nil path and query, respectively. * lisp/url/url-expand.el (url-expander-remove-relative-links): do not turn empty path into an absolute ("/") path. * lisp/url/url-expand.el (url-expand-file-name): properly resolve fragment-only URIs. Do not just return them unchanged. * lisp/url/url-expand.el (url-default-expander): an empty path in the relative reference URI should not drop the last segment. --- lisp/url/url-expand.el | 84 +++++++++---------- lisp/url/url-parse.el | 5 +- test/automated/url-expand-tests.el | 105 +++++++++++++++++++++++ test/automated/url-parse-tests.el | 167 +++++++++++++++++++++++++++++++++++++ 4 files changed, 313 insertions(+), 48 deletions(-) create mode 100644 test/automated/url-expand-tests.el create mode 100644 test/automated/url-parse-tests.el diff --git a/lisp/url/url-expand.el b/lisp/url/url-expand.el index c468a79..600a36d 100644 --- a/lisp/url/url-expand.el +++ b/lisp/url/url-expand.el @@ -26,32 +26,35 @@ (require 'url-parse) (defun url-expander-remove-relative-links (name) - ;; Strip . and .. from pathnames - (let ((new (if (not (string-match "^/" name)) - (concat "/" name) - name))) - - ;; If it ends with a '/.' or '/..', tack on a trailing '/' sot hat - ;; the tests that follow are not too complicated in terms of - ;; looking for '..' or '../', etc. - (if (string-match "/\\.+$" new) - (setq new (concat new "/"))) - - ;; Remove '/./' first - (while (string-match "/\\(\\./\\)" new) - (setq new (concat (substring new 0 (match-beginning 1)) - (substring new (match-end 1))))) - - ;; Then remove '/../' - (while (string-match "/\\([^/]*/\\.\\./\\)" new) - (setq new (concat (substring new 0 (match-beginning 1)) - (substring new (match-end 1))))) - - ;; Remove cruft at the beginning of the string, so people that put - ;; in extraneous '..' because they are morons won't lose. - (while (string-match "^/\\.\\.\\(/\\)" new) - (setq new (substring new (match-beginning 1) nil))) - new)) + (if (equal name "") + ;; An empty name is a properly valid relative URL reference/path. + "" + ;; Strip . and .. from pathnames + (let ((new (if (not (string-match "^/" name)) + (concat "/" name) + name))) + + ;; If it ends with a '/.' or '/..', tack on a trailing '/' sot hat + ;; the tests that follow are not too complicated in terms of + ;; looking for '..' or '../', etc. + (if (string-match "/\\.+$" new) + (setq new (concat new "/"))) + + ;; Remove '/./' first + (while (string-match "/\\(\\./\\)" new) + (setq new (concat (substring new 0 (match-beginning 1)) + (substring new (match-end 1))))) + + ;; Then remove '/../' + (while (string-match "/\\([^/]*/\\.\\./\\)" new) + (setq new (concat (substring new 0 (match-beginning 1)) + (substring new (match-end 1))))) + + ;; Remove cruft at the beginning of the string, so people that put + ;; in extraneous '..' because they are morons won't lose. + (while (string-match "^/\\.\\.\\(/\\)" new) + (setq new (substring new (match-beginning 1) nil))) + new))) (defun url-expand-file-name (url &optional default) "Convert URL to a fully specified URL, and canonicalize it. @@ -89,8 +92,6 @@ path components followed by `..' are removed, along with the `..' itself." (cond ((= (length url) 0) ; nil or empty string (url-recreate-url default)) - ((string-match "^#" url) ; Offset link, use it raw - url) ((string-match url-nonrelative-link url) ; Fully-qualified URL, return it immediately url) (t @@ -120,29 +121,24 @@ path components followed by `..' are removed, along with the `..' itself." (setf (url-host urlobj) (or (url-host urlobj) (url-host defobj)))) (if (string= "ftp" (url-type urlobj)) (setf (url-user urlobj) (or (url-user urlobj) (url-user defobj)))) - (if (string= (url-filename urlobj) "") - (setf (url-filename urlobj) "/")) ;; If the object we're expanding from is full, then we are now ;; full. (unless (url-fullness urlobj) (setf (url-fullness urlobj) (url-fullness defobj))) - (if (string-match "^/" (url-filename urlobj)) - nil - (let ((query nil) - (file nil) - (sepchar nil)) - (if (string-match "[?#]" (url-filename urlobj)) - (setq query (substring (url-filename urlobj) (match-end 0)) - file (substring (url-filename urlobj) 0 (match-beginning 0)) - sepchar (substring (url-filename urlobj) (match-beginning 0) (match-end 0))) - (setq file (url-filename urlobj))) + (let* ((pathandquery (url-path-and-query urlobj)) + (defpathandquery (url-path-and-query defobj)) + (file (car pathandquery)) + (query (or (cdr pathandquery) (and (equal file "") (cdr defpathandquery))))) + (if (string-match "^/" (url-filename urlobj)) + (setq file (url-expander-remove-relative-links file)) ;; We use concat rather than expand-file-name to combine ;; directory and file name, since urls do not follow the same ;; rules as local files on all platforms. - (setq file (url-expander-remove-relative-links - (concat (url-file-directory (url-filename defobj)) file))) - (setf (url-filename urlobj) - (if query (concat file sepchar query) file)))))) + (setq file (url-expander-remove-relative-links + (if (equal file "") + (or (car (url-path-and-query defobj)) "") + (concat (url-file-directory (url-filename defobj)) file))))) + (setf (url-filename urlobj) (if query (concat file "?" query) file))))) (provide 'url-expand) diff --git a/lisp/url/url-parse.el b/lisp/url/url-parse.el index dbf0c38..c3159a7 100644 --- a/lisp/url/url-parse.el +++ b/lisp/url/url-parse.el @@ -59,8 +59,6 @@ where each of PATH and QUERY are strings or nil." (setq path (substring name 0 (match-beginning 0)) query (substring name (match-end 0))) (setq path name))) - (if (equal path "") (setq path nil)) - (if (equal query "") (setq query nil)) (cons path query))) (defun url-port-if-non-default (urlobj) @@ -217,8 +215,7 @@ parses to (when (looking-at "#") (let ((opoint (point))) (forward-char 1) - (unless (eobp) - (setq fragment (buffer-substring (point) (point-max)))) + (setq fragment (buffer-substring (point) (point-max))) (delete-region opoint (point-max))))) (if (and host (string-match "%[0-9][0-9]" host)) diff --git a/test/automated/url-expand-tests.el b/test/automated/url-expand-tests.el new file mode 100644 index 0000000..88c9b3b --- /dev/null +++ b/test/automated/url-expand-tests.el @@ -0,0 +1,105 @@ +;;; url-expand-tests.el --- Test suite for relative URI/URL resolution. + +;; Copyright (C) 2012-2015 Free Software Foundation, Inc. + +;; Author: Alain Nicolas Schneble <a.s@realize.ch> +;; Version: 1.0 + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>. + +;;; Commentary: + +;; Test cases covering URI reference resolution as described in RFC3986, +;; section 5. Reference Resolution and especially the relative resolution +;; rules specified in section 5.2. Relative Resolution. + +;; Each test calls `url-expand-file-name', typically with a relative +;; reference URI and a base URI as string and compares the result (Actual) +;; against a manually specified URI (Expected) + +;;; Code: + +(require 'url-expand) +(require 'ert) + +(ert-deftest url-expand-file-name/relative-resolution-normal-examples () + "RFC 3986, Section 5.4 Reference Resolution Examples / Section 5.4.1. Normal Examples" + (should (equal (url-expand-file-name "g:h" "http://a/b/c/d;p?q") "g:h")) + (should (equal (url-expand-file-name "g" "http://a/b/c/d;p?q") "http://a/b/c/g")) + (should (equal (url-expand-file-name "./g" "http://a/b/c/d;p?q") "http://a/b/c/g")) + (should (equal (url-expand-file-name "g/" "http://a/b/c/d;p?q") "http://a/b/c/g/")) + (should (equal (url-expand-file-name "/g" "http://a/b/c/d;p?q") "http://a/g")) + (should (equal (url-expand-file-name "//g" "http://a/b/c/d;p?q") "http://g")) + (should (equal (url-expand-file-name "?y" "http://a/b/c/d;p?q") "http://a/b/c/d;p?y")) + (should (equal (url-expand-file-name "g?y" "http://a/b/c/d;p?q") "http://a/b/c/g?y")) + (should (equal (url-expand-file-name "#s" "http://a/b/c/d;p?q") "http://a/b/c/d;p?q#s")) + (should (equal (url-expand-file-name "g#s" "http://a/b/c/d;p?q") "http://a/b/c/g#s")) + (should (equal (url-expand-file-name "g?y#s" "http://a/b/c/d;p?q") "http://a/b/c/g?y#s")) + (should (equal (url-expand-file-name ";x" "http://a/b/c/d;p?q") "http://a/b/c/;x")) + (should (equal (url-expand-file-name "g;x" "http://a/b/c/d;p?q") "http://a/b/c/g;x")) + (should (equal (url-expand-file-name "g;x?y#s" "http://a/b/c/d;p?q") "http://a/b/c/g;x?y#s")) + (should (equal (url-expand-file-name "" "http://a/b/c/d;p?q") "http://a/b/c/d;p?q")) + (should (equal (url-expand-file-name "." "http://a/b/c/d;p?q") "http://a/b/c/")) + (should (equal (url-expand-file-name "./" "http://a/b/c/d;p?q") "http://a/b/c/")) + (should (equal (url-expand-file-name ".." "http://a/b/c/d;p?q") "http://a/b/")) + (should (equal (url-expand-file-name "../" "http://a/b/c/d;p?q") "http://a/b/")) + (should (equal (url-expand-file-name "../g" "http://a/b/c/d;p?q") "http://a/b/g")) + (should (equal (url-expand-file-name "../.." "http://a/b/c/d;p?q") "http://a/")) + (should (equal (url-expand-file-name "../../" "http://a/b/c/d;p?q") "http://a/")) + (should (equal (url-expand-file-name "../../g" "http://a/b/c/d;p?q") "http://a/g"))) + +(ert-deftest url-expand-file-name/relative-resolution-absolute-examples () + "RFC 3986, Section 5.4 Reference Resolution Examples / Section 5.4.2. Abnormal Examples" + (should (equal (url-expand-file-name "../../../g" "http://a/b/c/d;p?q") "http://a/g")) + (should (equal (url-expand-file-name "../../../../g" "http://a/b/c/d;p?q") "http://a/g")) + + (should (equal (url-expand-file-name "/./g" "http://a/b/c/d;p?q") "http://a/g")) + (should (equal (url-expand-file-name "/../g" "http://a/b/c/d;p?q") "http://a/g")) + (should (equal (url-expand-file-name "g." "http://a/b/c/d;p?q") "http://a/b/c/g.")) + (should (equal (url-expand-file-name ".g" "http://a/b/c/d;p?q") "http://a/b/c/.g")) + (should (equal (url-expand-file-name "g.." "http://a/b/c/d;p?q") "http://a/b/c/g..")) + (should (equal (url-expand-file-name "..g" "http://a/b/c/d;p?q") "http://a/b/c/..g")) + + (should (equal (url-expand-file-name "./../g" "http://a/b/c/d;p?q") "http://a/b/g")) + (should (equal (url-expand-file-name "./g/." "http://a/b/c/d;p?q") "http://a/b/c/g/")) + (should (equal (url-expand-file-name "g/./h" "http://a/b/c/d;p?q") "http://a/b/c/g/h")) + (should (equal (url-expand-file-name "g/../h" "http://a/b/c/d;p?q") "http://a/b/c/h")) + (should (equal (url-expand-file-name "g;x=1/./y" "http://a/b/c/d;p?q") "http://a/b/c/g;x=1/y")) + (should (equal (url-expand-file-name "g;x=1/../y" "http://a/b/c/d;p?q") "http://a/b/c/y")) + + (should (equal (url-expand-file-name "g?y/./x" "http://a/b/c/d;p?q") "http://a/b/c/g?y/./x")) + (should (equal (url-expand-file-name "g?y/../x" "http://a/b/c/d;p?q") "http://a/b/c/g?y/../x")) + (should (equal (url-expand-file-name "g#s/./x" "http://a/b/c/d;p?q") "http://a/b/c/g#s/./x")) + (should (equal (url-expand-file-name "g#s/../x" "http://a/b/c/d;p?q") "http://a/b/c/g#s/../x")) + + (should (equal (url-expand-file-name "http:g" "http://a/b/c/d;p?q") "http:g")) ; for strict parsers + ) + +(ert-deftest url-expand-file-name/relative-resolution-additional-examples () + "Reference Resolution Examples / Arbitrary Examples" + (should (equal (url-expand-file-name "" "http://host/foobar") "http://host/foobar")) + (should (equal (url-expand-file-name "?y" "http://a/b/c/d") "http://a/b/c/d?y")) + (should (equal (url-expand-file-name "?y" "http://a/b/c/d/") "http://a/b/c/d/?y")) + (should (equal (url-expand-file-name "?y#fragment" "http://a/b/c/d;p?q") "http://a/b/c/d;p?y#fragment")) + (should (equal (url-expand-file-name "#bar" "http://host") "http://host#bar")) + (should (equal (url-expand-file-name "#bar" "http://host/") "http://host/#bar")) + (should (equal (url-expand-file-name "#bar" "http://host/foo") "http://host/foo#bar")) + (should (equal (url-expand-file-name "foo#bar" "http://host/foobar") "http://host/foo#bar")) + (should (equal (url-expand-file-name "foo#bar" "http://host/foobar/") "http://host/foobar/foo#bar"))) + +(provide 'url-expand-tests) + +;;; url-expand-tests.el ends here diff --git a/test/automated/url-parse-tests.el b/test/automated/url-parse-tests.el new file mode 100644 index 0000000..cded361 --- /dev/null +++ b/test/automated/url-parse-tests.el @@ -0,0 +1,167 @@ +;;; url-parse-tests.el --- Test suite for URI/URL parsing. + +;; Copyright (C) 2012-2015 Free Software Foundation, Inc. + +;; Author: Alain Nicolas Schneble <a.s@realize.ch> +;; Version: 1.0 + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>. + +;;; Commentary: + +;; Test cases covering generic URI syntax as described in RFC3986, +;; section 3. Syntax Components and 4. Usage. See also appendix +;; A. Collected ABNF for URI, as the example given here are all +;; productions of this grammar. + +;; Each tests parses a given URI string - whether relative or absolute - +;; using `url-generic-parse-url' and compares the constructed +;; URL-struct (Actual) against a manually `url-parse-make-urlobj'- +;; constructed URL-struct (Expected). + +;;; Code: + +(require 'url-parse) +(require 'ert) + +(ert-deftest url-generic-parse-url/generic-uri-examples () + "RFC 3986, section 1.1.2. Examples / Example illustrating several URI schemes and variations in their common syntax components" + (should (equal (url-generic-parse-url "ftp://ftp.is.co.za/rfc/rfc1808.txt") (url-parse-make-urlobj "ftp" nil nil "ftp.is.co.za" nil "/rfc/rfc1808.txt" nil nil t))) + (should (equal (url-generic-parse-url "http://www.ietf.org/rfc/rfc2396.txt") (url-parse-make-urlobj "http" nil nil "www.ietf.org" nil "/rfc/rfc2396.txt" nil nil t))) + (should (equal (url-generic-parse-url "ldap://[2001:db8::7]/c=GB?objectClass?one") (url-parse-make-urlobj "ldap" nil nil "[2001:db8::7]" nil "/c=GB?objectClass?one" nil nil t))) + (should (equal (url-generic-parse-url "mailto:John.Doe@example.com") (url-parse-make-urlobj "mailto" nil nil nil nil "John.Doe@example.com" nil nil nil))) + (should (equal (url-generic-parse-url "news:comp.infosystems.www.servers.unix") (url-parse-make-urlobj "news" nil nil nil nil "comp.infosystems.www.servers.unix" nil nil nil))) + (should (equal (url-generic-parse-url "tel:+1-816-555-1212") (url-parse-make-urlobj "tel" nil nil nil nil "+1-816-555-1212" nil nil nil))) + (should (equal (url-generic-parse-url "telnet://192.0.2.16:80/") (url-parse-make-urlobj "telnet" nil nil "192.0.2.16" 80 "/" nil nil t))) + (should (equal (url-generic-parse-url "urn:oasis:names:specification:docbook:dtd:xml:4.1.2") (url-parse-make-urlobj "urn" nil nil nil nil "oasis:names:specification:docbook:dtd:xml:4.1.2" nil nil nil)))) + +(ert-deftest url-generic-parse-url/generic-uri () + "RFC 3986, section 3. Syntax Components / generic URI syntax" + ;; empty path + (should (equal (url-generic-parse-url "http://host#") (url-parse-make-urlobj "http" nil nil "host" nil "" "" nil t))) + (should (equal (url-generic-parse-url "http://host#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host?#") (url-parse-make-urlobj "http" nil nil "host" nil "?" "" nil t))) + (should (equal (url-generic-parse-url "http://host?query#") (url-parse-make-urlobj "http" nil nil "host" nil "?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "?query" "fragment" nil t))) + ;; absolute path / + (should (equal (url-generic-parse-url "http://host/#") (url-parse-make-urlobj "http" nil nil "host" nil "/" "" nil t))) + (should (equal (url-generic-parse-url "http://host/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/?" "" nil t))) + (should (equal (url-generic-parse-url "http://host/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" "fragment" nil t))) + ;; absolute path /foo + (should (equal (url-generic-parse-url "http://host/foo#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" "fragment" nil t))) + ;; absolute path /foo/ + (should (equal (url-generic-parse-url "http://host/foo/#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" "fragment" nil t))) + ;; absolute path /foo/bar + (should (equal (url-generic-parse-url "http://host/foo/bar#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" "fragment" nil t))) + ;; absolute path /foo/bar/ + (should (equal (url-generic-parse-url "http://host/foo/bar/#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" "fragment" nil t))) + ;; for more examples of URIs without fragments, see tests covering section 4.3. Absolute URI + ) + +(ert-deftest url-generic-parse-url/network-path-reference () + "RFC 3986, section 4.2. Relative Reference / network-path reference: a relative reference that begins with two slash characters" + (should (equal (url-generic-parse-url "//host") (url-parse-make-urlobj nil nil nil "host" nil "" nil nil t))) + (should (equal (url-generic-parse-url "//host/") (url-parse-make-urlobj nil nil nil "host" nil "/" nil nil t))) + (should (equal (url-generic-parse-url "//host/foo") (url-parse-make-urlobj nil nil nil "host" nil "/foo" nil nil t))) + (should (equal (url-generic-parse-url "//host/foo/bar") (url-parse-make-urlobj nil nil nil "host" nil "/foo/bar" nil nil t))) + (should (equal (url-generic-parse-url "//host/foo/bar/") (url-parse-make-urlobj nil nil nil "host" nil "/foo/bar/" nil nil t)))) + +(ert-deftest url-generic-parse-url/absolute-path-reference () + "RFC 3986, section 4.2. Relative Reference / absolute-path reference: a relative reference that begins with a single slash character" + (should (equal (url-generic-parse-url "/") (url-parse-make-urlobj nil nil nil nil nil "/" nil nil nil))) + (should (equal (url-generic-parse-url "/foo") (url-parse-make-urlobj nil nil nil nil nil "/foo" nil nil nil))) + (should (equal (url-generic-parse-url "/foo/bar") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar" nil nil nil))) + (should (equal (url-generic-parse-url "/foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar/" nil nil nil))) + (should (equal (url-generic-parse-url "/foo/bar#") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar" "" nil nil))) + (should (equal (url-generic-parse-url "/foo/bar/#") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar/" "" nil nil)))) + +(ert-deftest url-generic-parse-url/relative-path-reference () + "RFC 3986, section 4.2. Relative Reference / relative-path reference: a relative reference that does not begin with a slash character" + (should (equal (url-generic-parse-url "foo") (url-parse-make-urlobj nil nil nil nil nil "foo" nil nil nil))) + (should (equal (url-generic-parse-url "foo/bar") (url-parse-make-urlobj nil nil nil nil nil "foo/bar" nil nil nil))) + (should (equal (url-generic-parse-url "foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "foo/bar/" nil nil nil))) + (should (equal (url-generic-parse-url "./foo") (url-parse-make-urlobj nil nil nil nil nil "./foo" nil nil nil))) + (should (equal (url-generic-parse-url "./foo/bar") (url-parse-make-urlobj nil nil nil nil nil "./foo/bar" nil nil nil))) + (should (equal (url-generic-parse-url "./foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "./foo/bar/" nil nil nil))) + (should (equal (url-generic-parse-url "../foo") (url-parse-make-urlobj nil nil nil nil nil "../foo" nil nil nil))) + (should (equal (url-generic-parse-url "../foo/bar") (url-parse-make-urlobj nil nil nil nil nil "../foo/bar" nil nil nil))) + (should (equal (url-generic-parse-url "../foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "../foo/bar/" nil nil nil))) + (should (equal (url-generic-parse-url "./this:that") (url-parse-make-urlobj nil nil nil nil nil "./this:that" nil nil nil))) + ;; for more examples of relative-path references, see tests covering section 4.4. Same-Document Reference + ) + +(ert-deftest url-generic-parse-url/absolute-uri () + "RFC 3986, section 4.3. Absolute URI / absolute URI: absolute form of a URI without a fragment identifier" + ;; empty path + (should (equal (url-generic-parse-url "http://host") (url-parse-make-urlobj "http" nil nil "host" nil "" nil nil t))) + (should (equal (url-generic-parse-url "http://host?") (url-parse-make-urlobj "http" nil nil "host" nil "?" nil nil t))) + (should (equal (url-generic-parse-url "http://host?query") (url-parse-make-urlobj "http" nil nil "host" nil "?query" nil nil t))) + ;; absolute path / + (should (equal (url-generic-parse-url "http://host/") (url-parse-make-urlobj "http" nil nil "host" nil "/" nil nil t))) + (should (equal (url-generic-parse-url "http://host/?") (url-parse-make-urlobj "http" nil nil "host" nil "/?" nil nil t))) + (should (equal (url-generic-parse-url "http://host/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" nil nil t))) + ;; absolute path /foo + (should (equal (url-generic-parse-url "http://host/foo") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" nil nil t))) + ;; absolute path /foo/ + (should (equal (url-generic-parse-url "http://host/foo/") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" nil nil t))) + ;; absolute path /foo/bar + (should (equal (url-generic-parse-url "http://host/foo/bar") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" nil nil t))) + ;; absolute path /foo/bar/ + (should (equal (url-generic-parse-url "http://host/foo/bar/") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" nil nil t))) + ;; example mentioned in RFC3986, section 5.4. Reference Resolution Examples + (should (equal (url-generic-parse-url "http://a/b/c/d;p?q") (url-parse-make-urlobj "http" nil nil "a" nil "/b/c/d;p?q" nil nil t)))) + +(ert-deftest url-generic-parse-url/same-decument-reference () + "RFC 3986, section 4.4. Same-Document Reference / same-document reference: empty or number sign (\"#\") followed by a fragment identifier" + (should (equal (url-generic-parse-url "") (url-parse-make-urlobj nil nil nil nil nil "" nil nil nil))) + (should (equal (url-generic-parse-url "#") (url-parse-make-urlobj nil nil nil nil nil "" "" nil nil))) + (should (equal (url-generic-parse-url "#foo") (url-parse-make-urlobj nil nil nil nil nil "" "foo" nil nil)))) + +(provide 'url-parse-tests) + +;;; url-parse-tests.el ends here -- 2.6.2.windows.1 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-11-28 23:42 ` Alain Schneble @ 2015-12-25 18:01 ` Lars Ingebrigtsen 2015-12-25 19:45 ` Eli Zaretskii 2015-12-25 19:58 ` Lars Ingebrigtsen 1 sibling, 1 reply; 18+ messages in thread From: Lars Ingebrigtsen @ 2015-12-25 18:01 UTC (permalink / raw) To: Alain Schneble; +Cc: 22044 Alain Schneble <a.s@realize.ch> writes: > To address these issues and make the behavior consistent with RFC 3984, > I propose the following changes to url-parse.el and url-expand.el, see > attached patch. Looks good. Do you have Emacs copyright assignment papers on file with the FSF? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-25 18:01 ` Lars Ingebrigtsen @ 2015-12-25 19:45 ` Eli Zaretskii 2015-12-25 19:54 ` Lars Ingebrigtsen 0 siblings, 1 reply; 18+ messages in thread From: Eli Zaretskii @ 2015-12-25 19:45 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: a.s, 22044 > From: Lars Ingebrigtsen <larsi@gnus.org> > Date: Fri, 25 Dec 2015 19:01:33 +0100 > Cc: 22044@debbugs.gnu.org > > Alain Schneble <a.s@realize.ch> writes: > > > To address these issues and make the behavior consistent with RFC 3984, > > I propose the following changes to url-parse.el and url-expand.el, see > > attached patch. > > Looks good. Do you have Emacs copyright assignment papers on file with > the FSF? He does. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-25 19:45 ` Eli Zaretskii @ 2015-12-25 19:54 ` Lars Ingebrigtsen 0 siblings, 0 replies; 18+ messages in thread From: Lars Ingebrigtsen @ 2015-12-25 19:54 UTC (permalink / raw) To: Eli Zaretskii; +Cc: a.s, 22044 Eli Zaretskii <eliz@gnu.org> writes: >> From: Lars Ingebrigtsen <larsi@gnus.org> >> Date: Fri, 25 Dec 2015 19:01:33 +0100 >> Cc: 22044@debbugs.gnu.org >> >> Alain Schneble <a.s@realize.ch> writes: >> >> > To address these issues and make the behavior consistent with RFC 3984, >> > I propose the following changes to url-parse.el and url-expand.el, see >> > attached patch. >> >> Looks good. Do you have Emacs copyright assignment papers on file with >> the FSF? > > He does. Great; I'll apply the patches. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-11-28 23:42 ` Alain Schneble 2015-12-25 18:01 ` Lars Ingebrigtsen @ 2015-12-25 19:58 ` Lars Ingebrigtsen 2015-12-25 22:27 ` Alain Schneble 1 sibling, 1 reply; 18+ messages in thread From: Lars Ingebrigtsen @ 2015-12-25 19:58 UTC (permalink / raw) To: Alain Schneble; +Cc: 22044 Alain Schneble <a.s@realize.ch> writes: > * test/automated/url-parse-tests.el: add tests covering url-generic-parse-url. > * test/automated/url-expand-tests.el: add tests covering url-expand-file-name. > * test/automated/url-parse.el (url-generic-parse-url): keep empty fragment > information in URL-struct. This is apparently a new top-level directory. Are these supposed to be in this directory, is is this an error? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-25 19:58 ` Lars Ingebrigtsen @ 2015-12-25 22:27 ` Alain Schneble 2015-12-25 22:39 ` Lars Ingebrigtsen 0 siblings, 1 reply; 18+ messages in thread From: Alain Schneble @ 2015-12-25 22:27 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: 22044 Lars Ingebrigtsen <larsi@gnus.org> writes: Hello Lars > Alain Schneble <a.s@realize.ch> writes: > >> * test/automated/url-parse-tests.el: add tests covering url-generic-parse-url. >> * test/automated/url-expand-tests.el: add tests covering url-expand-file-name. >> * test/automated/url-parse.el (url-generic-parse-url): keep empty fragment >> information in URL-struct. > > This is apparently a new top-level directory. Are these supposed to be > in this directory, is is this an error? When I created the patch, the new test directory layout (see 75336a2 - Merge branch 'feature/standard-test-location') has not yet been merged to master. I guess that these tests should now reside in test/lisp/url/ if we follow the new convention. Shall I create a new patch or will you just move the files there? Many thanks for looking into my changes. Alain ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-25 22:27 ` Alain Schneble @ 2015-12-25 22:39 ` Lars Ingebrigtsen 2015-12-25 22:48 ` Alain Schneble 2015-12-25 23:44 ` Alain Schneble 0 siblings, 2 replies; 18+ messages in thread From: Lars Ingebrigtsen @ 2015-12-25 22:39 UTC (permalink / raw) To: Alain Schneble; +Cc: 22044 Alain Schneble <a.s@realize.ch> writes: > When I created the patch, the new test directory layout (see 75336a2 - > Merge branch 'feature/standard-test-location') has not yet been merged > to master. I guess that these tests should now reside in test/lisp/url/ > if we follow the new convention. Shall I create a new patch or will you > just move the files there? If you could roll a new patch, that would be easier for me. :-) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-25 22:39 ` Lars Ingebrigtsen @ 2015-12-25 22:48 ` Alain Schneble 2015-12-25 23:44 ` Alain Schneble 1 sibling, 0 replies; 18+ messages in thread From: Alain Schneble @ 2015-12-25 22:48 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: 22044 Lars Ingebrigtsen <larsi@gnus.org> writes: > Alain Schneble <a.s@realize.ch> writes: > >> When I created the patch, the new test directory layout (see 75336a2 - >> Merge branch 'feature/standard-test-location') has not yet been merged >> to master. I guess that these tests should now reside in test/lisp/url/ >> if we follow the new convention. Shall I create a new patch or will you >> just move the files there? > > If you could roll a new patch, that would be easier for me. :-) Of course, I'll do it. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-25 22:39 ` Lars Ingebrigtsen 2015-12-25 22:48 ` Alain Schneble @ 2015-12-25 23:44 ` Alain Schneble 2015-12-25 23:51 ` Lars Ingebrigtsen 1 sibling, 1 reply; 18+ messages in thread From: Alain Schneble @ 2015-12-25 23:44 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: 22044 [-- Attachment #1: Type: text/plain, Size: 514 bytes --] Lars Ingebrigtsen <larsi@gnus.org> writes: > Alain Schneble <a.s@realize.ch> writes: > >> When I created the patch, the new test directory layout (see 75336a2 - >> Merge branch 'feature/standard-test-location') has not yet been merged >> to master. I guess that these tests should now reside in test/lisp/url/ >> if we follow the new convention. Shall I create a new patch or will you >> just move the files there? > > If you could roll a new patch, that would be easier for me. :-) Here is the adapted patch. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: Make relative URL parsing and resolution consistent with RFC 3986 (bug#22044) --] [-- Type: text/x-patch, Size: 29607 bytes --] From 6f676f50f08b398c5de57126af8c67f1f0501137 Mon Sep 17 00:00:00 2001 From: Alain Schneble <a.s@realize.ch> Date: Sat, 26 Dec 2015 00:25:56 +0100 Subject: [PATCH] Make relative URL parsing and resolution consistent with RFC 3986 (bug#22044) * test/lisp/url/url-parse-tests.el: add tests covering url-generic-parse-url. * test/lisp/url/url-expand-tests.el: add tests covering url-expand-file-name. * lisp/url/url-parse.el (url-generic-parse-url): keep empty fragment information in URL-struct. * lisp/url/url-parse.el (url-path-and-query): do not artificially turn empty path and query into nil path and query, respectively. * lisp/url/url-expand.el (url-expander-remove-relative-links): do not turn empty path into an absolute ("/") path. * lisp/url/url-expand.el (url-expand-file-name): properly resolve fragment-only URIs. Do not just return them unchanged. * lisp/url/url-expand.el (url-default-expander): an empty path in the relative reference URI should not drop the last segment. --- lisp/url/url-expand.el | 84 +++++++++---------- lisp/url/url-parse.el | 5 +- test/lisp/url/url-expand-tests.el | 105 ++++++++++++++++++++++++ test/lisp/url/url-parse-tests.el | 167 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 313 insertions(+), 48 deletions(-) create mode 100644 test/lisp/url/url-expand-tests.el create mode 100644 test/lisp/url/url-parse-tests.el diff --git a/lisp/url/url-expand.el b/lisp/url/url-expand.el index c468a79..600a36d 100644 --- a/lisp/url/url-expand.el +++ b/lisp/url/url-expand.el @@ -26,32 +26,35 @@ (require 'url-parse) (defun url-expander-remove-relative-links (name) - ;; Strip . and .. from pathnames - (let ((new (if (not (string-match "^/" name)) - (concat "/" name) - name))) - - ;; If it ends with a '/.' or '/..', tack on a trailing '/' sot hat - ;; the tests that follow are not too complicated in terms of - ;; looking for '..' or '../', etc. - (if (string-match "/\\.+$" new) - (setq new (concat new "/"))) - - ;; Remove '/./' first - (while (string-match "/\\(\\./\\)" new) - (setq new (concat (substring new 0 (match-beginning 1)) - (substring new (match-end 1))))) - - ;; Then remove '/../' - (while (string-match "/\\([^/]*/\\.\\./\\)" new) - (setq new (concat (substring new 0 (match-beginning 1)) - (substring new (match-end 1))))) - - ;; Remove cruft at the beginning of the string, so people that put - ;; in extraneous '..' because they are morons won't lose. - (while (string-match "^/\\.\\.\\(/\\)" new) - (setq new (substring new (match-beginning 1) nil))) - new)) + (if (equal name "") + ;; An empty name is a properly valid relative URL reference/path. + "" + ;; Strip . and .. from pathnames + (let ((new (if (not (string-match "^/" name)) + (concat "/" name) + name))) + + ;; If it ends with a '/.' or '/..', tack on a trailing '/' sot hat + ;; the tests that follow are not too complicated in terms of + ;; looking for '..' or '../', etc. + (if (string-match "/\\.+$" new) + (setq new (concat new "/"))) + + ;; Remove '/./' first + (while (string-match "/\\(\\./\\)" new) + (setq new (concat (substring new 0 (match-beginning 1)) + (substring new (match-end 1))))) + + ;; Then remove '/../' + (while (string-match "/\\([^/]*/\\.\\./\\)" new) + (setq new (concat (substring new 0 (match-beginning 1)) + (substring new (match-end 1))))) + + ;; Remove cruft at the beginning of the string, so people that put + ;; in extraneous '..' because they are morons won't lose. + (while (string-match "^/\\.\\.\\(/\\)" new) + (setq new (substring new (match-beginning 1) nil))) + new))) (defun url-expand-file-name (url &optional default) "Convert URL to a fully specified URL, and canonicalize it. @@ -89,8 +92,6 @@ path components followed by `..' are removed, along with the `..' itself." (cond ((= (length url) 0) ; nil or empty string (url-recreate-url default)) - ((string-match "^#" url) ; Offset link, use it raw - url) ((string-match url-nonrelative-link url) ; Fully-qualified URL, return it immediately url) (t @@ -120,29 +121,24 @@ path components followed by `..' are removed, along with the `..' itself." (setf (url-host urlobj) (or (url-host urlobj) (url-host defobj)))) (if (string= "ftp" (url-type urlobj)) (setf (url-user urlobj) (or (url-user urlobj) (url-user defobj)))) - (if (string= (url-filename urlobj) "") - (setf (url-filename urlobj) "/")) ;; If the object we're expanding from is full, then we are now ;; full. (unless (url-fullness urlobj) (setf (url-fullness urlobj) (url-fullness defobj))) - (if (string-match "^/" (url-filename urlobj)) - nil - (let ((query nil) - (file nil) - (sepchar nil)) - (if (string-match "[?#]" (url-filename urlobj)) - (setq query (substring (url-filename urlobj) (match-end 0)) - file (substring (url-filename urlobj) 0 (match-beginning 0)) - sepchar (substring (url-filename urlobj) (match-beginning 0) (match-end 0))) - (setq file (url-filename urlobj))) + (let* ((pathandquery (url-path-and-query urlobj)) + (defpathandquery (url-path-and-query defobj)) + (file (car pathandquery)) + (query (or (cdr pathandquery) (and (equal file "") (cdr defpathandquery))))) + (if (string-match "^/" (url-filename urlobj)) + (setq file (url-expander-remove-relative-links file)) ;; We use concat rather than expand-file-name to combine ;; directory and file name, since urls do not follow the same ;; rules as local files on all platforms. - (setq file (url-expander-remove-relative-links - (concat (url-file-directory (url-filename defobj)) file))) - (setf (url-filename urlobj) - (if query (concat file sepchar query) file)))))) + (setq file (url-expander-remove-relative-links + (if (equal file "") + (or (car (url-path-and-query defobj)) "") + (concat (url-file-directory (url-filename defobj)) file))))) + (setf (url-filename urlobj) (if query (concat file "?" query) file))))) (provide 'url-expand) diff --git a/lisp/url/url-parse.el b/lisp/url/url-parse.el index dbf0c38..c3159a7 100644 --- a/lisp/url/url-parse.el +++ b/lisp/url/url-parse.el @@ -59,8 +59,6 @@ where each of PATH and QUERY are strings or nil." (setq path (substring name 0 (match-beginning 0)) query (substring name (match-end 0))) (setq path name))) - (if (equal path "") (setq path nil)) - (if (equal query "") (setq query nil)) (cons path query))) (defun url-port-if-non-default (urlobj) @@ -217,8 +215,7 @@ parses to (when (looking-at "#") (let ((opoint (point))) (forward-char 1) - (unless (eobp) - (setq fragment (buffer-substring (point) (point-max)))) + (setq fragment (buffer-substring (point) (point-max))) (delete-region opoint (point-max))))) (if (and host (string-match "%[0-9][0-9]" host)) diff --git a/test/lisp/url/url-expand-tests.el b/test/lisp/url/url-expand-tests.el new file mode 100644 index 0000000..2bd2868 --- /dev/null +++ b/test/lisp/url/url-expand-tests.el @@ -0,0 +1,105 @@ +;;; url-expand-tests.el --- Test suite for relative URI/URL resolution. + +;; Copyright (C) 2012-2015 Free Software Foundation, Inc. + +;; Author: Alain Schneble <a.s@realize.ch> +;; Version: 1.0 + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>. + +;;; Commentary: + +;; Test cases covering URI reference resolution as described in RFC3986, +;; section 5. Reference Resolution and especially the relative resolution +;; rules specified in section 5.2. Relative Resolution. + +;; Each test calls `url-expand-file-name', typically with a relative +;; reference URI and a base URI as string and compares the result (Actual) +;; against a manually specified URI (Expected) + +;;; Code: + +(require 'url-expand) +(require 'ert) + +(ert-deftest url-expand-file-name/relative-resolution-normal-examples () + "RFC 3986, Section 5.4 Reference Resolution Examples / Section 5.4.1. Normal Examples" + (should (equal (url-expand-file-name "g:h" "http://a/b/c/d;p?q") "g:h")) + (should (equal (url-expand-file-name "g" "http://a/b/c/d;p?q") "http://a/b/c/g")) + (should (equal (url-expand-file-name "./g" "http://a/b/c/d;p?q") "http://a/b/c/g")) + (should (equal (url-expand-file-name "g/" "http://a/b/c/d;p?q") "http://a/b/c/g/")) + (should (equal (url-expand-file-name "/g" "http://a/b/c/d;p?q") "http://a/g")) + (should (equal (url-expand-file-name "//g" "http://a/b/c/d;p?q") "http://g")) + (should (equal (url-expand-file-name "?y" "http://a/b/c/d;p?q") "http://a/b/c/d;p?y")) + (should (equal (url-expand-file-name "g?y" "http://a/b/c/d;p?q") "http://a/b/c/g?y")) + (should (equal (url-expand-file-name "#s" "http://a/b/c/d;p?q") "http://a/b/c/d;p?q#s")) + (should (equal (url-expand-file-name "g#s" "http://a/b/c/d;p?q") "http://a/b/c/g#s")) + (should (equal (url-expand-file-name "g?y#s" "http://a/b/c/d;p?q") "http://a/b/c/g?y#s")) + (should (equal (url-expand-file-name ";x" "http://a/b/c/d;p?q") "http://a/b/c/;x")) + (should (equal (url-expand-file-name "g;x" "http://a/b/c/d;p?q") "http://a/b/c/g;x")) + (should (equal (url-expand-file-name "g;x?y#s" "http://a/b/c/d;p?q") "http://a/b/c/g;x?y#s")) + (should (equal (url-expand-file-name "" "http://a/b/c/d;p?q") "http://a/b/c/d;p?q")) + (should (equal (url-expand-file-name "." "http://a/b/c/d;p?q") "http://a/b/c/")) + (should (equal (url-expand-file-name "./" "http://a/b/c/d;p?q") "http://a/b/c/")) + (should (equal (url-expand-file-name ".." "http://a/b/c/d;p?q") "http://a/b/")) + (should (equal (url-expand-file-name "../" "http://a/b/c/d;p?q") "http://a/b/")) + (should (equal (url-expand-file-name "../g" "http://a/b/c/d;p?q") "http://a/b/g")) + (should (equal (url-expand-file-name "../.." "http://a/b/c/d;p?q") "http://a/")) + (should (equal (url-expand-file-name "../../" "http://a/b/c/d;p?q") "http://a/")) + (should (equal (url-expand-file-name "../../g" "http://a/b/c/d;p?q") "http://a/g"))) + +(ert-deftest url-expand-file-name/relative-resolution-absolute-examples () + "RFC 3986, Section 5.4 Reference Resolution Examples / Section 5.4.2. Abnormal Examples" + (should (equal (url-expand-file-name "../../../g" "http://a/b/c/d;p?q") "http://a/g")) + (should (equal (url-expand-file-name "../../../../g" "http://a/b/c/d;p?q") "http://a/g")) + + (should (equal (url-expand-file-name "/./g" "http://a/b/c/d;p?q") "http://a/g")) + (should (equal (url-expand-file-name "/../g" "http://a/b/c/d;p?q") "http://a/g")) + (should (equal (url-expand-file-name "g." "http://a/b/c/d;p?q") "http://a/b/c/g.")) + (should (equal (url-expand-file-name ".g" "http://a/b/c/d;p?q") "http://a/b/c/.g")) + (should (equal (url-expand-file-name "g.." "http://a/b/c/d;p?q") "http://a/b/c/g..")) + (should (equal (url-expand-file-name "..g" "http://a/b/c/d;p?q") "http://a/b/c/..g")) + + (should (equal (url-expand-file-name "./../g" "http://a/b/c/d;p?q") "http://a/b/g")) + (should (equal (url-expand-file-name "./g/." "http://a/b/c/d;p?q") "http://a/b/c/g/")) + (should (equal (url-expand-file-name "g/./h" "http://a/b/c/d;p?q") "http://a/b/c/g/h")) + (should (equal (url-expand-file-name "g/../h" "http://a/b/c/d;p?q") "http://a/b/c/h")) + (should (equal (url-expand-file-name "g;x=1/./y" "http://a/b/c/d;p?q") "http://a/b/c/g;x=1/y")) + (should (equal (url-expand-file-name "g;x=1/../y" "http://a/b/c/d;p?q") "http://a/b/c/y")) + + (should (equal (url-expand-file-name "g?y/./x" "http://a/b/c/d;p?q") "http://a/b/c/g?y/./x")) + (should (equal (url-expand-file-name "g?y/../x" "http://a/b/c/d;p?q") "http://a/b/c/g?y/../x")) + (should (equal (url-expand-file-name "g#s/./x" "http://a/b/c/d;p?q") "http://a/b/c/g#s/./x")) + (should (equal (url-expand-file-name "g#s/../x" "http://a/b/c/d;p?q") "http://a/b/c/g#s/../x")) + + (should (equal (url-expand-file-name "http:g" "http://a/b/c/d;p?q") "http:g")) ; for strict parsers + ) + +(ert-deftest url-expand-file-name/relative-resolution-additional-examples () + "Reference Resolution Examples / Arbitrary Examples" + (should (equal (url-expand-file-name "" "http://host/foobar") "http://host/foobar")) + (should (equal (url-expand-file-name "?y" "http://a/b/c/d") "http://a/b/c/d?y")) + (should (equal (url-expand-file-name "?y" "http://a/b/c/d/") "http://a/b/c/d/?y")) + (should (equal (url-expand-file-name "?y#fragment" "http://a/b/c/d;p?q") "http://a/b/c/d;p?y#fragment")) + (should (equal (url-expand-file-name "#bar" "http://host") "http://host#bar")) + (should (equal (url-expand-file-name "#bar" "http://host/") "http://host/#bar")) + (should (equal (url-expand-file-name "#bar" "http://host/foo") "http://host/foo#bar")) + (should (equal (url-expand-file-name "foo#bar" "http://host/foobar") "http://host/foo#bar")) + (should (equal (url-expand-file-name "foo#bar" "http://host/foobar/") "http://host/foobar/foo#bar"))) + +(provide 'url-expand-tests) + +;;; url-expand-tests.el ends here diff --git a/test/lisp/url/url-parse-tests.el b/test/lisp/url/url-parse-tests.el new file mode 100644 index 0000000..154e091 --- /dev/null +++ b/test/lisp/url/url-parse-tests.el @@ -0,0 +1,167 @@ +;;; url-parse-tests.el --- Test suite for URI/URL parsing. + +;; Copyright (C) 2012-2015 Free Software Foundation, Inc. + +;; Author: Alain Schneble <a.s@realize.ch> +;; Version: 1.0 + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>. + +;;; Commentary: + +;; Test cases covering generic URI syntax as described in RFC3986, +;; section 3. Syntax Components and 4. Usage. See also appendix +;; A. Collected ABNF for URI, as the example given here are all +;; productions of this grammar. + +;; Each tests parses a given URI string - whether relative or absolute - +;; using `url-generic-parse-url' and compares the constructed +;; URL-struct (Actual) against a manually `url-parse-make-urlobj'- +;; constructed URL-struct (Expected). + +;;; Code: + +(require 'url-parse) +(require 'ert) + +(ert-deftest url-generic-parse-url/generic-uri-examples () + "RFC 3986, section 1.1.2. Examples / Example illustrating several URI schemes and variations in their common syntax components" + (should (equal (url-generic-parse-url "ftp://ftp.is.co.za/rfc/rfc1808.txt") (url-parse-make-urlobj "ftp" nil nil "ftp.is.co.za" nil "/rfc/rfc1808.txt" nil nil t))) + (should (equal (url-generic-parse-url "http://www.ietf.org/rfc/rfc2396.txt") (url-parse-make-urlobj "http" nil nil "www.ietf.org" nil "/rfc/rfc2396.txt" nil nil t))) + (should (equal (url-generic-parse-url "ldap://[2001:db8::7]/c=GB?objectClass?one") (url-parse-make-urlobj "ldap" nil nil "[2001:db8::7]" nil "/c=GB?objectClass?one" nil nil t))) + (should (equal (url-generic-parse-url "mailto:John.Doe@example.com") (url-parse-make-urlobj "mailto" nil nil nil nil "John.Doe@example.com" nil nil nil))) + (should (equal (url-generic-parse-url "news:comp.infosystems.www.servers.unix") (url-parse-make-urlobj "news" nil nil nil nil "comp.infosystems.www.servers.unix" nil nil nil))) + (should (equal (url-generic-parse-url "tel:+1-816-555-1212") (url-parse-make-urlobj "tel" nil nil nil nil "+1-816-555-1212" nil nil nil))) + (should (equal (url-generic-parse-url "telnet://192.0.2.16:80/") (url-parse-make-urlobj "telnet" nil nil "192.0.2.16" 80 "/" nil nil t))) + (should (equal (url-generic-parse-url "urn:oasis:names:specification:docbook:dtd:xml:4.1.2") (url-parse-make-urlobj "urn" nil nil nil nil "oasis:names:specification:docbook:dtd:xml:4.1.2" nil nil nil)))) + +(ert-deftest url-generic-parse-url/generic-uri () + "RFC 3986, section 3. Syntax Components / generic URI syntax" + ;; empty path + (should (equal (url-generic-parse-url "http://host#") (url-parse-make-urlobj "http" nil nil "host" nil "" "" nil t))) + (should (equal (url-generic-parse-url "http://host#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host?#") (url-parse-make-urlobj "http" nil nil "host" nil "?" "" nil t))) + (should (equal (url-generic-parse-url "http://host?query#") (url-parse-make-urlobj "http" nil nil "host" nil "?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "?query" "fragment" nil t))) + ;; absolute path / + (should (equal (url-generic-parse-url "http://host/#") (url-parse-make-urlobj "http" nil nil "host" nil "/" "" nil t))) + (should (equal (url-generic-parse-url "http://host/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/?" "" nil t))) + (should (equal (url-generic-parse-url "http://host/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" "fragment" nil t))) + ;; absolute path /foo + (should (equal (url-generic-parse-url "http://host/foo#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" "fragment" nil t))) + ;; absolute path /foo/ + (should (equal (url-generic-parse-url "http://host/foo/#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" "fragment" nil t))) + ;; absolute path /foo/bar + (should (equal (url-generic-parse-url "http://host/foo/bar#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" "fragment" nil t))) + ;; absolute path /foo/bar/ + (should (equal (url-generic-parse-url "http://host/foo/bar/#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" "" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" "fragment" nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" "fragment" nil t))) + ;; for more examples of URIs without fragments, see tests covering section 4.3. Absolute URI + ) + +(ert-deftest url-generic-parse-url/network-path-reference () + "RFC 3986, section 4.2. Relative Reference / network-path reference: a relative reference that begins with two slash characters" + (should (equal (url-generic-parse-url "//host") (url-parse-make-urlobj nil nil nil "host" nil "" nil nil t))) + (should (equal (url-generic-parse-url "//host/") (url-parse-make-urlobj nil nil nil "host" nil "/" nil nil t))) + (should (equal (url-generic-parse-url "//host/foo") (url-parse-make-urlobj nil nil nil "host" nil "/foo" nil nil t))) + (should (equal (url-generic-parse-url "//host/foo/bar") (url-parse-make-urlobj nil nil nil "host" nil "/foo/bar" nil nil t))) + (should (equal (url-generic-parse-url "//host/foo/bar/") (url-parse-make-urlobj nil nil nil "host" nil "/foo/bar/" nil nil t)))) + +(ert-deftest url-generic-parse-url/absolute-path-reference () + "RFC 3986, section 4.2. Relative Reference / absolute-path reference: a relative reference that begins with a single slash character" + (should (equal (url-generic-parse-url "/") (url-parse-make-urlobj nil nil nil nil nil "/" nil nil nil))) + (should (equal (url-generic-parse-url "/foo") (url-parse-make-urlobj nil nil nil nil nil "/foo" nil nil nil))) + (should (equal (url-generic-parse-url "/foo/bar") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar" nil nil nil))) + (should (equal (url-generic-parse-url "/foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar/" nil nil nil))) + (should (equal (url-generic-parse-url "/foo/bar#") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar" "" nil nil))) + (should (equal (url-generic-parse-url "/foo/bar/#") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar/" "" nil nil)))) + +(ert-deftest url-generic-parse-url/relative-path-reference () + "RFC 3986, section 4.2. Relative Reference / relative-path reference: a relative reference that does not begin with a slash character" + (should (equal (url-generic-parse-url "foo") (url-parse-make-urlobj nil nil nil nil nil "foo" nil nil nil))) + (should (equal (url-generic-parse-url "foo/bar") (url-parse-make-urlobj nil nil nil nil nil "foo/bar" nil nil nil))) + (should (equal (url-generic-parse-url "foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "foo/bar/" nil nil nil))) + (should (equal (url-generic-parse-url "./foo") (url-parse-make-urlobj nil nil nil nil nil "./foo" nil nil nil))) + (should (equal (url-generic-parse-url "./foo/bar") (url-parse-make-urlobj nil nil nil nil nil "./foo/bar" nil nil nil))) + (should (equal (url-generic-parse-url "./foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "./foo/bar/" nil nil nil))) + (should (equal (url-generic-parse-url "../foo") (url-parse-make-urlobj nil nil nil nil nil "../foo" nil nil nil))) + (should (equal (url-generic-parse-url "../foo/bar") (url-parse-make-urlobj nil nil nil nil nil "../foo/bar" nil nil nil))) + (should (equal (url-generic-parse-url "../foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "../foo/bar/" nil nil nil))) + (should (equal (url-generic-parse-url "./this:that") (url-parse-make-urlobj nil nil nil nil nil "./this:that" nil nil nil))) + ;; for more examples of relative-path references, see tests covering section 4.4. Same-Document Reference + ) + +(ert-deftest url-generic-parse-url/absolute-uri () + "RFC 3986, section 4.3. Absolute URI / absolute URI: absolute form of a URI without a fragment identifier" + ;; empty path + (should (equal (url-generic-parse-url "http://host") (url-parse-make-urlobj "http" nil nil "host" nil "" nil nil t))) + (should (equal (url-generic-parse-url "http://host?") (url-parse-make-urlobj "http" nil nil "host" nil "?" nil nil t))) + (should (equal (url-generic-parse-url "http://host?query") (url-parse-make-urlobj "http" nil nil "host" nil "?query" nil nil t))) + ;; absolute path / + (should (equal (url-generic-parse-url "http://host/") (url-parse-make-urlobj "http" nil nil "host" nil "/" nil nil t))) + (should (equal (url-generic-parse-url "http://host/?") (url-parse-make-urlobj "http" nil nil "host" nil "/?" nil nil t))) + (should (equal (url-generic-parse-url "http://host/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" nil nil t))) + ;; absolute path /foo + (should (equal (url-generic-parse-url "http://host/foo") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" nil nil t))) + ;; absolute path /foo/ + (should (equal (url-generic-parse-url "http://host/foo/") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" nil nil t))) + ;; absolute path /foo/bar + (should (equal (url-generic-parse-url "http://host/foo/bar") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" nil nil t))) + ;; absolute path /foo/bar/ + (should (equal (url-generic-parse-url "http://host/foo/bar/") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" nil nil t))) + (should (equal (url-generic-parse-url "http://host/foo/bar/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" nil nil t))) + ;; example mentioned in RFC3986, section 5.4. Reference Resolution Examples + (should (equal (url-generic-parse-url "http://a/b/c/d;p?q") (url-parse-make-urlobj "http" nil nil "a" nil "/b/c/d;p?q" nil nil t)))) + +(ert-deftest url-generic-parse-url/same-decument-reference () + "RFC 3986, section 4.4. Same-Document Reference / same-document reference: empty or number sign (\"#\") followed by a fragment identifier" + (should (equal (url-generic-parse-url "") (url-parse-make-urlobj nil nil nil nil nil "" nil nil nil))) + (should (equal (url-generic-parse-url "#") (url-parse-make-urlobj nil nil nil nil nil "" "" nil nil))) + (should (equal (url-generic-parse-url "#foo") (url-parse-make-urlobj nil nil nil nil nil "" "foo" nil nil)))) + +(provide 'url-parse-tests) + +;;; url-parse-tests.el ends here -- 2.6.2.windows.1 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-25 23:44 ` Alain Schneble @ 2015-12-25 23:51 ` Lars Ingebrigtsen 2015-12-26 0:23 ` Alain Schneble 2015-12-26 9:57 ` Eli Zaretskii 0 siblings, 2 replies; 18+ messages in thread From: Lars Ingebrigtsen @ 2015-12-25 23:51 UTC (permalink / raw) To: Alain Schneble; +Cc: 22044 Alain Schneble <a.s@realize.ch> writes: > Here is the adapted patch. Thanks; applied and pushed (to trunk). It might make sense to backport to Emacs 25.1 -- do you think that's a good idea? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-25 23:51 ` Lars Ingebrigtsen @ 2015-12-26 0:23 ` Alain Schneble 2015-12-26 0:51 ` John Wiegley 2015-12-26 1:06 ` Lars Ingebrigtsen 2015-12-26 9:57 ` Eli Zaretskii 1 sibling, 2 replies; 18+ messages in thread From: Alain Schneble @ 2015-12-26 0:23 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: 22044 Lars Ingebrigtsen <larsi@gnus.org> writes: > Alain Schneble <a.s@realize.ch> writes: > >> Here is the adapted patch. > > Thanks; applied and pushed (to trunk). It might make sense to backport > to Emacs 25.1 -- do you think that's a good idea? Thanks, great! I see nothing against backporting it to Emacs 25.1. But I must be honest, I'm pretty new to Emacs and its hard for me to judge the risks of doing so. But with all the new tests in place, I don't expect it breaks anything fundamentally; the changes were mainly related to how fragments and relative URLs are handled. These are more edge cases and I guess it would also only affect some rare edge cases - if at all. Shall I to elaborate more on this? What do you think? By the way, the 25.1 branch still uses the old test directory layout... ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-26 0:23 ` Alain Schneble @ 2015-12-26 0:51 ` John Wiegley 2015-12-26 10:51 ` Alain Schneble 2015-12-26 1:06 ` Lars Ingebrigtsen 1 sibling, 1 reply; 18+ messages in thread From: John Wiegley @ 2015-12-26 0:51 UTC (permalink / raw) To: Alain Schneble; +Cc: Lars Ingebrigtsen, 22044 >>>>> Alain Schneble <a.s@realize.ch> writes: > Thanks, great! I see nothing against backporting it to Emacs 25.1. But I > must be honest, I'm pretty new to Emacs and its hard for me to judge the > risks of doing so. At this point mainly ask yourself: 1. Would the change require new, or change existing, documentation or APIs? If so, it should go to master. 2. If it corrects existing behavior, refines that behavior, or removes a stumbling block, it should go to emacs-25. The length of time it takes to get emacs-25 fully ready should not be seen as an opening for "features that could get ready during that time". I haven't seen any attempts to commit such features, I just wanted to make this one point clear. -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2 ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-26 0:51 ` John Wiegley @ 2015-12-26 10:51 ` Alain Schneble 0 siblings, 0 replies; 18+ messages in thread From: Alain Schneble @ 2015-12-26 10:51 UTC (permalink / raw) To: John Wiegley; +Cc: Lars Ingebrigtsen, 22044 John Wiegley <jwiegley@gmail.com> writes: >>>>>> Alain Schneble <a.s@realize.ch> writes: > >> Thanks, great! I see nothing against backporting it to Emacs 25.1. But I >> must be honest, I'm pretty new to Emacs and its hard for me to judge the >> risks of doing so. > > At this point mainly ask yourself: > > 1. Would the change require new, or change existing, documentation or APIs? > If so, it should go to master. The bugfix does not change any public interface. And the documented behavior is expected to remain the same. > 2. If it corrects existing behavior, refines that behavior, or removes a > stumbling block, it should go to emacs-25. It clearly corrects existing behavior and adds a bunch of new test cases. From that point of view, it shoud go to emacs-25. > The length of time it takes to get emacs-25 fully ready should not be seen as > an opening for "features that could get ready during that time". I haven't > seen any attempts to commit such features, I just wanted to make this one > point clear. I see your point - thanks for clarifying this. But as stated above, I don't think it falls into this category of changes. Thanks! Alain ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-26 0:23 ` Alain Schneble 2015-12-26 0:51 ` John Wiegley @ 2015-12-26 1:06 ` Lars Ingebrigtsen 2015-12-26 10:12 ` Eli Zaretskii 1 sibling, 1 reply; 18+ messages in thread From: Lars Ingebrigtsen @ 2015-12-26 1:06 UTC (permalink / raw) To: Alain Schneble; +Cc: 22044 Alain Schneble <a.s@realize.ch> writes: > Thanks, great! I see nothing against backporting it to Emacs 25.1. But > I must be honest, I'm pretty new to Emacs and its hard for me to judge > the risks of doing so. But with all the new tests in place, I don't > expect it breaks anything fundamentally; the changes were mainly related > to how fragments and relative URLs are handled. These are more edge > cases and I guess it would also only affect some rare edge cases - if at > all. > > Shall I to elaborate more on this? What do you think? I think it should probably be backported, but: > By the way, the 25.1 branch still uses the old test directory layout... How does one do that, then? :-) If somebody feels like doing it, the two commits are b792ecea1715e080ad8e232d3d154b8a25d2edfb 1dee11d874de5ff3d5634e1629054c4398b27b72 It's two commits because I hit the wrong buttons in vc-mode. :-/ -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-26 1:06 ` Lars Ingebrigtsen @ 2015-12-26 10:12 ` Eli Zaretskii 2015-12-26 13:53 ` Lars Ingebrigtsen 0 siblings, 1 reply; 18+ messages in thread From: Eli Zaretskii @ 2015-12-26 10:12 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: a.s, 22044 > From: Lars Ingebrigtsen <larsi@gnus.org> > Date: Sat, 26 Dec 2015 02:06:20 +0100 > Cc: 22044@debbugs.gnu.org > > > By the way, the 25.1 branch still uses the old test directory layout... > > How does one do that, then? :-) I'd simply apply the test suite changes by hand using "M-|". Please don't forget including "backport" in the commit log message. Thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-26 10:12 ` Eli Zaretskii @ 2015-12-26 13:53 ` Lars Ingebrigtsen 0 siblings, 0 replies; 18+ messages in thread From: Lars Ingebrigtsen @ 2015-12-26 13:53 UTC (permalink / raw) To: Eli Zaretskii; +Cc: a.s, 22044 Eli Zaretskii <eliz@gnu.org> writes: > I'd simply apply the test suite changes by hand using "M-|". > > Please don't forget including "backport" in the commit log message. Aargh. Guess what I forgot. :-/ -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 2015-12-25 23:51 ` Lars Ingebrigtsen 2015-12-26 0:23 ` Alain Schneble @ 2015-12-26 9:57 ` Eli Zaretskii 1 sibling, 0 replies; 18+ messages in thread From: Eli Zaretskii @ 2015-12-26 9:57 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: a.s, 22044 > From: Lars Ingebrigtsen <larsi@gnus.org> > Date: Sat, 26 Dec 2015 00:51:14 +0100 > Cc: 22044@debbugs.gnu.org > > Thanks; applied and pushed (to trunk). It might make sense to backport > to Emacs 25.1 -- do you think that's a good idea? Yes, I think so. Thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2015-12-26 13:53 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-11-28 21:53 bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 Alain Schneble 2015-11-28 23:42 ` Alain Schneble 2015-12-25 18:01 ` Lars Ingebrigtsen 2015-12-25 19:45 ` Eli Zaretskii 2015-12-25 19:54 ` Lars Ingebrigtsen 2015-12-25 19:58 ` Lars Ingebrigtsen 2015-12-25 22:27 ` Alain Schneble 2015-12-25 22:39 ` Lars Ingebrigtsen 2015-12-25 22:48 ` Alain Schneble 2015-12-25 23:44 ` Alain Schneble 2015-12-25 23:51 ` Lars Ingebrigtsen 2015-12-26 0:23 ` Alain Schneble 2015-12-26 0:51 ` John Wiegley 2015-12-26 10:51 ` Alain Schneble 2015-12-26 1:06 ` Lars Ingebrigtsen 2015-12-26 10:12 ` Eli Zaretskii 2015-12-26 13:53 ` Lars Ingebrigtsen 2015-12-26 9:57 ` Eli Zaretskii
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).