* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
@ 2015-11-28 21:53 Alain Schneble
2015-11-28 23:42 ` Alain Schneble
0 siblings, 1 reply; 18+ messages in thread
From: Alain Schneble @ 2015-11-28 21:53 UTC (permalink / raw)
To: 22044
url-expand.el and url-parse.el seem to not conform to RFC 3986 "Uniform
Resource Identifier (URI): Generic Syntax" in some cases. But I assume
they should. Here is a list of issues found in url-expand-file-name
and url-generic-parse-url, respectively:
1. resolving relative "fragment-only" URIs against a given absolute
base URI (see RFC3986, section 5. Reference Resolution, and
especially 5.2.2. Transform References):
(url-expand-file-name "#s" "http://a/b/c/d;p?q")
=> "#s" but should be http://a/b/c/d;p?q#s"
(url-expand-file-name "#bar" "http://host")
=> "#bar" but should be "http://host#bar"
(url-expand-file-name "#bar" "http://host/")
=> "#bar" but should be "http://host/#bar"
(url-expand-file-name "#bar" "http://host/foo")
=> "#bar" but should be "http://host/foo#bar"
2. resolving relative "query-only" URIs against a given absolute base
URI (see RFC3986, same sections as mentioned in point 1.):
(url-expand-file-name "?y" "http://a/b/c/d;p?q")
=> "http://a/b/c/?y" but should be "http://a/b/c/d;p?y"
(url-expand-file-name "?y" "http://a/b/c/d")
=> "http://a/b/c/?y" but should be "http://a/b/c/d?y")
3. removing dot segments (see RFC3986, section 5.2.4. Remove Dot
Segments):
(url-expand-file-name "/./g" "http://a/b/c/d;p?q")
=> "http://a/./g" but should be "http://a/g"
(url-expand-file-name "/../g" "http://a/b/c/d;p?q")
=> "http://a/../g" but should be "http://a/g"
4. empty fragment information is lost after parsing URI:
(equal (url-generic-parse-url "#")
(url-parse-make-urlobj nil nil nil nil nil "" "" nil nil))
^
=> nil but should be t (fragment component is actually nil instead
of an empty string)
Same issue with URLs having a number sign (#) as suffix:
"/foo/bar#"
"/foo/bar/#"
"http://host#"
"http://host?#"
"http://host?query#"
"http://host/#"
"http://host/?#"
"http://host/?query#"
"http://host/foo#"
"http://host/foo?#"
"http://host/foo?query#"
... and so forth
With the current behavior, the inverse function url-recreate-url
won't be able to reconstruct exactly the same URI. For example:
(url-recreate-url (url-generic-parse-url "#"))
=> "" but should be "#"
Alain
In GNU Emacs 24.4.1 (i686-pc-mingw32)
of 2014-10-24 on LEG570
Windowing system distributor `Microsoft Corp.', version 6.3.9600
Configured using:
`configure --prefix=/c/usr'
Important settings:
value of $LANG: DES
locale-coding-system: cp1252
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-11-28 21:53 bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 Alain Schneble
@ 2015-11-28 23:42 ` Alain Schneble
2015-12-25 18:01 ` Lars Ingebrigtsen
2015-12-25 19:58 ` Lars Ingebrigtsen
0 siblings, 2 replies; 18+ messages in thread
From: Alain Schneble @ 2015-11-28 23:42 UTC (permalink / raw)
To: 22044
[-- Attachment #1: Type: text/plain, Size: 179 bytes --]
Tags: patch
To address these issues and make the behavior consistent with RFC 3984,
I propose the following changes to url-parse.el and url-expand.el, see
attached patch.
Alain
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-patch, Size: 29649 bytes --]
From 5b6e6db3c31be8ed63f36b9f82e66610c86f6cb0 Mon Sep 17 00:00:00 2001
From: Alain Nicolas Schneble <a.s@realize.ch>
Date: Fri, 27 Nov 2015 15:54:12 +0100
Subject: [PATCH] Make relative URL parsing and resolution consistent with RFC
3986 (bug#22044)
* test/automated/url-parse-tests.el: add tests covering url-generic-parse-url.
* test/automated/url-expand-tests.el: add tests covering url-expand-file-name.
* test/automated/url-parse.el (url-generic-parse-url): keep empty fragment
information in URL-struct.
* lisp/url/url-parse.el (url-path-and-query): do not artificially turn empty
path and query into nil path and query, respectively.
* lisp/url/url-expand.el (url-expander-remove-relative-links): do not turn
empty path into an absolute ("/") path.
* lisp/url/url-expand.el (url-expand-file-name): properly resolve
fragment-only URIs. Do not just return them unchanged.
* lisp/url/url-expand.el (url-default-expander): an empty path in the relative
reference URI should not drop the last segment.
---
lisp/url/url-expand.el | 84 +++++++++----------
lisp/url/url-parse.el | 5 +-
test/automated/url-expand-tests.el | 105 +++++++++++++++++++++++
test/automated/url-parse-tests.el | 167 +++++++++++++++++++++++++++++++++++++
4 files changed, 313 insertions(+), 48 deletions(-)
create mode 100644 test/automated/url-expand-tests.el
create mode 100644 test/automated/url-parse-tests.el
diff --git a/lisp/url/url-expand.el b/lisp/url/url-expand.el
index c468a79..600a36d 100644
--- a/lisp/url/url-expand.el
+++ b/lisp/url/url-expand.el
@@ -26,32 +26,35 @@
(require 'url-parse)
(defun url-expander-remove-relative-links (name)
- ;; Strip . and .. from pathnames
- (let ((new (if (not (string-match "^/" name))
- (concat "/" name)
- name)))
-
- ;; If it ends with a '/.' or '/..', tack on a trailing '/' sot hat
- ;; the tests that follow are not too complicated in terms of
- ;; looking for '..' or '../', etc.
- (if (string-match "/\\.+$" new)
- (setq new (concat new "/")))
-
- ;; Remove '/./' first
- (while (string-match "/\\(\\./\\)" new)
- (setq new (concat (substring new 0 (match-beginning 1))
- (substring new (match-end 1)))))
-
- ;; Then remove '/../'
- (while (string-match "/\\([^/]*/\\.\\./\\)" new)
- (setq new (concat (substring new 0 (match-beginning 1))
- (substring new (match-end 1)))))
-
- ;; Remove cruft at the beginning of the string, so people that put
- ;; in extraneous '..' because they are morons won't lose.
- (while (string-match "^/\\.\\.\\(/\\)" new)
- (setq new (substring new (match-beginning 1) nil)))
- new))
+ (if (equal name "")
+ ;; An empty name is a properly valid relative URL reference/path.
+ ""
+ ;; Strip . and .. from pathnames
+ (let ((new (if (not (string-match "^/" name))
+ (concat "/" name)
+ name)))
+
+ ;; If it ends with a '/.' or '/..', tack on a trailing '/' sot hat
+ ;; the tests that follow are not too complicated in terms of
+ ;; looking for '..' or '../', etc.
+ (if (string-match "/\\.+$" new)
+ (setq new (concat new "/")))
+
+ ;; Remove '/./' first
+ (while (string-match "/\\(\\./\\)" new)
+ (setq new (concat (substring new 0 (match-beginning 1))
+ (substring new (match-end 1)))))
+
+ ;; Then remove '/../'
+ (while (string-match "/\\([^/]*/\\.\\./\\)" new)
+ (setq new (concat (substring new 0 (match-beginning 1))
+ (substring new (match-end 1)))))
+
+ ;; Remove cruft at the beginning of the string, so people that put
+ ;; in extraneous '..' because they are morons won't lose.
+ (while (string-match "^/\\.\\.\\(/\\)" new)
+ (setq new (substring new (match-beginning 1) nil)))
+ new)))
(defun url-expand-file-name (url &optional default)
"Convert URL to a fully specified URL, and canonicalize it.
@@ -89,8 +92,6 @@ path components followed by `..' are removed, along with the `..' itself."
(cond
((= (length url) 0) ; nil or empty string
(url-recreate-url default))
- ((string-match "^#" url) ; Offset link, use it raw
- url)
((string-match url-nonrelative-link url) ; Fully-qualified URL, return it immediately
url)
(t
@@ -120,29 +121,24 @@ path components followed by `..' are removed, along with the `..' itself."
(setf (url-host urlobj) (or (url-host urlobj) (url-host defobj))))
(if (string= "ftp" (url-type urlobj))
(setf (url-user urlobj) (or (url-user urlobj) (url-user defobj))))
- (if (string= (url-filename urlobj) "")
- (setf (url-filename urlobj) "/"))
;; If the object we're expanding from is full, then we are now
;; full.
(unless (url-fullness urlobj)
(setf (url-fullness urlobj) (url-fullness defobj)))
- (if (string-match "^/" (url-filename urlobj))
- nil
- (let ((query nil)
- (file nil)
- (sepchar nil))
- (if (string-match "[?#]" (url-filename urlobj))
- (setq query (substring (url-filename urlobj) (match-end 0))
- file (substring (url-filename urlobj) 0 (match-beginning 0))
- sepchar (substring (url-filename urlobj) (match-beginning 0) (match-end 0)))
- (setq file (url-filename urlobj)))
+ (let* ((pathandquery (url-path-and-query urlobj))
+ (defpathandquery (url-path-and-query defobj))
+ (file (car pathandquery))
+ (query (or (cdr pathandquery) (and (equal file "") (cdr defpathandquery)))))
+ (if (string-match "^/" (url-filename urlobj))
+ (setq file (url-expander-remove-relative-links file))
;; We use concat rather than expand-file-name to combine
;; directory and file name, since urls do not follow the same
;; rules as local files on all platforms.
- (setq file (url-expander-remove-relative-links
- (concat (url-file-directory (url-filename defobj)) file)))
- (setf (url-filename urlobj)
- (if query (concat file sepchar query) file))))))
+ (setq file (url-expander-remove-relative-links
+ (if (equal file "")
+ (or (car (url-path-and-query defobj)) "")
+ (concat (url-file-directory (url-filename defobj)) file)))))
+ (setf (url-filename urlobj) (if query (concat file "?" query) file)))))
(provide 'url-expand)
diff --git a/lisp/url/url-parse.el b/lisp/url/url-parse.el
index dbf0c38..c3159a7 100644
--- a/lisp/url/url-parse.el
+++ b/lisp/url/url-parse.el
@@ -59,8 +59,6 @@ where each of PATH and QUERY are strings or nil."
(setq path (substring name 0 (match-beginning 0))
query (substring name (match-end 0)))
(setq path name)))
- (if (equal path "") (setq path nil))
- (if (equal query "") (setq query nil))
(cons path query)))
(defun url-port-if-non-default (urlobj)
@@ -217,8 +215,7 @@ parses to
(when (looking-at "#")
(let ((opoint (point)))
(forward-char 1)
- (unless (eobp)
- (setq fragment (buffer-substring (point) (point-max))))
+ (setq fragment (buffer-substring (point) (point-max)))
(delete-region opoint (point-max)))))
(if (and host (string-match "%[0-9][0-9]" host))
diff --git a/test/automated/url-expand-tests.el b/test/automated/url-expand-tests.el
new file mode 100644
index 0000000..88c9b3b
--- /dev/null
+++ b/test/automated/url-expand-tests.el
@@ -0,0 +1,105 @@
+;;; url-expand-tests.el --- Test suite for relative URI/URL resolution.
+
+;; Copyright (C) 2012-2015 Free Software Foundation, Inc.
+
+;; Author: Alain Nicolas Schneble <a.s@realize.ch>
+;; Version: 1.0
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; Test cases covering URI reference resolution as described in RFC3986,
+;; section 5. Reference Resolution and especially the relative resolution
+;; rules specified in section 5.2. Relative Resolution.
+
+;; Each test calls `url-expand-file-name', typically with a relative
+;; reference URI and a base URI as string and compares the result (Actual)
+;; against a manually specified URI (Expected)
+
+;;; Code:
+
+(require 'url-expand)
+(require 'ert)
+
+(ert-deftest url-expand-file-name/relative-resolution-normal-examples ()
+ "RFC 3986, Section 5.4 Reference Resolution Examples / Section 5.4.1. Normal Examples"
+ (should (equal (url-expand-file-name "g:h" "http://a/b/c/d;p?q") "g:h"))
+ (should (equal (url-expand-file-name "g" "http://a/b/c/d;p?q") "http://a/b/c/g"))
+ (should (equal (url-expand-file-name "./g" "http://a/b/c/d;p?q") "http://a/b/c/g"))
+ (should (equal (url-expand-file-name "g/" "http://a/b/c/d;p?q") "http://a/b/c/g/"))
+ (should (equal (url-expand-file-name "/g" "http://a/b/c/d;p?q") "http://a/g"))
+ (should (equal (url-expand-file-name "//g" "http://a/b/c/d;p?q") "http://g"))
+ (should (equal (url-expand-file-name "?y" "http://a/b/c/d;p?q") "http://a/b/c/d;p?y"))
+ (should (equal (url-expand-file-name "g?y" "http://a/b/c/d;p?q") "http://a/b/c/g?y"))
+ (should (equal (url-expand-file-name "#s" "http://a/b/c/d;p?q") "http://a/b/c/d;p?q#s"))
+ (should (equal (url-expand-file-name "g#s" "http://a/b/c/d;p?q") "http://a/b/c/g#s"))
+ (should (equal (url-expand-file-name "g?y#s" "http://a/b/c/d;p?q") "http://a/b/c/g?y#s"))
+ (should (equal (url-expand-file-name ";x" "http://a/b/c/d;p?q") "http://a/b/c/;x"))
+ (should (equal (url-expand-file-name "g;x" "http://a/b/c/d;p?q") "http://a/b/c/g;x"))
+ (should (equal (url-expand-file-name "g;x?y#s" "http://a/b/c/d;p?q") "http://a/b/c/g;x?y#s"))
+ (should (equal (url-expand-file-name "" "http://a/b/c/d;p?q") "http://a/b/c/d;p?q"))
+ (should (equal (url-expand-file-name "." "http://a/b/c/d;p?q") "http://a/b/c/"))
+ (should (equal (url-expand-file-name "./" "http://a/b/c/d;p?q") "http://a/b/c/"))
+ (should (equal (url-expand-file-name ".." "http://a/b/c/d;p?q") "http://a/b/"))
+ (should (equal (url-expand-file-name "../" "http://a/b/c/d;p?q") "http://a/b/"))
+ (should (equal (url-expand-file-name "../g" "http://a/b/c/d;p?q") "http://a/b/g"))
+ (should (equal (url-expand-file-name "../.." "http://a/b/c/d;p?q") "http://a/"))
+ (should (equal (url-expand-file-name "../../" "http://a/b/c/d;p?q") "http://a/"))
+ (should (equal (url-expand-file-name "../../g" "http://a/b/c/d;p?q") "http://a/g")))
+
+(ert-deftest url-expand-file-name/relative-resolution-absolute-examples ()
+ "RFC 3986, Section 5.4 Reference Resolution Examples / Section 5.4.2. Abnormal Examples"
+ (should (equal (url-expand-file-name "../../../g" "http://a/b/c/d;p?q") "http://a/g"))
+ (should (equal (url-expand-file-name "../../../../g" "http://a/b/c/d;p?q") "http://a/g"))
+
+ (should (equal (url-expand-file-name "/./g" "http://a/b/c/d;p?q") "http://a/g"))
+ (should (equal (url-expand-file-name "/../g" "http://a/b/c/d;p?q") "http://a/g"))
+ (should (equal (url-expand-file-name "g." "http://a/b/c/d;p?q") "http://a/b/c/g."))
+ (should (equal (url-expand-file-name ".g" "http://a/b/c/d;p?q") "http://a/b/c/.g"))
+ (should (equal (url-expand-file-name "g.." "http://a/b/c/d;p?q") "http://a/b/c/g.."))
+ (should (equal (url-expand-file-name "..g" "http://a/b/c/d;p?q") "http://a/b/c/..g"))
+
+ (should (equal (url-expand-file-name "./../g" "http://a/b/c/d;p?q") "http://a/b/g"))
+ (should (equal (url-expand-file-name "./g/." "http://a/b/c/d;p?q") "http://a/b/c/g/"))
+ (should (equal (url-expand-file-name "g/./h" "http://a/b/c/d;p?q") "http://a/b/c/g/h"))
+ (should (equal (url-expand-file-name "g/../h" "http://a/b/c/d;p?q") "http://a/b/c/h"))
+ (should (equal (url-expand-file-name "g;x=1/./y" "http://a/b/c/d;p?q") "http://a/b/c/g;x=1/y"))
+ (should (equal (url-expand-file-name "g;x=1/../y" "http://a/b/c/d;p?q") "http://a/b/c/y"))
+
+ (should (equal (url-expand-file-name "g?y/./x" "http://a/b/c/d;p?q") "http://a/b/c/g?y/./x"))
+ (should (equal (url-expand-file-name "g?y/../x" "http://a/b/c/d;p?q") "http://a/b/c/g?y/../x"))
+ (should (equal (url-expand-file-name "g#s/./x" "http://a/b/c/d;p?q") "http://a/b/c/g#s/./x"))
+ (should (equal (url-expand-file-name "g#s/../x" "http://a/b/c/d;p?q") "http://a/b/c/g#s/../x"))
+
+ (should (equal (url-expand-file-name "http:g" "http://a/b/c/d;p?q") "http:g")) ; for strict parsers
+ )
+
+(ert-deftest url-expand-file-name/relative-resolution-additional-examples ()
+ "Reference Resolution Examples / Arbitrary Examples"
+ (should (equal (url-expand-file-name "" "http://host/foobar") "http://host/foobar"))
+ (should (equal (url-expand-file-name "?y" "http://a/b/c/d") "http://a/b/c/d?y"))
+ (should (equal (url-expand-file-name "?y" "http://a/b/c/d/") "http://a/b/c/d/?y"))
+ (should (equal (url-expand-file-name "?y#fragment" "http://a/b/c/d;p?q") "http://a/b/c/d;p?y#fragment"))
+ (should (equal (url-expand-file-name "#bar" "http://host") "http://host#bar"))
+ (should (equal (url-expand-file-name "#bar" "http://host/") "http://host/#bar"))
+ (should (equal (url-expand-file-name "#bar" "http://host/foo") "http://host/foo#bar"))
+ (should (equal (url-expand-file-name "foo#bar" "http://host/foobar") "http://host/foo#bar"))
+ (should (equal (url-expand-file-name "foo#bar" "http://host/foobar/") "http://host/foobar/foo#bar")))
+
+(provide 'url-expand-tests)
+
+;;; url-expand-tests.el ends here
diff --git a/test/automated/url-parse-tests.el b/test/automated/url-parse-tests.el
new file mode 100644
index 0000000..cded361
--- /dev/null
+++ b/test/automated/url-parse-tests.el
@@ -0,0 +1,167 @@
+;;; url-parse-tests.el --- Test suite for URI/URL parsing.
+
+;; Copyright (C) 2012-2015 Free Software Foundation, Inc.
+
+;; Author: Alain Nicolas Schneble <a.s@realize.ch>
+;; Version: 1.0
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; Test cases covering generic URI syntax as described in RFC3986,
+;; section 3. Syntax Components and 4. Usage. See also appendix
+;; A. Collected ABNF for URI, as the example given here are all
+;; productions of this grammar.
+
+;; Each tests parses a given URI string - whether relative or absolute -
+;; using `url-generic-parse-url' and compares the constructed
+;; URL-struct (Actual) against a manually `url-parse-make-urlobj'-
+;; constructed URL-struct (Expected).
+
+;;; Code:
+
+(require 'url-parse)
+(require 'ert)
+
+(ert-deftest url-generic-parse-url/generic-uri-examples ()
+ "RFC 3986, section 1.1.2. Examples / Example illustrating several URI schemes and variations in their common syntax components"
+ (should (equal (url-generic-parse-url "ftp://ftp.is.co.za/rfc/rfc1808.txt") (url-parse-make-urlobj "ftp" nil nil "ftp.is.co.za" nil "/rfc/rfc1808.txt" nil nil t)))
+ (should (equal (url-generic-parse-url "http://www.ietf.org/rfc/rfc2396.txt") (url-parse-make-urlobj "http" nil nil "www.ietf.org" nil "/rfc/rfc2396.txt" nil nil t)))
+ (should (equal (url-generic-parse-url "ldap://[2001:db8::7]/c=GB?objectClass?one") (url-parse-make-urlobj "ldap" nil nil "[2001:db8::7]" nil "/c=GB?objectClass?one" nil nil t)))
+ (should (equal (url-generic-parse-url "mailto:John.Doe@example.com") (url-parse-make-urlobj "mailto" nil nil nil nil "John.Doe@example.com" nil nil nil)))
+ (should (equal (url-generic-parse-url "news:comp.infosystems.www.servers.unix") (url-parse-make-urlobj "news" nil nil nil nil "comp.infosystems.www.servers.unix" nil nil nil)))
+ (should (equal (url-generic-parse-url "tel:+1-816-555-1212") (url-parse-make-urlobj "tel" nil nil nil nil "+1-816-555-1212" nil nil nil)))
+ (should (equal (url-generic-parse-url "telnet://192.0.2.16:80/") (url-parse-make-urlobj "telnet" nil nil "192.0.2.16" 80 "/" nil nil t)))
+ (should (equal (url-generic-parse-url "urn:oasis:names:specification:docbook:dtd:xml:4.1.2") (url-parse-make-urlobj "urn" nil nil nil nil "oasis:names:specification:docbook:dtd:xml:4.1.2" nil nil nil))))
+
+(ert-deftest url-generic-parse-url/generic-uri ()
+ "RFC 3986, section 3. Syntax Components / generic URI syntax"
+ ;; empty path
+ (should (equal (url-generic-parse-url "http://host#") (url-parse-make-urlobj "http" nil nil "host" nil "" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host?#") (url-parse-make-urlobj "http" nil nil "host" nil "?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host?query#") (url-parse-make-urlobj "http" nil nil "host" nil "?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "?query" "fragment" nil t)))
+ ;; absolute path /
+ (should (equal (url-generic-parse-url "http://host/#") (url-parse-make-urlobj "http" nil nil "host" nil "/" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" "fragment" nil t)))
+ ;; absolute path /foo
+ (should (equal (url-generic-parse-url "http://host/foo#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" "fragment" nil t)))
+ ;; absolute path /foo/
+ (should (equal (url-generic-parse-url "http://host/foo/#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" "fragment" nil t)))
+ ;; absolute path /foo/bar
+ (should (equal (url-generic-parse-url "http://host/foo/bar#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" "fragment" nil t)))
+ ;; absolute path /foo/bar/
+ (should (equal (url-generic-parse-url "http://host/foo/bar/#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" "fragment" nil t)))
+ ;; for more examples of URIs without fragments, see tests covering section 4.3. Absolute URI
+ )
+
+(ert-deftest url-generic-parse-url/network-path-reference ()
+ "RFC 3986, section 4.2. Relative Reference / network-path reference: a relative reference that begins with two slash characters"
+ (should (equal (url-generic-parse-url "//host") (url-parse-make-urlobj nil nil nil "host" nil "" nil nil t)))
+ (should (equal (url-generic-parse-url "//host/") (url-parse-make-urlobj nil nil nil "host" nil "/" nil nil t)))
+ (should (equal (url-generic-parse-url "//host/foo") (url-parse-make-urlobj nil nil nil "host" nil "/foo" nil nil t)))
+ (should (equal (url-generic-parse-url "//host/foo/bar") (url-parse-make-urlobj nil nil nil "host" nil "/foo/bar" nil nil t)))
+ (should (equal (url-generic-parse-url "//host/foo/bar/") (url-parse-make-urlobj nil nil nil "host" nil "/foo/bar/" nil nil t))))
+
+(ert-deftest url-generic-parse-url/absolute-path-reference ()
+ "RFC 3986, section 4.2. Relative Reference / absolute-path reference: a relative reference that begins with a single slash character"
+ (should (equal (url-generic-parse-url "/") (url-parse-make-urlobj nil nil nil nil nil "/" nil nil nil)))
+ (should (equal (url-generic-parse-url "/foo") (url-parse-make-urlobj nil nil nil nil nil "/foo" nil nil nil)))
+ (should (equal (url-generic-parse-url "/foo/bar") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar" nil nil nil)))
+ (should (equal (url-generic-parse-url "/foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar/" nil nil nil)))
+ (should (equal (url-generic-parse-url "/foo/bar#") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar" "" nil nil)))
+ (should (equal (url-generic-parse-url "/foo/bar/#") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar/" "" nil nil))))
+
+(ert-deftest url-generic-parse-url/relative-path-reference ()
+ "RFC 3986, section 4.2. Relative Reference / relative-path reference: a relative reference that does not begin with a slash character"
+ (should (equal (url-generic-parse-url "foo") (url-parse-make-urlobj nil nil nil nil nil "foo" nil nil nil)))
+ (should (equal (url-generic-parse-url "foo/bar") (url-parse-make-urlobj nil nil nil nil nil "foo/bar" nil nil nil)))
+ (should (equal (url-generic-parse-url "foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "foo/bar/" nil nil nil)))
+ (should (equal (url-generic-parse-url "./foo") (url-parse-make-urlobj nil nil nil nil nil "./foo" nil nil nil)))
+ (should (equal (url-generic-parse-url "./foo/bar") (url-parse-make-urlobj nil nil nil nil nil "./foo/bar" nil nil nil)))
+ (should (equal (url-generic-parse-url "./foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "./foo/bar/" nil nil nil)))
+ (should (equal (url-generic-parse-url "../foo") (url-parse-make-urlobj nil nil nil nil nil "../foo" nil nil nil)))
+ (should (equal (url-generic-parse-url "../foo/bar") (url-parse-make-urlobj nil nil nil nil nil "../foo/bar" nil nil nil)))
+ (should (equal (url-generic-parse-url "../foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "../foo/bar/" nil nil nil)))
+ (should (equal (url-generic-parse-url "./this:that") (url-parse-make-urlobj nil nil nil nil nil "./this:that" nil nil nil)))
+ ;; for more examples of relative-path references, see tests covering section 4.4. Same-Document Reference
+ )
+
+(ert-deftest url-generic-parse-url/absolute-uri ()
+ "RFC 3986, section 4.3. Absolute URI / absolute URI: absolute form of a URI without a fragment identifier"
+ ;; empty path
+ (should (equal (url-generic-parse-url "http://host") (url-parse-make-urlobj "http" nil nil "host" nil "" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host?") (url-parse-make-urlobj "http" nil nil "host" nil "?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host?query") (url-parse-make-urlobj "http" nil nil "host" nil "?query" nil nil t)))
+ ;; absolute path /
+ (should (equal (url-generic-parse-url "http://host/") (url-parse-make-urlobj "http" nil nil "host" nil "/" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/?") (url-parse-make-urlobj "http" nil nil "host" nil "/?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" nil nil t)))
+ ;; absolute path /foo
+ (should (equal (url-generic-parse-url "http://host/foo") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" nil nil t)))
+ ;; absolute path /foo/
+ (should (equal (url-generic-parse-url "http://host/foo/") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" nil nil t)))
+ ;; absolute path /foo/bar
+ (should (equal (url-generic-parse-url "http://host/foo/bar") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" nil nil t)))
+ ;; absolute path /foo/bar/
+ (should (equal (url-generic-parse-url "http://host/foo/bar/") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" nil nil t)))
+ ;; example mentioned in RFC3986, section 5.4. Reference Resolution Examples
+ (should (equal (url-generic-parse-url "http://a/b/c/d;p?q") (url-parse-make-urlobj "http" nil nil "a" nil "/b/c/d;p?q" nil nil t))))
+
+(ert-deftest url-generic-parse-url/same-decument-reference ()
+ "RFC 3986, section 4.4. Same-Document Reference / same-document reference: empty or number sign (\"#\") followed by a fragment identifier"
+ (should (equal (url-generic-parse-url "") (url-parse-make-urlobj nil nil nil nil nil "" nil nil nil)))
+ (should (equal (url-generic-parse-url "#") (url-parse-make-urlobj nil nil nil nil nil "" "" nil nil)))
+ (should (equal (url-generic-parse-url "#foo") (url-parse-make-urlobj nil nil nil nil nil "" "foo" nil nil))))
+
+(provide 'url-parse-tests)
+
+;;; url-parse-tests.el ends here
--
2.6.2.windows.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-11-28 23:42 ` Alain Schneble
@ 2015-12-25 18:01 ` Lars Ingebrigtsen
2015-12-25 19:45 ` Eli Zaretskii
2015-12-25 19:58 ` Lars Ingebrigtsen
1 sibling, 1 reply; 18+ messages in thread
From: Lars Ingebrigtsen @ 2015-12-25 18:01 UTC (permalink / raw)
To: Alain Schneble; +Cc: 22044
Alain Schneble <a.s@realize.ch> writes:
> To address these issues and make the behavior consistent with RFC 3984,
> I propose the following changes to url-parse.el and url-expand.el, see
> attached patch.
Looks good. Do you have Emacs copyright assignment papers on file with
the FSF?
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-25 18:01 ` Lars Ingebrigtsen
@ 2015-12-25 19:45 ` Eli Zaretskii
2015-12-25 19:54 ` Lars Ingebrigtsen
0 siblings, 1 reply; 18+ messages in thread
From: Eli Zaretskii @ 2015-12-25 19:45 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: a.s, 22044
> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Fri, 25 Dec 2015 19:01:33 +0100
> Cc: 22044@debbugs.gnu.org
>
> Alain Schneble <a.s@realize.ch> writes:
>
> > To address these issues and make the behavior consistent with RFC 3984,
> > I propose the following changes to url-parse.el and url-expand.el, see
> > attached patch.
>
> Looks good. Do you have Emacs copyright assignment papers on file with
> the FSF?
He does.
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-25 19:45 ` Eli Zaretskii
@ 2015-12-25 19:54 ` Lars Ingebrigtsen
0 siblings, 0 replies; 18+ messages in thread
From: Lars Ingebrigtsen @ 2015-12-25 19:54 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: a.s, 22044
Eli Zaretskii <eliz@gnu.org> writes:
>> From: Lars Ingebrigtsen <larsi@gnus.org>
>> Date: Fri, 25 Dec 2015 19:01:33 +0100
>> Cc: 22044@debbugs.gnu.org
>>
>> Alain Schneble <a.s@realize.ch> writes:
>>
>> > To address these issues and make the behavior consistent with RFC 3984,
>> > I propose the following changes to url-parse.el and url-expand.el, see
>> > attached patch.
>>
>> Looks good. Do you have Emacs copyright assignment papers on file with
>> the FSF?
>
> He does.
Great; I'll apply the patches.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-11-28 23:42 ` Alain Schneble
2015-12-25 18:01 ` Lars Ingebrigtsen
@ 2015-12-25 19:58 ` Lars Ingebrigtsen
2015-12-25 22:27 ` Alain Schneble
1 sibling, 1 reply; 18+ messages in thread
From: Lars Ingebrigtsen @ 2015-12-25 19:58 UTC (permalink / raw)
To: Alain Schneble; +Cc: 22044
Alain Schneble <a.s@realize.ch> writes:
> * test/automated/url-parse-tests.el: add tests covering url-generic-parse-url.
> * test/automated/url-expand-tests.el: add tests covering url-expand-file-name.
> * test/automated/url-parse.el (url-generic-parse-url): keep empty fragment
> information in URL-struct.
This is apparently a new top-level directory. Are these supposed to be
in this directory, is is this an error?
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-25 19:58 ` Lars Ingebrigtsen
@ 2015-12-25 22:27 ` Alain Schneble
2015-12-25 22:39 ` Lars Ingebrigtsen
0 siblings, 1 reply; 18+ messages in thread
From: Alain Schneble @ 2015-12-25 22:27 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: 22044
Lars Ingebrigtsen <larsi@gnus.org> writes:
Hello Lars
> Alain Schneble <a.s@realize.ch> writes:
>
>> * test/automated/url-parse-tests.el: add tests covering url-generic-parse-url.
>> * test/automated/url-expand-tests.el: add tests covering url-expand-file-name.
>> * test/automated/url-parse.el (url-generic-parse-url): keep empty fragment
>> information in URL-struct.
>
> This is apparently a new top-level directory. Are these supposed to be
> in this directory, is is this an error?
When I created the patch, the new test directory layout (see 75336a2 -
Merge branch 'feature/standard-test-location') has not yet been merged
to master. I guess that these tests should now reside in test/lisp/url/
if we follow the new convention. Shall I create a new patch or will you
just move the files there?
Many thanks for looking into my changes.
Alain
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-25 22:27 ` Alain Schneble
@ 2015-12-25 22:39 ` Lars Ingebrigtsen
2015-12-25 22:48 ` Alain Schneble
2015-12-25 23:44 ` Alain Schneble
0 siblings, 2 replies; 18+ messages in thread
From: Lars Ingebrigtsen @ 2015-12-25 22:39 UTC (permalink / raw)
To: Alain Schneble; +Cc: 22044
Alain Schneble <a.s@realize.ch> writes:
> When I created the patch, the new test directory layout (see 75336a2 -
> Merge branch 'feature/standard-test-location') has not yet been merged
> to master. I guess that these tests should now reside in test/lisp/url/
> if we follow the new convention. Shall I create a new patch or will you
> just move the files there?
If you could roll a new patch, that would be easier for me. :-)
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-25 22:39 ` Lars Ingebrigtsen
@ 2015-12-25 22:48 ` Alain Schneble
2015-12-25 23:44 ` Alain Schneble
1 sibling, 0 replies; 18+ messages in thread
From: Alain Schneble @ 2015-12-25 22:48 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: 22044
Lars Ingebrigtsen <larsi@gnus.org> writes:
> Alain Schneble <a.s@realize.ch> writes:
>
>> When I created the patch, the new test directory layout (see 75336a2 -
>> Merge branch 'feature/standard-test-location') has not yet been merged
>> to master. I guess that these tests should now reside in test/lisp/url/
>> if we follow the new convention. Shall I create a new patch or will you
>> just move the files there?
>
> If you could roll a new patch, that would be easier for me. :-)
Of course, I'll do it.
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-25 22:39 ` Lars Ingebrigtsen
2015-12-25 22:48 ` Alain Schneble
@ 2015-12-25 23:44 ` Alain Schneble
2015-12-25 23:51 ` Lars Ingebrigtsen
1 sibling, 1 reply; 18+ messages in thread
From: Alain Schneble @ 2015-12-25 23:44 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: 22044
[-- Attachment #1: Type: text/plain, Size: 514 bytes --]
Lars Ingebrigtsen <larsi@gnus.org> writes:
> Alain Schneble <a.s@realize.ch> writes:
>
>> When I created the patch, the new test directory layout (see 75336a2 -
>> Merge branch 'feature/standard-test-location') has not yet been merged
>> to master. I guess that these tests should now reside in test/lisp/url/
>> if we follow the new convention. Shall I create a new patch or will you
>> just move the files there?
>
> If you could roll a new patch, that would be easier for me. :-)
Here is the adapted patch.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Make relative URL parsing and resolution consistent with RFC 3986 (bug#22044) --]
[-- Type: text/x-patch, Size: 29607 bytes --]
From 6f676f50f08b398c5de57126af8c67f1f0501137 Mon Sep 17 00:00:00 2001
From: Alain Schneble <a.s@realize.ch>
Date: Sat, 26 Dec 2015 00:25:56 +0100
Subject: [PATCH] Make relative URL parsing and resolution consistent with RFC
3986 (bug#22044)
* test/lisp/url/url-parse-tests.el: add tests covering url-generic-parse-url.
* test/lisp/url/url-expand-tests.el: add tests covering url-expand-file-name.
* lisp/url/url-parse.el (url-generic-parse-url): keep empty fragment
information in URL-struct.
* lisp/url/url-parse.el (url-path-and-query): do not artificially turn empty
path and query into nil path and query, respectively.
* lisp/url/url-expand.el (url-expander-remove-relative-links): do not turn
empty path into an absolute ("/") path.
* lisp/url/url-expand.el (url-expand-file-name): properly resolve
fragment-only URIs. Do not just return them unchanged.
* lisp/url/url-expand.el (url-default-expander): an empty path in the relative
reference URI should not drop the last segment.
---
lisp/url/url-expand.el | 84 +++++++++----------
lisp/url/url-parse.el | 5 +-
test/lisp/url/url-expand-tests.el | 105 ++++++++++++++++++++++++
test/lisp/url/url-parse-tests.el | 167 ++++++++++++++++++++++++++++++++++++++
4 files changed, 313 insertions(+), 48 deletions(-)
create mode 100644 test/lisp/url/url-expand-tests.el
create mode 100644 test/lisp/url/url-parse-tests.el
diff --git a/lisp/url/url-expand.el b/lisp/url/url-expand.el
index c468a79..600a36d 100644
--- a/lisp/url/url-expand.el
+++ b/lisp/url/url-expand.el
@@ -26,32 +26,35 @@
(require 'url-parse)
(defun url-expander-remove-relative-links (name)
- ;; Strip . and .. from pathnames
- (let ((new (if (not (string-match "^/" name))
- (concat "/" name)
- name)))
-
- ;; If it ends with a '/.' or '/..', tack on a trailing '/' sot hat
- ;; the tests that follow are not too complicated in terms of
- ;; looking for '..' or '../', etc.
- (if (string-match "/\\.+$" new)
- (setq new (concat new "/")))
-
- ;; Remove '/./' first
- (while (string-match "/\\(\\./\\)" new)
- (setq new (concat (substring new 0 (match-beginning 1))
- (substring new (match-end 1)))))
-
- ;; Then remove '/../'
- (while (string-match "/\\([^/]*/\\.\\./\\)" new)
- (setq new (concat (substring new 0 (match-beginning 1))
- (substring new (match-end 1)))))
-
- ;; Remove cruft at the beginning of the string, so people that put
- ;; in extraneous '..' because they are morons won't lose.
- (while (string-match "^/\\.\\.\\(/\\)" new)
- (setq new (substring new (match-beginning 1) nil)))
- new))
+ (if (equal name "")
+ ;; An empty name is a properly valid relative URL reference/path.
+ ""
+ ;; Strip . and .. from pathnames
+ (let ((new (if (not (string-match "^/" name))
+ (concat "/" name)
+ name)))
+
+ ;; If it ends with a '/.' or '/..', tack on a trailing '/' sot hat
+ ;; the tests that follow are not too complicated in terms of
+ ;; looking for '..' or '../', etc.
+ (if (string-match "/\\.+$" new)
+ (setq new (concat new "/")))
+
+ ;; Remove '/./' first
+ (while (string-match "/\\(\\./\\)" new)
+ (setq new (concat (substring new 0 (match-beginning 1))
+ (substring new (match-end 1)))))
+
+ ;; Then remove '/../'
+ (while (string-match "/\\([^/]*/\\.\\./\\)" new)
+ (setq new (concat (substring new 0 (match-beginning 1))
+ (substring new (match-end 1)))))
+
+ ;; Remove cruft at the beginning of the string, so people that put
+ ;; in extraneous '..' because they are morons won't lose.
+ (while (string-match "^/\\.\\.\\(/\\)" new)
+ (setq new (substring new (match-beginning 1) nil)))
+ new)))
(defun url-expand-file-name (url &optional default)
"Convert URL to a fully specified URL, and canonicalize it.
@@ -89,8 +92,6 @@ path components followed by `..' are removed, along with the `..' itself."
(cond
((= (length url) 0) ; nil or empty string
(url-recreate-url default))
- ((string-match "^#" url) ; Offset link, use it raw
- url)
((string-match url-nonrelative-link url) ; Fully-qualified URL, return it immediately
url)
(t
@@ -120,29 +121,24 @@ path components followed by `..' are removed, along with the `..' itself."
(setf (url-host urlobj) (or (url-host urlobj) (url-host defobj))))
(if (string= "ftp" (url-type urlobj))
(setf (url-user urlobj) (or (url-user urlobj) (url-user defobj))))
- (if (string= (url-filename urlobj) "")
- (setf (url-filename urlobj) "/"))
;; If the object we're expanding from is full, then we are now
;; full.
(unless (url-fullness urlobj)
(setf (url-fullness urlobj) (url-fullness defobj)))
- (if (string-match "^/" (url-filename urlobj))
- nil
- (let ((query nil)
- (file nil)
- (sepchar nil))
- (if (string-match "[?#]" (url-filename urlobj))
- (setq query (substring (url-filename urlobj) (match-end 0))
- file (substring (url-filename urlobj) 0 (match-beginning 0))
- sepchar (substring (url-filename urlobj) (match-beginning 0) (match-end 0)))
- (setq file (url-filename urlobj)))
+ (let* ((pathandquery (url-path-and-query urlobj))
+ (defpathandquery (url-path-and-query defobj))
+ (file (car pathandquery))
+ (query (or (cdr pathandquery) (and (equal file "") (cdr defpathandquery)))))
+ (if (string-match "^/" (url-filename urlobj))
+ (setq file (url-expander-remove-relative-links file))
;; We use concat rather than expand-file-name to combine
;; directory and file name, since urls do not follow the same
;; rules as local files on all platforms.
- (setq file (url-expander-remove-relative-links
- (concat (url-file-directory (url-filename defobj)) file)))
- (setf (url-filename urlobj)
- (if query (concat file sepchar query) file))))))
+ (setq file (url-expander-remove-relative-links
+ (if (equal file "")
+ (or (car (url-path-and-query defobj)) "")
+ (concat (url-file-directory (url-filename defobj)) file)))))
+ (setf (url-filename urlobj) (if query (concat file "?" query) file)))))
(provide 'url-expand)
diff --git a/lisp/url/url-parse.el b/lisp/url/url-parse.el
index dbf0c38..c3159a7 100644
--- a/lisp/url/url-parse.el
+++ b/lisp/url/url-parse.el
@@ -59,8 +59,6 @@ where each of PATH and QUERY are strings or nil."
(setq path (substring name 0 (match-beginning 0))
query (substring name (match-end 0)))
(setq path name)))
- (if (equal path "") (setq path nil))
- (if (equal query "") (setq query nil))
(cons path query)))
(defun url-port-if-non-default (urlobj)
@@ -217,8 +215,7 @@ parses to
(when (looking-at "#")
(let ((opoint (point)))
(forward-char 1)
- (unless (eobp)
- (setq fragment (buffer-substring (point) (point-max))))
+ (setq fragment (buffer-substring (point) (point-max)))
(delete-region opoint (point-max)))))
(if (and host (string-match "%[0-9][0-9]" host))
diff --git a/test/lisp/url/url-expand-tests.el b/test/lisp/url/url-expand-tests.el
new file mode 100644
index 0000000..2bd2868
--- /dev/null
+++ b/test/lisp/url/url-expand-tests.el
@@ -0,0 +1,105 @@
+;;; url-expand-tests.el --- Test suite for relative URI/URL resolution.
+
+;; Copyright (C) 2012-2015 Free Software Foundation, Inc.
+
+;; Author: Alain Schneble <a.s@realize.ch>
+;; Version: 1.0
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; Test cases covering URI reference resolution as described in RFC3986,
+;; section 5. Reference Resolution and especially the relative resolution
+;; rules specified in section 5.2. Relative Resolution.
+
+;; Each test calls `url-expand-file-name', typically with a relative
+;; reference URI and a base URI as string and compares the result (Actual)
+;; against a manually specified URI (Expected)
+
+;;; Code:
+
+(require 'url-expand)
+(require 'ert)
+
+(ert-deftest url-expand-file-name/relative-resolution-normal-examples ()
+ "RFC 3986, Section 5.4 Reference Resolution Examples / Section 5.4.1. Normal Examples"
+ (should (equal (url-expand-file-name "g:h" "http://a/b/c/d;p?q") "g:h"))
+ (should (equal (url-expand-file-name "g" "http://a/b/c/d;p?q") "http://a/b/c/g"))
+ (should (equal (url-expand-file-name "./g" "http://a/b/c/d;p?q") "http://a/b/c/g"))
+ (should (equal (url-expand-file-name "g/" "http://a/b/c/d;p?q") "http://a/b/c/g/"))
+ (should (equal (url-expand-file-name "/g" "http://a/b/c/d;p?q") "http://a/g"))
+ (should (equal (url-expand-file-name "//g" "http://a/b/c/d;p?q") "http://g"))
+ (should (equal (url-expand-file-name "?y" "http://a/b/c/d;p?q") "http://a/b/c/d;p?y"))
+ (should (equal (url-expand-file-name "g?y" "http://a/b/c/d;p?q") "http://a/b/c/g?y"))
+ (should (equal (url-expand-file-name "#s" "http://a/b/c/d;p?q") "http://a/b/c/d;p?q#s"))
+ (should (equal (url-expand-file-name "g#s" "http://a/b/c/d;p?q") "http://a/b/c/g#s"))
+ (should (equal (url-expand-file-name "g?y#s" "http://a/b/c/d;p?q") "http://a/b/c/g?y#s"))
+ (should (equal (url-expand-file-name ";x" "http://a/b/c/d;p?q") "http://a/b/c/;x"))
+ (should (equal (url-expand-file-name "g;x" "http://a/b/c/d;p?q") "http://a/b/c/g;x"))
+ (should (equal (url-expand-file-name "g;x?y#s" "http://a/b/c/d;p?q") "http://a/b/c/g;x?y#s"))
+ (should (equal (url-expand-file-name "" "http://a/b/c/d;p?q") "http://a/b/c/d;p?q"))
+ (should (equal (url-expand-file-name "." "http://a/b/c/d;p?q") "http://a/b/c/"))
+ (should (equal (url-expand-file-name "./" "http://a/b/c/d;p?q") "http://a/b/c/"))
+ (should (equal (url-expand-file-name ".." "http://a/b/c/d;p?q") "http://a/b/"))
+ (should (equal (url-expand-file-name "../" "http://a/b/c/d;p?q") "http://a/b/"))
+ (should (equal (url-expand-file-name "../g" "http://a/b/c/d;p?q") "http://a/b/g"))
+ (should (equal (url-expand-file-name "../.." "http://a/b/c/d;p?q") "http://a/"))
+ (should (equal (url-expand-file-name "../../" "http://a/b/c/d;p?q") "http://a/"))
+ (should (equal (url-expand-file-name "../../g" "http://a/b/c/d;p?q") "http://a/g")))
+
+(ert-deftest url-expand-file-name/relative-resolution-absolute-examples ()
+ "RFC 3986, Section 5.4 Reference Resolution Examples / Section 5.4.2. Abnormal Examples"
+ (should (equal (url-expand-file-name "../../../g" "http://a/b/c/d;p?q") "http://a/g"))
+ (should (equal (url-expand-file-name "../../../../g" "http://a/b/c/d;p?q") "http://a/g"))
+
+ (should (equal (url-expand-file-name "/./g" "http://a/b/c/d;p?q") "http://a/g"))
+ (should (equal (url-expand-file-name "/../g" "http://a/b/c/d;p?q") "http://a/g"))
+ (should (equal (url-expand-file-name "g." "http://a/b/c/d;p?q") "http://a/b/c/g."))
+ (should (equal (url-expand-file-name ".g" "http://a/b/c/d;p?q") "http://a/b/c/.g"))
+ (should (equal (url-expand-file-name "g.." "http://a/b/c/d;p?q") "http://a/b/c/g.."))
+ (should (equal (url-expand-file-name "..g" "http://a/b/c/d;p?q") "http://a/b/c/..g"))
+
+ (should (equal (url-expand-file-name "./../g" "http://a/b/c/d;p?q") "http://a/b/g"))
+ (should (equal (url-expand-file-name "./g/." "http://a/b/c/d;p?q") "http://a/b/c/g/"))
+ (should (equal (url-expand-file-name "g/./h" "http://a/b/c/d;p?q") "http://a/b/c/g/h"))
+ (should (equal (url-expand-file-name "g/../h" "http://a/b/c/d;p?q") "http://a/b/c/h"))
+ (should (equal (url-expand-file-name "g;x=1/./y" "http://a/b/c/d;p?q") "http://a/b/c/g;x=1/y"))
+ (should (equal (url-expand-file-name "g;x=1/../y" "http://a/b/c/d;p?q") "http://a/b/c/y"))
+
+ (should (equal (url-expand-file-name "g?y/./x" "http://a/b/c/d;p?q") "http://a/b/c/g?y/./x"))
+ (should (equal (url-expand-file-name "g?y/../x" "http://a/b/c/d;p?q") "http://a/b/c/g?y/../x"))
+ (should (equal (url-expand-file-name "g#s/./x" "http://a/b/c/d;p?q") "http://a/b/c/g#s/./x"))
+ (should (equal (url-expand-file-name "g#s/../x" "http://a/b/c/d;p?q") "http://a/b/c/g#s/../x"))
+
+ (should (equal (url-expand-file-name "http:g" "http://a/b/c/d;p?q") "http:g")) ; for strict parsers
+ )
+
+(ert-deftest url-expand-file-name/relative-resolution-additional-examples ()
+ "Reference Resolution Examples / Arbitrary Examples"
+ (should (equal (url-expand-file-name "" "http://host/foobar") "http://host/foobar"))
+ (should (equal (url-expand-file-name "?y" "http://a/b/c/d") "http://a/b/c/d?y"))
+ (should (equal (url-expand-file-name "?y" "http://a/b/c/d/") "http://a/b/c/d/?y"))
+ (should (equal (url-expand-file-name "?y#fragment" "http://a/b/c/d;p?q") "http://a/b/c/d;p?y#fragment"))
+ (should (equal (url-expand-file-name "#bar" "http://host") "http://host#bar"))
+ (should (equal (url-expand-file-name "#bar" "http://host/") "http://host/#bar"))
+ (should (equal (url-expand-file-name "#bar" "http://host/foo") "http://host/foo#bar"))
+ (should (equal (url-expand-file-name "foo#bar" "http://host/foobar") "http://host/foo#bar"))
+ (should (equal (url-expand-file-name "foo#bar" "http://host/foobar/") "http://host/foobar/foo#bar")))
+
+(provide 'url-expand-tests)
+
+;;; url-expand-tests.el ends here
diff --git a/test/lisp/url/url-parse-tests.el b/test/lisp/url/url-parse-tests.el
new file mode 100644
index 0000000..154e091
--- /dev/null
+++ b/test/lisp/url/url-parse-tests.el
@@ -0,0 +1,167 @@
+;;; url-parse-tests.el --- Test suite for URI/URL parsing.
+
+;; Copyright (C) 2012-2015 Free Software Foundation, Inc.
+
+;; Author: Alain Schneble <a.s@realize.ch>
+;; Version: 1.0
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; Test cases covering generic URI syntax as described in RFC3986,
+;; section 3. Syntax Components and 4. Usage. See also appendix
+;; A. Collected ABNF for URI, as the example given here are all
+;; productions of this grammar.
+
+;; Each tests parses a given URI string - whether relative or absolute -
+;; using `url-generic-parse-url' and compares the constructed
+;; URL-struct (Actual) against a manually `url-parse-make-urlobj'-
+;; constructed URL-struct (Expected).
+
+;;; Code:
+
+(require 'url-parse)
+(require 'ert)
+
+(ert-deftest url-generic-parse-url/generic-uri-examples ()
+ "RFC 3986, section 1.1.2. Examples / Example illustrating several URI schemes and variations in their common syntax components"
+ (should (equal (url-generic-parse-url "ftp://ftp.is.co.za/rfc/rfc1808.txt") (url-parse-make-urlobj "ftp" nil nil "ftp.is.co.za" nil "/rfc/rfc1808.txt" nil nil t)))
+ (should (equal (url-generic-parse-url "http://www.ietf.org/rfc/rfc2396.txt") (url-parse-make-urlobj "http" nil nil "www.ietf.org" nil "/rfc/rfc2396.txt" nil nil t)))
+ (should (equal (url-generic-parse-url "ldap://[2001:db8::7]/c=GB?objectClass?one") (url-parse-make-urlobj "ldap" nil nil "[2001:db8::7]" nil "/c=GB?objectClass?one" nil nil t)))
+ (should (equal (url-generic-parse-url "mailto:John.Doe@example.com") (url-parse-make-urlobj "mailto" nil nil nil nil "John.Doe@example.com" nil nil nil)))
+ (should (equal (url-generic-parse-url "news:comp.infosystems.www.servers.unix") (url-parse-make-urlobj "news" nil nil nil nil "comp.infosystems.www.servers.unix" nil nil nil)))
+ (should (equal (url-generic-parse-url "tel:+1-816-555-1212") (url-parse-make-urlobj "tel" nil nil nil nil "+1-816-555-1212" nil nil nil)))
+ (should (equal (url-generic-parse-url "telnet://192.0.2.16:80/") (url-parse-make-urlobj "telnet" nil nil "192.0.2.16" 80 "/" nil nil t)))
+ (should (equal (url-generic-parse-url "urn:oasis:names:specification:docbook:dtd:xml:4.1.2") (url-parse-make-urlobj "urn" nil nil nil nil "oasis:names:specification:docbook:dtd:xml:4.1.2" nil nil nil))))
+
+(ert-deftest url-generic-parse-url/generic-uri ()
+ "RFC 3986, section 3. Syntax Components / generic URI syntax"
+ ;; empty path
+ (should (equal (url-generic-parse-url "http://host#") (url-parse-make-urlobj "http" nil nil "host" nil "" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host?#") (url-parse-make-urlobj "http" nil nil "host" nil "?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host?query#") (url-parse-make-urlobj "http" nil nil "host" nil "?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "?query" "fragment" nil t)))
+ ;; absolute path /
+ (should (equal (url-generic-parse-url "http://host/#") (url-parse-make-urlobj "http" nil nil "host" nil "/" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" "fragment" nil t)))
+ ;; absolute path /foo
+ (should (equal (url-generic-parse-url "http://host/foo#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" "fragment" nil t)))
+ ;; absolute path /foo/
+ (should (equal (url-generic-parse-url "http://host/foo/#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" "fragment" nil t)))
+ ;; absolute path /foo/bar
+ (should (equal (url-generic-parse-url "http://host/foo/bar#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" "fragment" nil t)))
+ ;; absolute path /foo/bar/
+ (should (equal (url-generic-parse-url "http://host/foo/bar/#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?query#") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" "" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" "fragment" nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?query#fragment") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" "fragment" nil t)))
+ ;; for more examples of URIs without fragments, see tests covering section 4.3. Absolute URI
+ )
+
+(ert-deftest url-generic-parse-url/network-path-reference ()
+ "RFC 3986, section 4.2. Relative Reference / network-path reference: a relative reference that begins with two slash characters"
+ (should (equal (url-generic-parse-url "//host") (url-parse-make-urlobj nil nil nil "host" nil "" nil nil t)))
+ (should (equal (url-generic-parse-url "//host/") (url-parse-make-urlobj nil nil nil "host" nil "/" nil nil t)))
+ (should (equal (url-generic-parse-url "//host/foo") (url-parse-make-urlobj nil nil nil "host" nil "/foo" nil nil t)))
+ (should (equal (url-generic-parse-url "//host/foo/bar") (url-parse-make-urlobj nil nil nil "host" nil "/foo/bar" nil nil t)))
+ (should (equal (url-generic-parse-url "//host/foo/bar/") (url-parse-make-urlobj nil nil nil "host" nil "/foo/bar/" nil nil t))))
+
+(ert-deftest url-generic-parse-url/absolute-path-reference ()
+ "RFC 3986, section 4.2. Relative Reference / absolute-path reference: a relative reference that begins with a single slash character"
+ (should (equal (url-generic-parse-url "/") (url-parse-make-urlobj nil nil nil nil nil "/" nil nil nil)))
+ (should (equal (url-generic-parse-url "/foo") (url-parse-make-urlobj nil nil nil nil nil "/foo" nil nil nil)))
+ (should (equal (url-generic-parse-url "/foo/bar") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar" nil nil nil)))
+ (should (equal (url-generic-parse-url "/foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar/" nil nil nil)))
+ (should (equal (url-generic-parse-url "/foo/bar#") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar" "" nil nil)))
+ (should (equal (url-generic-parse-url "/foo/bar/#") (url-parse-make-urlobj nil nil nil nil nil "/foo/bar/" "" nil nil))))
+
+(ert-deftest url-generic-parse-url/relative-path-reference ()
+ "RFC 3986, section 4.2. Relative Reference / relative-path reference: a relative reference that does not begin with a slash character"
+ (should (equal (url-generic-parse-url "foo") (url-parse-make-urlobj nil nil nil nil nil "foo" nil nil nil)))
+ (should (equal (url-generic-parse-url "foo/bar") (url-parse-make-urlobj nil nil nil nil nil "foo/bar" nil nil nil)))
+ (should (equal (url-generic-parse-url "foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "foo/bar/" nil nil nil)))
+ (should (equal (url-generic-parse-url "./foo") (url-parse-make-urlobj nil nil nil nil nil "./foo" nil nil nil)))
+ (should (equal (url-generic-parse-url "./foo/bar") (url-parse-make-urlobj nil nil nil nil nil "./foo/bar" nil nil nil)))
+ (should (equal (url-generic-parse-url "./foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "./foo/bar/" nil nil nil)))
+ (should (equal (url-generic-parse-url "../foo") (url-parse-make-urlobj nil nil nil nil nil "../foo" nil nil nil)))
+ (should (equal (url-generic-parse-url "../foo/bar") (url-parse-make-urlobj nil nil nil nil nil "../foo/bar" nil nil nil)))
+ (should (equal (url-generic-parse-url "../foo/bar/") (url-parse-make-urlobj nil nil nil nil nil "../foo/bar/" nil nil nil)))
+ (should (equal (url-generic-parse-url "./this:that") (url-parse-make-urlobj nil nil nil nil nil "./this:that" nil nil nil)))
+ ;; for more examples of relative-path references, see tests covering section 4.4. Same-Document Reference
+ )
+
+(ert-deftest url-generic-parse-url/absolute-uri ()
+ "RFC 3986, section 4.3. Absolute URI / absolute URI: absolute form of a URI without a fragment identifier"
+ ;; empty path
+ (should (equal (url-generic-parse-url "http://host") (url-parse-make-urlobj "http" nil nil "host" nil "" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host?") (url-parse-make-urlobj "http" nil nil "host" nil "?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host?query") (url-parse-make-urlobj "http" nil nil "host" nil "?query" nil nil t)))
+ ;; absolute path /
+ (should (equal (url-generic-parse-url "http://host/") (url-parse-make-urlobj "http" nil nil "host" nil "/" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/?") (url-parse-make-urlobj "http" nil nil "host" nil "/?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/?query" nil nil t)))
+ ;; absolute path /foo
+ (should (equal (url-generic-parse-url "http://host/foo") (url-parse-make-urlobj "http" nil nil "host" nil "/foo" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo?query" nil nil t)))
+ ;; absolute path /foo/
+ (should (equal (url-generic-parse-url "http://host/foo/") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/?query" nil nil t)))
+ ;; absolute path /foo/bar
+ (should (equal (url-generic-parse-url "http://host/foo/bar") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar?query" nil nil t)))
+ ;; absolute path /foo/bar/
+ (should (equal (url-generic-parse-url "http://host/foo/bar/") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?" nil nil t)))
+ (should (equal (url-generic-parse-url "http://host/foo/bar/?query") (url-parse-make-urlobj "http" nil nil "host" nil "/foo/bar/?query" nil nil t)))
+ ;; example mentioned in RFC3986, section 5.4. Reference Resolution Examples
+ (should (equal (url-generic-parse-url "http://a/b/c/d;p?q") (url-parse-make-urlobj "http" nil nil "a" nil "/b/c/d;p?q" nil nil t))))
+
+(ert-deftest url-generic-parse-url/same-decument-reference ()
+ "RFC 3986, section 4.4. Same-Document Reference / same-document reference: empty or number sign (\"#\") followed by a fragment identifier"
+ (should (equal (url-generic-parse-url "") (url-parse-make-urlobj nil nil nil nil nil "" nil nil nil)))
+ (should (equal (url-generic-parse-url "#") (url-parse-make-urlobj nil nil nil nil nil "" "" nil nil)))
+ (should (equal (url-generic-parse-url "#foo") (url-parse-make-urlobj nil nil nil nil nil "" "foo" nil nil))))
+
+(provide 'url-parse-tests)
+
+;;; url-parse-tests.el ends here
--
2.6.2.windows.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-25 23:44 ` Alain Schneble
@ 2015-12-25 23:51 ` Lars Ingebrigtsen
2015-12-26 0:23 ` Alain Schneble
2015-12-26 9:57 ` Eli Zaretskii
0 siblings, 2 replies; 18+ messages in thread
From: Lars Ingebrigtsen @ 2015-12-25 23:51 UTC (permalink / raw)
To: Alain Schneble; +Cc: 22044
Alain Schneble <a.s@realize.ch> writes:
> Here is the adapted patch.
Thanks; applied and pushed (to trunk). It might make sense to backport
to Emacs 25.1 -- do you think that's a good idea?
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-25 23:51 ` Lars Ingebrigtsen
@ 2015-12-26 0:23 ` Alain Schneble
2015-12-26 0:51 ` John Wiegley
2015-12-26 1:06 ` Lars Ingebrigtsen
2015-12-26 9:57 ` Eli Zaretskii
1 sibling, 2 replies; 18+ messages in thread
From: Alain Schneble @ 2015-12-26 0:23 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: 22044
Lars Ingebrigtsen <larsi@gnus.org> writes:
> Alain Schneble <a.s@realize.ch> writes:
>
>> Here is the adapted patch.
>
> Thanks; applied and pushed (to trunk). It might make sense to backport
> to Emacs 25.1 -- do you think that's a good idea?
Thanks, great! I see nothing against backporting it to Emacs 25.1. But
I must be honest, I'm pretty new to Emacs and its hard for me to judge
the risks of doing so. But with all the new tests in place, I don't
expect it breaks anything fundamentally; the changes were mainly related
to how fragments and relative URLs are handled. These are more edge
cases and I guess it would also only affect some rare edge cases - if at
all.
Shall I to elaborate more on this? What do you think?
By the way, the 25.1 branch still uses the old test directory layout...
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-26 0:23 ` Alain Schneble
@ 2015-12-26 0:51 ` John Wiegley
2015-12-26 10:51 ` Alain Schneble
2015-12-26 1:06 ` Lars Ingebrigtsen
1 sibling, 1 reply; 18+ messages in thread
From: John Wiegley @ 2015-12-26 0:51 UTC (permalink / raw)
To: Alain Schneble; +Cc: Lars Ingebrigtsen, 22044
>>>>> Alain Schneble <a.s@realize.ch> writes:
> Thanks, great! I see nothing against backporting it to Emacs 25.1. But I
> must be honest, I'm pretty new to Emacs and its hard for me to judge the
> risks of doing so.
At this point mainly ask yourself:
1. Would the change require new, or change existing, documentation or APIs?
If so, it should go to master.
2. If it corrects existing behavior, refines that behavior, or removes a
stumbling block, it should go to emacs-25.
The length of time it takes to get emacs-25 fully ready should not be seen as
an opening for "features that could get ready during that time". I haven't
seen any attempts to commit such features, I just wanted to make this one
point clear.
--
John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-26 0:23 ` Alain Schneble
2015-12-26 0:51 ` John Wiegley
@ 2015-12-26 1:06 ` Lars Ingebrigtsen
2015-12-26 10:12 ` Eli Zaretskii
1 sibling, 1 reply; 18+ messages in thread
From: Lars Ingebrigtsen @ 2015-12-26 1:06 UTC (permalink / raw)
To: Alain Schneble; +Cc: 22044
Alain Schneble <a.s@realize.ch> writes:
> Thanks, great! I see nothing against backporting it to Emacs 25.1. But
> I must be honest, I'm pretty new to Emacs and its hard for me to judge
> the risks of doing so. But with all the new tests in place, I don't
> expect it breaks anything fundamentally; the changes were mainly related
> to how fragments and relative URLs are handled. These are more edge
> cases and I guess it would also only affect some rare edge cases - if at
> all.
>
> Shall I to elaborate more on this? What do you think?
I think it should probably be backported, but:
> By the way, the 25.1 branch still uses the old test directory layout...
How does one do that, then? :-)
If somebody feels like doing it, the two commits are
b792ecea1715e080ad8e232d3d154b8a25d2edfb
1dee11d874de5ff3d5634e1629054c4398b27b72
It's two commits because I hit the wrong buttons in vc-mode. :-/
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-25 23:51 ` Lars Ingebrigtsen
2015-12-26 0:23 ` Alain Schneble
@ 2015-12-26 9:57 ` Eli Zaretskii
1 sibling, 0 replies; 18+ messages in thread
From: Eli Zaretskii @ 2015-12-26 9:57 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: a.s, 22044
> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 26 Dec 2015 00:51:14 +0100
> Cc: 22044@debbugs.gnu.org
>
> Thanks; applied and pushed (to trunk). It might make sense to backport
> to Emacs 25.1 -- do you think that's a good idea?
Yes, I think so.
Thanks.
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-26 1:06 ` Lars Ingebrigtsen
@ 2015-12-26 10:12 ` Eli Zaretskii
2015-12-26 13:53 ` Lars Ingebrigtsen
0 siblings, 1 reply; 18+ messages in thread
From: Eli Zaretskii @ 2015-12-26 10:12 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: a.s, 22044
> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 26 Dec 2015 02:06:20 +0100
> Cc: 22044@debbugs.gnu.org
>
> > By the way, the 25.1 branch still uses the old test directory layout...
>
> How does one do that, then? :-)
I'd simply apply the test suite changes by hand using "M-|".
Please don't forget including "backport" in the commit log message.
Thanks.
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-26 0:51 ` John Wiegley
@ 2015-12-26 10:51 ` Alain Schneble
0 siblings, 0 replies; 18+ messages in thread
From: Alain Schneble @ 2015-12-26 10:51 UTC (permalink / raw)
To: John Wiegley; +Cc: Lars Ingebrigtsen, 22044
John Wiegley <jwiegley@gmail.com> writes:
>>>>>> Alain Schneble <a.s@realize.ch> writes:
>
>> Thanks, great! I see nothing against backporting it to Emacs 25.1. But I
>> must be honest, I'm pretty new to Emacs and its hard for me to judge the
>> risks of doing so.
>
> At this point mainly ask yourself:
>
> 1. Would the change require new, or change existing, documentation or APIs?
> If so, it should go to master.
The bugfix does not change any public interface. And the documented
behavior is expected to remain the same.
> 2. If it corrects existing behavior, refines that behavior, or removes a
> stumbling block, it should go to emacs-25.
It clearly corrects existing behavior and adds a bunch of new test
cases. From that point of view, it shoud go to emacs-25.
> The length of time it takes to get emacs-25 fully ready should not be seen as
> an opening for "features that could get ready during that time". I haven't
> seen any attempts to commit such features, I just wanted to make this one
> point clear.
I see your point - thanks for clarifying this. But as stated above, I
don't think it falls into this category of changes.
Thanks!
Alain
^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986
2015-12-26 10:12 ` Eli Zaretskii
@ 2015-12-26 13:53 ` Lars Ingebrigtsen
0 siblings, 0 replies; 18+ messages in thread
From: Lars Ingebrigtsen @ 2015-12-26 13:53 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: a.s, 22044
Eli Zaretskii <eliz@gnu.org> writes:
> I'd simply apply the test suite changes by hand using "M-|".
>
> Please don't forget including "backport" in the commit log message.
Aargh. Guess what I forgot. :-/
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2015-12-26 13:53 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-28 21:53 bug#22044: 24.4; url-expand.el and url-parse.el not conforming to RFC 3986 Alain Schneble
2015-11-28 23:42 ` Alain Schneble
2015-12-25 18:01 ` Lars Ingebrigtsen
2015-12-25 19:45 ` Eli Zaretskii
2015-12-25 19:54 ` Lars Ingebrigtsen
2015-12-25 19:58 ` Lars Ingebrigtsen
2015-12-25 22:27 ` Alain Schneble
2015-12-25 22:39 ` Lars Ingebrigtsen
2015-12-25 22:48 ` Alain Schneble
2015-12-25 23:44 ` Alain Schneble
2015-12-25 23:51 ` Lars Ingebrigtsen
2015-12-26 0:23 ` Alain Schneble
2015-12-26 0:51 ` John Wiegley
2015-12-26 10:51 ` Alain Schneble
2015-12-26 1:06 ` Lars Ingebrigtsen
2015-12-26 10:12 ` Eli Zaretskii
2015-12-26 13:53 ` Lars Ingebrigtsen
2015-12-26 9:57 ` Eli Zaretskii
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).