Add support for base64url variant

Add support for base64url variant

[Pierre Téchoueyres]

[-- Attachment #1: Type: text/plain, Size: 439 bytes --]


I'm actually working on an emacs library to work with JWT (JSON Web
Token) as described in RFC 7519 / 7797.  To achive that I must use an
variant of base 64 encoding named base64url as described in RFC4648.

Actually Emacs doesn't provide this variant.  I've attached an attempt
to implement it at C level.

I'm avare that my patch is far from perfect and I would welcome any
help or comments to improve it.

Thanks in advance.  Pierre.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: base64url --]
[-- Type: text/x-patch, Size: 24812 bytes --]

From 55a38a1d32758ac31746c2c9c8d54934f36c55ed Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Pierre=20T=C3=A9choueyres?= <pierre.techoueyres@free.fr>
Date: Tue, 21 May 2019 23:00:13 +0200
Subject: [PATCH] Add support for base64url variant.

Implement the RFC4648 variant of base64 encoding used in URL (base64url).

* doc/lispref/text.texi: add notice for optionals parameters of
  base64-(encode|decode)-(string|region).
* etc/NEWS: announce variants for previous functions.
* src/fns.c (base64-encode-region, base64-encode-string) : add
  optional parameters to manage padding and url variant.
  (base64-decode-region, base64-decode-string): add optional parameter
  to indicate use of url-variant.
  (base64-encode-1): add parameters to manage padding and url variant.
  (base64-decode-1): add parameter to manage url variant.
* test/src/fns-tests.el: add tests for encoding / decoding variants.
---
 doc/lispref/text.texi |  28 +++++++--
 etc/NEWS              |   7 +++
 src/fns.c             | 129 ++++++++++++++++++++++++++++++-----------
 test/src/fns-tests.el | 131 ++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 257 insertions(+), 38 deletions(-)

diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi
index 278bc3c268..30b458c19e 100644
--- a/doc/lispref/text.texi
+++ b/doc/lispref/text.texi
@@ -4541,10 +4541,10 @@ Base 64
 usually written by technical experts acting on their own initiative,
 and are traditionally written in a pragmatic, experience-driven
 manner.
-}2045.  This section describes the functions for
+}2045 and also in RFC4648.  This section describes the functions for
 converting to and from this code.
 
-@deffn Command base64-encode-region beg end &optional no-line-break
+@deffn Command base64-encode-region beg end &optional no-line-break no-pad url-variant
 This function converts the region from @var{beg} to @var{end} into base
 64 code.  It returns the length of the encoded text.  An error is
 signaled if a character in the region is multibyte, i.e., in a
@@ -4556,9 +4556,15 @@ Base 64
 text, to avoid overlong lines.  However, if the optional argument
 @var{no-line-break} is non-@code{nil}, these newlines are not added, so
 the output is just one long line.
+
+If optional argument @var{no-pad} is set then padding isn't
+generated.
+
+If optional argument @var{url-variant} is set, then chars @code{+} and
+@code{/} are replaced by @code{-} and @code{_} (See RFC4648).
 @end deffn
 
-@defun base64-encode-string string &optional no-line-break
+@defun base64-encode-string string &optional no-line-break no-pad url-variant
 This function converts the string @var{string} into base 64 code.  It
 returns a string containing the encoded text.  As for
 @code{base64-encode-region}, an error is signaled if a character in the
@@ -4568,22 +4574,34 @@ Base 64
 text, to avoid overlong lines.  However, if the optional argument
 @var{no-line-break} is non-@code{nil}, these newlines are not added, so
 the result string is just one long line.
+
+If optional argument @var{no-pad} is set then padding isn't
+generated.
+
+If optional argument @var{url-variant} is set then chars @code{+} and
+@code{/} are replaced by @code{-} abd @code{_} (See RFC4648).
 @end defun
 
-@deffn Command base64-decode-region beg end
+@deffn Command base64-decode-region beg end &optional url-variant
 This function converts the region from @var{beg} to @var{end} from base
 64 code into the corresponding decoded text.  It returns the length of
 the decoded text.
 
 The decoding functions ignore newline characters in the encoded text.
+
+If optional argument @var{url-variant} is set then padding become
+optionnal and url variant is used (See RFC4648).
 @end deffn
 
-@defun base64-decode-string string
+@defun base64-decode-string string &optional url-variant
 This function converts the string @var{string} from base 64 code into
 the corresponding decoded text.  It returns a unibyte string containing the
 decoded text.
 
 The decoding functions ignore newline characters in the encoded text.
+
+If optional argument @var{url-variant} is set then padding become
+optionnal and url variant is used (See RFC4648).
 @end defun
 
 @node Checksum/Hash
diff --git a/etc/NEWS b/etc/NEWS
index 72702a9aaa..a87c7d140a 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -364,6 +364,13 @@ in tooltips, as it is not useful there.
 There are 2 new buffer local variables and 1 face to customize this
 mode they are described in the manual "(emacs) Display".
 
++++
+** Functions 'base64-(encode|decode)-(string|region)' now manage url variant (RFC4648)
+The functions 'base64-encode-(region|string)' now have optionals
+arguments to mange padding and url-variant.
+Mirror function 'base64-decode-(region|string)' now have an optional
+argument url-variant to manage this RFC.
+
 \f
 * Editing Changes in Emacs 27.1
 
diff --git a/src/fns.c b/src/fns.c
index c3202495da..f868f13214 100644
--- a/src/fns.c
+++ b/src/fns.c
@@ -3171,7 +3171,7 @@ #define MIME_LINE_LENGTH 76
 #define IS_ASCII(Character) \
   ((Character) < 128)
 #define IS_BASE64(Character) \
-  (IS_ASCII (Character) && base64_char_to_value[Character] >= 0)
+  (IS_ASCII (Character) && b64_char_to_value[Character] >= 0)
 #define IS_BASE64_IGNORABLE(Character) \
   ((Character) == ' ' || (Character) == '\t' || (Character) == '\n' \
    || (Character) == '\f' || (Character) == '\r')
@@ -3204,6 +3204,17 @@ #define READ_QUADRUPLET_BYTE(retval)	\
   '8', '9', '+', '/'					/* 60-63 */
 };
 
+static const char base64url_value_to_char[64] =
+{
+  'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J',	/*  0- 9 */
+  'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T',	/* 10-19 */
+  'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd',	/* 20-29 */
+  'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n',	/* 30-39 */
+  'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x',	/* 40-49 */
+  'y', 'z', '0', '1', '2', '3', '4', '5', '6', '7',	/* 50-59 */
+  '8', '9', '-', '_'					/* 60-63 */
+};
+
 /* Table of base64 values for first 128 characters.  */
 static const short base64_char_to_value[128] =
 {
@@ -3222,6 +3233,23 @@ #define READ_QUADRUPLET_BYTE(retval)	\
   49,  50,  51,  -1,  -1,  -1,  -1,  -1			/* 120-127 */
 };
 
+static const short base64url_char_to_value[128] =
+{
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*   0-  9 */
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*  10- 19 */
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*  20- 29 */
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*  30- 39 */
+  -1,  -1,  -1,  -1,  -1,  62,  -1,  -1,  52,  53,	/*  40- 49 */
+  54,  55,  56,  57,  58,  59,  60,  61,  -1,  -1,	/*  50- 59 */
+  -1,  -1,  -1,  -1,  -1,  0,   1,   2,   3,   4,	/*  60- 69 */
+  5,   6,   7,   8,   9,   10,  11,  12,  13,  14,	/*  70- 79 */
+  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,	/*  80- 89 */
+  25,  -1,  -1,  -1,  -1,  63,  -1,  26,  27,  28,	/*  90- 99 */
+  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,	/* 100-109 */
+  39,  40,  41,  42,  43,  44,  45,  46,  47,  48,	/* 110-119 */
+  49,  50,  51,  -1,  -1,  -1,  -1,  -1                 /* 120-127 */
+};
+
 /* The following diagram shows the logical steps by which three octets
    get transformed into four base64 characters.
 
@@ -3241,17 +3269,21 @@ #define READ_QUADRUPLET_BYTE(retval)	\
    base64 characters.  */
 
 
-static ptrdiff_t base64_encode_1 (const char *, char *, ptrdiff_t, bool, bool);
+static ptrdiff_t base64_encode_1 (const char *, char *, ptrdiff_t, bool, bool,
+				  bool, bool);
 static ptrdiff_t base64_decode_1 (const char *, char *, ptrdiff_t, bool,
-				  ptrdiff_t *);
+				  bool, ptrdiff_t *);
 
 DEFUN ("base64-encode-region", Fbase64_encode_region, Sbase64_encode_region,
-       2, 3, "r",
+       2, 5, "r",
        doc: /* Base64-encode the region between BEG and END.
 Return the length of the encoded text.
 Optional third argument NO-LINE-BREAK means do not break long lines
-into shorter lines.  */)
-  (Lisp_Object beg, Lisp_Object end, Lisp_Object no_line_break)
+into shorter lines.
+Optional fourth argument NO-PAD means do not add padding char =.
+Optional fifth argument URL-VARIANT means use Url variant (RFC4648).  */)
+  (Lisp_Object beg, Lisp_Object end, Lisp_Object no_line_break,
+   Lisp_Object no_pad, Lisp_Object url_variant)
 {
   char *encoded;
   ptrdiff_t allength, length;
@@ -3275,6 +3307,7 @@ DEFUN ("base64-encode-region", Fbase64_encode_region, Sbase64_encode_region,
   encoded = SAFE_ALLOCA (allength);
   encoded_length = base64_encode_1 ((char *) BYTE_POS_ADDR (ibeg),
 				    encoded, length, NILP (no_line_break),
+				    NILP(no_pad), !NILP(url_variant),
 				    !NILP (BVAR (current_buffer, enable_multibyte_characters)));
   if (encoded_length > allength)
     emacs_abort ();
@@ -3306,11 +3339,14 @@ DEFUN ("base64-encode-region", Fbase64_encode_region, Sbase64_encode_region,
 }
 
 DEFUN ("base64-encode-string", Fbase64_encode_string, Sbase64_encode_string,
-       1, 2, 0,
+       1, 4, 0,
        doc: /* Base64-encode STRING and return the result.
 Optional second argument NO-LINE-BREAK means do not break long lines
-into shorter lines.  */)
-  (Lisp_Object string, Lisp_Object no_line_break)
+into shorter lines.
+Optional third argument NO-PAD means do not add padding char =.
+Optional fourth argument URL-VARIANT means use Url variant (RFC4648).  */)
+  (Lisp_Object string, Lisp_Object no_line_break,
+   Lisp_Object no_pad, Lisp_Object url_variant)
 {
   ptrdiff_t allength, length, encoded_length;
   char *encoded;
@@ -3331,6 +3367,7 @@ DEFUN ("base64-encode-string", Fbase64_encode_string, Sbase64_encode_string,
 
   encoded_length = base64_encode_1 (SSDATA (string),
 				    encoded, length, NILP (no_line_break),
+				    NILP(no_pad), !NILP(url_variant),
 				    STRING_MULTIBYTE (string));
   if (encoded_length > allength)
     emacs_abort ();
@@ -3349,7 +3386,8 @@ DEFUN ("base64-encode-string", Fbase64_encode_string, Sbase64_encode_string,
 
 static ptrdiff_t
 base64_encode_1 (const char *from, char *to, ptrdiff_t length,
-		 bool line_break, bool multibyte)
+		 bool line_break, bool pad, bool url_variant,
+		 bool multibyte)
 {
   int counter = 0;
   ptrdiff_t i = 0;
@@ -3357,6 +3395,7 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
   int c;
   unsigned int value;
   int bytes;
+  char const *b64_value_to_char = (url_variant) ? base64url_value_to_char : base64_value_to_char;
 
   while (i < length)
     {
@@ -3387,16 +3426,18 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
 
       /* Process first byte of a triplet.  */
 
-      *e++ = base64_value_to_char[0x3f & c >> 2];
+      *e++ = b64_value_to_char[0x3f & c >> 2];
       value = (0x03 & c) << 4;
 
       /* Process second byte of a triplet.  */
 
       if (i == length)
 	{
-	  *e++ = base64_value_to_char[value];
-	  *e++ = '=';
-	  *e++ = '=';
+	  *e++ = b64_value_to_char[value];
+	  if (pad) {
+	    *e++ = '=';
+	    *e++ = '=';
+	  }
 	  break;
 	}
 
@@ -3412,15 +3453,17 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
       else
 	c = from[i++];
 
-      *e++ = base64_value_to_char[value | (0x0f & c >> 4)];
+      *e++ = b64_value_to_char[value | (0x0f & c >> 4)];
       value = (0x0f & c) << 2;
 
       /* Process third byte of a triplet.  */
 
       if (i == length)
 	{
-	  *e++ = base64_value_to_char[value];
-	  *e++ = '=';
+	  *e++ = b64_value_to_char[value];
+	  if (pad) {
+	    *e++ = '=';
+	  }
 	  break;
 	}
 
@@ -3436,8 +3479,8 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
       else
 	c = from[i++];
 
-      *e++ = base64_value_to_char[value | (0x03 & c >> 6)];
-      *e++ = base64_value_to_char[0x3f & c];
+      *e++ = b64_value_to_char[value | (0x03 & c >> 6)];
+      *e++ = b64_value_to_char[0x3f & c];
     }
 
   return e - to;
@@ -3445,11 +3488,13 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
 
 
 DEFUN ("base64-decode-region", Fbase64_decode_region, Sbase64_decode_region,
-       2, 2, "r",
+       2, 3, "r",
        doc: /* Base64-decode the region between BEG and END.
 Return the length of the decoded text.
-If the region can't be decoded, signal an error and don't modify the buffer.  */)
-  (Lisp_Object beg, Lisp_Object end)
+If the region can't be decoded, signal an error and don't modify the buffer.
+Optional third argument URL-VARIANT define if base64Url variant will be used
+see RFC4648.  */)
+     (Lisp_Object beg, Lisp_Object end, Lisp_Object url_variant)
 {
   ptrdiff_t ibeg, iend, length, allength;
   char *decoded;
@@ -3474,7 +3519,7 @@ DEFUN ("base64-decode-region", Fbase64_decode_region, Sbase64_decode_region,
 
   move_gap_both (XFIXNAT (beg), ibeg);
   decoded_length = base64_decode_1 ((char *) BYTE_POS_ADDR (ibeg),
-				    decoded, length,
+				    decoded, length, !NILP(url_variant),
 				    multibyte, &inserted_chars);
   if (decoded_length > allength)
     emacs_abort ();
@@ -3508,9 +3553,11 @@ DEFUN ("base64-decode-region", Fbase64_decode_region, Sbase64_decode_region,
 }
 
 DEFUN ("base64-decode-string", Fbase64_decode_string, Sbase64_decode_string,
-       1, 1, 0,
-       doc: /* Base64-decode STRING and return the result.  */)
-  (Lisp_Object string)
+       1, 2, 0,
+       doc: /* Base64-decode STRING and return the result
+Optional argument URL-VARIANT define if base64Url variant will be used
+see RFC4648.  */)
+     (Lisp_Object string, Lisp_Object url_variant)
 {
   char *decoded;
   ptrdiff_t length, decoded_length;
@@ -3525,7 +3572,7 @@ DEFUN ("base64-decode-string", Fbase64_decode_string, Sbase64_decode_string,
 
   /* The decoded result should be unibyte. */
   decoded_length = base64_decode_1 (SSDATA (string), decoded, length,
-				    0, NULL);
+				    !NILP(url_variant), 0, NULL);
   if (decoded_length > length)
     emacs_abort ();
   else if (decoded_length >= 0)
@@ -3547,6 +3594,7 @@ DEFUN ("base64-decode-string", Fbase64_decode_string, Sbase64_decode_string,
 
 static ptrdiff_t
 base64_decode_1 (const char *from, char *to, ptrdiff_t length,
+		 bool url_variant,
 		 bool multibyte, ptrdiff_t *nchars_return)
 {
   ptrdiff_t i = 0;		/* Used inside READ_QUADRUPLET_BYTE */
@@ -3554,6 +3602,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
   unsigned char c;
   unsigned long value;
   ptrdiff_t nchars = 0;
+  short const *b64_char_to_value = (url_variant) ? base64url_char_to_value : base64_char_to_value;
 
   while (1)
     {
@@ -3563,7 +3612,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       if (!IS_BASE64 (c))
 	return -1;
-      value = base64_char_to_value[c] << 18;
+      value = b64_char_to_value[c] << 18;
 
       /* Process second byte of a quadruplet.  */
 
@@ -3571,7 +3620,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       if (!IS_BASE64 (c))
 	return -1;
-      value |= base64_char_to_value[c] << 12;
+      value |= b64_char_to_value[c] << 12;
 
       c = (unsigned char) (value >> 16);
       if (multibyte && c >= 128)
@@ -3582,7 +3631,14 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       /* Process third byte of a quadruplet.  */
 
-      READ_QUADRUPLET_BYTE (-1);
+      if (!url_variant)
+	{
+	  READ_QUADRUPLET_BYTE (-1);
+	}
+      else
+	{
+	  READ_QUADRUPLET_BYTE (e-to);
+	}
 
       if (c == '=')
 	{
@@ -3595,7 +3651,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       if (!IS_BASE64 (c))
 	return -1;
-      value |= base64_char_to_value[c] << 6;
+      value |= b64_char_to_value[c] << 6;
 
       c = (unsigned char) (0xff & value >> 8);
       if (multibyte && c >= 128)
@@ -3606,14 +3662,21 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       /* Process fourth byte of a quadruplet.  */
 
-      READ_QUADRUPLET_BYTE (-1);
+      if (!url_variant)
+	{
+	  READ_QUADRUPLET_BYTE (-1);
+	}
+      else
+	{
+	  READ_QUADRUPLET_BYTE (e-to);
+	}
 
       if (c == '=')
 	continue;
 
       if (!IS_BASE64 (c))
 	return -1;
-      value |= base64_char_to_value[c];
+      value |= b64_char_to_value[c];
 
       c = (unsigned char) (0xff & value);
       if (multibyte && c >= 128)
diff --git a/test/src/fns-tests.el b/test/src/fns-tests.el
index 6ebab4287f..3301129567 100644
--- a/test/src/fns-tests.el
+++ b/test/src/fns-tests.el
@@ -233,6 +233,137 @@ fns-tests-func-arity
   (should (equal (func-arity (eval (lambda (x &optional y)) t)) '(1 . 2)))
   (should (equal (func-arity 'let) '(1 . unevalled))))
 
+(defun string-repeat (s o)
+  (apply 'concat (make-list o s)))
+
+(ert-deftest fns-tests-base64-encode-string ()
+  ;; standard variant RFC2045
+  (should (equal (base64-encode-string "") ""))
+  (should (equal (base64-encode-string "f") "Zg=="))
+  (should (equal (base64-encode-string "fo") "Zm8="))
+  (should (equal (base64-encode-string "foo") "Zm9v"))
+  (should (equal (base64-encode-string "foob") "Zm9vYg=="))
+  (should (equal (base64-encode-string "fooba") "Zm9vYmE="))
+  (should (equal (base64-encode-string "foobar") "Zm9vYmFy"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7e") "FPucA9l+"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7f") "FPucA9l/"))
+
+  ;; no line break
+  (should (equal (base64-encode-string "") ""))
+  (should (equal (base64-encode-string (string-repeat "f" 100) t) (concat (string-repeat "Zm" 66) "Zg==")))
+  (should (equal (base64-encode-string (string-repeat "fo" 50) t) (concat (string-repeat "Zm9mb2Zv" 16) "Zm9mbw==")))
+  (should (equal (base64-encode-string (string-repeat "foo" 25) t) (string-repeat "Zm9v" 25)))
+  (should (equal (base64-encode-string (string-repeat "foob" 15) t) (string-repeat "Zm9vYmZvb2Jmb29i" 5)))
+  (should (equal (base64-encode-string (string-repeat "fooba" 15) t) (string-repeat "Zm9vYmFmb29iYWZvb2Jh" 5)))
+  (should (equal (base64-encode-string (string-repeat "foobar" 15) t) (concat (string-repeat "Zm9vYmFyZm9vYmFy" 7) "Zm9vYmFy")))
+  (should (equal (base64-encode-string (string-repeat "\x14\xfb\x9c\x03\xd9\x7e" 10) t) (string-repeat "FPucA9l+" 10)))
+  (should (equal (base64-encode-string (string-repeat "\x14\xfb\x9c\x03\xd9\x7f" 10) t) (string-repeat "FPucA9l/" 10)))
+
+  ;; no paddign
+  (should (equal (base64-encode-string "") ""))
+  (should (equal (base64-encode-string "f" nil t) "Zg"))
+  (should (equal (base64-encode-string "fo" nil t) "Zm8"))
+  (should (equal (base64-encode-string "foo" nil t) "Zm9v"))
+  (should (equal (base64-encode-string "foob" nil t) "Zm9vYg"))
+  (should (equal (base64-encode-string "fooba" nil t) "Zm9vYmE"))
+  (should (equal (base64-encode-string "foobar" nil t) "Zm9vYmFy"))
+
+  ;; url variant wih padding
+  (should (equal (base64-encode-string "" nil nil t) ""))
+  (should (equal (base64-encode-string "f" nil nil t) "Zg=="))
+  (should (equal (base64-encode-string "fo" nil nil t) "Zm8="))
+  (should (equal (base64-encode-string "foo" nil nil t) "Zm9v"))
+  (should (equal (base64-encode-string "foob" nil nil t) "Zm9vYg=="))
+  (should (equal (base64-encode-string "fooba" nil nil t) "Zm9vYmE="))
+  (should (equal (base64-encode-string "foobar" nil nil t) "Zm9vYmFy"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7e" nil nil t) "FPucA9l-"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7f" nil nil t) "FPucA9l_"))
+
+  ;; url variant no padding
+  (should (equal (base64-encode-string "" nil t t) ""))
+  (should (equal (base64-encode-string "f" nil t t) "Zg"))
+  (should (equal (base64-encode-string "fo" nil t t) "Zm8"))
+  (should (equal (base64-encode-string "foo" nil t t) "Zm9v"))
+  (should (equal (base64-encode-string "foob" nil t t) "Zm9vYg"))
+  (should (equal (base64-encode-string "fooba" nil t t) "Zm9vYmE"))
+  (should (equal (base64-encode-string "foobar" nil t t) "Zm9vYmFy"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7e" nil t t) "FPucA9l-"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7f" nil t t) "FPucA9l_"))
+
+
+  ;; url variant no line break no padding
+  (should (equal (base64-encode-string (string-repeat "f" 100) t t t) (concat (string-repeat "Zm" 66) "Zg")))
+  (should (equal (base64-encode-string (string-repeat "fo" 50) t t t) (concat (string-repeat "Zm9mb2Zv" 16) "Zm9mbw")))
+  (should (equal (base64-encode-string (string-repeat "foo" 25) t t t) (string-repeat "Zm9v" 25)))
+  (should (equal (base64-encode-string (string-repeat "foob" 15) t t t) (string-repeat "Zm9vYmZvb2Jmb29i" 5)))
+  (should (equal (base64-encode-string (string-repeat "fooba" 15) t t t) (string-repeat "Zm9vYmFmb29iYWZvb2Jh" 5)))
+  (should (equal (base64-encode-string (string-repeat "foobar" 15) t t t) (concat (string-repeat "Zm9vYmFyZm9vYmFy" 7) "Zm9vYmFy")))
+  (should (equal (base64-encode-string (string-repeat "\x14\xfb\x9c\x03\xd9\x7e" 10) t t t) (string-repeat "FPucA9l-" 10)))
+  (should (equal (base64-encode-string (string-repeat "\x14\xfb\x9c\x03\xd9\x7f" 10) t t t) (string-repeat "FPucA9l_" 10)))
+
+  )
+
+(ert-deftest fns-tests-base64-decode-string ()
+  ;; standard variant RFC2045
+  (should (equal (base64-decode-string "") ""))
+  (should (equal (base64-decode-string "Zg==") "f"))
+  (should (equal (base64-decode-string "Zm8=") "fo"))
+  (should (equal (base64-decode-string "Zm9v") "foo"))
+  (should (equal (base64-decode-string "Zm9vYg==") "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE=") "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy") "foobar"))
+  (should (equal (base64-decode-string "FPucA9l+") "\x14\xfb\x9c\x03\xd9\x7e"))
+  (should (equal (base64-decode-string "FPucA9l/") "\x14\xfb\x9c\x03\xd9\x7f"))
+
+  ;; no paddign
+  (should (equal (base64-decode-string "" t) ""))
+  (should (equal (base64-decode-string "Zg" t) "f"))
+  (should (equal (base64-decode-string "Zm8" t) "fo"))
+  (should (equal (base64-decode-string "Zm9v" t) "foo"))
+  (should (equal (base64-decode-string "Zm9vYg" t) "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE" t) "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy" t) "foobar"))
+
+  ;; url variant wih padding
+  (should (equal (base64-decode-string "") ""))
+  (should (equal (base64-decode-string "Zg==" t) "f") )
+  (should (equal (base64-decode-string "Zm8=" t) "fo"))
+  (should (equal (base64-decode-string "Zm9v" t) "foo"))
+  (should (equal (base64-decode-string "Zm9vYg==" t) "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE=" t) "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy" t) "foobar"))
+  (should (equal (base64-decode-string "FPucA9l-" t) "\x14\xfb\x9c\x03\xd9\x7e"))
+  (should (equal (base64-decode-string "FPucA9l_" t) "\x14\xfb\x9c\x03\xd9\x7f"))
+
+  ;; url variant no padding
+  (should (equal (base64-decode-string "") ""))
+  (should (equal (base64-decode-string "Zg" t) "f"))
+  (should (equal (base64-decode-string "Zm8" t) "fo"))
+  (should (equal (base64-decode-string "Zm9v" t) "foo"))
+  (should (equal (base64-decode-string "Zm9vYg" t) "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE" t) "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy" t) "foobar"))
+  (should (equal (base64-decode-string "FPucA9l-" t) "\x14\xfb\x9c\x03\xd9\x7e"))
+  (should (equal (base64-decode-string "FPucA9l_" t) "\x14\xfb\x9c\x03\xd9\x7f"))
+
+
+  ;; url variant no line break no padding
+  (should (equal (base64-decode-string (concat (string-repeat "Zm" 66) "Zg") t) (string-repeat "f" 100)))
+  (should (equal (base64-decode-string (concat (string-repeat "Zm9mb2Zv" 16) "Zm9mbw") t) (string-repeat "fo" 50)))
+  (should (equal (base64-decode-string (string-repeat "Zm9v" 25) t) (string-repeat "foo" 25)))
+  (should (equal (base64-decode-string (string-repeat "Zm9vYmZvb2Jmb29i" 5) t) (string-repeat "foob" 15)))
+  (should (equal (base64-decode-string (string-repeat "Zm9vYmFmb29iYWZvb2Jh" 5) t) (string-repeat "fooba" 15)))
+  (should (equal (base64-decode-string (concat (string-repeat "Zm9vYmFyZm9vYmFy" 7) "Zm9vYmFy") t) (string-repeat "foobar" 15)))
+  (should (equal (base64-decode-string (string-repeat "FPucA9l-" 10) t) (string-repeat "\x14\xfb\x9c\x03\xd9\x7e" 10)))
+  (should (equal (base64-decode-string (string-repeat "FPucA9l_" 10) t) (string-repeat "\x14\xfb\x9c\x03\xd9\x7f" 10)))
+
+  ;; errors check
+  (should (eq :got-error (condition-case () (base64-decode-string "Zg=") (error :got-error))))
+  (should (eq :got-error (condition-case () (base64-decode-string "Zm9vYmE") (error :got-error))))
+  (should (eq :got-error (condition-case () (base64-decode-string "Zm9vYmFy=") (error :got-error))))
+  (should (eq :got-error (condition-case () (base64-decode-string "Zg=Zg=") (error :got-error))))
+  )
+
 (ert-deftest fns-tests-hash-buffer ()
   (should (equal (sha1 "foo") "0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"))
   (should (equal (with-temp-buffer
-- 
2.21.0

Re: Add support for base64url variant

[Eli Zaretskii]

> From: Pierre Téchoueyres <pierre.techoueyres@free.fr>
> Date: Wed, 22 May 2019 00:32:19 +0200
> 
> I'm actually working on an emacs library to work with JWT (JSON Web
> Token) as described in RFC 7519 / 7797.  To achive that I must use an
> variant of base 64 encoding named base64url as described in RFC4648.
> 
> Actually Emacs doesn't provide this variant.  I've attached an attempt
> to implement it at C level.
> 
> I'm avare that my patch is far from perfect and I would welcome any
> help or comments to improve it.

Thanks, but please avoid posting such large patches as long as you
don't have a copyright assignment for Emacs on file.  That's because
just by reading the patch, people will remember some of the
implementation, and will then have hard time to come up with a
different one, should your copyright assignment paperwork not come
through.

If you'd like to sign the forms, I will send them to you.  Then we
could revisit the patch when the legal paperwork is done.

Thanks again for working on this.

Re: Add support for base64url variant

[Pierre Téchoueyres]

Hello Eli,
> 
> Thanks, but please avoid posting such large patches as long as you
> don't have a copyright assignment for Emacs on file.

I have already signed paperwork for TRAMP and, I think, for Emacs.
But maybe there is something wrong with them ?

>  That's because
> just by reading the patch, people will remember some of the
> implementation, and will then have hard time to come up with a
> different one, should your copyright assignment paperwork not come
> through.
> 
> If you'd like to sign the forms, I will send them to you.  Then we
> could revisit the patch when the legal paperwork is done.

If you can't find my previous assignment, then yes send me them back.

> Thanks again for working on this.
Hope this could help.

I would like to have some discussion on how to improve some points :
- Is adding parameter to existing functions the way to go or is it 
better to add new ones for base64url
- I would like to improve base64-decode* in a way it could detect the 
variant but actually don't know how to do that.
- The documentations is not really what I would like to see (and there 
my English is faulty)

Thanks in advance.  Pierre.

Re: Add support for base64url variant

[Eli Zaretskii]

> Date: Wed, 22 May 2019 11:25:34 +0200
> From: Pierre Téchoueyres <pierre.techoueyres@free.fr>
> Cc: emacs-devel@gnu.org
> 
> Hello Eli,
> > 
> > Thanks, but please avoid posting such large patches as long as you
> > don't have a copyright assignment for Emacs on file.
> 
> I have already signed paperwork for TRAMP and, I think, for Emacs.
> But maybe there is something wrong with them ?

I do see the assignment for Tramp, but not for Emacs.

> If you can't find my previous assignment, then yes send me them back.

Will send off-list.

> I would like to have some discussion on how to improve some points :
> - Is adding parameter to existing functions the way to go or is it 
> better to add new ones for base64url

I think an optional argument will be fine.

> - I would like to improve base64-decode* in a way it could detect the 
> variant but actually don't know how to do that.

Maybe someone else will have an idea.  Is such an algorithmic
detection described someplace?

> - The documentations is not really what I would like to see (and there 
> my English is faulty)

Don't worry about that, we will polish it when the time comes.  Thanks
for providing the documentation in the first place.

Re: Add support for base64url variant

[Pierre Téchoueyres]

Eli Zaretskii <eliz@gnu.org> writes:

>> ...
>> I would like to have some discussion on how to improve some points :
>> - Is adding parameter to existing functions the way to go or is it 
>> better to add new ones for base64url
>
> I think an optional argument will be fine.

Except here I've added two, and one that doesn't really mean anything if
the second isn't set.  ie. you should not (at least without breaking the
RFC) generate an base64 string without padding.  At first many
parameters seemed to me a good thing : limitted patch and flexibility.
But now I'm no more sure ...


>> - I would like to improve base64-decode* in a way it could detect the 
>> variant but actually don't know how to do that.
>
> Maybe someone else will have an idea.  Is such an algorithmic
> detection described someplace?
>
None I'm aware of.  I was thinking to something like that :
- define two boolean.  One saying you're on crude base64 another saying
  you're on base64url variant. 
- initialize them as false.
- start decoding.
- when finding crude base64  chars (/ or +) set the base64 to true,
- when finding specific url variant chars (- or _) set base64url to true,

try to decode until the end of data.  On parts where padding is checked
do it only if base64 is true.

But this approch could fail on the following cases :
- a mix of chars from both variant without padding (no checks but
  obviously wrong)
- absence of chars from any variant (here I can't decide for the
  necessity of padding).

I would also bring your attention on the part where I dynamically assign
pointers on specialized arrays for encoding (resp. decoding).

ex: line 244 of patch

char const *b64_value_to_char = (url_variant) ? base64url_value_to_char : base64_value_to_char;


Before my change there were static const, so I suppose compiler could
have inlined them or at least stored on some cache.  But now ...
So I'm a little scarried by the possible lost of performance.  If anyone
has some hint on how I could benchmark this (other than by the naive way
which could result in my data where all in cache ...)

Pierre.

Re: Add support for base64url variant

[Noam Postavsky]

On Thu, 23 May 2019 at 13:51, Pierre Téchoueyres
<pierre.techoueyres@free.fr> wrote:
>
> Eli Zaretskii <eliz@gnu.org> writes:
>
> >> ...
> >> I would like to have some discussion on how to improve some points :
> >> - Is adding parameter to existing functions the way to go or is it
> >> better to add new ones for base64url
> >
> > I think an optional argument will be fine.
>
> Except here I've added two, and one that doesn't really mean anything if
> the second isn't set.  ie. you should not (at least without breaking the
> RFC) generate an base64 string without padding.  At first many
> parameters seemed to me a good thing : limitted patch and flexibility.
> But now I'm no more sure ...

You could leave out the NO-PAD argument, it's easy enough for the
calling Lisp code to delete a couple of "=" chars if really needed.

> I would also bring your attention on the part where I dynamically assign
> pointers on specialized arrays for encoding (resp. decoding).
>
> ex: line 244 of patch
>
> char const *b64_value_to_char = (url_variant) ? base64url_value_to_char : base64_value_to_char;
>
>
> Before my change there were static const, so I suppose compiler could
> have inlined them or at least stored on some cache.  But now ...
> So I'm a little scarried by the possible lost of performance.  If anyone
> has some hint on how I could benchmark this

I doubt it will have a measurable impact on performance. And even if
it did have some very tiny effect, what would you do about it?
Duplicate the whole encoding function, just to avoid a dynamic choice
of array? The difference would have to be pretty drastic to be worth
that, IMO.

Re: Add support for base64url variant

[Eli Zaretskii]

> From: Pierre Téchoueyres <pierre.techoueyres@free.fr>
> Date: Wed, 22 May 2019 00:32:19 +0200
> 
> +				    NILP(no_pad), !NILP(url_variant),

Please leave one blank between the name of a macro or function and the
following opening parenthesis.

> +	  if (pad) {
> +	    *e++ = '=';
> +	    *e++ = '=';
> +	  }

This is not our style of writing blocks in braces.  We use this style:

   if (pad)
     {
       *e++ = '=';
       ...

> +	  *e++ = b64_value_to_char[value];
> +	  if (pad) {
> +	    *e++ = '=';
> +	  }

Likewise.

I'd suggest to call the new argument base64url or somesuch, since
this is trhe official name.

Re: Add support for base64url variant

[Eli Zaretskii]

> From: pierre.techoueyres@free.fr (Pierre Téchoueyres)
> Cc: emacs-devel@gnu.org
> Date: Thu, 23 May 2019 19:51:00 +0200
> 
> > I think an optional argument will be fine.
> 
> Except here I've added two, and one that doesn't really mean anything if
> the second isn't set.

Then perhaps switch their order.

> >> - I would like to improve base64-decode* in a way it could detect the 
> >> variant but actually don't know how to do that.
> >
> > Maybe someone else will have an idea.  Is such an algorithmic
> > detection described someplace?
> >
> None I'm aware of.  I was thinking to something like that :
> - define two boolean.  One saying you're on crude base64 another saying
>   you're on base64url variant. 
> - initialize them as false.
> - start decoding.
> - when finding crude base64  chars (/ or +) set the base64 to true,
> - when finding specific url variant chars (- or _) set base64url to true,

Doesn't sound worth it.  Just an argument passed by the caller should
be fine.  The caller will have to know how the text is encoded.

> I would also bring your attention on the part where I dynamically assign
> pointers on specialized arrays for encoding (resp. decoding).

I don't see a problem with that.

> Before my change there were static const, so I suppose compiler could
> have inlined them or at least stored on some cache.  But now ...
> So I'm a little scarried by the possible lost of performance.  If anyone
> has some hint on how I could benchmark this (other than by the naive way
> which could result in my data where all in cache ...)

Emacs has benchmarking facilities, so you could time the code and see
if there's some performance penalty.  If the performance suffers too
much, we could always have a helper function which receives the array
as a parameter.

Re: Add support for base64url variant

[Pierre Téchoueyres]

Noam Postavsky <npostavs@gmail.com> writes:

> On Thu, 23 May 2019 at 13:51, Pierre Téchoueyres
> <pierre.techoueyres@free.fr> wrote:
>>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>> >> ...
>> >> I would like to have some discussion on how to improve some points :
>> >> - Is adding parameter to existing functions the way to go or is it
>> >> better to add new ones for base64url
>> >
>> > I think an optional argument will be fine.
>>
>> Except here I've added two, and one that doesn't really mean anything if
>> the second isn't set.  ie. you should not (at least without breaking the
>> RFC) generate an base64 string without padding.  At first many
>> parameters seemed to me a good thing : limitted patch and flexibility.
>> But now I'm no more sure ...
>
> You could leave out the NO-PAD argument, it's easy enough for the
> calling Lisp code to delete a couple of "=" chars if really needed.
>
This could be an option ... but I do see this as ugly adding characters
on one side to remove them from the other.

Maybe it could be better to split the functions then ? What about

(base64-encode-string STRING &optional NO-LINE-BREAK)
(base64url-encode-string STRING &optional NO-PAD)

and

(base64-encode-region BEG END &optional NO-LINE-BREAK)
(base64url-encode-region BEG END &optional NO-PAD)



>> I would also bring your attention on the part where I dynamically assign
>> pointers on specialized arrays for encoding (resp. decoding).
>>
>> ex: line 244 of patch
>>
>> char const *b64_value_to_char = (url_variant) ? base64url_value_to_char : base64_value_to_char;
>>
>>
>> Before my change there were static const, so I suppose compiler could
>> have inlined them or at least stored on some cache.  But now ...
>> So I'm a little scarried by the possible lost of performance.  If anyone
>> has some hint on how I could benchmark this
>
> I doubt it will have a measurable impact on performance. And even if
> it did have some very tiny effect, what would you do about it?
> Duplicate the whole encoding function, just to avoid a dynamic choice
> of array? The difference would have to be pretty drastic to be worth
> that, IMO.

Thanks, I hoped for something like your answer.

Re: Add support for base64url variant

[Pierre Téchoueyres]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Pierre Téchoueyres <pierre.techoueyres@free.fr>
>> Date: Wed, 22 May 2019 00:32:19 +0200
>> 
>> +				    NILP(no_pad), !NILP(url_variant),
>
> Please leave one blank between the name of a macro or function and the
> following opening parenthesis.

Fixed.

>
>> +	  if (pad) {
>> +	    *e++ = '=';
>> +	    *e++ = '=';
>> +	  }
>
> This is not our style of writing blocks in braces.  We use this style:
>
>    if (pad)
>      {
>        *e++ = '=';
>        ...
>
>> +	  *e++ = b64_value_to_char[value];
>> +	  if (pad) {
>> +	    *e++ = '=';
>> +	  }
>
> Likewise.
>

Fixed.

> I'd suggest to call the new argument base64url or somesuch, since
> this is trhe official name.
>
You mean in replacement of url_variant or b64_value_to_char ?

How should I send new versions of the patch ? As a full patch in
attachment like previously ?

Re: Add support for base64url variant

[Eli Zaretskii]

> From: pierre.techoueyres@free.fr (Pierre Téchoueyres)
> Cc: emacs-devel@gnu.org
> Date: Thu, 23 May 2019 21:37:01 +0200
> 
> > I'd suggest to call the new argument base64url or somesuch, since
> > this is trhe official name.
> >
> You mean in replacement of url_variant or b64_value_to_char ?

The former.

> How should I send new versions of the patch ? As a full patch in
> attachment like previously ?

Yes.  And please include the log messages (in the ChangeLog style).

Re: Add support for base64url variant

[Pierre Téchoueyres]

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: pierre.techoueyres@free.fr (Pierre Téchoueyres)
>> Cc: emacs-devel@gnu.org
>> Date: Thu, 23 May 2019 21:37:01 +0200
>> 
>> > I'd suggest to call the new argument base64url or somesuch, since
>> > this is trhe official name.
>> >
>> You mean in replacement of url_variant or b64_value_to_char ?
>
> The former.
>
>> How should I send new versions of the patch ? As a full patch in
>> attachment like previously ?
>
> Yes.  And please include the log messages (in the ChangeLog style).
>
>
You'll find two patches attached.
First one contains requested changes and, hoppefully, a valid Changelog

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: with parameters --]
[-- Type: text/x-patch, Size: 24899 bytes --]

From dcd03d67d4d7fa9d1f46554a6703ae562ef39ce5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Pierre=20T=C3=A9choueyres?= <pierre.techoueyres@free.fr>
Date: Tue, 21 May 2019 23:00:13 +0200
Subject: [PATCH] Add support for base64url variant.

Implement the RFC4648 variant of base64 encoding used with url.

* doc/lispref/text.texi (base64-encode-region, base64-encode-string,
  base64-decode-region, base64-decode-string): Adds notice for
  optionals parameters.

* etc/NEWS: Announce support for URL variant of base 64 functions.

* src/fns.c (base64-encode-region, base64-encode-string) : Adds
  optional parameters to manage padding and url variant.
  (base64-decode-region, base64-decode-string): Adds optional
  parameter to indicate use of url-variant.
  (base64-encode-1): Adds parameters to manage padding and url
  variant.
  (base64-decode-1): Adds parameter to manage url variant.
* test/src/fns-tests.el: Adds tests for encoding / decoding variants.
---
 doc/lispref/text.texi |  29 ++++++++--
 etc/NEWS              |   7 +++
 src/fns.c             | 131 +++++++++++++++++++++++++++++++-----------
 test/src/fns-tests.el | 131 ++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 260 insertions(+), 38 deletions(-)

diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi
index 278bc3c268..ca1bc4e2df 100644
--- a/doc/lispref/text.texi
+++ b/doc/lispref/text.texi
@@ -4541,10 +4541,10 @@ Base 64
 usually written by technical experts acting on their own initiative,
 and are traditionally written in a pragmatic, experience-driven
 manner.
-}2045.  This section describes the functions for
+}2045 and also in RFC4648.  This section describes the functions for
 converting to and from this code.
 
-@deffn Command base64-encode-region beg end &optional no-line-break
+@deffn Command base64-encode-region beg end &optional no-line-break no-pad base64url
 This function converts the region from @var{beg} to @var{end} into base
 64 code.  It returns the length of the encoded text.  An error is
 signaled if a character in the region is multibyte, i.e., in a
@@ -4556,9 +4556,15 @@ Base 64
 text, to avoid overlong lines.  However, if the optional argument
 @var{no-line-break} is non-@code{nil}, these newlines are not added, so
 the output is just one long line.
+
+If optional argument @var{no-pad} is set then padding isn't
+generated.
+
+If optional argument @var{base64url} is set, then chars @code{+} and
+@code{/} are replaced by @code{-} and @code{_} (See RFC4648).
 @end deffn
 
-@defun base64-encode-string string &optional no-line-break
+@defun base64-encode-string string &optional no-line-break no-pad base64url
 This function converts the string @var{string} into base 64 code.  It
 returns a string containing the encoded text.  As for
 @code{base64-encode-region}, an error is signaled if a character in the
@@ -4568,22 +4574,35 @@ Base 64
 text, to avoid overlong lines.  However, if the optional argument
 @var{no-line-break} is non-@code{nil}, these newlines are not added, so
 the result string is just one long line.
+
+If optional argument @var{no-pad} is non-@code{nil} then padding isn't
+generated.
+
+If optional argument @var{base64url} is non-@code{nil} then chars
+@code{+} and @code{/} are replaced by @code{-} abd @code{_} (See
+RFC4648).
 @end defun
 
-@deffn Command base64-decode-region beg end
+@deffn Command base64-decode-region beg end &optional base64url
 This function converts the region from @var{beg} to @var{end} from base
 64 code into the corresponding decoded text.  It returns the length of
 the decoded text.
 
 The decoding functions ignore newline characters in the encoded text.
+
+If optional argument @var{base64url} is non-@code{nil} then padding
+become optionnal and url variant is used (See RFC4648).
 @end deffn
 
-@defun base64-decode-string string
+@defun base64-decode-string string &optional base64url
 This function converts the string @var{string} from base 64 code into
 the corresponding decoded text.  It returns a unibyte string containing the
 decoded text.
 
 The decoding functions ignore newline characters in the encoded text.
+
+If optional argument @var{base64url} is non-@code{nil} then padding become
+optionnal and url variant is used (See RFC4648).
 @end defun
 
 @node Checksum/Hash
diff --git a/etc/NEWS b/etc/NEWS
index 222b86ee2b..9d7bfe5e82 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -370,6 +370,13 @@ in tooltips, as it is not useful there.
 There are 2 new buffer local variables and 1 face to customize this
 mode they are described in the manual "(emacs) Display".
 
++++
+** Functions 'base64-(encode|decode)-(string|region)' now manage url variant (RFC4648)
+The functions 'base64-encode-(region|string)' now have optionals
+arguments to mange padding and url variant.
+Mirror function 'base64-decode-(region|string)' now have an optional
+argument url-variant to manage this RFC.
+
 \f
 * Editing Changes in Emacs 27.1
 
diff --git a/src/fns.c b/src/fns.c
index 6b1f7331f5..963b54fb86 100644
--- a/src/fns.c
+++ b/src/fns.c
@@ -3169,7 +3169,7 @@ #define MIME_LINE_LENGTH 76
 #define IS_ASCII(Character) \
   ((Character) < 128)
 #define IS_BASE64(Character) \
-  (IS_ASCII (Character) && base64_char_to_value[Character] >= 0)
+  (IS_ASCII (Character) && b64_char_to_value[Character] >= 0)
 #define IS_BASE64_IGNORABLE(Character) \
   ((Character) == ' ' || (Character) == '\t' || (Character) == '\n' \
    || (Character) == '\f' || (Character) == '\r')
@@ -3202,6 +3202,17 @@ #define READ_QUADRUPLET_BYTE(retval)	\
   '8', '9', '+', '/'					/* 60-63 */
 };
 
+static const char base64url_value_to_char[64] =
+{
+  'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J',	/*  0- 9 */
+  'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T',	/* 10-19 */
+  'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd',	/* 20-29 */
+  'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n',	/* 30-39 */
+  'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x',	/* 40-49 */
+  'y', 'z', '0', '1', '2', '3', '4', '5', '6', '7',	/* 50-59 */
+  '8', '9', '-', '_'					/* 60-63 */
+};
+
 /* Table of base64 values for first 128 characters.  */
 static const short base64_char_to_value[128] =
 {
@@ -3220,6 +3231,23 @@ #define READ_QUADRUPLET_BYTE(retval)	\
   49,  50,  51,  -1,  -1,  -1,  -1,  -1			/* 120-127 */
 };
 
+static const short base64url_char_to_value[128] =
+{
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*   0-  9 */
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*  10- 19 */
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*  20- 29 */
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*  30- 39 */
+  -1,  -1,  -1,  -1,  -1,  62,  -1,  -1,  52,  53,	/*  40- 49 */
+  54,  55,  56,  57,  58,  59,  60,  61,  -1,  -1,	/*  50- 59 */
+  -1,  -1,  -1,  -1,  -1,  0,   1,   2,   3,   4,	/*  60- 69 */
+  5,   6,   7,   8,   9,   10,  11,  12,  13,  14,	/*  70- 79 */
+  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,	/*  80- 89 */
+  25,  -1,  -1,  -1,  -1,  63,  -1,  26,  27,  28,	/*  90- 99 */
+  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,	/* 100-109 */
+  39,  40,  41,  42,  43,  44,  45,  46,  47,  48,	/* 110-119 */
+  49,  50,  51,  -1,  -1,  -1,  -1,  -1                 /* 120-127 */
+};
+
 /* The following diagram shows the logical steps by which three octets
    get transformed into four base64 characters.
 
@@ -3239,17 +3267,21 @@ #define READ_QUADRUPLET_BYTE(retval)	\
    base64 characters.  */
 
 
-static ptrdiff_t base64_encode_1 (const char *, char *, ptrdiff_t, bool, bool);
+static ptrdiff_t base64_encode_1 (const char *, char *, ptrdiff_t, bool, bool,
+				  bool, bool);
 static ptrdiff_t base64_decode_1 (const char *, char *, ptrdiff_t, bool,
-				  ptrdiff_t *);
+				  bool, ptrdiff_t *);
 
 DEFUN ("base64-encode-region", Fbase64_encode_region, Sbase64_encode_region,
-       2, 3, "r",
+       2, 5, "r",
        doc: /* Base64-encode the region between BEG and END.
 Return the length of the encoded text.
 Optional third argument NO-LINE-BREAK means do not break long lines
-into shorter lines.  */)
-  (Lisp_Object beg, Lisp_Object end, Lisp_Object no_line_break)
+into shorter lines.
+Optional fourth argument NO-PAD means do not add padding char =.
+Optional fifth argument URL-VARIANT means use Url variant (RFC4648).  */)
+  (Lisp_Object beg, Lisp_Object end, Lisp_Object no_line_break,
+   Lisp_Object no_pad, Lisp_Object base64url)
 {
   char *encoded;
   ptrdiff_t allength, length;
@@ -3273,6 +3305,7 @@ DEFUN ("base64-encode-region", Fbase64_encode_region, Sbase64_encode_region,
   encoded = SAFE_ALLOCA (allength);
   encoded_length = base64_encode_1 ((char *) BYTE_POS_ADDR (ibeg),
 				    encoded, length, NILP (no_line_break),
+				    NILP (no_pad), !NILP (base64url),
 				    !NILP (BVAR (current_buffer, enable_multibyte_characters)));
   if (encoded_length > allength)
     emacs_abort ();
@@ -3304,11 +3337,14 @@ DEFUN ("base64-encode-region", Fbase64_encode_region, Sbase64_encode_region,
 }
 
 DEFUN ("base64-encode-string", Fbase64_encode_string, Sbase64_encode_string,
-       1, 2, 0,
+       1, 4, 0,
        doc: /* Base64-encode STRING and return the result.
 Optional second argument NO-LINE-BREAK means do not break long lines
-into shorter lines.  */)
-  (Lisp_Object string, Lisp_Object no_line_break)
+into shorter lines.
+Optional third argument NO-PAD means do not add padding char =.
+Optional fourth argument BASE64URL means use Url variant (RFC4648).  */)
+  (Lisp_Object string, Lisp_Object no_line_break,
+   Lisp_Object no_pad, Lisp_Object base64url)
 {
   ptrdiff_t allength, length, encoded_length;
   char *encoded;
@@ -3329,6 +3365,7 @@ DEFUN ("base64-encode-string", Fbase64_encode_string, Sbase64_encode_string,
 
   encoded_length = base64_encode_1 (SSDATA (string),
 				    encoded, length, NILP (no_line_break),
+				    NILP (no_pad), !NILP (base64url),
 				    STRING_MULTIBYTE (string));
   if (encoded_length > allength)
     emacs_abort ();
@@ -3347,7 +3384,8 @@ DEFUN ("base64-encode-string", Fbase64_encode_string, Sbase64_encode_string,
 
 static ptrdiff_t
 base64_encode_1 (const char *from, char *to, ptrdiff_t length,
-		 bool line_break, bool multibyte)
+		 bool line_break, bool pad, bool base64url,
+		 bool multibyte)
 {
   int counter = 0;
   ptrdiff_t i = 0;
@@ -3355,6 +3393,7 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
   int c;
   unsigned int value;
   int bytes;
+  char const *b64_value_to_char = (base64url) ? base64url_value_to_char : base64_value_to_char;
 
   while (i < length)
     {
@@ -3385,16 +3424,19 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
 
       /* Process first byte of a triplet.  */
 
-      *e++ = base64_value_to_char[0x3f & c >> 2];
+      *e++ = b64_value_to_char[0x3f & c >> 2];
       value = (0x03 & c) << 4;
 
       /* Process second byte of a triplet.  */
 
       if (i == length)
 	{
-	  *e++ = base64_value_to_char[value];
-	  *e++ = '=';
-	  *e++ = '=';
+	  *e++ = b64_value_to_char[value];
+	  if (pad)
+	    {
+	      *e++ = '=';
+	      *e++ = '=';
+	    }
 	  break;
 	}
 
@@ -3410,15 +3452,18 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
       else
 	c = from[i++];
 
-      *e++ = base64_value_to_char[value | (0x0f & c >> 4)];
+      *e++ = b64_value_to_char[value | (0x0f & c >> 4)];
       value = (0x0f & c) << 2;
 
       /* Process third byte of a triplet.  */
 
       if (i == length)
 	{
-	  *e++ = base64_value_to_char[value];
-	  *e++ = '=';
+	  *e++ = b64_value_to_char[value];
+	  if (pad)
+	    {
+	      *e++ = '=';
+	    }
 	  break;
 	}
 
@@ -3434,8 +3479,8 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
       else
 	c = from[i++];
 
-      *e++ = base64_value_to_char[value | (0x03 & c >> 6)];
-      *e++ = base64_value_to_char[0x3f & c];
+      *e++ = b64_value_to_char[value | (0x03 & c >> 6)];
+      *e++ = b64_value_to_char[0x3f & c];
     }
 
   return e - to;
@@ -3443,11 +3488,13 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
 
 
 DEFUN ("base64-decode-region", Fbase64_decode_region, Sbase64_decode_region,
-       2, 2, "r",
+       2, 3, "r",
        doc: /* Base64-decode the region between BEG and END.
 Return the length of the decoded text.
-If the region can't be decoded, signal an error and don't modify the buffer.  */)
-  (Lisp_Object beg, Lisp_Object end)
+If the region can't be decoded, signal an error and don't modify the buffer.
+Optional third argument BASE64URL define if base64Url variant will be used
+see RFC4648.  */)
+     (Lisp_Object beg, Lisp_Object end, Lisp_Object base64url)
 {
   ptrdiff_t ibeg, iend, length, allength;
   char *decoded;
@@ -3472,7 +3519,7 @@ DEFUN ("base64-decode-region", Fbase64_decode_region, Sbase64_decode_region,
 
   move_gap_both (XFIXNAT (beg), ibeg);
   decoded_length = base64_decode_1 ((char *) BYTE_POS_ADDR (ibeg),
-				    decoded, length,
+				    decoded, length, !NILP (base64url),
 				    multibyte, &inserted_chars);
   if (decoded_length > allength)
     emacs_abort ();
@@ -3506,9 +3553,11 @@ DEFUN ("base64-decode-region", Fbase64_decode_region, Sbase64_decode_region,
 }
 
 DEFUN ("base64-decode-string", Fbase64_decode_string, Sbase64_decode_string,
-       1, 1, 0,
-       doc: /* Base64-decode STRING and return the result.  */)
-  (Lisp_Object string)
+       1, 2, 0,
+       doc: /* Base64-decode STRING and return the result
+Optional argument BASE64URL define if base64Url variant will be used
+see RFC4648.  */)
+     (Lisp_Object string, Lisp_Object base64url)
 {
   char *decoded;
   ptrdiff_t length, decoded_length;
@@ -3523,7 +3572,7 @@ DEFUN ("base64-decode-string", Fbase64_decode_string, Sbase64_decode_string,
 
   /* The decoded result should be unibyte. */
   decoded_length = base64_decode_1 (SSDATA (string), decoded, length,
-				    0, NULL);
+				    !NILP (base64url), 0, NULL);
   if (decoded_length > length)
     emacs_abort ();
   else if (decoded_length >= 0)
@@ -3545,6 +3594,7 @@ DEFUN ("base64-decode-string", Fbase64_decode_string, Sbase64_decode_string,
 
 static ptrdiff_t
 base64_decode_1 (const char *from, char *to, ptrdiff_t length,
+		 bool base64url,
 		 bool multibyte, ptrdiff_t *nchars_return)
 {
   ptrdiff_t i = 0;		/* Used inside READ_QUADRUPLET_BYTE */
@@ -3552,6 +3602,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
   unsigned char c;
   unsigned long value;
   ptrdiff_t nchars = 0;
+  short const *b64_char_to_value = (base64url) ? base64url_char_to_value : base64_char_to_value;
 
   while (1)
     {
@@ -3561,7 +3612,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       if (!IS_BASE64 (c))
 	return -1;
-      value = base64_char_to_value[c] << 18;
+      value = b64_char_to_value[c] << 18;
 
       /* Process second byte of a quadruplet.  */
 
@@ -3569,7 +3620,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       if (!IS_BASE64 (c))
 	return -1;
-      value |= base64_char_to_value[c] << 12;
+      value |= b64_char_to_value[c] << 12;
 
       c = (unsigned char) (value >> 16);
       if (multibyte && c >= 128)
@@ -3580,7 +3631,14 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       /* Process third byte of a quadruplet.  */
 
-      READ_QUADRUPLET_BYTE (-1);
+      if (!base64url)
+	{
+	  READ_QUADRUPLET_BYTE (-1);
+	}
+      else
+	{
+	  READ_QUADRUPLET_BYTE (e-to);
+	}
 
       if (c == '=')
 	{
@@ -3593,7 +3651,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       if (!IS_BASE64 (c))
 	return -1;
-      value |= base64_char_to_value[c] << 6;
+      value |= b64_char_to_value[c] << 6;
 
       c = (unsigned char) (0xff & value >> 8);
       if (multibyte && c >= 128)
@@ -3604,14 +3662,21 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       /* Process fourth byte of a quadruplet.  */
 
-      READ_QUADRUPLET_BYTE (-1);
+      if (!base64url)
+	{
+	  READ_QUADRUPLET_BYTE (-1);
+	}
+      else
+	{
+	  READ_QUADRUPLET_BYTE (e-to);
+	}
 
       if (c == '=')
 	continue;
 
       if (!IS_BASE64 (c))
 	return -1;
-      value |= base64_char_to_value[c];
+      value |= b64_char_to_value[c];
 
       c = (unsigned char) (0xff & value);
       if (multibyte && c >= 128)
diff --git a/test/src/fns-tests.el b/test/src/fns-tests.el
index 6ebab4287f..3301129567 100644
--- a/test/src/fns-tests.el
+++ b/test/src/fns-tests.el
@@ -233,6 +233,137 @@ fns-tests-func-arity
   (should (equal (func-arity (eval (lambda (x &optional y)) t)) '(1 . 2)))
   (should (equal (func-arity 'let) '(1 . unevalled))))
 
+(defun string-repeat (s o)
+  (apply 'concat (make-list o s)))
+
+(ert-deftest fns-tests-base64-encode-string ()
+  ;; standard variant RFC2045
+  (should (equal (base64-encode-string "") ""))
+  (should (equal (base64-encode-string "f") "Zg=="))
+  (should (equal (base64-encode-string "fo") "Zm8="))
+  (should (equal (base64-encode-string "foo") "Zm9v"))
+  (should (equal (base64-encode-string "foob") "Zm9vYg=="))
+  (should (equal (base64-encode-string "fooba") "Zm9vYmE="))
+  (should (equal (base64-encode-string "foobar") "Zm9vYmFy"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7e") "FPucA9l+"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7f") "FPucA9l/"))
+
+  ;; no line break
+  (should (equal (base64-encode-string "") ""))
+  (should (equal (base64-encode-string (string-repeat "f" 100) t) (concat (string-repeat "Zm" 66) "Zg==")))
+  (should (equal (base64-encode-string (string-repeat "fo" 50) t) (concat (string-repeat "Zm9mb2Zv" 16) "Zm9mbw==")))
+  (should (equal (base64-encode-string (string-repeat "foo" 25) t) (string-repeat "Zm9v" 25)))
+  (should (equal (base64-encode-string (string-repeat "foob" 15) t) (string-repeat "Zm9vYmZvb2Jmb29i" 5)))
+  (should (equal (base64-encode-string (string-repeat "fooba" 15) t) (string-repeat "Zm9vYmFmb29iYWZvb2Jh" 5)))
+  (should (equal (base64-encode-string (string-repeat "foobar" 15) t) (concat (string-repeat "Zm9vYmFyZm9vYmFy" 7) "Zm9vYmFy")))
+  (should (equal (base64-encode-string (string-repeat "\x14\xfb\x9c\x03\xd9\x7e" 10) t) (string-repeat "FPucA9l+" 10)))
+  (should (equal (base64-encode-string (string-repeat "\x14\xfb\x9c\x03\xd9\x7f" 10) t) (string-repeat "FPucA9l/" 10)))
+
+  ;; no paddign
+  (should (equal (base64-encode-string "") ""))
+  (should (equal (base64-encode-string "f" nil t) "Zg"))
+  (should (equal (base64-encode-string "fo" nil t) "Zm8"))
+  (should (equal (base64-encode-string "foo" nil t) "Zm9v"))
+  (should (equal (base64-encode-string "foob" nil t) "Zm9vYg"))
+  (should (equal (base64-encode-string "fooba" nil t) "Zm9vYmE"))
+  (should (equal (base64-encode-string "foobar" nil t) "Zm9vYmFy"))
+
+  ;; url variant wih padding
+  (should (equal (base64-encode-string "" nil nil t) ""))
+  (should (equal (base64-encode-string "f" nil nil t) "Zg=="))
+  (should (equal (base64-encode-string "fo" nil nil t) "Zm8="))
+  (should (equal (base64-encode-string "foo" nil nil t) "Zm9v"))
+  (should (equal (base64-encode-string "foob" nil nil t) "Zm9vYg=="))
+  (should (equal (base64-encode-string "fooba" nil nil t) "Zm9vYmE="))
+  (should (equal (base64-encode-string "foobar" nil nil t) "Zm9vYmFy"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7e" nil nil t) "FPucA9l-"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7f" nil nil t) "FPucA9l_"))
+
+  ;; url variant no padding
+  (should (equal (base64-encode-string "" nil t t) ""))
+  (should (equal (base64-encode-string "f" nil t t) "Zg"))
+  (should (equal (base64-encode-string "fo" nil t t) "Zm8"))
+  (should (equal (base64-encode-string "foo" nil t t) "Zm9v"))
+  (should (equal (base64-encode-string "foob" nil t t) "Zm9vYg"))
+  (should (equal (base64-encode-string "fooba" nil t t) "Zm9vYmE"))
+  (should (equal (base64-encode-string "foobar" nil t t) "Zm9vYmFy"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7e" nil t t) "FPucA9l-"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7f" nil t t) "FPucA9l_"))
+
+
+  ;; url variant no line break no padding
+  (should (equal (base64-encode-string (string-repeat "f" 100) t t t) (concat (string-repeat "Zm" 66) "Zg")))
+  (should (equal (base64-encode-string (string-repeat "fo" 50) t t t) (concat (string-repeat "Zm9mb2Zv" 16) "Zm9mbw")))
+  (should (equal (base64-encode-string (string-repeat "foo" 25) t t t) (string-repeat "Zm9v" 25)))
+  (should (equal (base64-encode-string (string-repeat "foob" 15) t t t) (string-repeat "Zm9vYmZvb2Jmb29i" 5)))
+  (should (equal (base64-encode-string (string-repeat "fooba" 15) t t t) (string-repeat "Zm9vYmFmb29iYWZvb2Jh" 5)))
+  (should (equal (base64-encode-string (string-repeat "foobar" 15) t t t) (concat (string-repeat "Zm9vYmFyZm9vYmFy" 7) "Zm9vYmFy")))
+  (should (equal (base64-encode-string (string-repeat "\x14\xfb\x9c\x03\xd9\x7e" 10) t t t) (string-repeat "FPucA9l-" 10)))
+  (should (equal (base64-encode-string (string-repeat "\x14\xfb\x9c\x03\xd9\x7f" 10) t t t) (string-repeat "FPucA9l_" 10)))
+
+  )
+
+(ert-deftest fns-tests-base64-decode-string ()
+  ;; standard variant RFC2045
+  (should (equal (base64-decode-string "") ""))
+  (should (equal (base64-decode-string "Zg==") "f"))
+  (should (equal (base64-decode-string "Zm8=") "fo"))
+  (should (equal (base64-decode-string "Zm9v") "foo"))
+  (should (equal (base64-decode-string "Zm9vYg==") "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE=") "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy") "foobar"))
+  (should (equal (base64-decode-string "FPucA9l+") "\x14\xfb\x9c\x03\xd9\x7e"))
+  (should (equal (base64-decode-string "FPucA9l/") "\x14\xfb\x9c\x03\xd9\x7f"))
+
+  ;; no paddign
+  (should (equal (base64-decode-string "" t) ""))
+  (should (equal (base64-decode-string "Zg" t) "f"))
+  (should (equal (base64-decode-string "Zm8" t) "fo"))
+  (should (equal (base64-decode-string "Zm9v" t) "foo"))
+  (should (equal (base64-decode-string "Zm9vYg" t) "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE" t) "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy" t) "foobar"))
+
+  ;; url variant wih padding
+  (should (equal (base64-decode-string "") ""))
+  (should (equal (base64-decode-string "Zg==" t) "f") )
+  (should (equal (base64-decode-string "Zm8=" t) "fo"))
+  (should (equal (base64-decode-string "Zm9v" t) "foo"))
+  (should (equal (base64-decode-string "Zm9vYg==" t) "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE=" t) "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy" t) "foobar"))
+  (should (equal (base64-decode-string "FPucA9l-" t) "\x14\xfb\x9c\x03\xd9\x7e"))
+  (should (equal (base64-decode-string "FPucA9l_" t) "\x14\xfb\x9c\x03\xd9\x7f"))
+
+  ;; url variant no padding
+  (should (equal (base64-decode-string "") ""))
+  (should (equal (base64-decode-string "Zg" t) "f"))
+  (should (equal (base64-decode-string "Zm8" t) "fo"))
+  (should (equal (base64-decode-string "Zm9v" t) "foo"))
+  (should (equal (base64-decode-string "Zm9vYg" t) "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE" t) "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy" t) "foobar"))
+  (should (equal (base64-decode-string "FPucA9l-" t) "\x14\xfb\x9c\x03\xd9\x7e"))
+  (should (equal (base64-decode-string "FPucA9l_" t) "\x14\xfb\x9c\x03\xd9\x7f"))
+
+
+  ;; url variant no line break no padding
+  (should (equal (base64-decode-string (concat (string-repeat "Zm" 66) "Zg") t) (string-repeat "f" 100)))
+  (should (equal (base64-decode-string (concat (string-repeat "Zm9mb2Zv" 16) "Zm9mbw") t) (string-repeat "fo" 50)))
+  (should (equal (base64-decode-string (string-repeat "Zm9v" 25) t) (string-repeat "foo" 25)))
+  (should (equal (base64-decode-string (string-repeat "Zm9vYmZvb2Jmb29i" 5) t) (string-repeat "foob" 15)))
+  (should (equal (base64-decode-string (string-repeat "Zm9vYmFmb29iYWZvb2Jh" 5) t) (string-repeat "fooba" 15)))
+  (should (equal (base64-decode-string (concat (string-repeat "Zm9vYmFyZm9vYmFy" 7) "Zm9vYmFy") t) (string-repeat "foobar" 15)))
+  (should (equal (base64-decode-string (string-repeat "FPucA9l-" 10) t) (string-repeat "\x14\xfb\x9c\x03\xd9\x7e" 10)))
+  (should (equal (base64-decode-string (string-repeat "FPucA9l_" 10) t) (string-repeat "\x14\xfb\x9c\x03\xd9\x7f" 10)))
+
+  ;; errors check
+  (should (eq :got-error (condition-case () (base64-decode-string "Zg=") (error :got-error))))
+  (should (eq :got-error (condition-case () (base64-decode-string "Zm9vYmE") (error :got-error))))
+  (should (eq :got-error (condition-case () (base64-decode-string "Zm9vYmFy=") (error :got-error))))
+  (should (eq :got-error (condition-case () (base64-decode-string "Zg=Zg=") (error :got-error))))
+  )
+
 (ert-deftest fns-tests-hash-buffer ()
   (should (equal (sha1 "foo") "0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"))
   (should (equal (with-temp-buffer
-- 
2.21.0


[-- Attachment #3: Type: text/plain, Size: 281 bytes --]


The second patch contains an reworked version wich instead of adding
many parameters to base64-encode-region (resp. base64-encode-string)
function create the base64url-encode-region
(resp. base64url-encode-string) function. Documentation and tests are
also updated consequently.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: new functions --]
[-- Type: text/x-patch, Size: 30190 bytes --]

From 8acbb905aeb49fb0bfb747acc6b4965cf120e23b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Pierre=20T=C3=A9choueyres?= <pierre.techoueyres@free.fr>
Date: Tue, 21 May 2019 23:00:13 +0200
Subject: [PATCH] Add support for base64url variant.

Implements the RFC4648 variant of base64 encoding used by URLs.

* doc/lispref/text.texi (base64url-encode-region,
  base64url-encode-string): Documents new functions.
  (base64-decode-region, base64-decode-string): Documents new optional
  parameter 'base64url' used to use url variant when decoding data.

* etc/NEWS: Announce new functions and optionals parameters.

* src/fns.c (base64url-encode-region, base64url-encode-region): New
  functions to manage url variant.
  (base64-decode-region, base64-decode-string): Adds optional
  parameter to indicate use of url-variant.
  (base64_encode_region_1, base64_encode_string_1): Internal functions
  with extracted code from 'base64_encode_region' and
  'base64_encode_string' and optional parameters to manage padding and
  url variant.
  (base64-encode-region, base64-encode-string) : Use internal
  functions previously defined.
  (base64-encode-1): Adds parameters to manage padding and url variant.
  (base64-decode-1): Adds parameter to manage url variant.

* test/src/fns-tests.el (fns-tests--with-region): New helper macro to
  test region variant of base64 encode / decode functions.
  (fns-tests--string-repeat): Helper function used in base64 tests.
  (fns-tests-base64-encode-region, fns-tests-base64-encode-string):
  Tests for standard base64 function.
  (fns-test-base64url-encode-region,
  fns-test-base64url-encode-string): Tests for url variant.
  (fns-tests-base64-decode-string): Tests for decoding part.
---
 doc/lispref/text.texi |  42 +++++++++-
 etc/NEWS              |   8 ++
 src/fns.c             | 166 ++++++++++++++++++++++++++++++++-------
 test/src/fns-tests.el | 179 ++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 363 insertions(+), 32 deletions(-)

diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi
index 278bc3c268..3e0cf4c06f 100644
--- a/doc/lispref/text.texi
+++ b/doc/lispref/text.texi
@@ -4541,7 +4541,7 @@ Base 64
 usually written by technical experts acting on their own initiative,
 and are traditionally written in a pragmatic, experience-driven
 manner.
-}2045.  This section describes the functions for
+}2045 and also in RFC4648.  This section describes the functions for
 converting to and from this code.
 
 @deffn Command base64-encode-region beg end &optional no-line-break
@@ -4558,6 +4558,22 @@ Base 64
 the output is just one long line.
 @end deffn
 
+@deffn Command base64url-encode-region beg end &optional no-pad
+This function converts the region from @var{beg} to @var{end} into base
+64 code.  It returns the length of the encoded text.  An error is
+signaled if a character in the region is multibyte, i.e., in a
+multibyte buffer the region must contain only characters from the
+charsets @code{ascii}, @code{eight-bit-control} and
+@code{eight-bit-graphic}.
+
+Contrary to the function @code{base64-encode-region}, this function
+doesnt inserts newline characters into the encoded text, so the output
+is just one long line.
+
+If the optional argument @var{no-pad} is non-@code{nil} then padding
+(@code{=}) isn't generated.
+@end deffn
+
 @defun base64-encode-string string &optional no-line-break
 This function converts the string @var{string} into base 64 code.  It
 returns a string containing the encoded text.  As for
@@ -4570,20 +4586,40 @@ Base 64
 the result string is just one long line.
 @end defun
 
-@deffn Command base64-decode-region beg end
+@defun base64url-encode-string string &optional no-pad
+This function converts the string @var{string} into base 64 url code
+(see RFC4648).  It returns a string containing the encoded text.  As
+for @code{base64url-encode-region}, an error is signaled if a
+character in the string is multibyte.
+
+Contrary to @code{base64-encode-string}, this function doesnt inserts
+newline characters into the encoded text, so the result string is just
+one long line.
+
+If the optional argument @var{no-pad} is non-@code{nil} then padding
+(@code{=}) isn't generated.
+@end defun
+
+@deffn Command base64-decode-region beg end &optional base64url
 This function converts the region from @var{beg} to @var{end} from base
 64 code into the corresponding decoded text.  It returns the length of
 the decoded text.
 
 The decoding functions ignore newline characters in the encoded text.
+
+If optional argument @var{base64url} is is non-@code{nil} then padding
+become optionnal and url variant is used (see RFC4648).
 @end deffn
 
-@defun base64-decode-string string
+@defun base64-decode-string string &optional base64url
 This function converts the string @var{string} from base 64 code into
 the corresponding decoded text.  It returns a unibyte string containing the
 decoded text.
 
 The decoding functions ignore newline characters in the encoded text.
+
+If optional argument @var{base64url} is is non-@code{nil} then padding
+become optionnal and url variant is used (see RFC4648).
 @end defun
 
 @node Checksum/Hash
diff --git a/etc/NEWS b/etc/NEWS
index 222b86ee2b..cae6b25aca 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -370,6 +370,14 @@ in tooltips, as it is not useful there.
 There are 2 new buffer local variables and 1 face to customize this
 mode they are described in the manual "(emacs) Display".
 
++++
+** New functions 'base64url-encode-(string|region)' manage url variant (RFC4648)
+The new functions 'base64url-encode-region' and
+'base64url-encode-region' now implements url-variant as defined in RFC.
+
+Mirror functions 'base64-decode-(region|string)' now have an optional
+argument base64url to manage this RFC.
+
 \f
 * Editing Changes in Emacs 27.1
 
diff --git a/src/fns.c b/src/fns.c
index 6b1f7331f5..8c6894b77d 100644
--- a/src/fns.c
+++ b/src/fns.c
@@ -3169,7 +3169,7 @@ #define MIME_LINE_LENGTH 76
 #define IS_ASCII(Character) \
   ((Character) < 128)
 #define IS_BASE64(Character) \
-  (IS_ASCII (Character) && base64_char_to_value[Character] >= 0)
+  (IS_ASCII (Character) && b64_char_to_value[Character] >= 0)
 #define IS_BASE64_IGNORABLE(Character) \
   ((Character) == ' ' || (Character) == '\t' || (Character) == '\n' \
    || (Character) == '\f' || (Character) == '\r')
@@ -3202,6 +3202,17 @@ #define READ_QUADRUPLET_BYTE(retval)	\
   '8', '9', '+', '/'					/* 60-63 */
 };
 
+static const char base64url_value_to_char[64] =
+{
+  'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J',	/*  0- 9 */
+  'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T',	/* 10-19 */
+  'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd',	/* 20-29 */
+  'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n',	/* 30-39 */
+  'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x',	/* 40-49 */
+  'y', 'z', '0', '1', '2', '3', '4', '5', '6', '7',	/* 50-59 */
+  '8', '9', '-', '_'					/* 60-63 */
+};
+
 /* Table of base64 values for first 128 characters.  */
 static const short base64_char_to_value[128] =
 {
@@ -3220,6 +3231,23 @@ #define READ_QUADRUPLET_BYTE(retval)	\
   49,  50,  51,  -1,  -1,  -1,  -1,  -1			/* 120-127 */
 };
 
+static const short base64url_char_to_value[128] =
+{
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*   0-  9 */
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*  10- 19 */
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*  20- 29 */
+  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,  -1,	/*  30- 39 */
+  -1,  -1,  -1,  -1,  -1,  62,  -1,  -1,  52,  53,	/*  40- 49 */
+  54,  55,  56,  57,  58,  59,  60,  61,  -1,  -1,	/*  50- 59 */
+  -1,  -1,  -1,  -1,  -1,  0,   1,   2,   3,   4,	/*  60- 69 */
+  5,   6,   7,   8,   9,   10,  11,  12,  13,  14,	/*  70- 79 */
+  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,	/*  80- 89 */
+  25,  -1,  -1,  -1,  -1,  63,  -1,  26,  27,  28,	/*  90- 99 */
+  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,	/* 100-109 */
+  39,  40,  41,  42,  43,  44,  45,  46,  47,  48,	/* 110-119 */
+  49,  50,  51,  -1,  -1,  -1,  -1,  -1                 /* 120-127 */
+};
+
 /* The following diagram shows the logical steps by which three octets
    get transformed into four base64 characters.
 
@@ -3239,9 +3267,17 @@ #define READ_QUADRUPLET_BYTE(retval)	\
    base64 characters.  */
 
 
-static ptrdiff_t base64_encode_1 (const char *, char *, ptrdiff_t, bool, bool);
+static ptrdiff_t base64_encode_1 (const char *, char *, ptrdiff_t, bool, bool,
+				  bool, bool);
 static ptrdiff_t base64_decode_1 (const char *, char *, ptrdiff_t, bool,
-				  ptrdiff_t *);
+				  bool, ptrdiff_t *);
+
+Lisp_Object base64_encode_region_1 (Lisp_Object, Lisp_Object, bool,
+				    bool, bool);
+
+Lisp_Object base64_encode_string_1(Lisp_Object, bool,
+				   bool, bool);
+
 
 DEFUN ("base64-encode-region", Fbase64_encode_region, Sbase64_encode_region,
        2, 3, "r",
@@ -3250,6 +3286,26 @@ DEFUN ("base64-encode-region", Fbase64_encode_region, Sbase64_encode_region,
 Optional third argument NO-LINE-BREAK means do not break long lines
 into shorter lines.  */)
   (Lisp_Object beg, Lisp_Object end, Lisp_Object no_line_break)
+{
+  return base64_encode_region_1(beg, end, NILP (no_line_break), true, false);
+}
+
+
+DEFUN ("base64url-encode-region", Fbase64url_encode_region, Sbase64url_encode_region,
+       2, 3, "r",
+       doc: /* Base64url-encode the region between BEG and END.
+Return the length of the encoded text.
+Optional second argument NO-PAD means do not add padding char =.
+
+This is the variant defined in RFC4648.  */)
+  (Lisp_Object beg, Lisp_Object end, Lisp_Object no_pad)
+{
+  return base64_encode_region_1(beg, end, false, NILP(no_pad), true);
+}
+
+Lisp_Object
+base64_encode_region_1 (Lisp_Object beg, Lisp_Object end, bool line_break,
+			bool pad, bool base64url)
 {
   char *encoded;
   ptrdiff_t allength, length;
@@ -3272,7 +3328,8 @@ DEFUN ("base64-encode-region", Fbase64_encode_region, Sbase64_encode_region,
 
   encoded = SAFE_ALLOCA (allength);
   encoded_length = base64_encode_1 ((char *) BYTE_POS_ADDR (ibeg),
-				    encoded, length, NILP (no_line_break),
+				    encoded, length, line_break,
+				    pad, base64url,
 				    !NILP (BVAR (current_buffer, enable_multibyte_characters)));
   if (encoded_length > allength)
     emacs_abort ();
@@ -3310,6 +3367,26 @@ DEFUN ("base64-encode-string", Fbase64_encode_string, Sbase64_encode_string,
 into shorter lines.  */)
   (Lisp_Object string, Lisp_Object no_line_break)
 {
+
+  return base64_encode_string_1(string, NILP (no_line_break), true, false);
+}
+
+DEFUN ("base64url-encode-string", Fbase64url_encode_string, Sbase64url_encode_string,
+       1, 2, 0,
+       doc: /* Base64url-encode STRING and return the result.
+Optional second argument NO-PAD means do not add padding char =.
+
+This is the variant defined in RFC4648.  */)
+  (Lisp_Object string, Lisp_Object no_pad)
+{
+
+  return base64_encode_string_1(string, false, NILP(no_pad), true);
+}
+
+Lisp_Object
+base64_encode_string_1(Lisp_Object string, bool line_break,
+		       bool pad, bool base64url)
+{
   ptrdiff_t allength, length, encoded_length;
   char *encoded;
   Lisp_Object encoded_string;
@@ -3328,7 +3405,8 @@ DEFUN ("base64-encode-string", Fbase64_encode_string, Sbase64_encode_string,
   encoded = SAFE_ALLOCA (allength);
 
   encoded_length = base64_encode_1 (SSDATA (string),
-				    encoded, length, NILP (no_line_break),
+				    encoded, length, line_break,
+				    pad, base64url,
 				    STRING_MULTIBYTE (string));
   if (encoded_length > allength)
     emacs_abort ();
@@ -3347,7 +3425,8 @@ DEFUN ("base64-encode-string", Fbase64_encode_string, Sbase64_encode_string,
 
 static ptrdiff_t
 base64_encode_1 (const char *from, char *to, ptrdiff_t length,
-		 bool line_break, bool multibyte)
+		 bool line_break, bool pad, bool base64url,
+		 bool multibyte)
 {
   int counter = 0;
   ptrdiff_t i = 0;
@@ -3355,6 +3434,7 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
   int c;
   unsigned int value;
   int bytes;
+  char const *b64_value_to_char = (base64url) ? base64url_value_to_char : base64_value_to_char;
 
   while (i < length)
     {
@@ -3385,16 +3465,19 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
 
       /* Process first byte of a triplet.  */
 
-      *e++ = base64_value_to_char[0x3f & c >> 2];
+      *e++ = b64_value_to_char[0x3f & c >> 2];
       value = (0x03 & c) << 4;
 
       /* Process second byte of a triplet.  */
 
       if (i == length)
 	{
-	  *e++ = base64_value_to_char[value];
-	  *e++ = '=';
-	  *e++ = '=';
+	  *e++ = b64_value_to_char[value];
+	  if (pad)
+	    {
+	      *e++ = '=';
+	      *e++ = '=';
+	    }
 	  break;
 	}
 
@@ -3410,15 +3493,18 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
       else
 	c = from[i++];
 
-      *e++ = base64_value_to_char[value | (0x0f & c >> 4)];
+      *e++ = b64_value_to_char[value | (0x0f & c >> 4)];
       value = (0x0f & c) << 2;
 
       /* Process third byte of a triplet.  */
 
       if (i == length)
 	{
-	  *e++ = base64_value_to_char[value];
-	  *e++ = '=';
+	  *e++ = b64_value_to_char[value];
+	  if (pad)
+	    {
+	      *e++ = '=';
+	    }
 	  break;
 	}
 
@@ -3434,8 +3520,8 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
       else
 	c = from[i++];
 
-      *e++ = base64_value_to_char[value | (0x03 & c >> 6)];
-      *e++ = base64_value_to_char[0x3f & c];
+      *e++ = b64_value_to_char[value | (0x03 & c >> 6)];
+      *e++ = b64_value_to_char[0x3f & c];
     }
 
   return e - to;
@@ -3443,11 +3529,13 @@ base64_encode_1 (const char *from, char *to, ptrdiff_t length,
 
 
 DEFUN ("base64-decode-region", Fbase64_decode_region, Sbase64_decode_region,
-       2, 2, "r",
+       2, 3, "r",
        doc: /* Base64-decode the region between BEG and END.
 Return the length of the decoded text.
-If the region can't be decoded, signal an error and don't modify the buffer.  */)
-  (Lisp_Object beg, Lisp_Object end)
+If the region can't be decoded, signal an error and don't modify the buffer.
+Optional third argument BASE64URL define if base64Url variant will be used
+see RFC4648.  */)
+     (Lisp_Object beg, Lisp_Object end, Lisp_Object base64url)
 {
   ptrdiff_t ibeg, iend, length, allength;
   char *decoded;
@@ -3472,7 +3560,7 @@ DEFUN ("base64-decode-region", Fbase64_decode_region, Sbase64_decode_region,
 
   move_gap_both (XFIXNAT (beg), ibeg);
   decoded_length = base64_decode_1 ((char *) BYTE_POS_ADDR (ibeg),
-				    decoded, length,
+				    decoded, length, !NILP (base64url),
 				    multibyte, &inserted_chars);
   if (decoded_length > allength)
     emacs_abort ();
@@ -3506,9 +3594,11 @@ DEFUN ("base64-decode-region", Fbase64_decode_region, Sbase64_decode_region,
 }
 
 DEFUN ("base64-decode-string", Fbase64_decode_string, Sbase64_decode_string,
-       1, 1, 0,
-       doc: /* Base64-decode STRING and return the result.  */)
-  (Lisp_Object string)
+       1, 2, 0,
+       doc: /* Base64-decode STRING and return the result
+Optional argument BASE64URL define if base64Url variant will be used
+see RFC4648.  */)
+     (Lisp_Object string, Lisp_Object base64url)
 {
   char *decoded;
   ptrdiff_t length, decoded_length;
@@ -3523,7 +3613,7 @@ DEFUN ("base64-decode-string", Fbase64_decode_string, Sbase64_decode_string,
 
   /* The decoded result should be unibyte. */
   decoded_length = base64_decode_1 (SSDATA (string), decoded, length,
-				    0, NULL);
+				    !NILP (base64url), 0, NULL);
   if (decoded_length > length)
     emacs_abort ();
   else if (decoded_length >= 0)
@@ -3545,6 +3635,7 @@ DEFUN ("base64-decode-string", Fbase64_decode_string, Sbase64_decode_string,
 
 static ptrdiff_t
 base64_decode_1 (const char *from, char *to, ptrdiff_t length,
+		 bool base64url,
 		 bool multibyte, ptrdiff_t *nchars_return)
 {
   ptrdiff_t i = 0;		/* Used inside READ_QUADRUPLET_BYTE */
@@ -3552,6 +3643,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
   unsigned char c;
   unsigned long value;
   ptrdiff_t nchars = 0;
+  short const *b64_char_to_value = (base64url) ? base64url_char_to_value : base64_char_to_value;
 
   while (1)
     {
@@ -3561,7 +3653,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       if (!IS_BASE64 (c))
 	return -1;
-      value = base64_char_to_value[c] << 18;
+      value = b64_char_to_value[c] << 18;
 
       /* Process second byte of a quadruplet.  */
 
@@ -3569,7 +3661,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       if (!IS_BASE64 (c))
 	return -1;
-      value |= base64_char_to_value[c] << 12;
+      value |= b64_char_to_value[c] << 12;
 
       c = (unsigned char) (value >> 16);
       if (multibyte && c >= 128)
@@ -3580,7 +3672,14 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       /* Process third byte of a quadruplet.  */
 
-      READ_QUADRUPLET_BYTE (-1);
+      if (!base64url)
+	{
+	  READ_QUADRUPLET_BYTE (-1);
+	}
+      else
+	{
+	  READ_QUADRUPLET_BYTE (e-to);
+	}
 
       if (c == '=')
 	{
@@ -3593,7 +3692,7 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       if (!IS_BASE64 (c))
 	return -1;
-      value |= base64_char_to_value[c] << 6;
+      value |= b64_char_to_value[c] << 6;
 
       c = (unsigned char) (0xff & value >> 8);
       if (multibyte && c >= 128)
@@ -3604,14 +3703,21 @@ base64_decode_1 (const char *from, char *to, ptrdiff_t length,
 
       /* Process fourth byte of a quadruplet.  */
 
-      READ_QUADRUPLET_BYTE (-1);
+      if (!base64url)
+	{
+	  READ_QUADRUPLET_BYTE (-1);
+	}
+      else
+	{
+	  READ_QUADRUPLET_BYTE (e-to);
+	}
 
       if (c == '=')
 	continue;
 
       if (!IS_BASE64 (c))
 	return -1;
-      value |= base64_char_to_value[c];
+      value |= b64_char_to_value[c];
 
       c = (unsigned char) (0xff & value);
       if (multibyte && c >= 128)
@@ -5445,6 +5551,8 @@ syms_of_fns (void)
   defsubr (&Sbase64_decode_region);
   defsubr (&Sbase64_encode_string);
   defsubr (&Sbase64_decode_string);
+  defsubr (&Sbase64url_encode_region);
+  defsubr (&Sbase64url_encode_string);
   defsubr (&Smd5);
   defsubr (&Ssecure_hash_algorithms);
   defsubr (&Ssecure_hash);
diff --git a/test/src/fns-tests.el b/test/src/fns-tests.el
index 6ebab4287f..dab43212f5 100644
--- a/test/src/fns-tests.el
+++ b/test/src/fns-tests.el
@@ -233,6 +233,185 @@ fns-tests-func-arity
   (should (equal (func-arity (eval (lambda (x &optional y)) t)) '(1 . 2)))
   (should (equal (func-arity 'let) '(1 . unevalled))))
 
+(defun fns-tests--string-repeat (s o)
+  (apply 'concat (make-list o s)))
+
+(defmacro fns-tests--with-region (funcname string &rest args)
+  "Apply FUNCNAME in a temp bufer on the region produced by STRING."
+  (declare (indent 1))
+  `(with-temp-buffer
+     (insert ,string)
+     (,funcname (point-min) (point-max) ,@args)
+     (buffer-string)))
+
+(ert-deftest fns-tests-base64-encode-region ()
+  ;; standard variant RFC2045
+  (should (equal (fns-tests--with-region base64-encode-region "") ""))
+  (should (equal (fns-tests--with-region base64-encode-region "f") "Zg=="))
+  (should (equal (fns-tests--with-region base64-encode-region "fo") "Zm8="))
+  (should (equal (fns-tests--with-region base64-encode-region "foo") "Zm9v"))
+  (should (equal (fns-tests--with-region base64-encode-region "foob") "Zm9vYg=="))
+  (should (equal (fns-tests--with-region base64-encode-region "fooba") "Zm9vYmE="))
+  (should (equal (fns-tests--with-region base64-encode-region "foobar") "Zm9vYmFy"))
+  (should (equal (fns-tests--with-region base64-encode-region "\x14\xfb\x9c\x03\xd9\x7e") "FPucA9l+"))
+  (should (equal (fns-tests--with-region base64-encode-region "\x14\xfb\x9c\x03\xd9\x7f") "FPucA9l/")))
+
+(ert-deftest fns-tests-base64-encode-string ()
+  ;; standard variant RFC2045
+  (should (equal (base64-encode-string "") ""))
+  (should (equal (base64-encode-string "f") "Zg=="))
+  (should (equal (base64-encode-string "fo") "Zm8="))
+  (should (equal (base64-encode-string "foo") "Zm9v"))
+  (should (equal (base64-encode-string "foob") "Zm9vYg=="))
+  (should (equal (base64-encode-string "fooba") "Zm9vYmE="))
+  (should (equal (base64-encode-string "foobar") "Zm9vYmFy"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7e") "FPucA9l+"))
+  (should (equal (base64-encode-string "\x14\xfb\x9c\x03\xd9\x7f") "FPucA9l/")))
+
+(ert-deftest fns-test-base64url-encode-region ()
+  ;; url variant wih padding
+  (should (equal (fns-tests--with-region base64url-encode-region "") ""))
+  (should (equal (fns-tests--with-region base64url-encode-region "f") "Zg=="))
+  (should (equal (fns-tests--with-region base64url-encode-region "fo") "Zm8="))
+  (should (equal (fns-tests--with-region base64url-encode-region "foo") "Zm9v"))
+  (should (equal (fns-tests--with-region base64url-encode-region "foob") "Zm9vYg=="))
+  (should (equal (fns-tests--with-region base64url-encode-region "fooba") "Zm9vYmE="))
+  (should (equal (fns-tests--with-region base64url-encode-region "foobar") "Zm9vYmFy"))
+  (should (equal (fns-tests--with-region base64url-encode-region "\x14\xfb\x9c\x03\xd9\x7e") "FPucA9l-"))
+  (should (equal (fns-tests--with-region base64url-encode-region "\x14\xfb\x9c\x03\xd9\x7f") "FPucA9l_"))
+
+  ;; url variant no padding
+  (should (equal (fns-tests--with-region base64url-encode-region "" t) ""))
+  (should (equal (fns-tests--with-region base64url-encode-region "f" t) "Zg"))
+  (should (equal (fns-tests--with-region base64url-encode-region "fo" t) "Zm8"))
+  (should (equal (fns-tests--with-region base64url-encode-region "foo" t) "Zm9v"))
+  (should (equal (fns-tests--with-region base64url-encode-region "foob" t) "Zm9vYg"))
+  (should (equal (fns-tests--with-region base64url-encode-region "fooba" t) "Zm9vYmE"))
+  (should (equal (fns-tests--with-region base64url-encode-region "foobar" t) "Zm9vYmFy"))
+  (should (equal (fns-tests--with-region base64url-encode-region "\x14\xfb\x9c\x03\xd9\x7e" t) "FPucA9l-"))
+  (should (equal (fns-tests--with-region base64url-encode-region "\x14\xfb\x9c\x03\xd9\x7f" t) "FPucA9l_"))
+
+
+  ;; url variant no line break no padding
+  (should (equal (fns-tests--with-region base64url-encode-region (fns-tests--string-repeat "f" 100) t)
+                 (concat (fns-tests--string-repeat "Zm" 66) "Zg")))
+  (should (equal (fns-tests--with-region base64url-encode-region (fns-tests--string-repeat "fo" 50) t)
+                 (concat (fns-tests--string-repeat "Zm9mb2Zv" 16) "Zm9mbw")))
+  (should (equal (fns-tests--with-region base64url-encode-region (fns-tests--string-repeat "foo" 25) t)
+                 (fns-tests--string-repeat "Zm9v" 25)))
+  (should (equal (fns-tests--with-region base64url-encode-region (fns-tests--string-repeat "foob" 15) t)
+                 (fns-tests--string-repeat "Zm9vYmZvb2Jmb29i" 5)))
+  (should (equal (fns-tests--with-region base64url-encode-region (fns-tests--string-repeat "fooba" 15) t)
+                 (fns-tests--string-repeat "Zm9vYmFmb29iYWZvb2Jh" 5)))
+  (should (equal (fns-tests--with-region base64url-encode-region (fns-tests--string-repeat "foobar" 15) t)
+                 (concat (fns-tests--string-repeat "Zm9vYmFyZm9vYmFy" 7) "Zm9vYmFy")))
+  (should (equal (fns-tests--with-region base64url-encode-region (fns-tests--string-repeat "\x14\xfb\x9c\x03\xd9\x7e" 10) t)
+                 (fns-tests--string-repeat "FPucA9l-" 10)))
+  (should (equal (fns-tests--with-region base64url-encode-region (fns-tests--string-repeat "\x14\xfb\x9c\x03\xd9\x7f" 10) t)
+                 (fns-tests--string-repeat "FPucA9l_" 10))))
+
+(ert-deftest fns-test-base64url-encode-string ()
+  ;; url variant wih padding
+  (should (equal (base64url-encode-string "") ""))
+  (should (equal (base64url-encode-string "f") "Zg=="))
+  (should (equal (base64url-encode-string "fo") "Zm8="))
+  (should (equal (base64url-encode-string "foo") "Zm9v"))
+  (should (equal (base64url-encode-string "foob") "Zm9vYg=="))
+  (should (equal (base64url-encode-string "fooba") "Zm9vYmE="))
+  (should (equal (base64url-encode-string "foobar") "Zm9vYmFy"))
+  (should (equal (base64url-encode-string "\x14\xfb\x9c\x03\xd9\x7e") "FPucA9l-"))
+  (should (equal (base64url-encode-string "\x14\xfb\x9c\x03\xd9\x7f") "FPucA9l_"))
+
+  ;; url variant no padding
+  (should (equal (base64url-encode-string "" t) ""))
+  (should (equal (base64url-encode-string "f" t) "Zg"))
+  (should (equal (base64url-encode-string "fo" t) "Zm8"))
+  (should (equal (base64url-encode-string "foo" t) "Zm9v"))
+  (should (equal (base64url-encode-string "foob" t) "Zm9vYg"))
+  (should (equal (base64url-encode-string "fooba" t) "Zm9vYmE"))
+  (should (equal (base64url-encode-string "foobar" t) "Zm9vYmFy"))
+  (should (equal (base64url-encode-string "\x14\xfb\x9c\x03\xd9\x7e" t) "FPucA9l-"))
+  (should (equal (base64url-encode-string "\x14\xfb\x9c\x03\xd9\x7f" t) "FPucA9l_"))
+
+
+  ;; url variant no line break no padding
+  (should (equal (base64url-encode-string (fns-tests--string-repeat "f" 100) t) (concat (fns-tests--string-repeat "Zm" 66) "Zg")))
+  (should (equal (base64url-encode-string (fns-tests--string-repeat "fo" 50) t) (concat (fns-tests--string-repeat "Zm9mb2Zv" 16) "Zm9mbw")))
+  (should (equal (base64url-encode-string (fns-tests--string-repeat "foo" 25) t) (fns-tests--string-repeat "Zm9v" 25)))
+  (should (equal (base64url-encode-string (fns-tests--string-repeat "foob" 15) t) (fns-tests--string-repeat "Zm9vYmZvb2Jmb29i" 5)))
+  (should (equal (base64url-encode-string (fns-tests--string-repeat "fooba" 15) t) (fns-tests--string-repeat "Zm9vYmFmb29iYWZvb2Jh" 5)))
+  (should (equal (base64url-encode-string (fns-tests--string-repeat "foobar" 15) t) (concat (fns-tests--string-repeat "Zm9vYmFyZm9vYmFy" 7) "Zm9vYmFy")))
+  (should (equal (base64url-encode-string (fns-tests--string-repeat "\x14\xfb\x9c\x03\xd9\x7e" 10) t) (fns-tests--string-repeat "FPucA9l-" 10)))
+  (should (equal (base64url-encode-string (fns-tests--string-repeat "\x14\xfb\x9c\x03\xd9\x7f" 10) t) (fns-tests--string-repeat "FPucA9l_" 10))))
+
+(ert-deftest fns-tests-base64-decode-string ()
+  ;; standard variant RFC2045
+  (should (equal (base64-decode-string "") ""))
+  (should (equal (base64-decode-string "Zg==") "f"))
+  (should (equal (base64-decode-string "Zm8=") "fo"))
+  (should (equal (base64-decode-string "Zm9v") "foo"))
+  (should (equal (base64-decode-string "Zm9vYg==") "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE=") "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy") "foobar"))
+  (should (equal (base64-decode-string "FPucA9l+") "\x14\xfb\x9c\x03\xd9\x7e"))
+  (should (equal (base64-decode-string "FPucA9l/") "\x14\xfb\x9c\x03\xd9\x7f"))
+
+  ;; no paddign
+  (should (equal (base64-decode-string "" t) ""))
+  (should (equal (base64-decode-string "Zg" t) "f"))
+  (should (equal (base64-decode-string "Zm8" t) "fo"))
+  (should (equal (base64-decode-string "Zm9v" t) "foo"))
+  (should (equal (base64-decode-string "Zm9vYg" t) "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE" t) "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy" t) "foobar"))
+
+  ;; url variant wih padding
+  (should (equal (base64-decode-string "") ""))
+  (should (equal (base64-decode-string "Zg==" t) "f") )
+  (should (equal (base64-decode-string "Zm8=" t) "fo"))
+  (should (equal (base64-decode-string "Zm9v" t) "foo"))
+  (should (equal (base64-decode-string "Zm9vYg==" t) "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE=" t) "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy" t) "foobar"))
+  (should (equal (base64-decode-string "FPucA9l-" t) "\x14\xfb\x9c\x03\xd9\x7e"))
+  (should (equal (base64-decode-string "FPucA9l_" t) "\x14\xfb\x9c\x03\xd9\x7f"))
+
+  ;; url variant no padding
+  (should (equal (base64-decode-string "") ""))
+  (should (equal (base64-decode-string "Zg" t) "f"))
+  (should (equal (base64-decode-string "Zm8" t) "fo"))
+  (should (equal (base64-decode-string "Zm9v" t) "foo"))
+  (should (equal (base64-decode-string "Zm9vYg" t) "foob"))
+  (should (equal (base64-decode-string "Zm9vYmE" t) "fooba"))
+  (should (equal (base64-decode-string "Zm9vYmFy" t) "foobar"))
+  (should (equal (base64-decode-string "FPucA9l-" t) "\x14\xfb\x9c\x03\xd9\x7e"))
+  (should (equal (base64-decode-string "FPucA9l_" t) "\x14\xfb\x9c\x03\xd9\x7f"))
+
+
+  ;; url variant no line break no padding
+  (should (equal (base64-decode-string (concat (fns-tests--string-repeat "Zm" 66) "Zg") t)
+                 (fns-tests--string-repeat "f" 100)))
+  (should (equal (base64-decode-string (concat (fns-tests--string-repeat "Zm9mb2Zv" 16) "Zm9mbw") t)
+                 (fns-tests--string-repeat "fo" 50)))
+  (should (equal (base64-decode-string (fns-tests--string-repeat "Zm9v" 25) t)
+                 (fns-tests--string-repeat "foo" 25)))
+  (should (equal (base64-decode-string (fns-tests--string-repeat "Zm9vYmZvb2Jmb29i" 5) t)
+                 (fns-tests--string-repeat "foob" 15)))
+  (should (equal (base64-decode-string (fns-tests--string-repeat "Zm9vYmFmb29iYWZvb2Jh" 5) t)
+                 (fns-tests--string-repeat "fooba" 15)))
+  (should (equal (base64-decode-string (concat (fns-tests--string-repeat "Zm9vYmFyZm9vYmFy" 7) "Zm9vYmFy") t)
+                 (fns-tests--string-repeat "foobar" 15)))
+  (should (equal (base64-decode-string (fns-tests--string-repeat "FPucA9l-" 10) t)
+                 (fns-tests--string-repeat "\x14\xfb\x9c\x03\xd9\x7e" 10)))
+  (should (equal (base64-decode-string (fns-tests--string-repeat "FPucA9l_" 10) t)
+                 (fns-tests--string-repeat "\x14\xfb\x9c\x03\xd9\x7f" 10)))
+
+  ;; errors check
+  (should (eq :got-error (condition-case () (base64-decode-string "Zg=") (error :got-error))))
+  (should (eq :got-error (condition-case () (base64-decode-string "Zm9vYmE") (error :got-error))))
+  (should (eq :got-error (condition-case () (base64-decode-string "Zm9vYmFy=") (error :got-error))))
+  (should (eq :got-error (condition-case () (base64-decode-string "Zg=Zg=") (error :got-error)))))
+
 (ert-deftest fns-tests-hash-buffer ()
   (should (equal (sha1 "foo") "0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"))
   (should (equal (with-temp-buffer
-- 
2.21.0


[-- Attachment #5: Type: text/plain, Size: 42 bytes --]


Tell me what you thing is the way to go.

Re: Add support for base64url variant

[Pierre Téchoueyres]

Had you have a chance to look at my patches ?
Should/Could I do something ?

pierre.techoueyres@free.fr (Pierre Téchoueyres) writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: pierre.techoueyres@free.fr (Pierre Téchoueyres)
>>> Cc: emacs-devel@gnu.org
>>> Date: Thu, 23 May 2019 21:37:01 +0200
>>> 
>>> > I'd suggest to call the new argument base64url or somesuch, since
>>> > this is trhe official name.
>>> >
>>> You mean in replacement of url_variant or b64_value_to_char ?
>>
>> The former.
>>
>>> How should I send new versions of the patch ? As a full patch in
>>> attachment like previously ?
>>
>> Yes.  And please include the log messages (in the ChangeLog style).
>>
>>
> You'll find two patches attached.
> First one contains requested changes and, hoppefully, a valid Changelog
>
>
>
> The second patch contains an reworked version wich instead of adding
> many parameters to base64-encode-region (resp. base64-encode-string)
> function create the base64url-encode-region
> (resp. base64url-encode-string) function. Documentation and tests are
> also updated consequently.
>
>
>
> Tell me what you thing is the way to go.

Re: Add support for base64url variant

[Eli Zaretskii]

> From: pierre.techoueyres@free.fr (Pierre Téchoueyres)
> Cc: emacs-devel@gnu.org
> Date: Fri, 07 Jun 2019 23:04:44 +0200
> 
> Had you have a chance to look at my patches ?
> Should/Could I do something ?

Your patches are in my queue (which was unusually long the past 2
weeks).  Sorry for the delay, I will get to that in a day or two.

Re: Add support for base64url variant

[Eli Zaretskii]

> From: pierre.techoueyres@free.fr (Pierre Téchoueyres)
> Cc: emacs-devel@gnu.org
> Date: Mon, 27 May 2019 22:30:14 +0200
> 
> You'll find two patches attached.

Thanks, I liked the second one better, so I pushed it.

Please note that the doc strings and the manual text included a few
typos, and also used the passive tense too much.  See the follow-up
changes I pushed after your changeset.  In the future, I suggest to
use the Emacs spell-checking capabilities to spell-check the text in
the manual and also the comments/strings in the C/Lisp sources.

Re: Add support for base64url variant

[Pierre Téchoueyres]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: pierre.techoueyres@free.fr (Pierre Téchoueyres)
>> Cc: emacs-devel@gnu.org
>> Date: Mon, 27 May 2019 22:30:14 +0200
>> 
>> You'll find two patches attached.
>
> Thanks, I liked the second one better, so I pushed it.

Thank you.  Honestly I didn't expect it was accepted without further
modifications.

>
> Please note that the doc strings and the manual text included a few
> typos, and also used the passive tense too much.  See the follow-up
> changes I pushed after your changeset.  In the future, I suggest to
> use the Emacs spell-checking capabilities to spell-check the text in
> the manual and also the comments/strings in the C/Lisp sources.

This was the part I was the less confident (and for good reasons).
Thanks again to have (completely) rewritten it.  I'll try to be
more rigorous in the future with the doc, NEWS, etc..

I'll also thanks Paul Eggert for his optimizations of the new functions
in commit 5abaea334cf4c0e004fca2b8b272e091eb5b5444.

Pierre.

P.S.
Would it be any interest if I try to add others bases encoding as
defined in RFC 4648 ?  I think, first, to BASE32 and maybe on BASE16
later.

Re: Add support for base64url variant

[Eli Zaretskii]

> From: pierre.techoueyres@free.fr (Pierre Téchoueyres)
> Cc: emacs-devel@gnu.org, Paul Eggert <eggert@Penguin.CS.UCLA.EDU>
> Date: Tue, 11 Jun 2019 20:36:52 +0200
> 
> Would it be any interest if I try to add others bases encoding as
> defined in RFC 4648 ?  I think, first, to BASE32 and maybe on BASE16
> later.

I have never needed those.  Let's hear what others have to say.

Re: Add support for base64url variant

[Achim Gratz]

Eli Zaretskii writes:
>> From: pierre.techoueyres@free.fr (Pierre Téchoueyres)
>> Cc: emacs-devel@gnu.org, Paul Eggert <eggert@Penguin.CS.UCLA.EDU>
>> Date: Tue, 11 Jun 2019 20:36:52 +0200
>> 
>> Would it be any interest if I try to add others bases encoding as
>> defined in RFC 4648 ?  I think, first, to BASE32 and maybe on BASE16
>> later.
>
> I have never needed those.  Let's hear what others have to say.

Base32url is useful for use on case-insensitive filesystems or
with case-insensitive protocols.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Add support for base64url variant

[Richard Copley]

[-- Attachment #1: Type: text/plain, Size: 1097 bytes --]

Eli Zaretskii writes:

    Please note that the doc strings and the manual text included a few
    typos, and also used the passive tense too much.  See the follow-up
    changes I pushed after your changeset.  In the future, I suggest to
    use the Emacs spell-checking capabilities to spell-check the text in
    the manual and also the comments/strings in the C/Lisp sources.

Two more typos (if/of, inserts/insert):

diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi
index 55f5269849..2e7c497f57 100644
--- a/doc/lispref/text.texi
+++ b/doc/lispref/text.texi
@@ -4560,8 +4560,8 @@ Base 64

 @deffn Command base64url-encode-region beg end &optional no-pad
 This function is like @code{base64-encode-region}, but it implements
-the URL variant if base 64 encoding, per RFC 4648, and it doesn't
-inserts newline characters into the encoded text, so the output is
+the URL variant of base 64 encoding, per RFC 4648, and it doesn't
+insert newline characters into the encoded text, so the output is
 just one long line.

 If the optional argument @var{no-pad} is non-@code{nil} then this

[-- Attachment #2: Type: text/html, Size: 1249 bytes --]

Re: Add support for base64url variant

[Stefan Monnier]

>>> Would it be any interest if I try to add others bases encoding as
>>> defined in RFC 4648 ?  I think, first, to BASE32 and maybe on BASE16
>>> later.
>> I have never needed those.  Let's hear what others have to say.

Same here.

> Base32url is useful for use on case-insensitive filesystems or
> with case-insensitive protocols.

That's what it's designed for, yes, but I can't think of any example
where that's useful.  E.g. for case-insensitive filenames, using a hash
instead of a base32 encoding was a better option in all the cases I've
encountered because I only needed "one-way" and it conveniently bounds
the max file name length.


        Stefan

Re: Add support for base64url variant

[Eli Zaretskii]

> From: Richard Copley <rcopley@gmail.com>
> Date: Tue, 11 Jun 2019 21:14:11 +0100
> Cc: Emacs Development <emacs-devel@gnu.org>
> 
> Two more typos (if/of, inserts/insert):

Thanks, fixed.

Re: Add support for base64url variant

[Achim Gratz]

Stefan Monnier writes:
>> Base32url is useful for use on case-insensitive filesystems or
>> with case-insensitive protocols.
>
> That's what it's designed for, yes, but I can't think of any example
> where that's useful.  E.g. for case-insensitive filenames, using a hash
> instead of a base32 encoding was a better option in all the cases I've
> encountered because I only needed "one-way" and it conveniently bounds
> the max file name length.

I've used it for sharding into a blobstore on NTFS/SMB (two characters
for first level, so tree width of 1024).  The key is indeed a hash
(SHA512), so the probability of a collision would be low enough to just
ignore the case (for only 512 wide tree), but it felt cleaner to encode
via base32url instead.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: Add support for base64url variant

[Stefan Monnier]

> I've used it for sharding into a blobstore on NTFS/SMB (two characters
> for first level, so tree width of 1024).  The key is indeed a hash
> (SHA512), so the probability of a collision would be low enough to just
> ignore the case (for only 512 wide tree), but it felt cleaner to encode
> via base32url instead.

Ah, I was assuming a hex representation of the hash.
In that case, using base32 is basically an optimization that shortens
the filename by ~20% [ Using base64 would make the hash a bit weaker and
shorten the filename by ~33% instead.  ]


        Stefan