From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Alex Bochannek Newsgroups: gmane.emacs.bugs Subject: bug#43441: 27.1; [PATCH] Fix incorrectly base64-padded faces in gnus-convert-face-to-png Date: Sun, 27 Sep 2020 23:05:28 -0700 Message-ID: References: <875z8cpghq.fsf@gnus.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="21684"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (darwin) Cc: 43441@debbugs.gnu.org To: Lars Ingebrigtsen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Sep 28 08:06:15 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kMmIh-0005Sq-FR for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 28 Sep 2020 08:06:15 +0200 Original-Received: from localhost ([::1]:47196 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kMmIg-0000gh-F9 for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 28 Sep 2020 02:06:14 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:36888) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kMmIU-0000e5-Ba for bug-gnu-emacs@gnu.org; Mon, 28 Sep 2020 02:06:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:39674) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kMmIT-0005NR-Vt for bug-gnu-emacs@gnu.org; Mon, 28 Sep 2020 02:06:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kMmIT-0000nn-RM for bug-gnu-emacs@gnu.org; Mon, 28 Sep 2020 02:06:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Alex Bochannek Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 28 Sep 2020 06:06:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43441 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 43441-submit@debbugs.gnu.org id=B43441.16012731433049 (code B ref 43441); Mon, 28 Sep 2020 06:06:01 +0000 Original-Received: (at 43441) by debbugs.gnu.org; 28 Sep 2020 06:05:43 +0000 Original-Received: from localhost ([127.0.0.1]:51219 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kMmIA-0000n7-DS for submit@debbugs.gnu.org; Mon, 28 Sep 2020 02:05:42 -0400 Original-Received: from 50-0-39-243.dsl.static.fusionbroadband.com ([50.0.39.243]:60113 helo=mail.lapseofthought.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kMmI4-0000mp-Is for 43441@debbugs.gnu.org; Mon, 28 Sep 2020 02:05:41 -0400 Original-Received: from localhost (localhost [127.0.0.1]) by mail.lapseofthought.com (Postfix) with ESMTP id 4C0Bp22ZFCz3jj2G; Sun, 27 Sep 2020 23:05:30 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at lapseofthought.com Original-Received: from mail.lapseofthought.com ([127.0.0.1]) by localhost (mail.lapseofthought.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k8rr3eUIcmZ2; Sun, 27 Sep 2020 23:05:29 -0700 (PDT) Original-Received: from awb-mbp.local (unknown [IPv6:2601:646:4200:b470:6105:e647:b3d2:53cf]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by mail.lapseofthought.com (Postfix) with ESMTPSA id 4C0Bp11ccFz3jj6Z; Sun, 27 Sep 2020 23:05:29 -0700 (PDT) In-Reply-To: <875z8cpghq.fsf@gnus.org> (Lars Ingebrigtsen's message of "Thu, 17 Sep 2020 17:31:13 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:189179 Archived-At: --=-=-= Content-Type: text/plain Lars Ingebrigtsen writes: > Alex Bochannek writes: > >> I don't suspect this problem is widespread in other uses of the base64 >> decoder, so it seems appropriate to me to just patch >> gnus-convert-face-to-png to generate the right amount of padding. > > Hm... I think it would be nice to have a utility function to fix up > base64 padding, and then gnus-convert-face-to-png could just use that? > > It should work for both base64 that has newlines inserted and not. Took me a little while, but I broke this out into a utility function as you suggested. I also wrote some tests for it. As part of me getting familiar with ERT, I added a bunch of tests for gnus-string< and gnus-string>, which I hope are useful. They should probably be in a separate commit, but I leave that up to you. --=-=-= Content-Type: text/x-patch Content-Disposition: inline Content-Description: Fix incorrectly base64-padded faces in gnus-convert-face-to-png diff --git a/lisp/gnus/gnus-fun.el b/lisp/gnus/gnus-fun.el index c95449762e..2461fd45fd 100644 --- a/lisp/gnus/gnus-fun.el +++ b/lisp/gnus/gnus-fun.el @@ -205,11 +205,12 @@ gnus-face-encode (defun gnus-convert-face-to-png (face) "Convert FACE (which is base64-encoded) to a PNG. The PNG is returned as a string." - (mm-with-unibyte-buffer - (insert face) - (ignore-errors - (base64-decode-region (point-min) (point-max))) - (buffer-string))) + (let ((face (gnus-base64-repad face))) + (mm-with-unibyte-buffer + (insert face) + (ignore-errors + (base64-decode-region (point-min) (point-max))) + (buffer-string)))) ;;;###autoload (defun gnus-convert-png-to-face (file) --=-=-= Content-Type: text/x-patch Content-Disposition: inline Content-Description: Implement gnus-base64-repad utility function to create properly base64-padded strings diff --git a/lisp/gnus/gnus-util.el b/lisp/gnus/gnus-util.el index aa9f137e20..7e8dcaa47c 100644 --- a/lisp/gnus/gnus-util.el +++ b/lisp/gnus/gnus-util.el @@ -1343,6 +1343,58 @@ gnus-url-unhex-string (setq tmp (concat tmp str)) tmp)) +(defun gnus-base64-repad (str &optional reject-newlines line-length) + "Take a base 64-encoded string and return it padded correctly, +regardless of what padding the input string alread had. + +If any combination of CR and LF characters are present and +REJECT-NEWLINES is nil remove them, otherwise raise an error. If +LINE-LENGTH is set and the string (or any line in the string if +REJECT-NEWLINES is nil) is longer than that number, raise an +error. Common line length for input characters are 76 plus CRLF +(RFC 2045 MIME), 64 plus CRLF (RFC 1421 PEM), and 1000 including +CRLF (RFC 5321 SMTP). +" + ;;; RFC 4648 specifies that: + ;;; - three 8-bit inputs make up a 24-bit group + ;;; - the 24-bit group is broken up into four 6-bit values + ;;; - each 6-bit value is mapped to one character of the base 64 alphabet + ;;; - if the final 24-bit quantum is filled with only 8 bits the output + ;;; will be two base 64 characters followed by two "=" padding characters + ;;; - if the final 24-bit quantum is filled with only 16 bits the output + ;;; will be three base 64 character followed by one "=" padding character + ;;; + ;;; RFC 4648 section 3 considerations: + ;;; - if reject-newlines is nil (default), concatenate multi-line + ;;; input (3.1, 3.3) + ;;; - if line-length is set, error on input exceeding the limit (3.1) + ;;; - reject characters outside base encoding (3.3, also section 12) + + (let ((splitstr (split-string str "[\r\n]" t))) + (progn + (if (and reject-newlines (> (length splitstr) 1)) + (error "Invalid Base64 string")) + (dolist (substr splitstr) + (if (and line-length (> (length substr) line-length)) + (error "Base64 string exceeds line-length")) + (if (string-match "[^A-Za-z0-9+/=]" substr) + (error "Invalid Base64 string"))) + (let* ((str (string-join splitstr)) + (len (length str)) + (padding nil)) + (if (string-match "=" str) + (setq len (match-beginning 0))) + (progn (setq padding + (/ + (- 24 + (pcase (mod (* len 6) 24) + (`0 24) + (n n))) + 6)) + (concat + (substring str 0 len) + (make-string padding ?=))))))) + (defun gnus-make-predicate (spec) "Transform SPEC into a function that can be called. SPEC is a predicate specifier that contains stuff like `or', `and', --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Description: Implement test cases of gnus-string>, gnus-string<, and gnus-base64-repad diff --git a/test/lisp/gnus/gnus-util-tests.el b/test/lisp/gnus/gnus-util-t= ests.el index 7eadb0de71..ed33be46a3 100644 --- a/test/lisp/gnus/gnus-util-tests.el +++ b/test/lisp/gnus/gnus-util-tests.el @@ -25,6 +25,65 @@ (require 'ert) (require 'gnus-util) =20 +(ert-deftest gnus-string> () + ;; Failure paths + (should-error (gnus-string> "" 1) + :type 'wrong-type-argument) + (should-error (gnus-string> "") + :type 'wrong-number-of-arguments) + + ;; String tests + (should (gnus-string> "def" "abc")) + (should (gnus-string> 'def 'abc)) + (should (gnus-string> "abc" "DEF")) + (should (gnus-string> "abc" 'DEF)) + (should (gnus-string> "=CE=B1=CE=B2=CE=B3" "abc")) + (should (gnus-string> "=D7=90=D7=91=D7=92" "=CE=B1=CE=B2=CE=B3")) + (should (gnus-string> nil "")) + (should (gnus-string> "abc" "")) + (should (gnus-string> "abc" "ab")) + (should-not (gnus-string> "abc" "abc")) + (should-not (gnus-string> "abc" "def")) + (should-not (gnus-string> "DEF" "abc")) + (should-not (gnus-string> 'DEF "abc")) + (should-not (gnus-string> "123" "abc")) + (should-not (gnus-string> "" ""))) + +(ert-deftest gnus-string< () + ;; Failure paths + (should-error (gnus-string< "" 1) + :type 'wrong-type-argument) + (should-error (gnus-string< "") + :type 'wrong-number-of-arguments) + + ;; String tests + (setq case-fold-search nil) + (should (gnus-string< "abc" "def")) + (should (gnus-string< 'abc 'def)) + (should (gnus-string< "DEF" "abc")) + (should (gnus-string< "DEF" 'abc)) + (should (gnus-string< "abc" "=CE=B1=CE=B2=CE=B3")) + (should (gnus-string< "=CE=B1=CE=B2=CE=B3" "=D7=90=D7=91=D7=92")) + (should (gnus-string< "" nil)) + (should (gnus-string< "" "abc")) + (should (gnus-string< "ab" "abc")) + (should-not (gnus-string< "abc" "abc")) + (should-not (gnus-string< "def" "abc")) + (should-not (gnus-string< "abc" "DEF")) + (should-not (gnus-string< "abc" 'DEF)) + (should-not (gnus-string< "abc" "123")) + (should-not (gnus-string< "" "")) + + ;; gnus-string< checks case-fold-search + (setq case-fold-search t) + (should (gnus-string< "abc" "DEF")) + (should (gnus-string< "abc" 'GHI)) + (should (gnus-string< 'abc "DEF")) + (should (gnus-string< 'GHI 'JKL)) + (should (gnus-string< "abc" "=CE=91=CE=92=CE=93")) + (should-not (gnus-string< "ABC" "abc")) + (should-not (gnus-string< "def" "ABC"))) + (ert-deftest gnus-subsetp () ;; False for non-lists. (should-not (gnus-subsetp "1" "1")) @@ -73,4 +132,43 @@ gnus-setdiff (should (equal '("1") (gnus-setdiff '(2 "1" 2) '(2)))) (should (equal '("1" "1") (gnus-setdiff '(2 "1" 2 "1") '(2))))) =20 +(ert-deftest gnus-base64-repad () + (should-error (gnus-base64-repad "" nil nil nil) + :type 'wrong-number-of-arguments) + (should-error (gnus-base64-repad 1) + :type 'wrong-type-argument) + + ;; RFC4648 test vectors + (should (equal "" (gnus-base64-repad ""))) + (should (equal "Zg=3D=3D" (gnus-base64-repad "Zg=3D=3D"))) + (should (equal "Zm8=3D" (gnus-base64-repad "Zm8=3D"))) + (should (equal "Zm9v" (gnus-base64-repad "Zm9v"))) + (should (equal "Zm9vYg=3D=3D" (gnus-base64-repad "Zm9vYg=3D=3D"))) + (should (equal "Zm9vYmE=3D" (gnus-base64-repad "Zm9vYmE=3D"))) + (should (equal "Zm9vYmFy" (gnus-base64-repad "Zm9vYmFy"))) + + (should (equal "Zm8=3D" (gnus-base64-repad "Zm8"))) + (should (equal "Zg=3D=3D" (gnus-base64-repad "Zg"))) + (should (equal "Zg=3D=3D" (gnus-base64-repad "Zg=3D=3D=3D=3D"))) + + (should-error (gnus-base64-repad " ") + :type 'error) + (should-error (gnus-base64-repad "Zg=3D=3D ") + :type 'error) + (should-error (gnus-base64-repad "Z?\x00g=3D=3D") + :type 'error) + ;; line-length + (should-error (gnus-base64-repad "Zg=3D=3D=3D=3D" nil 4) + :type 'error) + ;; reject-newlines + (should-error (gnus-base64-repad "Zm9v\r\nYmFy" t) + :type 'error) + (should (equal "Zm9vYmFy" (gnus-base64-repad "Zm9vYmFy" t))) + (should (equal "Zm9vYmFy" (gnus-base64-repad "Zm9v\r\nYmFy" nil))) + (should (equal "Zm9vYmFy" (gnus-base64-repad "Zm9v\r\nYmFy\n" nil))) + (should-error (gnus-base64-repad "Zm9v\r\n YmFy\r\n" nil) + :type 'error) + (should-error (gnus-base64-repad "Zm9v\r\nYmFy" nil 3) + :type 'error)) + ;;; gnustest-gnus-util.el ends here --=-=-= Content-Type: text/plain -- Alex. --=-=-=--