From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.bugs Subject: bug#35739: Bad signature from GNU ELPA Date: Wed, 29 May 2019 14:45:50 -0400 Message-ID: References: <87mujog0ao.fsf@gmail.com> <837easa178.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="24221"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: 35739@debbugs.gnu.org, Noam Postavsky To: Richard Copley Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed May 29 20:46:42 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hW3az-00068W-LV for geb-bug-gnu-emacs@m.gmane.org; Wed, 29 May 2019 20:46:42 +0200 Original-Received: from localhost ([127.0.0.1]:59189 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hW3ay-0001dk-A4 for geb-bug-gnu-emacs@m.gmane.org; Wed, 29 May 2019 14:46:40 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:35648) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hW3am-0001Zf-FV for bug-gnu-emacs@gnu.org; Wed, 29 May 2019 14:46:32 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hW3al-0004fP-0L for bug-gnu-emacs@gnu.org; Wed, 29 May 2019 14:46:28 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:46320) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hW3aM-0004Vt-2M for bug-gnu-emacs@gnu.org; Wed, 29 May 2019 14:46:07 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hW3aL-0007Xw-VH for bug-gnu-emacs@gnu.org; Wed, 29 May 2019 14:46:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 29 May 2019 18:46:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35739 X-GNU-PR-Package: emacs Original-Received: via spool by 35739-submit@debbugs.gnu.org id=B35739.155915556028999 (code B ref 35739); Wed, 29 May 2019 18:46:01 +0000 Original-Received: (at 35739) by debbugs.gnu.org; 29 May 2019 18:46:00 +0000 Original-Received: from localhost ([127.0.0.1]:59864 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hW3aK-0007Xf-Fm for submit@debbugs.gnu.org; Wed, 29 May 2019 14:46:00 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:40298) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hW3aJ-0007XP-Dc for 35739@debbugs.gnu.org; Wed, 29 May 2019 14:45:59 -0400 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 06C88442DD8; Wed, 29 May 2019 14:45:54 -0400 (EDT) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 26033442DD5; Wed, 29 May 2019 14:45:52 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1559155552; bh=1EPo1D74GW7B9Umm62ENGFzfrA8iu3G+kmbN/65kGa8=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=RsEySvAmI/5oQf3uFNm/ot1Nz05UiEOf2giqhyNliI2ir/L8/vDxa++auMlQxS/VY NSXM/lSSt2EqLY9eHDqg4xo5WeLONLPWVJWIdjcFctgwFUZAzssTs5/dxfYVFMxTlS o1Fatp7xD1wrARN1M+PD4G/piW7sPtjDvuivvS7Mwzrf2aWUyMksXxtzhbe0gCrIfG EKI5LbTHAiesLG40ojX2tw9YudTQesmtmvHN/0UsuMm3Fo3Kf/lr1UQlnR3HuOw9Wu xo8ep99zlEHUZoScPkz7RJxm1LO6l9S1MY0RTU6rNpfaQeCha6PFp2xntz+jmp+4Db s6FE2hg13pedw== Original-Received: from alfajor (modemcable157.163-203-24.mc.videotron.ca [24.203.163.157]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 84DA0120B7F; Wed, 29 May 2019 14:45:51 -0400 (EDT) In-Reply-To: (Richard Copley's message of "Wed, 15 May 2019 07:46:12 +0100") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:159890 Archived-At: > Not in detail, it's not an area of expertise of mine. We call > (decode-coding-region (point-min) (point-max) 'undecided) on the > payload of "https://elpa.gnu.org/packages/archive-contents", > which is raw text. The resulting buffer's buffer-file-coding- > system is iso-latin-1-dos. Indeed, it seems that url-insert-file-contents sets buffer-file-coding-system. Maybe we can use that in the emacs-26 branch. Can you try the emacs-26 code with the patch below to see if it fixes the current problem? Stefan 2019-05-29 Stefan Monnier * lisp/emacs-lisp/package.el: Obey buffer-file-coding-system. `url-insert-file-contents` saves in buffer-file-coding-system the coding-system used to decode the contents. Preserve this as the contents is moved from buffer to string to buffer, and use it when saving the contents to file, so as to try and better preserve the original byte sequence (bug#35739). (package--buffer-string, package--cs): New functions. (package--check-signature): Encode `string` if a coding-system was specified in buffer-file-coding-system. (package--download-one-archive, package-install-from-archive): Obey and preserve the buffer-file-coding-system if specified. diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el index 1a185de4a5..46f7c91272 100644 --- a/lisp/emacs-lisp/package.el +++ b/lisp/emacs-lisp/package.el @@ -1241,6 +1241,17 @@ package--check-signature-content (signal 'bad-signature (list sig-file))) good-signatures))) +(defun package--buffer-string () + (let ((string (buffer-string))) + (when (and buffer-file-coding-system + (> (length string) 0)) + (put-text-property 0 1 'package--cs buffer-file-coding-system string)) + string)) + +(defun package--cs (string) + (and (> (length string) 0) + (get-text-property 0 'package--cs string))) + (defun package--check-signature (location file &optional string async callback unwind) "Check signature of the current buffer. Download the signature file from LOCATION by appending \".sig\" @@ -1260,8 +1271,12 @@ package--check-signature UNWIND, if provided, is a function to be called after everything else, even if an error is signaled." - (let ((sig-file (concat file ".sig")) - (string (or string (buffer-string)))) + (let* ((sig-file (concat file ".sig")) + (string (or string (package--buffer-string))) + (cs (package--cs string))) + ;; Re-encode the downloaded file with the coding-system with which + ;; it was decoded, so we (hopefully) get the exact same bytes back. + (when cs (setq string (encode-coding-string string cs))) (package--with-response-buffer location :file sig-file :async async :noerror t ;; Connection error is assumed to mean "no sig-file". @@ -1529,7 +1544,7 @@ package--download-one-archive :error-form (package--update-downloads-in-progress archive) (let* ((location (cdr archive)) (name (car archive)) - (content (buffer-string)) + (content (package--buffer-string)) (dir (expand-file-name (format "archives/%s" name) package-user-dir)) (local-file (expand-file-name file dir))) (when (listp (read content)) @@ -1538,7 +1553,8 @@ package--download-one-archive (member name package-unsigned-archives)) ;; If we don't care about the signature, save the file and ;; we're done. - (progn (let ((coding-system-for-write 'utf-8)) + (progn (let ((coding-system-for-write + (or (package--cs content) 'utf-8))) (write-region content nil local-file nil 'silent)) (package--update-downloads-in-progress archive)) ;; If we care, check it (perhaps async) and *then* write the file. @@ -1546,7 +1562,7 @@ package--download-one-archive location file content async ;; This function will be called after signature checking. (lambda (&optional good-sigs) - (let ((coding-system-for-write 'utf-8)) + (let ((coding-system-for-write (or (package--cs content) 'utf-8))) (write-region content nil local-file nil 'silent)) ;; Write out good signatures into archive-contents.signed file. (when good-sigs @@ -1838,15 +1854,17 @@ package-install-from-archive (let ((save-silently t)) (package-unpack pkg-desc)) ;; If we care, check it and *then* write the file. - (let ((content (buffer-string))) + (let ((content (package--buffer-string))) (package--check-signature location file content nil ;; This function will be called after signature checking. (lambda (&optional good-sigs) ;; Signature checked, unpack now. - (with-temp-buffer (insert content) - (let ((save-silently t)) - (package-unpack pkg-desc))) + (with-temp-buffer + (insert content) + (setq buffer-file-coding-system (package--cs content)) + (let ((save-silently t)) + (package-unpack pkg-desc))) ;; Here the package has been installed successfully, mark it as ;; signed if appropriate. (when good-sigs