From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Thomas Fitzsimmons Newsgroups: gmane.emacs.devel Subject: Re: emacs-25 b6b47AF: Properly encode/decode base64Binary data in SOAP Date: Mon, 14 Mar 2016 21:56:16 -0400 Message-ID: References: <20160106200404.17375.71733@vcs.savannah.gnu.org> <87k2l9xg19.fsf@linux-m68k.org> <87d1r1x84z.fsf@linux-m68k.org> <83lh5mfjsn.fsf@gnu.org> <83oaaidydx.fsf@gnu.org> <83io0qdtb0.fsf@gnu.org> <83egbdenx4.fsf@gnu.org> <87fuvtqv33.fsf@gmx.de> <87d1qx9qv7.fsf@gmx.de> <87a8m1cfmt.fsf@gmx.de> <83h9g9c566.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1458006995 19017 80.91.229.3 (15 Mar 2016 01:56:35 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 15 Mar 2016 01:56:35 +0000 (UTC) Cc: michael.albinus@gmx.de, monnier@iro.umontreal.ca, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Mar 15 02:56:29 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1afeDk-0004XP-Ms for ged-emacs-devel@m.gmane.org; Tue, 15 Mar 2016 02:56:28 +0100 Original-Received: from localhost ([::1]:44969 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1afeDj-0002oA-Fn for ged-emacs-devel@m.gmane.org; Mon, 14 Mar 2016 21:56:27 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:35743) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1afeDf-0002n4-Ao for emacs-devel@gnu.org; Mon, 14 Mar 2016 21:56:24 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1afeDb-0007Rm-AZ for emacs-devel@gnu.org; Mon, 14 Mar 2016 21:56:23 -0400 Original-Received: from mail-ig0-x233.google.com ([2607:f8b0:4001:c05::233]:35953) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1afeDb-0007RZ-3F for emacs-devel@gnu.org; Mon, 14 Mar 2016 21:56:19 -0400 Original-Received: by mail-ig0-x233.google.com with SMTP id nk17so54634012igb.1 for ; Mon, 14 Mar 2016 18:56:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fitzsim-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=WBMBXXcYrmpekdkX7FBoaSQ3WvbajDReSpr32DLDx6U=; b=hQJ1eJVZZ2z5fGLqSFELPx0pKGBUW2MdnIy8oY5QE9OHwvIH2cTmj/T6U7mF9Qn6dC OI4TQdFnzJBpDcVMcu7xmZQl9gRKgmpWXE0pkSpK1B6jZ/s2GiqwybVpxDHIOBVECuZ3 CMRVXtzZ51FWOKPdcGcvvadTmfZeoC2l46S5nXg5ZF1c6RW0QmyxhO+98phTp6Zu0+xB 3u5wQZr+lswciphl0JtRxfL/oAExZOhhO9Bi1jv7v0ANQgWZtJ8XD+391IWhWAryxJw4 l3VA8VRlOQkfQjxJKEH3Wu6teWckoxf9YfkpELjMjkC1+vE3lvAPmKcDLkjnhiM08KBV bSLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=WBMBXXcYrmpekdkX7FBoaSQ3WvbajDReSpr32DLDx6U=; b=apWym2ft2OkafxyFtPpQ5hJhm9nVC5JsDr6FhRwJ8T/vqPbciju8ukefBKKrq0EETB ikVV4yVgTjRyI4+s1lZsdZms9NNfxCC7SyTNvvLqtJgKkaT8MJxHTtHGgT3lOHdlkLFy qFeKofsBBMOlMuVMvKkDv0dWLxCtay95JK21XyE759ybmt3GTK2pR7SuSplPWkyjGIM0 2Il0aoVxsNyLj437qnFoJBNITkpnyM5RIZ114QjxQVuSbvuXNHAz9jMSzTEYgdcMpA2g KkTCcIW06XcVoxaZ4RRnjYjok5dWWXwoWDRI5MPJiMWsZQD/L1NXo/ve+a7cRsqudzgA rZBQ== X-Gm-Message-State: AD7BkJKYoPcdjRmNUz6DVDStksjOxwmL3n6yVPJkKfiVRH/krSD7fUKoOHRC8Xs8/rqWHQ== X-Received: by 10.50.61.69 with SMTP id n5mr20697829igr.28.1458006978136; Mon, 14 Mar 2016 18:56:18 -0700 (PDT) Original-Received: from hp-dv5t (69-165-165-189.dsl.teksavvy.com. [69.165.165.189]) by smtp.gmail.com with ESMTPSA id s101sm10676362ioe.24.2016.03.14.18.56.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 14 Mar 2016 18:56:17 -0700 (PDT) In-Reply-To: <83h9g9c566.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 14 Mar 2016 19:58:41 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2607:f8b0:4001:c05::233 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:201772 Archived-At: --=-=-= Content-Type: text/plain Eli Zaretskii writes: >> From: Thomas Fitzsimmons >> Date: Mon, 14 Mar 2016 10:58:44 -0400 >> Cc: Stefan Monnier , emacs-devel@gnu.org >> >> My plan is to replicate that without the soap-client patch, and try to >> fix it a different way, in Debbugs. From that thread, I think this will >> show it: >> >> (async-get (async-start >> `(lambda () >> (load ,(locate-library "debbugs")) >> (debbugs-get-status 22285)))) >> >> I'll work on this hopefully this evening unless you beat me to it. > > Thanks. But it would be good to discuss what you think should be done > in debbugs.el, before you actually do that. The important part is to > figure out how can debbugs.el know whether a byte stream is a text > string. With the soap-client patch reverted, this debbugs.el patch fixes the above test case, producing a correct multibyte string for originator. This is just one example, obviously. The new function should be applied to all values returned by the Debbugs server that may contain multibyte UTF-8 characters. I'll leave this part to Michael. Also, there's still something not quite right about async, for strings that don't contain extended characters: (multibyte-string-p (cdr (assq 'severity (car (async-get (async-start `(lambda () (load ,(locate-library "debbugs")) (debbugs-get-status 22285)))))))) => nil (multibyte-string-p (cdr (assq 'severity (car (debbugs-get-status 22285))))) => t but this looks like a side effect of async. I'll wait to revert the soap-client patch on master and emacs-25 until Debbugs has a new release to fix this, to avoid temporarily re-breaking Debbugs. (However, I'd really like to get this done before Emacs 25.1 so that it doesn't go out with an incompatible soap-client API.) Thomas --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=emacs-debbugs-soap-value-to-string.patch diff --git a/packages/debbugs/debbugs.el b/packages/debbugs/debbugs.el index f145280..e4c9667 100644 --- a/packages/debbugs/debbugs.el +++ b/packages/debbugs/debbugs.el @@ -264,6 +264,27 @@ (defun debbugs-newest-bugs (amount) "Return the list of bug numbers, according to AMOUNT (a number) latest bugs." (sort (car (soap-invoke debbugs-wsdl debbugs-port "newest_bugs" amount)) '<)) +(defun debbugs-convert-soap-value-to-string (string-value) + "If STRING-VALUE is unibyte, decode its contents as a UTF-8 string. +If STRING-VALUE is a multibyte string, then `soap-client' +received an xsd:string for this value, and will have decoded it +already. + +If STRING-VALUE is a unibyte string, then `soap-client' received +an xsd:base64Binary, and ran `base64-decode-string' on it to +produce a unibyte string of bytes. + +For some reason, the Debbugs server code base64-encodes strings +that contain UTF-8 characters, and returns them as +xsd:base64Binary, instead of just returning them as xsd:string. +Therefore, when STRING-VALUE is a unibyte string, we assume its +bytes represent a UTF-8 string and decode them accordingly." + (if (stringp string-value) + (if (not (multibyte-string-p string-value)) + (decode-coding-string string-value 'utf-8) + string-value) + (error "Invalid string value"))) + (defun debbugs-get-status (&rest bug-numbers) "Return a list of status entries for the bugs identified by BUG-NUMBERS. @@ -421,6 +442,11 @@ (defun debbugs-get-status (&rest bug-numbers) (when (stringp (cdr y)) (setcdr y (mapcar 'string-to-number (split-string (cdr y) " " t))))) + ;; "originator" may be an xsd:base64Binary value containing + ;; a UTF-8-encoded string. + (dolist (attribute '(originator)) + (setq y (assoc attribute (cdr (assoc 'value x)))) + (setcdr y (debbugs-convert-soap-value-to-string (cdr y)))) ;; "package" is a string, containing comma separated ;; package names. "keywords" and "tags" are strings, ;; containing blank separated package names. --=-=-=--