From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Joseph Turner Newsgroups: gmane.emacs.devel Subject: Re: How to get buffer byte length (not number of characters)? Date: Fri, 23 Aug 2024 00:07:54 -0700 Message-ID: <87ttfbg0a0.fsf@ushin.org> References: <87wmkbekjp.fsf@ushin.org> <86o75nwilg.fsf@gnu.org> <87bk1lhkvg.fsf@ushin.org> <86y14pu5rp.fsf@gnu.org> <871q2hfn7c.fsf@ushin.org> <86plq1td4n.fsf@gnu.org> <87ed6hdnpe.fsf@ushin.org> <86o75ksm2q.fsf@gnu.org> <203a830d-356e-4c0a-83c6-c2aeb8d1e135@alphapapa.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="22127"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , emacs-devel@gnu.org, schwab@suse.de To: Adam Porter Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Aug 23 14:26:40 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1shTNH-0005Wk-SW for ged-emacs-devel@m.gmane-mx.org; Fri, 23 Aug 2024 14:26:40 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1shTMr-0002XJ-Vx; Fri, 23 Aug 2024 08:26:17 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1shOp5-0008QD-I8 for emacs-devel@gnu.org; Fri, 23 Aug 2024 03:35:03 -0400 Original-Received: from out-173.mta0.migadu.com ([2001:41d0:1004:224b::ad]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1shOoy-0007fy-Eg for emacs-devel@gnu.org; Fri, 23 Aug 2024 03:35:02 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ushin.org; s=key1; t=1724398493; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=qHfGyDjN8gjgH6UVNDnjQQWtsvtvFheWRt4Tthnq0Jg=; b=kqw042JCFmN80sTfwbCol9mXdwPBbipjgBA23eiGTnJo/QpEFd1k2dAufA9fGoE3Y5PYmx O0Ig6xm9cmMdyjq7OKUpeH+zXnIl0rwaeyrzEC+GpOuix8RHC1F9PmgujN5dS7pM9tnEJ9 Brogo7zNIxMh/ZS5UufZnJMvkP7DI3uBsf0gZ0Yd3Zc9/ogbSsNY+tSWGmcEhB7U/IqIJS pPH1moYK80butVhoM4CCgGfcVAr7frCCgxtUq4SIGKdAZx0326TxtF1z0n2tgQMRhM0tiS yPJt9BMmOkWnLGtvgpHRbrW4JIFe8FN0vgnD3wGKS0WOZiQitO+B86Zde1Ye6w== In-Reply-To: <203a830d-356e-4c0a-83c6-c2aeb8d1e135@alphapapa.net> (Adam Porter's message of "Fri, 23 Aug 2024 01:31:16 -0500") X-Migadu-Flow: FLOW_OUT Received-SPF: pass client-ip=2001:41d0:1004:224b::ad; envelope-from=joseph@ushin.org; helo=out-173.mta0.migadu.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 23 Aug 2024 08:26:08 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:323074 Archived-At: --=-=-= Content-Type: text/plain Adam Porter writes: > On 8/22/24 08:50, Eli Zaretskii wrote: > >> But AFAICT, plz.el does decode the stuff it gets from curl, which >> doesn't seem to be consistent with what you say above. If plz.el >> would accept unibyte text and return unibyte text, that would be >> consistent: it would mean that any callers of plz.el need to do >> encoding and decoding themselves. But that doesn't seem to be the >> case now. >> Am I missing something? > > Yes, the :decode argument to plz. If :decode is nil (or if ":as > 'binary" is specified, which sets :decode to nil), plz does not decode > the response body. Currently, GET decodes by default while PUT does no encoding by default. IIUC, the suggestion is that GET and PUT requests either both handle coding by default or neither does by default. Currently, PUT requests which pass an unencoded buffer with multibyte characters currently send the internal Emacs multibyte representation. I'd be in favor of adding some automatic encoding handling for PUT requests so that most users don't have to think about it. Please see patch! (not tested yet) Best, Joseph --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Add-plz-ENCODE-argument-Add-default-encoding.patch >From 6a8f13fa799f4ba8b64effe229379dc54ef19c91 Mon Sep 17 00:00:00 2001 From: Joseph Turner Date: Thu, 22 Aug 2024 00:02:19 -0700 Subject: [PATCH] Add: (plz) ENCODE argument; Add default encoding Previously, PUT requests which pass an unencoded string or buffer with multibyte characters sent the internal Emacs multibyte representation. Now strings and buffers are encoded by default, and the ENCODE nil argument (or :BODY-TYPE 'binary) can used when the user wants to handle encoding. WIP --- plz.el | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/plz.el b/plz.el index 903d71e..ffbbe0b 100644 --- a/plz.el +++ b/plz.el @@ -325,7 +325,8 @@ (defalias 'plz--generate-new-buffer (cl-defun plz (method url &rest rest &key headers body else filter finally noquery timeout (as 'string) (then 'sync) - (body-type 'text) (decode t decode-s) + (body-type 'text) (encode t encode-s) + (decode t decode-s) (connect-timeout plz-connect-timeout)) "Request METHOD from URL with curl. Return the curl process object or, for a synchronous request, the @@ -340,6 +341,11 @@ (cl-defun plz (method url &rest rest &key headers body else filter finally noque BODY-TYPE may be `text' to send BODY as text, or `binary' to send it as binary. +If ENCODE is non-nil, BODY is encoded automatically. For binary +content, it should be nil. When BODY-TYPE is `binary', ENCODE is +automatically set to nil. ENCODE has no effect when BODY is a +list like `(file FILENAME)'. + AS selects the kind of result to pass to the callback function THEN, or the kind of result to return for synchronous requests. It may be: @@ -416,6 +422,8 @@ (cl-defun plz (method url &rest rest &key headers body else filter finally noque (declare (indent defun)) (setf decode (if (and decode-s (not decode)) nil decode)) + (setf encode (if (and encode-s (not encode)) + nil encode)) ;; NOTE: By default, for PUT requests and POST requests >1KB, curl sends an ;; "Expect:" header, which causes servers to send a "100 Continue" response, which ;; we don't want to have to deal with, so we disable it by setting the header to @@ -473,6 +481,9 @@ (cl-defun plz (method url &rest rest &key headers body else filter finally noque (decode (pcase as ('binary nil) (_ decode))) + (encode (pcase body-type + ('binary nil) + (_ encode))) (default-directory ;; Avoid making process in a nonexistent directory (in case the current ;; default-directory has since been removed). It's unclear what the best @@ -553,9 +564,18 @@ (cl-defun plz (method url &rest rest &key headers body else filter finally noque (process-send-string process curl-config) (when body (cl-typecase body - (string (process-send-string process body)) - (buffer (with-current-buffer body - (process-send-region process (point-min) (point-max)))))) + (string (process-send-string + process (if encode + (encode-coding-string + body (cdr default-process-coding-system)) + body))) + (buffer (if encode + (with-temp-buffer + (insert-buffer-substring-no-properties body) + (encode-coding-region (point-min) (point-max) body-coding) + (process-send-region process (point-min) (point-max))) + (with-current-buffer body + (process-send-region process (point-min) (point-max))))))) (process-send-eof process) (if sync-p (unwind-protect -- 2.41.0 --=-=-=--