From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?B?Sm/Do28gVMOhdm9yYQ==?= Newsgroups: gmane.emacs.devel Subject: Re: jsonrpc.el uses "Content-Length" headers which seem specific to LSP Date: Fri, 20 Oct 2023 01:02:49 +0100 Message-ID: <87leby9n4m.fsf@gmail.com> References: <04f0ef0b-7fcf-4b09-a1c8-94960d8184cf@alphapapa.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="12118"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: emacs-devel , Joseph Turner To: Adam Porter Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Oct 20 02:00:42 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qtcwU-00030F-1w for ged-emacs-devel@m.gmane-mx.org; Fri, 20 Oct 2023 02:00:42 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtcvm-0006UC-4R; Thu, 19 Oct 2023 19:59:58 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtcvk-0006K2-FB for emacs-devel@gnu.org; Thu, 19 Oct 2023 19:59:56 -0400 Original-Received: from mail-lj1-x22b.google.com ([2a00:1450:4864:20::22b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qtcvi-0003aC-I1 for emacs-devel@gnu.org; Thu, 19 Oct 2023 19:59:56 -0400 Original-Received: by mail-lj1-x22b.google.com with SMTP id 38308e7fff4ca-2c5039d4e88so3577951fa.3 for ; Thu, 19 Oct 2023 16:59:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697759992; x=1698364792; darn=gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aE819f4XxbiJRZD96XeKn+Fdjqe3a62RMEOwRo8iS0M=; b=R6vyYdxR/R0Tyihur0OVaWW8dj9W7jXCYL7Moxx5GEjBw2US9dgltSMJwe6nAFV2F6 9hPTk5X3tAPCDeVjgeO1Zz8mlBYUJfqQtLANukmu5a3XtO7AzuQreXWLF1aDoiShZ8WO P5l1OGV8pF/+lkWlbycr0l0VEkPW3wEPOqqqDCAVJzHo/ZGSKFvdpzixQoDEyqhDG3GE vugK9faAAMSRfWv49PEvOlvFG/Q13RKrq4edx47f9OxX67qaAoPDJadmWfX629RX3D96 MFZnoOnzOtytlx3ZvSwOZGuuLYEg4/Kz3KalzhLJzkj5QpUgLAYTWdOMNNZ4HF8n4mCN u1Eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697759992; x=1698364792; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=aE819f4XxbiJRZD96XeKn+Fdjqe3a62RMEOwRo8iS0M=; b=uoSJVXMuLaz8ke/ylqbA8sER03imkf0OV+RB1uh6UXEFTqx9z9NvGh8/hJm0zJpn1I 3MNLzYFIqnqme+DVj7JVav8MoFm5L31nADPdNt7Bn9PxplyzDbIhiZtQHw0yUi7BTu0W M8cDKLuyX/6smRYXrzu9j16xRTK4kJ7i2LvxIkMHjz0ZiUYXtEfsZHAIG7lCHT/4hkLg 6n2o/5mPUQH6+S5Vg6h/M6svTVfyNacXCOXQb6T5xUjowT33yel5dVwiozLDDA/T0qVM o9nRXjChuRg40ngLwaN8l1YHaN053dIRvGOfRLRF/k44lTmmkiMinu4vzlXGB2WIBhk7 JO3w== X-Gm-Message-State: AOJu0YxOok2A3352YKBgkKn6QT405vGCH8C7PncHhVD5TpdWnLC/3I+n 18OLpfV3MrkCnFL7ISJ1b54= X-Google-Smtp-Source: AGHT+IG1dehWbkjltiEJNrkHWZFM6xPw4K1fRGbu9JVFAReBwv7+ObirMVsylKK6sMZTrQ2OkNwWDg== X-Received: by 2002:a05:651c:1a29:b0:2b9:ee3e:2412 with SMTP id by41-20020a05651c1a2900b002b9ee3e2412mr351569ljb.22.1697759991427; Thu, 19 Oct 2023 16:59:51 -0700 (PDT) Original-Received: from krug ([87.196.80.249]) by smtp.gmail.com with ESMTPSA id fm12-20020a05600c0c0c00b004030e8ff964sm5589780wmb.34.2023.10.19.16.59.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 16:59:50 -0700 (PDT) In-Reply-To: (Adam Porter's message of "Thu, 19 Oct 2023 15:59:59 -0500") Received-SPF: pass client-ip=2a00:1450:4864:20::22b; envelope-from=joaotavora@gmail.com; helo=mail-lj1-x22b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:311609 Archived-At: Adam Porter writes: >> To give you an example, in a private project I subclassed >> jsonrpc-connection to make a websocket-based JSONRPC transport. >> It was really easy (I can show the code if you're interested). > > Sure, that could be helpful. Shown at the bottom of this message after my sig. I'm not showing the websocket library or the full thing, of course, but it should give you an idea. > Yes, we're using a network process connecting to a local TCP > server. Indeed, we didn't want to reinvent those things. > >> Now, the flaw I will concede about jsonrpc-process-connection is that it >> describes itself as a "jsonrpc connection over an Emacs process", but it >> would be more accurately described as "JSONRPC connection over HTTP-like >> over Emacs process". Presumably, you want the first and the last part >> but not the middle.=20=20 > > Right, the code we wrote (shared below) basically removes those middle > parts. After having looked at your code, I think you may be underestimating the use of the Content-Length header. It's not a spurious decoration. Rather it helps processes make sense of incomplete or multiple messages that may appear in the process buffer. I'd say it's only in toy examples or "lucky" cases in which you can safely jsonrpc--json-read one single full JSON objects to completeness. In other words, with or without refactorings or adjustments to jsonrpc.el's design, I don't think this code can work reliably within Emacs: (defun hyperdrive-jsonrpc--process-filter (proc string) (when (buffer-live-p (process-buffer proc)) (with-current-buffer (process-buffer proc) (let* ((inhibit-read-only t) (connection (process-get proc 'jsonrpc-connection))) ;; Insert the text, advancing the process marker. (save-excursion (goto-char (process-mark proc)) (insert string) (set-marker (process-mark proc) (point))) (condition-case err (when-let ((json-message (jsonrpc--json-read))) (with-temp-buffer (jsonrpc-connection-receive connection json-message))) (error (message "hyperdrive-jsonrpc--process-filter ERROR:%S PRO= C:%S STRING:%S" err proc string))))))) For example, what if two full messages appear in the process buffer suddenly and the other endpoint is waiting for a reply to the second one before it sends more stuff? Isn't that deadlock? What if a very large message appears in chunks? Are you going to waste lots of works trying to read it and failing each time? I'd be careful about these things, but if you can absolutely guarantee that this never happens or somehow change the code to deal with them reliably, then OK. In any case, I'm now starting to think your code doesn't motivate any significant reworking of the class hierarchy in jsonrpc.el. It's true it uses some jsonrpc.el internals when it shouldn't but it's quite OK to augment the jsonrpc.el API to export jsonrpc--json-read and the few other internal helpers you're using. That doesn't pose any backward-compatibility challenge and your code is indeed quite simple. > Of course, one of the considerations is backward compatibility with > existing users of jsonrpc. There don't appear to be very many yet, > other than Eglot, so some kind of transition might be feasible, > especially since jsonrpc.el is on GNU ELPA. There are definitely other users, some not open source, but some are open source. Last time I did a GitHub code search I came up with at least couple non-Eglot uses. Not sure what you mean by "transition", but any refactorings/redesigns must not introduce breaking changes. Jo=C3=A3o (defclass nih--connection (jsonrpc-connection) ((repl :accessor nih--repl :documentation "REPL buffer") (target-info :reader nih--target-info :initarg :target-info :documentation "Full target info. A JSON object returned by /json/list. May be augmented with nih-specific fields.") (socket :reader nih--socket :initarg :socket :documentation "Open WEBSOCKET object")) :documentation "Represents a NIH connection. `jsonrpc-name' is NOT unique.") (cl-defmethod jsonrpc-connection-send ((conn nih--connection) &rest args &key _id method _params _result _error _partial) "Send MESSAGE, a JSON object, to CONNECTION." ;; next form clearly something to put in an :around method (when method (plist-put args :method (cond ((keywordp method) (substring (symbol-name method) 1)) ((and method (symbolp method)) (symbol-name method))))) (let* ((message `(;; CDP isn't technically JSONRPC, so don't send ;; the `:jsonrpc' "2.0" version identifier which ;; trips up node's server, for example. ,@args)) (json (jsonrpc--json-encode message))) (with-slots (socket) conn (websocket-send-text socket json)) ;; also something for a generic :AFTER (jsonrpc--log-event conn message 'client))) (cl-defmethod jsonrpc-running-p ((conn nih--connection)) (with-slots (socket) conn (websocket-openp socket))) (cl-defmethod jsonrpc-shutdown ((conn nih--connection)) (with-slots (socket) conn (websocket-close socket))) (defun nih--on-websocket-message (ws frame) "Called when FRAME received on NIH WS." (jsonrpc-connection-receive (websocket-client-data ws) ; the connection (json-parse-string (websocket-frame-text frame) :object-type 'plist :null-object nil :false-object :json-false)))