From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Adam Porter Newsgroups: gmane.emacs.devel Subject: Re: jsonrpc.el uses "Content-Length" headers which seem specific to LSP Date: Wed, 22 Nov 2023 16:40:57 -0600 Message-ID: <4d3cd934-01a8-41ac-b50b-61ff729c9a2c@alphapapa.net> References: <04f0ef0b-7fcf-4b09-a1c8-94960d8184cf@alphapapa.net> <87leby9n4m.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="40187"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla Thunderbird Cc: emacs-devel , Joseph Turner To: =?UTF-8?B?Sm/Do28gVMOhdm9yYQ==?= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Nov 22 23:42:12 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1r5vv9-000ACw-Jr for ged-emacs-devel@m.gmane-mx.org; Wed, 22 Nov 2023 23:42:12 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r5vuG-0008Vx-T2; Wed, 22 Nov 2023 17:41:16 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r5vuF-0008Ug-CL for emacs-devel@gnu.org; Wed, 22 Nov 2023 17:41:15 -0500 Original-Received: from toucan.tulip.relay.mailchannels.net ([23.83.218.254]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r5vuD-0003jw-E9 for emacs-devel@gnu.org; Wed, 22 Nov 2023 17:41:15 -0500 X-Sender-Id: dreamhost|x-authsender|adam@alphapapa.net Original-Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 469BD90244C; Wed, 22 Nov 2023 22:41:10 +0000 (UTC) Original-Received: from pdx1-sub0-mail-a271.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id E6AA890231B; Wed, 22 Nov 2023 22:41:09 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1700692869; a=rsa-sha256; cv=none; b=T7x+63KDqfgKnCbKwc0AUXwQ2Qa7NklFQgcMLiFycZbuhpH9B5s9h7gBX0ViaWQuLNl7pg FnGjji6tYw3OUYymuKnNf3rPNwXJr8UJ/auRmevNQ7lEpX6pvGAoIB8w5iI78TifXRYIlB Pd6kFAfvm5Xy+iGv0GJDWSURjiPjycN3rpOhTFZhUzNp/2egf86N+ByRR7mHkKazByRDCs s0DvAnVDf7/iOmkchQz3LP9UH2rjpaKPfEaffYu3p7NKDtsb6K0ZeFRGYyXB0g8Totz8DN 6EaKVwnKmgLtYTXqS9/GHbE85HX36j5hYfz1eMVJTRGAm61YILVhTODmlv2C9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1700692869; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Q0EKJDWZ4AbY7pzHpWL9IKig3nrpN/ErC3xsc22lYpI=; b=OLxnHPCXRWn34QTaTH/jiDRxqV84OPvCGGZiyXrpF1Xwdgp5zB9YdppBxyEjrXXYdm6oNe IcM0gtA4Di8K24AqfJ09Scla+EAFWPBJDv6hZhsOTyqEEnyuZT12RUppwDauk+7F62NhuK huLilnP3fqs/qZ1j35dn24WJh3jTOg/sFGR4nWHjuRGOb7SMKikKZnJY88bkTBxD5O1n4g D2VB8mUOGzmZC94h4/C9XC9preirMvem39w2EB/leDfLKKsfudbtx6Mj+9pMHVZaM/cGmo lTv6kUWqU6o6tBBu2Tu9X2+QiHkKIkyPLaXTWCTbNX17rKdc8FWPoAu1VI9R9Q== ARC-Authentication-Results: i=1; rspamd-788bb65f6c-hmhfm; auth=pass smtp.auth=dreamhost smtp.mailfrom=adam@alphapapa.net X-Sender-Id: dreamhost|x-authsender|adam@alphapapa.net X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|adam@alphapapa.net X-MailChannels-Auth-Id: dreamhost X-Skirt-Illustrious: 2992410b06708f1a_1700692870130_3136116402 X-MC-Loop-Signature: 1700692870130:2134217345 X-MC-Ingress-Time: 1700692870130 Original-Received: from pdx1-sub0-mail-a271.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.108.22.8 (trex/6.9.2); Wed, 22 Nov 2023 22:41:10 +0000 Original-Received: from [10.66.1.30] (unknown [91.193.232.98]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: adam@alphapapa.net) by pdx1-sub0-mail-a271.dreamhost.com (Postfix) with ESMTPSA id 4SbGS446ZRzFT; Wed, 22 Nov 2023 14:41:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alphapapa.net; s=dreamhost; t=1700692869; bh=Q0EKJDWZ4AbY7pzHpWL9IKig3nrpN/ErC3xsc22lYpI=; h=Date:Subject:To:Cc:From:Content-Type:Content-Transfer-Encoding; b=F+mt+0JDxIMV683mJnB74rZVlvnbDtczGweJcCsPfgLsrtt1BpUAJcf18Tlmalxsm mWYfTxeGlH666O4Oj6Jz/Z1RCxQc/CpCJuzPXb00qX5XSAXe7w0bRaUGtWtOf/22tu EdSj2fFfRI5bPF+OgzJUgZzjhekZVewpjkWoM9rxlq3EVBRuCeOjUmsW2tBd+lvv86 2/vhObKPGdyKm/A0jyBBtAIzuKbb8TQ4o7aF4fFsFSky1jMNg4vVivub+utONsXkYK xxrCNzkN6B87gk/lv70UfbCWPRWJQGQDBoXdonSp2e2kcEUsA5MqHlF8/0HMpR7JiV H4uN5ar85vh9g== Content-Language: en-US In-Reply-To: <87leby9n4m.fsf@gmail.com> Received-SPF: neutral client-ip=23.83.218.254; envelope-from=adam@alphapapa.net; helo=toucan.tulip.relay.mailchannels.net X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NEUTRAL=0.779, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:313134 Archived-At: Hi João, Thanks for this reply of yours from last month. I hadn't had anything to add until now, after we've had more development in our project. On 10/19/23 19:02, João Távora wrote: > After having looked at your code, I think you may be underestimating > the use of the Content-Length header. It's not a spurious > decoration. Rather it helps processes make sense of incomplete or > multiple messages that may appear in the process buffer. I'd say > it's only in toy examples or "lucky" cases in which you can safely > jsonrpc--json-read one single full JSON objects to completeness. Agreed. Our intention was to use the hack of trying to parse potentially incomplete objects for only small responses, and only because our RPC server doesn't support the Content-Length header. We may be able to add support for that header to the RPC server, but we've been discussing another potential solution: to use something like the JSON Lines[0] (aka NDJSON[1]) format, which would mean that each JSON object would be sent as a single line (with any internal newlines being escaped). Then, in the process filter, we would look for unescaped newlines, and if we find any, we would parse the received data up to that newline. This would allow us to know when a complete object has been received. > It's true it uses some jsonrpc.el internals when it shouldn't but > it's quite OK to augment the jsonrpc.el API to export > jsonrpc--json-read and the few other internal helpers you're using. > That doesn't pose any backward-compatibility challenge and your code > is indeed quite simple. Of course, this means that we would still need to make our own version of the jsonrpc process filter function, similar to the code seen earlier in this thread, but it should require only minor changes. Given your experience with these things, does this seem like a reasonable solution to you? Or is there still strong reason to prefer using a Content-Length header to detect the end of responses? Thanks for your help! --Adam 0: https://jsonlines.org/ 1: https://ndjson.org/