Re: jsonrpc.el uses "Content-Length" headers which seem specific to LSP

[Adam Porter <adam@alphapapa.net>]

Hi João,

Thanks for this reply of yours from last month.  I hadn't had anything
to add until now, after we've had more development in our project.

On 10/19/23 19:02, João Távora wrote:

> After having looked at your code, I think you may be underestimating 
> the use of the Content-Length header.  It's not a spurious 
> decoration. Rather it helps processes make sense of incomplete or 
> multiple messages that may appear in the process buffer.  I'd say 
> it's only in toy examples or "lucky" cases in which you can safely 
> jsonrpc--json-read one single full JSON objects to completeness.

Agreed.  Our intention was to use the hack of trying to parse
potentially incomplete objects for only small responses, and only
because our RPC server doesn't support the Content-Length header.

We may be able to add support for that header to the RPC server, but
we've been discussing another potential solution: to use something like
the JSON Lines[0] (aka NDJSON[1]) format, which would mean that each
JSON object would be sent as a single line (with any internal newlines 
being escaped).  Then, in the process filter, we would look for 
unescaped newlines, and if we find any, we would parse the received data 
up to that newline.  This would allow us to know when a complete object 
has been received.

> It's true it uses some jsonrpc.el internals when it shouldn't but 
> it's quite OK to augment the jsonrpc.el API to export 
> jsonrpc--json-read and the few other internal helpers you're using. 
> That doesn't pose any backward-compatibility challenge and your code 
> is indeed quite simple.

Of course, this means that we would still need to make our own version
of the jsonrpc process filter function, similar to the code seen earlier
in this thread, but it should require only minor changes.

Given your experience with these things, does this seem like a
reasonable solution to you?  Or is there still strong reason to prefer
using a Content-Length header to detect the end of responses?

Thanks for your help!

--Adam

0: https://jsonlines.org/
1: https://ndjson.org/