From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Helmut Eller Newsgroups: gmane.emacs.help Subject: Re: Look for data serialisation format to implement communication between Emacs and external program. Date: Mon, 07 Jan 2013 08:53:34 +0100 Message-ID: References: NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1357545316 3054 80.91.229.3 (7 Jan 2013 07:55:16 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 7 Jan 2013 07:55:16 +0000 (UTC) To: help-gnu-emacs@gnu.org Original-X-From: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Mon Jan 07 08:55:33 2013 Return-path: Envelope-to: geh-help-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Ts7YO-0000UU-87 for geh-help-gnu-emacs@m.gmane.org; Mon, 07 Jan 2013 08:55:28 +0100 Original-Received: from localhost ([::1]:43769 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Ts7Y8-0002Pp-RL for geh-help-gnu-emacs@m.gmane.org; Mon, 07 Jan 2013 02:55:12 -0500 Original-Path: usenet.stanford.edu!news.kjsl.com!us.feeder.erje.net!feeder.erje.net!eu.feeder.erje.net!border3.nntp.ams.giganews.com!Xl.tags.giganews.com!border1.nntp.ams.giganews.com!nntp.giganews.com!local2.nntp.ams.giganews.com!nntp.kpnqwest.it!news.kpnqwest.it.POSTED!not-for-mail Original-NNTP-Posting-Date: Mon, 07 Jan 2013 01:53:35 -0600 Original-Newsgroups: gnu.emacs.help User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) Cancel-Lock: sha1:XQ87EjdRBEjcXE/H3GcF6OeSE60= Original-Lines: 35 X-Usenet-Provider: http://www.giganews.com Original-NNTP-Posting-Host: 212.46.177.238 Original-X-Trace: sv3-lG0TUJFW+AgjUHg9kL9jtFvkh1Rmxhnh6niZnB8finPs9NeRV2vncgajlswl9sHDfdOrL2LtjZnyGP8!6U9V4B/nbW1GSLxzJxO/AHtS5DGCEhEUEwT7D+bthXVaQDsuxTVuUUk01uUfAb4= X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly X-Postfilter: 1.3.40 X-Original-Bytes: 2865 Original-Xref: usenet.stanford.edu gnu.emacs.help:196191 X-BeenThere: help-gnu-emacs@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Users list for the GNU Emacs text editor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Original-Sender: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.help:88494 Archived-At: On Sun, Jan 06 2013, Oleksandr Gavenko wrote: > Is that right to use ASN.1 BER as serialisation data format for communication > between Emacs and external program? S-expressions is the only format that Emacs can write and parse quickly because the printer and reader are implemented in C. This is likely 10 times faster than any parser that you write in Emacs Lisp. The downside is that the external program needs to be able to do the same. Not such a bad tradeoff as S-expressions are fairly easy to parse. For communication with an external format I recommend a "framed" format: a frame is a fixed sized header followed by a variable length payload. The header describes the length of the frame. The length should be in bytes (not characters as counting characters in UTF8 strings is uneccessary complicated). Knowing the length of the frame is very useful because that makes it easy to wait for a complete frame. After you received a complete frame, parsing is simpler because you don't have to worry about incomplete input. I also recommend to limit the frame length to 24 bits (not 32 bit) because Emacs fixnums are limited to 29 bits on 32 bit machines. The payload can then be an S-expression printed with the Emacs prin1 and parsed back with the read function. The encoding of the payload can be utf-8. But use the Emacs 'binary coding system for communication with the external process and unibyte buffers for parsing. For the binary-to-utf8 conversion of the payload use something like decode-coding-string (which is written C and should be fast). If you like, you can also use extra bits in the header to indicate the format of the payload. E.g. it might be useful to have frames that contain only plain strings (not encoded as S-expr). Helmut