From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Newsgroups: gmane.emacs.help Subject: Re: how to calculate the size of string in bytes? Date: Tue, 18 Aug 2015 13:47:03 +0200 Message-ID: <20150818114703.GA9807@tuxteam.de> References: NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; x-action=pgp-signed Content-Transfer-Encoding: 8bit X-Trace: ger.gmane.org 1439898463 998 80.91.229.3 (18 Aug 2015 11:47:43 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 18 Aug 2015 11:47:43 +0000 (UTC) Cc: help-gnu-emacs@gnu.org To: Sam Halliday Original-X-From: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Tue Aug 18 13:47:41 2015 Return-path: Envelope-to: geh-help-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ZRfMd-0006TB-BU for geh-help-gnu-emacs@m.gmane.org; Tue, 18 Aug 2015 13:47:35 +0200 Original-Received: from localhost ([::1]:55954 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZRfMc-0005IU-Kk for geh-help-gnu-emacs@m.gmane.org; Tue, 18 Aug 2015 07:47:34 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:44526) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZRfMD-0005Dh-Dp for help-gnu-emacs@gnu.org; Tue, 18 Aug 2015 07:47:10 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZRfM9-0007NW-5k for help-gnu-emacs@gnu.org; Tue, 18 Aug 2015 07:47:09 -0400 Original-Received: from mail.tuxteam.de ([5.199.139.25]:35150 helo=tomasium.tuxteam.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZRfM8-0007ND-W1 for help-gnu-emacs@gnu.org; Tue, 18 Aug 2015 07:47:05 -0400 Original-Received: from tomas by tomasium.tuxteam.de with local (Exim 4.80) (envelope-from ) id 1ZRfM7-0002cL-Vh; Tue, 18 Aug 2015 13:47:04 +0200 In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 5.199.139.25 X-BeenThere: help-gnu-emacs@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Users list for the GNU Emacs text editor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Original-Sender: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.help:106665 Archived-At: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, Aug 18, 2015 at 03:43:44AM -0700, Sam Halliday wrote: > On Tuesday, 18 August 2015 11:14:04 UTC+1, to...@tuxteam.de wrote: > > On Tue, Aug 18, 2015 at 02:11:54AM -0700, Sam Halliday wrote: > > > We used to have a 6 character hex number at the start of each message that counted the number of multibyte characters, but we'd like to change it to be the number of bytes in the message. > > > > > > We're sending the string to `process-send-string' and `read'ing from the associated network buffer. But when calculating the outgoing length of the string that we want to send, we use `length' --- but we need this to be `length-in-bytes' not the number of multibyte chars. Is there a built in function to do this or am I going to have to iterate the string and count the byte size of each character? > > > > > > A quick test shows that > > > > > > (length (encode-coding-string "EURO" 'raw-text)) > > > > > > seems to give the correct result (1 for ASCII, 2 for Pound Sterling, 3 for Euro), but I am not 100% sure if this is correct. > > > > Raw is, afaik, Emacs's internal coding system. You don't want traces of it > > in the network :-) > > > We're not sending the message using raw, we're using UTF-8. But I need to calculate the length of the UTF-8 string IN BYTES as part of the payload (each messages begins with a 6 character hex encoding of the proceeding string's raw length). Yes, I get that. The way I understand encode-coding-string is that you give it the target encoding: (length (encode-coding-string foo 'raw-text)) would mean "transform this string to whatever Emacs uses as internal encoding and measure its length in bytes", whereas what you want is, AFAIU "transform this string to UTF-8 and measure its length in bytes", which would read as: (length (encode-coding-string foo 'utf-8)) > I'm using "raw" to calculate an approximation of the UTF-8 string's byte length, but I am aware that it might not actually be true in the general case :-/ Use utf-8 then? > I don't think what you've suggested would actually change the semantics, but it would allow us to use a different encoding on the wire than the encoding of the string. We don't really need to worry about that at this stage, because all our users are using UTF-8. We'll keep it in mind though. But, but... isn't that a bug lurking? And it would be so easy to fix... (that is unrelated to the above issue -- that I think you want utf-8 instead of raw) Regards - -- tomás -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlXTGzcACgkQBcgs9XrR2kbq/wCggTBpkebxoL9wIXzoFcSBZDAq RqQAmwTy3yopi8MdM3r1xn9iQDXYRYWa =ISij -----END PGP SIGNATURE-----