Extra character 194 appearing in network stream

Extra character 194 appearing in network stream

[Patrik Jonsson]

[-- Attachment #1.1: Type: text/plain, Size: 1113 bytes --]

Hi all,

I'm maintaining Emacs Voice Recognition mode, a module which connects
emacs to Dragon Naturallyspeaking and enables dictation and editing by
voice in emacs buffers. Now I'm trying to figure out why it's broken on
recent versions of emacs.

The emacs code sends buffer info over a network stream to
naturallyspeaking. This is opened with open-network-stream and then
stuff is sent with process-send-string. The problem appears to be that
with irregular intervals, emacs decides to insert a character 194 into
the stream, which wreaks havoc with the comm protocol. It often shows up
in the middle of a send, ie I try to send integer 172 in network format
as chars 0 0 0 172, and what I get at the other end is 0 0 0 194 172.  I
tried setting the coding system explicitly to no-conversion, thinking
that it might be the problem, but it did not help.

I'm sure there are experts here that can tell me why this might be
happening, because at this point I'm stumped. I'm positive this was not
a problem in emacs 20, but it now happens very frequently in 23.0.

Regards,

/Patrik J.



[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Extra character 194 appearing in network stream

[Jason Rumney]

> I'm sure there are experts here that can tell me why this might be
> happening, because at this point I'm stumped. I'm positive this was not
> a problem in emacs 20, but it now happens very frequently in 23.0.
>   

What is more interesting to us right now is whether Emacs 22 works, as 
we are close to releasing that.

Re: Extra character 194 appearing in network stream

[Patrik Jonsson]

[-- Attachment #1.1: Type: text/plain, Size: 1198 bytes --]

Jason Rumney wrote:
>
>> I'm sure there are experts here that can tell me why this might be
>> happening, because at this point I'm stumped. I'm positive this was not
>> a problem in emacs 20, but it now happens very frequently in 23.0.
>>   
>
> What is more interesting to us right now is whether Emacs 22 works, as
> we are close to releasing that.
I tracked the problem down to the piece of code that packed a number
into a network-order 4-byte string:

      (format "%c%c%c%c"
        (lsh (logand i 4278190080) -24)
        (lsh (logand i 16711680) -16)
        (lsh (logand i 65280) -8)
        (logand i 255))

which when (logand i 255) >127 apparently produces a 5-character string.
I tried it on both 22.0.92.1 and 23.0.0.1 (i386-mingw-nt5.1.2600) with
identical results. It seems emacs has abandoned the notion that a
character is a byte, but one would at least think a character <256
should be one byte? Interestingly it never seems to make 2-byte
characters with just a single "%c" in the format, it needs at least one
preceding character to happen.

In any case I solved the problem by replacing this with the bindat
functions.

Regards,

/Patrik



[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Extra character 194 appearing in network stream

[Andreas Schwab]

Patrik Jonsson <patrik@ucolick.org> writes:

> I tracked the problem down to the piece of code that packed a number
> into a network-order 4-byte string:
>
>       (format "%c%c%c%c"
>         (lsh (logand i 4278190080) -24)
>         (lsh (logand i 16711680) -16)
>         (lsh (logand i 65280) -8)
>         (logand i 255))
>
> which when (logand i 255) >127 apparently produces a 5-character string.

I cannot reproduce that.  Can you give a specific example?

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Extra character 194 appearing in network stream

[Patrik Jonsson]

[-- Attachment #1.1: Type: text/plain, Size: 773 bytes --]

Andreas Schwab wrote:
> Patrik Jonsson <patrik@ucolick.org> writes:
>
>   
>> I tracked the problem down to the piece of code that packed a number
>> into a network-order 4-byte string:
>>
>>       (format "%c%c%c%c"
>>         (lsh (logand i 4278190080) -24)
>>         (lsh (logand i 16711680) -16)
>>         (lsh (logand i 65280) -8)
>>         (logand i 255))
>>
>> which when (logand i 255) >127 apparently produces a 5-character string.
>>     
>
> I cannot reproduce that.  Can you give a specific example?
>   
(string-bytes (let ((i 200))
      (format "%c%c%c%c"
        (lsh (logand i 4278190080) -24)
        (lsh (logand i 16711680) -16)
        (lsh (logand i 65280) -8)
        (logand i 255))))
gives
5

Regards,

/Patrik



[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Extra character 194 appearing in network stream

[Andreas Schwab]

Patrik Jonsson <patrik@ucolick.org> writes:

> (string-bytes (let ((i 200))

string-bytes does _not_ give the number of characters in a string.  Since
you have a multibyte string, it may contain more bytes than characters.
You need to apply the appropriate encoding before sending it to the
network stream.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Extra character 194 appearing in network stream

[Patrik Jonsson]

[-- Attachment #1.1: Type: text/plain, Size: 569 bytes --]

Andreas Schwab wrote:
> Patrik Jonsson <patrik@ucolick.org> writes:
>
>   
>> (string-bytes (let ((i 200))
>>     
>
> string-bytes does _not_ give the number of characters in a string.  Since
> you have a multibyte string, it may contain more bytes than characters.
> You need to apply the appropriate encoding before sending it to the
> network stream.
>   

I see. Under what circumstances does it do this? I have

(setq default-enable-multibyte-characters nil)

which I thought would turn multibyte characters off?


Regards,

/Patrik J.



[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Extra character 194 appearing in network stream

[Stefan Monnier]

> I see. Under what circumstances does it do this? I have
> (setq default-enable-multibyte-characters nil)
> which I thought would turn multibyte characters off?

It only changes the default.  But more and more Elisp code *needs* multibyte
character support, so either it forcefully uses enable-multibyte-characters
even if you've turned it off globally, or it works incorrectly.
I *strongly* recommend to remove this setting from your .emacs.


        Stefan