23.0.60; Coding system troubles since two days

23.0.60; Coding system troubles since two days

[Tassilo Horn]

Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your bug report will be posted to the emacs-pretest-bug@gnu.org mailing list.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug:

Since I updated my emacs CVS checkout on march, 26th in the morning
(EST) I have some very frustrating coding system issues.  They appeared
first when using Gnus (I couldn't read mails of one of my colleagues),
then I got some errors with rcirc (which I cannot reproduce).

First I've thought it was a Gnus related problem, but now I tend more to
say it's something in the coding system handling of emacs.  There are a
few mails/postings of mine with more details, e.g. one of the mails I
cannot open on the ding@gnus.org list / gmane.emacs.gnus.general group.

  http://article.gmane.org/gmane.emacs.gnus.general/66565
  http://article.gmane.org/gmane.emacs.gnus.general/66567
  http://article.gmane.org/gmane.emacs.gnus.general/66568
  http://article.gmane.org/gmane.emacs.gnus.general/66570
  http://article.gmane.org/gmane.emacs.gnus.general/66573

Maybe the following change causes the troubles?

,----
| 2008-03-25  Stefan Monnier  <monnier@iro.umontreal.ca>
| 

[...]

| 	* process.h (struct Lisp_Process): Remove filter_multibyte.
| 	* process.c (QCfilter_multibyte): Remove.
| 	(setup_process_coding_systems): Don't use filter_multibyte.
| 	(Fstart_process, Fmake_network_process): Don't set filter_multibyte.
| 	(read_process_output): Don't adjust multibyteness to filter_multibyte.
| 	(Fset_process_filter_multibyte): Change the coding-system to
| 	approximate the previous behavior.
| 	(Fprocess_filter_multibyte_p): Get the multibyteness straight from the
| 	coding-system.
| 
| 	* coding.c (decode_coding_object): When not decoding into a buffer,
| 	obey the coding system's preference of (uni|multi)byte.
`----

Bye,
Tassilo

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
If you would like to further debug the crash, please read the file
/usr/share/emacs/23.0.60/etc/DEBUG for instructions.


In GNU Emacs 23.0.60.2 (i686-pc-linux-gnu, GTK+ Version 2.12.9)
 of 2008-03-27 on localhost
Windowing system distributor `The X.Org Foundation', version 11.0.10400090
configured using `configure  '--prefix=/usr' '--host=i686-pc-linux-gnu' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--datadir=/usr/share' '--sysconfdir=/etc' '--localstatedir=/var/lib' '--program-suffix=-emacs-23' '--infodir=/usr/share/info/emacs-23' '--without-carbon' '--with-sound' '--with-x' '--with-toolkit-scroll-bars' '--with-gif' '--with-jpeg' '--with-png' '--with-rsvg' '--with-tiff' '--with-xpm' '--enable-font-backend' '--with-freetype' '--with-xft' '--with-libotf' '--with-m17n-flt' '--with-x-toolkit=gtk' '--without-hesiod' '--with-kerberos' '--with-kerberos5' '--with-gpm' '--with-dbus' '--build=i686-pc-linux-gnu' 'build_alias=i686-pc-linux-gnu' 'host_alias=i686-pc-linux-gnu' 'CFLAGS=-march=i386 -O0 -g' 'LDFLAGS=''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.utf8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: Group

Minor modes in effect:
  gnus-topic-mode: t
  gnus-undo-mode: t
  yas/minor-mode: t
  shell-dirtrack-mode: t
  recentf-mode: t
  icomplete-mode: t
  window-number-meta-mode: t
  window-number-mode: t
  savehist-mode: t
  exec-abbrev-cmd-mode: t
  show-paren-mode: t
  delete-selection-mode: t
  which-function-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
. . . <right> <right> <delete> . . . <right> <right> 
<left> <delete> . . . <down> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <right> <right> <right> 
<delete> <delete> <delete> <delete> <delete> <delete> 
. . . . <right> <right> <delete> <right> <right> <right> 
<backspace> . . . <right> <delete> . . . <right> <right> 
<delete> . . . <right> <right> <delete> . . . <right> 
<right> <delete> . . . <right> <delete> . . . <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <up> <up> <up> <up> SPC SPC N o w SPC 
I SPC r e p l a c e d SPC t h e m SPC m a n u a l l 
y SPC t o SPC g e t SPC t h e SPC m a i l SPC s e n 
t . SPC SPC A n y o n e SPC k n o w s SPC w h a t SPC 
c o u l d SPC c a u s e SPC t h o s e SPC s p o n t 
a n e o s C-c t q <backspace> <backspace> <backspace> 
o u s C-c t q <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> t a n C-c t <down> <down> <down> 
<down> <return> C-c t <return> SPC c o d i n g SPC 
s y s t e m SPC t r o u b l e s ? C-c C-c y q <down-mouse-1> 
<mouse-1> <down-mouse-1> <mouse-1> M-x r e b <return> 
<return>

Recent messages:
250 2.1.5 Ok
354 End data with <CR><LF>.<CR><LF>
250 2.0.0 Ok: queued as D18AD439C
221 2.0.0 Bye
20080327T100446.982> Opening nnml server on archive...
20080327T100447.010> Opening nnml server on archive...done
Wrote /home/heimdall/News/archive/sent-messages-2008/138
Sending...done
20080327T100449.912> Exiting summary buffer and applying spam rules
Auto-saving...

Re: 23.0.60; Coding system troubles since two days

[Tassilo Horn]

Tassilo Horn <thorn+news@fastmail.fm> writes:

> Since I updated my emacs CVS checkout on march, 26th in the morning
> (EST) I have some very frustrating coding system issues.  They appeared
> first when using Gnus (I couldn't read mails of one of my colleagues),
> then I got some errors with rcirc (which I cannot reproduce).
>
> First I've thought it was a Gnus related problem, but now I tend more to
> say it's something in the coding system handling of emacs.  There are a
> few mails/postings of mine with more details, e.g. one of the mails I
> cannot open on the ding@gnus.org list / gmane.emacs.gnus.general group.
>
>   http://article.gmane.org/gmane.emacs.gnus.general/66565
>   http://article.gmane.org/gmane.emacs.gnus.general/66567
>   http://article.gmane.org/gmane.emacs.gnus.general/66568
>   http://article.gmane.org/gmane.emacs.gnus.general/66570
>   http://article.gmane.org/gmane.emacs.gnus.general/66573
>
> Maybe the following change causes the troubles?
>
> ,----
> | 2008-03-25  Stefan Monnier  <monnier@iro.umontreal.ca>
> | 
>
> [...]
>
> | 	* process.h (struct Lisp_Process): Remove filter_multibyte.
> | 	* process.c (QCfilter_multibyte): Remove.
> | 	(setup_process_coding_systems): Don't use filter_multibyte.
> | 	(Fstart_process, Fmake_network_process): Don't set filter_multibyte.
> | 	(read_process_output): Don't adjust multibyteness to filter_multibyte.
> | 	(Fset_process_filter_multibyte): Change the coding-system to
> | 	approximate the previous behavior.
> | 	(Fprocess_filter_multibyte_p): Get the multibyteness straight from the
> | 	coding-system.
> | 
> | 	* coding.c (decode_coding_object): When not decoding into a buffer,
> | 	obey the coding system's preference of (uni|multi)byte.
> `----

It seems, I am right.  I did

  % cvs up -D "2008-03-25 06:00"
  % make bootstrap

to get the last working version and the entry on top of src/ChangeLog
now is

,----
| 2008-03-24  Stefan Monnier  <monnier@iro.umontreal.ca>
| 
| 	* casefiddle.c (casify_object): Avoid pathological N^2 worst case if
| 	every char is changed and has a different byte-length.
| 	(Fupcase_word, Fdowncase_word, Fcapitalize_word, operate_on_word):
| 	Fix int -> EMACS_INT.
`----

Now my Gnus works again.  So could you please double-check if that
change was valid?

Bye,
Tassilo

Re: 23.0.60; Coding system troubles since two days

[Tom Rauchenwald]

Tassilo Horn <thorn+news@fastmail.fm> writes:

> Please write in English if possible, because the Emacs maintainers
> usually do not have translators to read other languages for them.
>
> Your bug report will be posted to the emacs-pretest-bug@gnu.org mailing list.
>
> Please describe exactly what actions triggered the bug
> and the precise symptoms of the bug:
>
> Since I updated my emacs CVS checkout on march, 26th in the morning
> (EST) I have some very frustrating coding system issues.  They appeared
> first when using Gnus (I couldn't read mails of one of my colleagues),
> then I got some errors with rcirc (which I cannot reproduce).
>
> First I've thought it was a Gnus related problem, but now I tend more to
> say it's something in the coding system handling of emacs.  There are a
> few mails/postings of mine with more details, e.g. one of the mails I
> cannot open on the ding@gnus.org list / gmane.emacs.gnus.general group.
>
>   http://article.gmane.org/gmane.emacs.gnus.general/66565
>   http://article.gmane.org/gmane.emacs.gnus.general/66567
>   http://article.gmane.org/gmane.emacs.gnus.general/66568
>   http://article.gmane.org/gmane.emacs.gnus.general/66570
>   http://article.gmane.org/gmane.emacs.gnus.general/66573
>
> Maybe the following change causes the troubles?

in the last mail you say you get these:
Debugger entered--Lisp error: (wrong-type-argument characterp 12269800)
with rcirc.
I get similar ones with ERC. So there seems to be a problem somewhere.

Tom

-- 
Then I drew in a breath, and my renewed will with it, lifted the rod
in my right hand, murmured a phrase in a language I didn't know, and
blew the tires off his fucking truck.
        -- Harry Dresden

Re: 23.0.60; Coding system troubles since two days

[Stefan Monnier]

> Since I updated my emacs CVS checkout on march, 26th in the morning
> (EST) I have some very frustrating coding system issues.  They appeared
> first when using Gnus (I couldn't read mails of one of my colleagues),
> then I got some errors with rcirc (which I cannot reproduce).
[...]
> Maybe the following change causes the troubles?

Have you tried to revert the change to see if that's indeed the culprit?
Also, would you be able to make up a reproducible test case?


        Stefan

Re: 23.0.60; Coding system troubles since two days

[Tassilo Horn]

[-- Attachment #1: Type: text/plain, Size: 2315 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Since I updated my emacs CVS checkout on march, 26th in the morning
>> (EST) I have some very frustrating coding system issues.  They
>> appeared first when using Gnus (I couldn't read mails of one of my
>> colleagues), then I got some errors with rcirc (which I cannot
>> reproduce).
> [...]
>> Maybe the following change causes the troubles?
>
> Have you tried to revert the change to see if that's indeed the
> culprit?

Yes, please see my followup to my original bug report on emacs-devel.

http://lists.gnu.org/archive/html/emacs-devel/2008-03/msg02677.html

cvs up -D "2008-03-25 06:00" fixes the problem for me.

> Also, would you be able to make up a reproducible test case?

I don't know how.  I'll attach one of the mails that triggered the
problem.  After reverting my emacs to ""2008-03-25 06:00" I can view
them normally with Gnus, but if I try to save the mail to a file (`O f'
in the *Summary* buffer), I get this:

,----
| These default coding systems were tried to encode text
| in the buffer ` *temp*':
|   (utf-8-unix (403 . 4194294) (749 . 4194300) (788 . 4194276) (840 . 4194276)
|   (853 . 4194276) (892 . 4194271) (1701 . 4194300) (1937 . 4194300) (1938
|   . 4194271))
| However, each of them encountered characters it couldn't encode:
|   utf-8-unix cannot encode these: \366 \374 \344 \344 \344 \337 \374
|                                   \374 \337
| 
| Click on a character (or switch to this window by `C-x o'
| and select the characters by RET) to jump to the place it appears,
| where `C-u C-x =' will give information about it.
| 
| Select one of the safe coding systems listed below,
| or cancel the writing with C-g and edit the buffer
|    to remove or modify the problematic characters,
| or specify any other coding system (and risk losing
|    the problematic characters).
| 
|   utf-8-emacs
`----

I saved it as raw-text and utf-8-emacs, both files are attached.

One interesting thing: I marked the text above with the mouse and
inserted it into another emacs window with a middle mouse click.  There,
instead of \NNN I get: ö ü ä ä ä ß ü ü ß

This other emacs window I pasted into is a current CVS emacs from today
which includes your change, if that matters.


[-- Attachment #2: broken-mail.raw-text --]
[-- Type: application/octet-stream, Size: 2007 bytes --]

X-Gnus-Coding-System: -*- coding: raw-text; -*-

From: Claudia Obermaier <obermaie@uni-koblenz.de>
Subject: Re: Frage zur Dissolution
To: Tassilo Horn <heimdall@uni-koblenz.de>
Date: Wed, 26 Mar 2008 08:13:20 +0100

Hallo,

ich bin auch etwas verwirrt von dem Beispiel. Irgendwas ist da faul.
Der Subgraph relativ zu {C,~A} sollte eigentlich

         ~A
C  v    &
           C

sein. Aber auch wenn der Subgraph so aussieht und nicht wie in deiner 
Lösung, ist das ganz kein c-Block. Denn wie du richtig bemerkt hast, 
macht der c-Pfad {~A,D} Probleme. Der schneidet den Subgraphen zwar, 
aber ist kein Cpfad. Also ist der obige Subgraph noch nicht einmal ein 
C-Block.

Ich vermute, dass sich hier ein kleiner Fehler eingeschlichen hat.
Wenn man das rechte C durch ein anderes Literal ersetzen würde, dann 
klappt das Beispiel. Dann wäre C v ~A der Subgraph relativ zu {C,~A} und 
das wäre dann tatsächlich auch ein "strong c-block".

Gruß
  Claudia

Tassilo Horn schrieb:
> Hi Claudia,
>
> ich hab mal eine Frage zum Dissolutions-Paper.  Auf Seite 508 unten
> geht's um "strong blocks".  Demnach ist ein Subgraph ein c-block, wenn
> alle c-paths, die mindestens einen Knoten von H beinhalten, "durch H
> durchgehen" (pass through H).  Das ist der Fall, wenn die Knoten des
> c-path, welche in H liegen, wiederum einen c-path in H bilden.
>
> So weit, so gut.  Jetzt soll ein starker c-Block ein solcher sein, durch
> den alle c-paths des Graphen laufen.  Als Beispiel wird der Subgraph
> relativ zu {C, ~A} angegeben.
>
> Wenn ich alles korrekt verstanden habe, so ist dieser Graph folgender
> (~ = nicht, & = and):
>
>                   ~A
>                    &
>                    C
>
> Wenn man jetzt aber den c-path {~A, D} betrachtet, so würde ich meinen,
> dass das ein Gegenbeispiel ist.  ~A ist in dem vermeintlichen Block
> enthalten, ist aber nur ein partieller c-path darin, denn er ist im
> c-path {~A, C} komplett enthalten.
>
> Wo liegt mein Denkfehler?
>
> Viele Grüße,
> Tassilo
>   



[-- Attachment #3: broken-mail.utf8-emacs --]
[-- Type: application/octet-stream, Size: 2010 bytes --]

X-Gnus-Coding-System: -*- coding: utf-8-emacs; -*-

From: Claudia Obermaier <obermaie@uni-koblenz.de>
Subject: Re: Frage zur Dissolution
To: Tassilo Horn <heimdall@uni-koblenz.de>
Date: Wed, 26 Mar 2008 08:13:20 +0100

Hallo,

ich bin auch etwas verwirrt von dem Beispiel. Irgendwas ist da faul.
Der Subgraph relativ zu {C,~A} sollte eigentlich

         ~A
C  v    &
           C

sein. Aber auch wenn der Subgraph so aussieht und nicht wie in deiner 
Lösung, ist das ganz kein c-Block. Denn wie du richtig bemerkt hast, 
macht der c-Pfad {~A,D} Probleme. Der schneidet den Subgraphen zwar, 
aber ist kein Cpfad. Also ist der obige Subgraph noch nicht einmal ein 
C-Block.

Ich vermute, dass sich hier ein kleiner Fehler eingeschlichen hat.
Wenn man das rechte C durch ein anderes Literal ersetzen würde, dann 
klappt das Beispiel. Dann wäre C v ~A der Subgraph relativ zu {C,~A} und 
das wäre dann tatsächlich auch ein "strong c-block".

Gruß
  Claudia

Tassilo Horn schrieb:
> Hi Claudia,
>
> ich hab mal eine Frage zum Dissolutions-Paper.  Auf Seite 508 unten
> geht's um "strong blocks".  Demnach ist ein Subgraph ein c-block, wenn
> alle c-paths, die mindestens einen Knoten von H beinhalten, "durch H
> durchgehen" (pass through H).  Das ist der Fall, wenn die Knoten des
> c-path, welche in H liegen, wiederum einen c-path in H bilden.
>
> So weit, so gut.  Jetzt soll ein starker c-Block ein solcher sein, durch
> den alle c-paths des Graphen laufen.  Als Beispiel wird der Subgraph
> relativ zu {C, ~A} angegeben.
>
> Wenn ich alles korrekt verstanden habe, so ist dieser Graph folgender
> (~ = nicht, & = and):
>
>                   ~A
>                    &
>                    C
>
> Wenn man jetzt aber den c-path {~A, D} betrachtet, so würde ich meinen,
> dass das ein Gegenbeispiel ist.  ~A ist in dem vermeintlichen Block
> enthalten, ist aber nur ein partieller c-path darin, denn er ist im
> c-path {~A, C} komplett enthalten.
>
> Wo liegt mein Denkfehler?
>
> Viele Grüße,
> Tassilo
>   



[-- Attachment #4: Type: text/plain, Size: 14 bytes --]


Bye,
Tassilo