all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* regex encoding
@ 2006-08-01 19:09 Chip Coldwell
  2006-08-01 19:46 ` Reiner Steib
  0 siblings, 1 reply; 4+ messages in thread
From: Chip Coldwell @ 2006-08-01 19:09 UTC (permalink / raw)


[-- Attachment #1: Type: TEXT/PLAIN, Size: 790 bytes --]


Looking at lisp/textmodes/ispell.el, there are a number of regular
expressions such as this one taken from the "german8" entry of
ispell-dictionary-alist-4:

[a-zA-Z\304\326\334\344\366\337\374]

The last seven numeric character codes in the above are the iso-8859-1
(ISO Latin-1) encodings for letters used in German that are not
encoded by ASCII.  If your mail reader can handle UTF-8, the
characters represented by these iso-8859-1 codes look like this:

[a-zA-ZÄÖÜäößü]

My question is: are emacs regex character classes limited to the
iso-8859-1 character set, or is there some way to represent Unicode
(such as UTF-8) characters in a character class?

Thanks,

Chip

-- 
Charles M. "Chip" Coldwell
Senior Software Engineer
Red Hat, Inc
978-392-2426

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: regex encoding
  2006-08-01 19:09 regex encoding Chip Coldwell
@ 2006-08-01 19:46 ` Reiner Steib
  2006-08-01 20:38   ` Chip Coldwell
  0 siblings, 1 reply; 4+ messages in thread
From: Reiner Steib @ 2006-08-01 19:46 UTC (permalink / raw)
  Cc: emacs-devel

On Tue, Aug 01 2006, Chip Coldwell wrote:

> If your mail reader can handle UTF-8, the characters represented by
> these iso-8859-1 codes look like this:
>
> [a-zA-ZÄÖÜäößü]

| [a-zA-ZÄÖÜäößü]

If you intend to send UTF-8, you MUA should not declare it as
"Content-Type: TEXT/PLAIN; charset=ISO-8859-1".  ;-)

> My question is: are emacs regex character classes limited to the
> iso-8859-1 character set, or is there some way to represent Unicode
> (such as UTF-8) characters in a character class?

AFAIK, you can write the chars in UTF-8 if you specify the encoding of
the lisp file, cf. (info "(emacs)Specify Coding"):

--8<---------------cut here---------------start------------->8---
;; -*- coding: utf-8 -*-
(defun rs-test ()
  (interactive)
  (re-search-forward "[ÄÖÜäößü]"))
--8<---------------cut here---------------end--------------->8---

I don't know if there's a reason why isn't used in `ispell.el'.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: regex encoding
  2006-08-01 19:46 ` Reiner Steib
@ 2006-08-01 20:38   ` Chip Coldwell
  2006-08-01 20:40     ` Chip Coldwell
  0 siblings, 1 reply; 4+ messages in thread
From: Chip Coldwell @ 2006-08-01 20:38 UTC (permalink / raw)
  Cc: emacs-devel

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2199 bytes --]

On Tue, 1 Aug 2006, Reiner Steib wrote:
>
> | [a-zA-ZÄÖÜäößü]
>
> If you intend to send UTF-8, you MUA should not declare it as
> "Content-Type: TEXT/PLAIN; charset=ISO-8859-1".  ;-)

Ooops.  I haven't tamed my MUA yet.  It's only been ten years.

>> My question is: are emacs regex character classes limited to the
>> iso-8859-1 character set, or is there some way to represent Unicode
>> (such as UTF-8) characters in a character class?
>
> AFAIK, you can write the chars in UTF-8 if you specify the encoding of
> the lisp file, cf. (info "(emacs)Specify Coding"):
>
> --8<---------------cut here---------------start------------->8---
> ;; -*- coding: utf-8 -*-
> (defun rs-test ()
>  (interactive)
>  (re-search-forward "[ÄÖÜäößü]"))
> --8<---------------cut here---------------end--------------->8---
>
> I don't know if there's a reason why isn't used in `ispell.el'.

The particular issue is that ispell is currently broken if your LANG
environment variable specifies UTF-8 encoding, your buffer is UTF-8
encoded and contains one of these non-ASCII characters, and you
specify the "deutsch8" dictionary.  ispell-word generates the error:

"Ispell and its process have different character maps"

What happens is that emacs transcodes the word to iso-8859-1 before
sending it to the aspell process, which most likely is respecting a
the LANG environment variable.  If you change the value of the
ispell-dictionary-alist for "deutch8" from

    ("deutsch8"
     "[a-zA-Z\304\326\334\344\366\337\374]"
     "[^a-zA-Z\304\326\334\344\366\337\374]"
     "[']" t ("-C" "-d" "deutsch") "~latin1" iso-8859-1)

to

    ("deutsch8"
     "[a-zA-Z\304\326\334\344\366\337\374]"
     "[^a-zA-Z\304\326\334\344\366\337\374]"
     "[']" t ("-C" "-d" "deutsch") "~latin1" utf-8)

the regex doesn't match words in the buffer properly.  Changing it to

    ("deutsch8"
     "[a-zA-Z\304\326\334\344\366\337\374]"
     "[^a-zA-Z\304\326\334\344\366\337\374]"
     "[']" t ("-C" "-d" "deutsch" "--encoding=iso8859-1") "~latin1" iso-8859-1)

does seem to work.

Chip

-- 
Charles M. "Chip" Coldwell
Senior Software Engineer
Red Hat, Inc
978-392-2426

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: regex encoding
  2006-08-01 20:38   ` Chip Coldwell
@ 2006-08-01 20:40     ` Chip Coldwell
  0 siblings, 0 replies; 4+ messages in thread
From: Chip Coldwell @ 2006-08-01 20:40 UTC (permalink / raw)
  Cc: emacs-devel

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1531 bytes --]

On Tue, 1 Aug 2006, Chip Coldwell wrote:

> On Tue, 1 Aug 2006, Reiner Steib wrote:
>> 
>> | [a-zA-ZÄÖÜäößü]
>> 
>> If you intend to send UTF-8, you MUA should not declare it as
>> "Content-Type: TEXT/PLAIN; charset=ISO-8859-1".  ;-)
>
> Ooops.  I haven't tamed my MUA yet.  It's only been ten years.
>
>>> My question is: are emacs regex character classes limited to the
>>> iso-8859-1 character set, or is there some way to represent Unicode
>>> (such as UTF-8) characters in a character class?
>> 
>> AFAIK, you can write the chars in UTF-8 if you specify the encoding of
>> the lisp file, cf. (info "(emacs)Specify Coding"):
>> 
>> --8<---------------cut here---------------start------------->8---
>> ;; -*- coding: utf-8 -*-
>> (defun rs-test ()
>>  (interactive)
>>  (re-search-forward "[ÄÖÜäößü]"))
>> --8<---------------cut here---------------end--------------->8---
>> 
>> I don't know if there's a reason why isn't used in `ispell.el'.
>
> The particular issue is that ispell is currently broken if your LANG
> environment variable specifies UTF-8 encoding, your buffer is UTF-8
> encoded and contains one of these non-ASCII characters, and you
> specify the "deutsch8" dictionary.  ispell-word generates the error:
>
> "Ispell and its process have different character maps"

I should also mention that this is with aspell version 0.6; version
0.5 seems to do the right thing regardless.

Chip

-- 
Charles M. "Chip" Coldwell
Senior Software Engineer
Red Hat, Inc
978-392-2426

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2006-08-01 20:40 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-08-01 19:09 regex encoding Chip Coldwell
2006-08-01 19:46 ` Reiner Steib
2006-08-01 20:38   ` Chip Coldwell
2006-08-01 20:40     ` Chip Coldwell

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.