* Better encoding handling for aspell in ispell.el and flyspell.el
@ 2007-11-13 13:04 Agustin Martin
2007-11-18 13:00 ` Richard Stallman
0 siblings, 1 reply; 3+ messages in thread
From: Agustin Martin @ 2007-11-13 13:04 UTC (permalink / raw)
To: emacs-devel
[-- Attachment #1: Type: text/plain, Size: 2165 bytes --]
Hi,
There are some problems with the way emacs {ispell,flyspell}.el currently
handle encodings for aspell.
For ispell.el, when creating the dicts list from the present aspell dicts an
--encoding string is added, but this is not done for the default entries not
having an equivalent in the parsed list. For that reason no encoding is
forced for them and it is selected after current locale, which might not
match the dict encoding. For instance, that is not done for castellano8, the
old way for calling 8 bit spanish, which is not usually shipped as an aspell
alias.
Attached 'ispell.el_aspell-encoding.diff' patch tries to address this
problem by adding the --encoding string when the ispell process is started,
so is done for all dicts.
ispell.el patch [ispell.el_aspell-encoding.diff] proposed changelog entries
------------------- 8< ----------------------------------------------------
(ispell-aspell-find-dictionary): Do not set aspell encoding here.
(ispell-start-process): Explicitly set encoding here if we are using aspell.
------------------- 8< ----------------------------------------------------
The other call that should involve --encoding (currently not set) is in
flyspell.el (flyspell-large-region) function. Here, the proposed patch
also makes sure that communication with the process is done in the dict
encoding (since this only wraps the ispell process I just put a generic
regexp here) and addresses some other issues, like using
ispell-current-{personal-,}dictionary and making sure no "-d" string is
added if one is already present. Things are also rearranged in a way
closer to that of (ispell-start-process).
flyspell.el patch [flyspell.el_aspell-encoding.diff] proposed changelog entries
------------------- 8< ----------------------------------------------------
(flyspell-large-region):
- Explicitly set encoding if we are using aspell, for process and for
communication with the process.
- Do not add "-d" string if one is already present.
- Use ispell-current-dictionary and ispell-current-personal-dictionary.
- Reorganize code.
------------------- 8< ----------------------------------------------------
--
Agustin
[-- Attachment #2: flyspell.el_aspell-encoding.diff --]
[-- Type: text/x-diff, Size: 2382 bytes --]
--- flyspell.el.orig 2007-11-12 14:05:54.000000000 +0100
+++ flyspell.el 2007-11-13 13:51:50.000000000 +0100
@@ -1531,29 +1531,42 @@
(if flyspell-issue-message-flag (message "Checking region..."))
(set-buffer curbuf)
(ispell-check-version)
- (let ((c (apply 'ispell-call-process-region beg
- end
- ispell-program-name
- nil
- buffer
- nil
- (if ispell-really-aspell "list" "-l")
- (let (args)
- ;; Local dictionary becomes the global dictionary in use.
- (if ispell-local-dictionary
- (setq ispell-dictionary ispell-local-dictionary))
- (setq args (ispell-get-ispell-args))
- (if ispell-dictionary ; use specified dictionary
- (setq args
- (append (list "-d" ispell-dictionary) args)))
- (if ispell-personal-dictionary ; use specified pers dict
- (setq args
- (append args
- (list "-p"
- (expand-file-name
- ispell-personal-dictionary)))))
- (setq args (append args ispell-extra-args))
- args))))
+ ;; Local dictionary becomes the global dictionary in use.
+ (setq ispell-current-dictionary
+ (or ispell-local-dictionary ispell-dictionary))
+ (setq ispell-current-personal-dictionary
+ (or ispell-local-pdict ispell-personal-dictionary))
+ (let ((args (ispell-get-ispell-args))
+ (encoding (ispell-get-coding-system))
+ c)
+ (if (and ispell-current-dictionary ; use specified dictionary
+ (not (member "-d" args))) ; only define if not overridden
+ (setq args
+ (append (list "-d" ispell-current-dictionary) args)))
+ (if ispell-current-personal-dictionary ; use specified pers dict
+ (setq args
+ (append args
+ (list "-p"
+ (expand-file-name
+ ispell-current-personal-dictionary)))))
+ (setq args (append args ispell-extra-args))
+ (if (and ispell-really-aspell
+ ispell-aspell-supports-utf8)
+ (setq args
+ (append args
+ (list
+ (concat "--encoding="
+ (symbol-name
+ encoding))))))
+ (let ((process-coding-system-alist (list (cons "\\.*" encoding))))
+ (setq c (apply 'ispell-call-process-region beg
+ end
+ ispell-program-name
+ nil
+ buffer
+ nil
+ (if ispell-really-aspell "list" "-l")
+ args)))
(if (eq c 0)
(progn
(flyspell-process-localwords buffer)
[-- Attachment #3: ispell.el_aspell-encoding.diff --]
[-- Type: text/x-diff, Size: 904 bytes --]
--- ispell.el.orig 2007-11-12 14:06:00.000000000 +0100
+++ ispell.el 2007-11-13 13:50:34.000000000 +0100
@@ -981,7 +981,7 @@
"[^[:alpha:]]"
(regexp-opt otherchars)
t ; We can't tell, so set this to t
- (list "-d" dict-name "--encoding=utf-8")
+ (list "-d" dict-name)
nil ; aspell doesn't support this
;; Here we specify the encoding to use while communicating with
;; aspell. This doesn't apply to command line arguments, so
@@ -2508,6 +2508,13 @@
(append args
(list "-p"
(expand-file-name ispell-current-personal-dictionary)))))
+ (if (and ispell-really-aspell
+ ispell-aspell-supports-utf8)
+ (setq args
+ (append args
+ (list
+ (concat "--encoding="
+ (symbol-name (ispell-get-coding-system)))))))
(setq args (append args ispell-extra-args))
;; Initially we don't know any buffer's local words.
[-- Attachment #4: Type: text/plain, Size: 142 bytes --]
_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Better encoding handling for aspell in ispell.el and flyspell.el
2007-11-13 13:04 Better encoding handling for aspell in ispell.el and flyspell.el Agustin Martin
@ 2007-11-18 13:00 ` Richard Stallman
2007-12-02 18:19 ` Glenn Morris
0 siblings, 1 reply; 3+ messages in thread
From: Richard Stallman @ 2007-11-18 13:00 UTC (permalink / raw)
To: emacs-devel; +Cc: Agustin Martin
Would someone please install this patch, then ack?
I have not carefully studied it, but I think Agustin knows
what he's doing.
Date: Tue, 13 Nov 2007 14:04:01 +0100
From: Agustin Martin <agustin.martin@hispalinux.es>
To: emacs-devel@gnu.org
Mail-Followup-To: emacs-devel@gnu.org
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="5mCyUwZo2JvN/JJP"
Content-Disposition: inline
Subject: Better encoding handling for aspell in ispell.el and flyspell.el
--5mCyUwZo2JvN/JJP
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Hi,
There are some problems with the way emacs {ispell,flyspell}.el currently
handle encodings for aspell.
For ispell.el, when creating the dicts list from the present aspell dicts an
--encoding string is added, but this is not done for the default entries not
having an equivalent in the parsed list. For that reason no encoding is
forced for them and it is selected after current locale, which might not
match the dict encoding. For instance, that is not done for castellano8, the
old way for calling 8 bit spanish, which is not usually shipped as an aspell
alias.
Attached 'ispell.el_aspell-encoding.diff' patch tries to address this
problem by adding the --encoding string when the ispell process is started,
so is done for all dicts.
ispell.el patch [ispell.el_aspell-encoding.diff] proposed changelog entries
------------------- 8< ----------------------------------------------------
(ispell-aspell-find-dictionary): Do not set aspell encoding here.
(ispell-start-process): Explicitly set encoding here if we are using aspell.
------------------- 8< ----------------------------------------------------
The other call that should involve --encoding (currently not set) is in
flyspell.el (flyspell-large-region) function. Here, the proposed patch
also makes sure that communication with the process is done in the dict
encoding (since this only wraps the ispell process I just put a generic
regexp here) and addresses some other issues, like using
ispell-current-{personal-,}dictionary and making sure no "-d" string is
added if one is already present. Things are also rearranged in a way
closer to that of (ispell-start-process).
flyspell.el patch [flyspell.el_aspell-encoding.diff] proposed changelog entries
------------------- 8< ----------------------------------------------------
(flyspell-large-region):
- Explicitly set encoding if we are using aspell, for process and for
communication with the process.
- Do not add "-d" string if one is already present.
- Use ispell-current-dictionary and ispell-current-personal-dictionary.
- Reorganize code.
------------------- 8< ----------------------------------------------------
--
Agustin
--5mCyUwZo2JvN/JJP
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="flyspell.el_aspell-encoding.diff"
--- flyspell.el.orig 2007-11-12 14:05:54.000000000 +0100
+++ flyspell.el 2007-11-13 13:51:50.000000000 +0100
@@ -1531,29 +1531,42 @@
(if flyspell-issue-message-flag (message "Checking region..."))
(set-buffer curbuf)
(ispell-check-version)
- (let ((c (apply 'ispell-call-process-region beg
- end
- ispell-program-name
- nil
- buffer
- nil
- (if ispell-really-aspell "list" "-l")
- (let (args)
- ;; Local dictionary becomes the global dictionary in use.
- (if ispell-local-dictionary
- (setq ispell-dictionary ispell-local-dictionary))
- (setq args (ispell-get-ispell-args))
- (if ispell-dictionary ; use specified dictionary
- (setq args
- (append (list "-d" ispell-dictionary) args)))
- (if ispell-personal-dictionary ; use specified pers dict
- (setq args
- (append args
- (list "-p"
- (expand-file-name
- ispell-personal-dictionary)))))
- (setq args (append args ispell-extra-args))
- args))))
+ ;; Local dictionary becomes the global dictionary in use.
+ (setq ispell-current-dictionary
+ (or ispell-local-dictionary ispell-dictionary))
+ (setq ispell-current-personal-dictionary
+ (or ispell-local-pdict ispell-personal-dictionary))
+ (let ((args (ispell-get-ispell-args))
+ (encoding (ispell-get-coding-system))
+ c)
+ (if (and ispell-current-dictionary ; use specified dictionary
+ (not (member "-d" args))) ; only define if not overridden
+ (setq args
+ (append (list "-d" ispell-current-dictionary) args)))
+ (if ispell-current-personal-dictionary ; use specified pers dict
+ (setq args
+ (append args
+ (list "-p"
+ (expand-file-name
+ ispell-current-personal-dictionary)))))
+ (setq args (append args ispell-extra-args))
+ (if (and ispell-really-aspell
+ ispell-aspell-supports-utf8)
+ (setq args
+ (append args
+ (list
+ (concat "--encoding="
+ (symbol-name
+ encoding))))))
+ (let ((process-coding-system-alist (list (cons "\\.*" encoding))))
+ (setq c (apply 'ispell-call-process-region beg
+ end
+ ispell-program-name
+ nil
+ buffer
+ nil
+ (if ispell-really-aspell "list" "-l")
+ args)))
(if (eq c 0)
(progn
(flyspell-process-localwords buffer)
--5mCyUwZo2JvN/JJP
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="ispell.el_aspell-encoding.diff"
--- ispell.el.orig 2007-11-12 14:06:00.000000000 +0100
+++ ispell.el 2007-11-13 13:50:34.000000000 +0100
@@ -981,7 +981,7 @@
"[^[:alpha:]]"
(regexp-opt otherchars)
t ; We can't tell, so set this to t
- (list "-d" dict-name "--encoding=utf-8")
+ (list "-d" dict-name)
nil ; aspell doesn't support this
;; Here we specify the encoding to use while communicating with
;; aspell. This doesn't apply to command line arguments, so
@@ -2508,6 +2508,13 @@
(append args
(list "-p"
(expand-file-name ispell-current-personal-dictionary)))))
+ (if (and ispell-really-aspell
+ ispell-aspell-supports-utf8)
+ (setq args
+ (append args
+ (list
+ (concat "--encoding="
+ (symbol-name (ispell-get-coding-system)))))))
(setq args (append args ispell-extra-args))
;; Initially we don't know any buffer's local words.
--5mCyUwZo2JvN/JJP
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel
--5mCyUwZo2JvN/JJP--
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Better encoding handling for aspell in ispell.el and flyspell.el
2007-11-18 13:00 ` Richard Stallman
@ 2007-12-02 18:19 ` Glenn Morris
0 siblings, 0 replies; 3+ messages in thread
From: Glenn Morris @ 2007-12-02 18:19 UTC (permalink / raw)
To: rms; +Cc: Agustin Martin, emacs-devel
Richard Stallman wrote:
> Would someone please install this patch, then ack?
ack.
> Date: Tue, 13 Nov 2007 14:04:01 +0100
> From: Agustin Martin <agustin.martin@hispalinux.es>
> To: emacs-devel@gnu.org
> Mail-Followup-To: emacs-devel@gnu.org
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2007-12-02 18:19 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-11-13 13:04 Better encoding handling for aspell in ispell.el and flyspell.el Agustin Martin
2007-11-18 13:00 ` Richard Stallman
2007-12-02 18:19 ` Glenn Morris
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).