Is it possible to transform text before feeding it to ispell?

Is it possible to transform text before feeding it to ispell?

[Marcin Borkowski]

Hi all,

assume that I have an \(i\|hun\)spell dictionary in encoding X, and my
document is in encoding Y.  Does Emacs have any facility for translating
the text before feeding it to ispell, and translating it back when
presenting alternatives?

TIA,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Is it possible to transform text before feeding it to ispell?

[Alexis]

Marcin Borkowski <mbork@mbork.pl> writes:

> assume that I have an \(i\|hun\)spell dictionary in encoding X, 
> and my document is in encoding Y.  Does Emacs have any facility 
> for translating the text before feeding it to ispell, and 
> translating it back when presenting alternatives?

i don't know about ready-made solutions for this, but perhaps 
something could be built around the `google-translate` package 
that's available from MELPA and Marmalade?

http://melpa.org/#/google-translate


Alexis.

Re: Is it possible to transform text before feeding it to ispell?

[Tassilo Horn]

Alexis <flexibeast@gmail.com> writes:

>> assume that I have an \(i\|hun\)spell dictionary in encoding X, and
>> my document is in encoding Y.  Does Emacs have any facility for
>> translating the text before feeding it to ispell, and translating it
>> back when presenting alternatives?
>
> i don't know about ready-made solutions for this, but perhaps
> something could be built around the `google-translate` package that's
> available from MELPA and Marmalade?

I'm pretty sure Marcin means something else when he says translating,
i.e., translating from one encoding to another encoding, not translating
between languages.  For example, one might have the German word
"m{\"a}{\ss}igen" in a ASCII-encoded TeX buffer and what should actually
be passed to ispell/hunspell is "mäßigen" in UTF-8 encoding where the
translation would be `iso-tex2iso' and `iso-iso2tex'.

Bye,
Tassilo

Re: Is it possible to transform text before feeding it to ispell?

[Michael Heerdegen]

Marcin Borkowski <mbork@mbork.pl> writes:

> assume that I have an \(i\|hun\)spell dictionary in encoding X, and my
> document is in encoding Y.  Does Emacs have any facility for translating
> the text before feeding it to ispell, and translating it back when
> presenting alternatives?

AFAIK ispell does that automatically (if it is not configured
incorrectly).  In particular, the encoding of dictionaries is part of
their representation in Emacs - see `ispell-dictionary-base-alist'.

What's your problem in concrete?


Michael.

Re: Is it possible to transform text before feeding it to ispell?

[Marcin Borkowski]

On 2015-09-08, at 15:06, Michael Heerdegen <michael_heerdegen@web.de> wrote:

> Marcin Borkowski <mbork@mbork.pl> writes:
>
>> assume that I have an \(i\|hun\)spell dictionary in encoding X, and my
>> document is in encoding Y.  Does Emacs have any facility for translating
>> the text before feeding it to ispell, and translating it back when
>> presenting alternatives?
>
> AFAIK ispell does that automatically (if it is not configured
> incorrectly).  In particular, the encoding of dictionaries is part of
> their representation in Emacs - see `ispell-dictionary-base-alist'.

I see.  (I very seldom use a spill chucker, maybe I should start?)

> What's your problem in concrete?

People writing "ą" (a letter existing in Polish, but outside ASCII) as
"/a" (i.e., slash and a, sometimes seen in old Polish-language TeX
documents) or ""a" (i.e., quotation mark and a, which is used by LaTeX
package "babel").  Not a typical "encoding", since it uses an "escape
char", but sometimes used (unfortunately).

> Michael.

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Is it possible to transform text before feeding it to ispell?

[Eli Zaretskii]

> From: Marcin Borkowski <mbork@mbork.pl>
> Date: Tue, 08 Sep 2015 10:29:00 +0200
> 
> assume that I have an \(i\|hun\)spell dictionary in encoding X, and my
> document is in encoding Y.  Does Emacs have any facility for translating
> the text before feeding it to ispell, and translating it back when
> presenting alternatives?

Emacs already does all that automatically.  Recall that text in the
buffer you are spell-checking is already decoded into the internal
representation, so the file's encoding is irrelevant.  As for the
dictionary encoding, ispell.el detects that automatically (and if it
fails, you can help it by customizing ispell-dictionary-alist), and
encodes/decodes the words as needed when it talks to the spelling
back-end.

Re: Is it possible to transform text before feeding it to ispell?

[Eli Zaretskii]

> From: Marcin Borkowski <mbork@mbork.pl>
> Date: Tue, 08 Sep 2015 17:24:50 +0200
> 
> > What's your problem in concrete?
> 
> People writing "ą" (a letter existing in Polish, but outside ASCII) as
> "/a" (i.e., slash and a, sometimes seen in old Polish-language TeX
> documents) or ""a" (i.e., quotation mark and a, which is used by LaTeX
> package "babel").  Not a typical "encoding", since it uses an "escape
> char", but sometimes used (unfortunately).

Then you'd need to extend ispell.el to support such "encoding".
Alternatively, define a new coding-system that makes these conversions
before the real encoding, and then use that in ispell-dictionary-alist.

google-translate (was: Re: Is it possible to transform text before feeding it to ispell?)

[Emanuel Berg]

Alexis <flexibeast@gmail.com> writes:

> i don't know about ready-made solutions for this,
> but perhaps something could be built around the
> `google-translate` package that's available from
> MELPA and Marmalade?

I have used that from the shell for some time but for
whatever reason it stopped working recently, so thank
you for this. I had to write this -

(require 'w3m-unisearch)

(defun swedish-to-english (&optional text)
  (interactive)
  (let ((text (get-search-string "ord")))
    (google-translate-translate "Swedish" "English" text 'kill-ring)
    (let ((translation (string-make-unibyte (current-kill 0 t))))
      (message "%s: %s" text translation) )))

(defalias 'sv 'swedish-to-english)

- to get the DWIM region/word-at-point/just-type
interface I also have for w3m Google searches (and
more); for the above to work, get it here:

    http://user.it.uu.se/~embe8573/conf/emacs-init/w3m/w3m-unisearch.el

But it seems I found a bug already. Try this line:

    (google-translate-translate "Swedish" "English" "i")

It gives an error, even tho such short words are
common in many languages and are likely to be
translated now and then.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: google-translate (was: Re: Is it possible to transform text before feeding it to ispell?)

[Andrey Tykhonov]

Hi Emanuel,

On 9 September 2015 at 01:40, Emanuel Berg <embe8573@student.uu.se> wrote:

> Alexis <flexibeast@gmail.com> writes:
>
> > i don't know about ready-made solutions for this,
> > but perhaps something could be built around the
> > `google-translate` package that's available from
> > MELPA and Marmalade?
>
> I have used that from the shell for some time but for
> whatever reason it stopped working recently, so thank
> you for this. I had to write this -
>
> (require 'w3m-unisearch)
>
> (defun swedish-to-english (&optional text)
>   (interactive)
>   (let ((text (get-search-string "ord")))
>     (google-translate-translate "Swedish" "English" text 'kill-ring)
>     (let ((translation (string-make-unibyte (current-kill 0 t))))
>       (message "%s: %s" text translation) )))
>
> (defalias 'sv 'swedish-to-english)
>
> - to get the DWIM region/word-at-point/just-type
> interface I also have for w3m Google searches (and
> more); for the above to work, get it here:
>
>     http://user.it.uu.se/~embe8573/conf/emacs-init/w3m/w3m-unisearch.el
>
> But it seems I found a bug already. Try this line:
>
>     (google-translate-translate "Swedish" "English" "i")
>
> It gives an error, even tho such short words are
> common in many languages and are likely to be
> translated now and then.
>


Thank you for the bug report! I'll fix it as soon as possible!


>
> --
> underground experts united
> http://user.it.uu.se/~embe8573
>
>
>

Re: google-translate (was: Re: Is it possible to transform text before feeding it to ispell?)

[Emanuel Berg]

Andrey Tykhonov <atykhonov@gmail.com> writes:

> Thank you for the bug report! I'll fix it as soon
> as possible!

My pleasure :)

Here is an improvement of the interface I posted the
other day. You might want to use it or do something
similar your way. The idea is to echo the translation,
only with no properties, and not have it stick on the
kill ring.

;; This file: http://user.it.uu.se/~embe8573/conf/emacs-init/translate.el

(require 'w3m-unisearch) ; http://user.it.uu.se/~embe8573/conf/emacs-init/w3m/w3m-unisearch.el

(require 'google-translate-core-ui)

(defun swedish-to-english (&optional text-noninteractive)
  (interactive)
  (let ((killed kill-ring)
        (text   (or text-noninteractive (get-search-string "ord"))) )
    (google-translate-translate "Swedish" "English" text 'kill-ring)
    (let ((translation (string-make-unibyte (current-kill 0))))
      (setq kill-ring-yank-pointer killed)
      (if text-noninteractive translation
        (message "%s: %s" text translation) ))))

(defalias 'sv 'swedish-to-english)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: google-translate

[Emanuel Berg]

Andrey Tykhonov <atykhonov@gmail.com> writes:

> Thank you for the bug report! I'll fix it as soon
> as possible!

Here are two other bugs:

(google-translate-translate "Swedish" "English" "bok")   ; "book" - correct, but:
(google-translate-translate "Swedish" "Danish"  "stuga") ; "cottage" - not Danish!
(google-translate-translate "Swedish" "German"  "stuga") ; (same)
;; etc.

;; also wierd caption: "Translate from nil to nil:" (all cases)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: google-translate

[Alex Kost]

Emanuel Berg (2015-09-16 06:01 +0300) wrote:

> Andrey Tykhonov <atykhonov@gmail.com> writes:
>
>> Thank you for the bug report! I'll fix it as soon
>> as possible!
>
> Here are two other bugs:
>
> (google-translate-translate "Swedish" "English" "bok")   ; "book" - correct, but:
> (google-translate-translate "Swedish" "Danish"  "stuga") ; "cottage" - not Danish!
> (google-translate-translate "Swedish" "German"  "stuga") ; (same)
> ;; etc.
>
> ;; also wierd caption: "Translate from nil to nil:" (all cases)

Because you are doing it wrong.  It should be:

(google-translate-translate "sv" "en" "bok")
(google-translate-translate "sv" "da" "stuga")
(google-translate-translate "sv" "de" "stuga")

-- 
Alex

Re: google-translate

[Emanuel Berg]

Alex Kost <alezost@gmail.com> writes:

> Because you are doing it wrong. It should be:
>
> (google-translate-translate "sv" "en" "bok")
> (google-translate-translate "sv" "da" "stuga")
> (google-translate-translate "sv" "de" "stuga")

The help says

    Translate TEXT from SOURCE-LANGUAGE to TARGET-LANGUAGE.

so it isn't stated what form should be used. "sv",
"en", "da", and "de" are not languages
but designations.

And regardless of whatever, when a language is inputed
that isn't recognized, the program should (fail to)
verify the input and say this doesn't make sense,
instead of displaying br0ken data.

Now it is even more confusing as it seems to work in
the first case, but not in the subsequent ones.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: google-translate

[Emanuel Berg]

Alex Kost <alezost@gmail.com> writes:

> Emanuel Berg (2015-09-16 06:01 +0300) wrote:
>
> (google-translate-translate "sv" "en" "bok")
> (google-translate-translate "sv" "da" "stuga")
> (google-translate-translate "sv" "de" "stuga")

OK, I looked this up! Let's see if I can it right
first try...

Now, if what is supposed to be used as inputs
(arguments) are the language codes that are among
other things in the Unix/C LC*/LANG* environmental
variables, then valid data with respect to language
input is defined in ISO-3166.

The Unix/C variables carry data with the following
syntax:

    xx_YY.CHARSET

where

    xx:       language code (e.g., "en") according to ISO-3166
    YY:       country  code (e.g., "FR") according to ISO-369
    CHARSET:  ASCII, UTF-8, or ISO-8859-1

[1, p. 317]

By the way, the domain of YY is the same as the list
of national, top-level domains ("se" -> ".se" etc.).

This can be included in the help and accompanied by
some examples. If the user sees the examples, even tho
he doesn't recognize their specific meaning
(i.e., what languages), probably he still knows *his*
locale and recognize that pattern and just add it.
Otherwise perhaps he would be benefited from an
ISO-3166 list to find out.

[2, pp. 25-26]

[1]

@book{practical-guide-to-linux,
    author    = {Mark Sobell},
    title     = {A Practical Guide to Linux},
    publisher = {Prentice Hall},
    edition   = {3rd edition},
    year      = 2013,
    ISBN      = {0-13-308504-X}
}

[2]

@book{internet-message,
  title      = {The Internet Message: Closing the Book with Electronic Mail},
  author     = {Marshall Rose},
  publisher  = {Prentice-Hall},
  year       = 1993,
  ISBN       = {0-13-092941-7}
}

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: google-translate

[Emanuel Berg]

Another issue seems to be the display of Swedish chars
in the echo area, tho that isn't a general issue.
I don't know where it sneaked in, but the fixer defun
below appears to have the situation in
a rock-stead grip.

;; This file: http://user.it.uu.se/~embe8573/conf/emacs-init/translate.el

;; w3m-unisearch from
;; http://user.it.uu.se/~embe8573/conf/emacs-init/w3m/w3m-unisearch.el
(require 'w3m-unisearch)

(require 'google-translate-core-ui)

(defun fix-swedish-chars-in-string (str)
  (loop for trans-pair in '(("\345" "å")
                            ("\344" "ä")
                            ("\366" "ö") ) do
          (setq str (replace-regexp-in-string (car  trans-pair)
                                              (cadr trans-pair) str)) )
    str)

(defun translate (src-lang dst-lang text)
  (interactive)
  (let ((killed kill-ring))
    (google-translate-translate src-lang dst-lang text 'kill-ring)
    (let ((translation (string-make-unibyte (current-kill 0))))
      (setq kill-ring-yank-pointer killed)
      (message "%s: %s" text (fix-swedish-chars-in-string translation)) )))

(defun swedish-to-english ()
  (interactive)
  (translate "sv" "en" (get-search-string "ord")) )

(defun english-to-swedish ()
  (interactive)
  (translate "en" "sv" (get-search-string "word(s)")) )

(defalias 'sv 'swedish-to-english)
(defalias 'en 'english-to-swedish)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: google-translate

[Ian Zimmerman]

On 2015-09-19 02:09 +0200, Emanuel Berg wrote:

>   (loop for trans-pair in '(("\345" "å")
>                             ("\344" "ä")
>                             ("\366" "ö") ) do

That looks like an encoding mismatch, not like a bug.  I bet these byte
values are the latin-1 codes for these letters.

> (defun translate (src-lang dst-lang text)
>   (interactive)
>   (let ((killed kill-ring))
>     (google-translate-translate src-lang dst-lang text 'kill-ring)
>     (let ((translation (string-make-unibyte (current-kill 0))))
                          ^^^^^^^^^^^^^^^^^^^
Why do you do that?  That may be your problem right there.

-- 
Please *no* private copies of mailing list or newsgroup messages.
Rule 420: All persons more than eight miles high to leave the court.

Re: google-translate

[Emanuel Berg]

Ian Zimmerman <itz@buug.org> writes:

>>   (loop for trans-pair in '(("\345" "å") ("\344"
>> "ä") ("\366" "ö") ) do
>
> That looks like an encoding mismatch, not like
> a bug. I bet these byte values are the latin-1 codes
> for these letters.

Now we have left the bug discussion and this is an
issue of my interface. But it may still be of interest
to the writer of google-translate because I didn't
find such an interface to begin with (of course,
otherwise I wouldn't have written mine). What I want,
and now have, is really simple:

    M-x sv RET ord RET

echos

    ord: word

and ditto English to Swedish. For additional languages
a less streamlined interface is OK since I do that
much less.

As for the quoted Elisp, without it, the string
"reader" is translated/displayed "l\344sare" and not
"läsare". And the same for the other chars.

>> (defun translate (src-lang dst-lang text)
>>   (interactive)
>>   (let ((killed kill-ring))
>>     (google-translate-translate src-lang dst-lang text 'kill-ring)
>>     (let ((translation (string-make-unibyte (current-kill 0))))
>                          ^^^^^^^^^^^^^^^^^^^
> Why do you do that? That may be your problem
> right there.

BRB.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: google-translate

[Emanuel Berg]

Ian Zimmerman <itz@buug.org> writes:

>> (defun translate (src-lang dst-lang text)
>>   (interactive)
>>   (let ((killed kill-ring))
>>     (google-translate-translate src-lang dst-lang text 'kill-ring)
>>     (let ((translation (string-make-unibyte (current-kill 0))))
>                           ^^^^^^^^^^^^^^^^^^^
> Why do you do that? That may be your problem
> right there.

That was it! The reason I put it there was to get rid
of the properties. I replaced it with
`substring-no-properties' but it seems that isn't
needed either. The reason I thought it was needed is:
put the region around this word: spiderweb - and hit
M-w (`kill-ring-save'). Now evaluate:

    (message "%s" (current-kill 0))

this will show:

    #("spiderweb" 0 9 (fontified t))

now evaluate:

    (message "%s" (substring-no-properties (current-kill 0)))

and get the desired:

    "spiderweb"

however this:

    (progn
      (kill-new "spiderwebs")
      (message "%s" (current-kill 0)) )

also gives no properties! So perhaps it is an
interactive/noninteractive issue as in a string (as
*data*), regardless how it is showed, there aren't
 any properties.

Anyway, thank you. Here is the most recent edition:

    (defun translate (src-lang dst-lang text)
      (interactive)
      (let ((killed kill-ring))
        (google-translate-translate src-lang dst-lang text 'kill-ring)
        (let ((translation (current-kill 0)))
          (setq kill-ring-yank-pointer killed)
          (message "%s: %s" text translation) )))

-- 
underground experts united
http://user.it.uu.se/~embe8573