announcing thaiword.el?

announcing thaiword.el?

[Werner LEMBERG]

Thanks for adding thaiword.el to the CVS.  Shouldn't it be announced
somehow in the NEWS file?  Additionally, I think it would be a good
idea to automatically use thai-forward-word and thai-backward-word
(which are supersets of forward-word and backward-word, respectively)
if `Thai' is selected as the language environment.


    Werner

Re: announcing thaiword.el?

[Eli Zaretskii]

> Date: Thu, 24 Mar 2005 08:29:23 +0100 (CET)
> From: Werner LEMBERG <wl@gnu.org>
> 
> Thanks for adding thaiword.el to the CVS.

You're welcome.  And thanks for working on it.

> Shouldn't it be announced somehow in the NEWS file?

I'd love to do that, but the file was handed to me without a single
word about its usage, not even where to put it in the Lisp hierarchy.
So I have no idea what to say in NEWS.  But if you (or someone else)
write up a NEWS entry, I will gladly install it.

> Additionally, I think it would be a good idea to automatically use
> thai-forward-word and thai-backward-word (which are supersets of
> forward-word and backward-word, respectively) if `Thai' is selected
> as the language environment.

Again, if you suggest a patch, I will install it.

Re: announcing thaiword.el?

[Werner LEMBERG]

> > Shouldn't it be announced somehow in the NEWS file?
> 
> I'd love to do that, but the file was handed to me without a single
> word about its usage, not even where to put it in the Lisp
> hierarchy.

It's in the right place IMHO.

> So I have no idea what to say in NEWS.  But if you (or someone else)
> write up a NEWS entry, I will gladly install it.

Assuming that selecting `Thai' as the language environment activates
thai-forward-word and thai-backward-word, I suggest something like
this:

  forward-word and backward-ward now also work for Thai text if the
  `Thai' language environment is active.  The functions
  thai-forward-word and thai-backward-word are mapped to forward-word
  and backward-word, respectively.  Another useful function (defined
  in thai-word.el) is thai-break-words which inserts a separator
  string at word boundaries.

> > Additionally, I think it would be a good idea to automatically use
> > thai-forward-word and thai-backward-word (which are supersets of
> > forward-word and backward-word, respectively) if `Thai' is
> > selected as the language environment.
> 
> Again, if you suggest a patch, I will install it.

Uh, oh, I hope Ken'ichi-san can help!  I don't know how to do that.
It seems to me that the `setup-function' property in the
language-info-alist for Thai would be the right place for a hook.


    Werner

Re: announcing thaiword.el?

[Richard Stallman]

      Additionally, I think it would be a good
    idea to automatically use thai-forward-word and thai-backward-word
    (which are supersets of forward-word and backward-word, respectively)
    if `Thai' is selected as the language environment.

That would require changes in a place that is too central to be
changed now.  We can't do it.

Re: announcing thaiword.el?

[Werner LEMBERG]

>       Additionally, I think it would be a good idea to automatically
>     use thai-forward-word and thai-backward-word (which are
>     supersets of forward-word and backward-word, respectively) if
>     `Thai' is selected as the language environment.
> 
> That would require changes in a place that is too central to be
> changed now.  We can't do it.

Which place?  Please explain.  There is *no* functional difference
between thai-forward-ward and forward-word except that the former
works for Thai also (it simply checks whether point is located on a
Thai character, then it starts a lexical word search to find the end
of the current Thai word).

BTW, exactly the same machinery as in thai-word.el could be used for
word search in Lao -- provided someone finds a free Lao word list.


    Werner

Re: announcing thaiword.el?

[Stefan Monnier]

>> Additionally, I think it would be a good idea to automatically
>> use thai-forward-word and thai-backward-word (which are
>> supersets of forward-word and backward-word, respectively) if
>> `Thai' is selected as the language environment.
>> 
>> That would require changes in a place that is too central to be
>> changed now.  We can't do it.

> Which place?  Please explain.

The explanation is that you're not talking about the same thing.
Changing the forward-word function is out of the question (it's a function
used at may places, so changgin its behavior has to be done extremely
carefully).  That's what Richard is talking about.

Whereas you're maybe just talking about M-f and M-b, which can be changed
with something like a thai-word-mode minor mode, to which nobody
would object.

Until someone provides an actual patch, we can't tell whether it's OK.


        Stefan

Re: announcing thaiword.el?

[Werner LEMBERG]

> The explanation is that you're not talking about the same thing.
> Changing the forward-word function is out of the question (it's a
> function used at may places, so changing its behavior has to be done
> extremely carefully).  That's what Richard is talking about.

OK.

> Whereas you're maybe just talking about M-f and M-b, which can be
> changed with something like a thai-word-mode minor mode, to which
> nobody would object.

Hmm, I really think that in a Thai environment the word movement
functions should automatically handle Thai, without special
activation.  In general, a hook into forward-word and friends to
handle scripts which don't use a space for word boundaries is a must
IMHO as soon as we have native Unicode support.

As a compromise I suggest that M-f and M-b are mapped to
thai-forward-word and thai-backward-word, respectively, as soon as we
enter the Thai environment.  This should be a two-liner or so for the
experienced Emacs Lisp hacker (which I'm not, unfortunately).

Any Thai users on the list which actually want to test those
functions?


    Werner

Re: announcing thaiword.el?

[Kenichi Handa]

In article <20050325.232613.73792307.wl@gnu.org>, Werner LEMBERG <wl@gnu.org> writes:

> As a compromise I suggest that M-f and M-b are mapped to
> thai-forward-word and thai-backward-word, respectively, as soon as we
> enter the Thai environment.  This should be a two-liner or so for the
> experienced Emacs Lisp hacker (which I'm not, unfortunately).

We can do that by setup-function and exit-function of a
language environment.

  setup-function     value is a function to call to switch to this
			language environment.
  exit-function      value is a function to call to leave this
		        language environment.

But, I'm not an expert of key-binding.

Could someone tell me what is the right way to rebind all
keys that are currently bound to forward-word to
thai-forward-word, and bind them back to the orignal later?

---
Ken'ichi HANDA
handa@m17n.org

Re: announcing thaiword.el?

[Stefan Monnier]

>> As a compromise I suggest that M-f and M-b are mapped to
>> thai-forward-word and thai-backward-word, respectively, as soon as we
>> enter the Thai environment.  This should be a two-liner or so for the
>> experienced Emacs Lisp hacker (which I'm not, unfortunately).

> We can do that by setup-function and exit-function of a
> language environment.

>   setup-function     value is a function to call to switch to this
> 			language environment.
>   exit-function      value is a function to call to leave this
> 		        language environment.

> But, I'm not an expert of key-binding.

> Could someone tell me what is the right way to rebind all
> keys that are currently bound to forward-word to
> thai-forward-word, and bind them back to the orignal later?

Something like

  (defvar thaiword-mode-map
    (let ((map (make-sparse-keymap)))
      (define-key [remap forward-word] 'thai-forward-word)
      (define-key [remap backward-word] 'thai-backward-word)
      map))
  (define-minor-mode thaiword-mode
    "Minor mode to make word-movement aware of thai words."
    :global t)

Then add (lambda () (thaiword-mode 1)) to setup-function
and (lambda () (thaiword-mode -1)) to exit-function


        Stefan

Re: announcing thaiword.el?

[Richard Stallman]

The "right" way to handle Thai words would not depend on
the language environment.  It would be based on the characters
themselves, and it would be inside forward-word.

We wouldn't want to install such a change now, but people
could work on implementing it for the Unicode-2 branch.

Re: announcing thaiword.el?

[Kenichi Handa]

In article <E1DFOpv-0000BQ-UI@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:

> The "right" way to handle Thai words would not depend on
> the language environment.  It would be based on the characters
> themselves, and it would be inside forward-word.

To handle the regular expression "\\b" and "\\B" correctly
for Thai, we need a bigger change in regex.c.  For the
moment, I have no idea how to do that.

> We wouldn't want to install such a change now, but people
> could work on implementing it for the Unicode-2 branch.

Is it ok to install a change that binds such word-oriented
commands as M-f, M-b, M-t, M-d, M-DEL to the functions
recognizing thai words when a user select Thai lang. env?

---
Ken'ichi HANDA
handa@m17n.org

Re: announcing thaiword.el?

[Richard Stallman]

    Is it ok to install a change that binds such word-oriented
    commands as M-f, M-b, M-t, M-d, M-DEL to the functions
    recognizing thai words when a user select Thai lang. env?

That is ok to install now, because it can't cause problems except for
people who use the Thai language environment.  If those people think
this is an improvement, there's nobody else to complain.

Re: announcing thaiword.el?

[Juri Linkov]

> Is it ok to install a change that binds such word-oriented
> commands as M-f, M-b, M-t, M-d, M-DEL to the functions
> recognizing thai words when a user select Thai lang. env?

BTW, there is also the variable `sentence-end-without-period' for
Thai texts.  Maybe it should be enabled in Thai lang.envs too.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: announcing thaiword.el?

[Kim F. Storm]

Richard Stallman <rms@gnu.org> writes:

>     Is it ok to install a change that binds such word-oriented
>     commands as M-f, M-b, M-t, M-d, M-DEL to the functions
>     recognizing thai words when a user select Thai lang. env?
>
> That is ok to install now, because it can't cause problems except for
> people who use the Thai language environment.  If those people think
> this is an improvement, there's nobody else to complain.

I haven't looked at the specific code, but rather than rebinding the
keys, it seems more appropriate to remap the commands instead!?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: announcing thaiword.el?

[Kenichi Handa]

In article <m1zmwqsaea.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>  Could someone tell me what is the right way to rebind all
>>  keys that are currently bound to forward-word to
>>  thai-forward-word, and bind them back to the orignal later?

> Something like

>   (defvar thaiword-mode-map
>     (let ((map (make-sparse-keymap)))
>       (define-key [remap forward-word] 'thai-forward-word)
>       (define-key [remap backward-word] 'thai-backward-word)
>       map))
>   (define-minor-mode thaiword-mode
>     "Minor mode to make word-movement aware of thai words."
>     :global t)

> Then add (lambda () (thaiword-mode 1)) to setup-function
> and (lambda () (thaiword-mode -1)) to exit-function

Thank you!

Richard Stallman <rms@gnu.org> writes:
>     Is it ok to install a change that binds such word-oriented
>     commands as M-f, M-b, M-t, M-d, M-DEL to the functions
>     recognizing thai words when a user select Thai lang. env?

> That is ok to install now, because it can't cause problems except for
> people who use the Thai language environment.  If those people think
> this is an improvement, there's nobody else to complain.

Ok, I've just installed such changes.

Juri Linkov <juri@jurta.org> writes:
> BTW, there is also the variable `sentence-end-without-period' for
> Thai texts.  Maybe it should be enabled in Thai lang.envs too.

Even in Thai lang. env., people read/write English text, and
in that case sentence-end-without-period should not be.
Perhaps, the better way is to make sentence-end return a
regexp that also matches with "\\ct +" (i.e. Thai character
followed by spaces).  But, I found that TUTORIAL.th ends
sentences with period.  I have to consult with Thai native
what to do.

Anyway, in thai-word-mode, I also enabled line-breaking at
Thai word boundary.

---
Ken'ichi HANDA
handa@m17n.org

Re: announcing thaiword.el?

[Miles Bader]

On Mon, 28 Mar 2005 09:47:09 +0900 (JST), Kenichi Handa <handa@m17n.org> wrote:
> To handle the regular expression "\\b" and "\\B" correctly
> for Thai, we need a bigger change in regex.c.  For the
> moment, I have no idea how to do that.

Current extensions to "word syntax", using `word-separating-categories'
etc., seem to do the correct thing with regexps.[*]  Perhaps some
extension to that mechanism would work.

For instance, what if entries in `word-separating-categories' could have an
optional predicate function -- in addition to the current (CAT1 . CAT2)
format, allow (CAT1 CAT2 PREDICATE-FUN), and only consider the entry to
match if PREDICATE-FUN fun (with some apropriate args) also returns true?

Then for a case like Thai, where you want to do more complicated tests
to establish word-boundaries inside sequences of non-delimited text,
could use a "degenerate" entry in `word-separating-categories' with both
CAT1 and CAT2 the same, but also with a predicate attached to do the
more complicated test.  I suppose that would slow down word matching
when the predicate is called, but it would only happen for text where
that is appropriate.

-Miles

[*] I was surprised that this is true, and I don't understand why from
    my quick look at regex.c :-/ ... But my simple tests seem to show
    that it does really work.  E.g., I can add '(?C . ?C) to
    `word-separating-categories', and then a regexp search will suddenly
    start considering every single kanji character as a standalone word.
-- 
Do not taunt Happy Fun Ball.

Re: announcing thaiword.el?

[Kenichi Handa]

In article <buo1x9yvoho.fsf@mctpc71.ucom.lsi.nec.co.jp>, Miles Bader <miles@lsi.nec.co.jp> writes:

> On Mon, 28 Mar 2005 09:47:09 +0900 (JST), Kenichi Handa <handa@m17n.org> wrote:
>>  To handle the regular expression "\\b" and "\\B" correctly
>>  for Thai, we need a bigger change in regex.c.  For the
>>  moment, I have no idea how to do that.

> Current extensions to "word syntax", using `word-separating-categories'
> etc., seem to do the correct thing with regexps.[*]  Perhaps some
> extension to that mechanism would work.

> For instance, what if entries in `word-separating-categories' could have an
> optional predicate function -- in addition to the current (CAT1 . CAT2)
> format, allow (CAT1 CAT2 PREDICATE-FUN), and only consider the entry to
> match if PREDICATE-FUN fun (with some apropriate args) also returns true?

The problem is that the innermost function
re_match_2_internal doesn't know about the original buffer
or Lisp string.  So, to make PREDICATE-FUN work, we must
generate a Lisp string each time and that will be extemely
slow.  And first of all, is re_match_2_internal a safe place
to call a Lisp function?

> [*] I was surprised that this is true, and I don't understand why from
>     my quick look at regex.c :-/ ... But my simple tests seem to show
>     that it does really work.  E.g., I can add '(?C . ?C) to
>     `word-separating-categories', and then a regexp search will suddenly
>     start considering every single kanji character as a standalone word.

I spent fairy long time to make it work. :-p
re_match_2_internal calls the macro WORD_BOUNDARY_P at
proper places.  It is also used in scan_words (syntax.c).

---
Ken'ichi HANDA
handa@m17n.org

Re: announcing thaiword.el?

[Miles Bader]

On Tue, 29 Mar 2005 18:02:51 +0900 (JST), Kenichi Handa <handa@m17n.org> wrote:
> The problem is that the innermost function
> re_match_2_internal doesn't know about the original buffer
> or Lisp string.  So, to make PREDICATE-FUN work, we must
> generate a Lisp string each time and that will be extemely
> slow.

Hmmm, I see the following in referenced in various places in regex.c,
and it seems to be set by Emacs callers to the regex functions:

   /* In Emacs, this is the string or buffer in which we
      are matching.  It is used for looking up syntax properties.	*/
   Lisp_Object re_match_object;

search.c sets it to nil for the current buffer (and the string in the
case of a string); if word_boundary_p were just passed this object and
the position as extra arguments...?

>  And first of all, is re_match_2_internal a safe place
> to call a Lisp function?

Good question ... :-/

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: announcing thaiword.el?

[Kenichi Handa]

In article <fc339e4a050329021417ba1fe5@mail.gmail.com>, Miles Bader <snogglethorpe@gmail.com> writes:

> Hmmm, I see the following in referenced in various places in regex.c,
> and it seems to be set by Emacs callers to the regex functions:

>    /* In Emacs, this is the string or buffer in which we
>       are matching.  It is used for looking up syntax properties.	*/
>    Lisp_Object re_match_object;

> search.c sets it to nil for the current buffer (and the string in the
> case of a string); if word_boundary_p were just passed this object and
> the position as extra arguments...?

Ah!  Thank you for the info.  I didn't know about that
variable.  Hmmm, it seems that we can use it.

>>   And first of all, is re_match_2_internal a safe place
>>  to call a Lisp function?

> Good question ... :-/

When I find a time, I'll implement and test it in the
Unicode branch.

---
Ken'ichi HANDA
handa@m17n.org

Re: announcing thaiword.el?

[Kenichi Handa]

In article <m3y8c6ykv0.fsf@kfs-l.imdomain.dk>, storm@cua.dk (Kim F. Storm) writes:

> Richard Stallman <rms@gnu.org> writes:
>>      Is it ok to install a change that binds such word-oriented
>>      commands as M-f, M-b, M-t, M-d, M-DEL to the functions
>>      recognizing thai words when a user select Thai lang. env?
>> 
>>  That is ok to install now, because it can't cause problems except for
>>  people who use the Thai language environment.  If those people think
>>  this is an improvement, there's nobody else to complain.

> I haven't looked at the specific code, but rather than rebinding the
> keys, it seems more appropriate to remap the commands instead!?

Yes, Stefan tought me the way of remapping which is
something like this:
  (define-key map [remap forward-word] 'thai-forward-word)
and I installed a code using that method.

---
Ken'ichi HANDA
handa@m17n.org

Re: announcing thaiword.el?

[Werner LEMBERG]

> > So I have no idea what to say in NEWS.  But if you (or someone else)
> > write up a NEWS entry, I will gladly install it.

Here a new try:

  Improved Thai support.  A new minor mode `thai-word-mode' (which is
  automatically activated if you select Thai as a language
  environment) changes key bindings of most word-oriented commands to
  versions which recognize Thai words.  Affected commands are

    M-f     (forward-word)
    M-b     (backward-word)
    M-d     (kill-word)
    M-DEL   (backward-kill-word)
    M-t     (transpose-words)
    M-q     (fill-paragraph)

Re: announcing thaiword.el?

[Kenichi Handa]

In article <20050330.091610.238262582.wl@gnu.org>, Werner LEMBERG <wl@gnu.org> writes:
> Here a new try:

>   Improved Thai support.  A new minor mode `thai-word-mode' (which is
>   automatically activated if you select Thai as a language
>   environment) changes key bindings of most word-oriented commands to
>   versions which recognize Thai words.  Affected commands are

>     M-f     (forward-word)
>     M-b     (backward-word)
>     M-d     (kill-word)
>     M-DEL   (backward-kill-word)
>     M-t     (transpose-words)
>     M-q     (fill-paragraph)

Thank you.  I've just changed my wording to the above in NEWS.

---
Ken'ichi HANDA
handa@m17n.org