[bug#66049] [PATCH 0/4] Add Voikko.

[bug#66049] [PATCH 0/4] Add Voikko.

[Jussi Timperi]

Hi Guix,

This series adds Voikko, a morphological analyzer, spelling and grammar checker,
hyphenator and collection of related linguistic data. It is probably the most
widely used libre spell checker for Finnish language.

'foma' dependency uses the latest git snapshot instead of a stable release to
remove the need for patches. Since the latest release, the project has switched
to cmake based build[1], fixing a number of issues with the old Makefile based
build. Other distributions, like Debian and Fedora, also use snapshots for this
reason.

Kind regards,
Jussi

Footnotes:
[1] https://github.com/mhulden/foma/commit/a76cb52bc83f9d27e44d4f77d10dbca4e0a9a034 

Jussi Timperi (4):
  gnu: Add hfst-ospell.
  gnu: Add foma.
  gnu: Add libvoikko.
  gnu: Add voikko-fi.

 gnu/packages/language.scm | 141 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 141 insertions(+)


base-commit: 9b78699744b0c4c8af28820cc4f0d59b9d317ead
-- 
2.41.0

[bug#66049] [PATCH 1/4] gnu: Add hfst-ospell.

[Jussi Timperi]

* gnu/packages/language.scm (hfst-ospell): New variable.
---
 gnu/packages/language.scm | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/gnu/packages/language.scm b/gnu/packages/language.scm
index faf3114b84..241b145440 100644
--- a/gnu/packages/language.scm
+++ b/gnu/packages/language.scm
@@ -8,6 +8,7 @@
 ;;; Copyright © 2022 Milran <milranmike@protonmail.com>
 ;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;; Copyright © 2023 gemmaro <gemmaro.dev@gmail.com>
+;;; Copyright © 2023 Jussi Timperi <jussi.timperi@iki.fi>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -29,6 +30,7 @@ (define-module (gnu packages language)
   #:use-module (gnu packages anthy)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages audio)
+  #:use-module (gnu packages backup)
   #:use-module (gnu packages base)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages dbm)
@@ -40,6 +42,7 @@ (define-module (gnu packages language)
   #:use-module (gnu packages gnome)
   #:use-module (gnu packages gtk)
   #:use-module (gnu packages ibus)
+  #:use-module (gnu packages icu4c)
   #:use-module (gnu packages java)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages llvm)
@@ -1179,3 +1182,33 @@ (define-public mecab-unidic
 from the database are used as entries (heading terms).")
     ;; triple-licensed (at the user’s choice)
     (license (list license:gpl2+ license:lgpl2.1 license:bsd-3))))
+
+(define-public hfst-ospell
+  (package
+    (name "hfst-ospell")
+    (version "0.5.3")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/hfst/hfst-ospell")
+                    (commit (string-append "v" version))))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "0n8zds3s3gj3fws6ghjr9cpgswasnjqr9rz4syxy2yq8n2fzb8fp"))))
+    (build-system gnu-build-system)
+    (arguments
+     (list
+      ;; Don't use obsolete XML libraries.
+      ;; see: https://github.com/hfst/hfst-ospell/issues/48
+      #:configure-flags #~'("--without-libxmlpp" "--without-tinyxml2"
+                            "--enable-zhfst")))
+    (inputs (list icu4c libarchive))
+    (native-inputs (list autoconf automake libtool pkg-config))
+    (home-page "https://github.com/hfst/hfst-ospell")
+    (synopsis "HFST spell checker library and command line tool")
+    (description
+     "Minimal @acronym{HFST, Helsinki Finite-State Technology}
+optimized-lookup format based spell checker library and a demonstrational
+implementation of command line based spell checker.")
+    (license license:asl2.0)))
-- 
2.41.0

[bug#66049] [PATCH 2/4] gnu: Add foma.

[Jussi Timperi]

* gnu/packages/language.scm (foma): New variable.
---
 gnu/packages/language.scm | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/gnu/packages/language.scm b/gnu/packages/language.scm
index 241b145440..625462fad0 100644
--- a/gnu/packages/language.scm
+++ b/gnu/packages/language.scm
@@ -32,10 +32,12 @@ (define-module (gnu packages language)
   #:use-module (gnu packages audio)
   #:use-module (gnu packages backup)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages bison)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages dbm)
   #:use-module (gnu packages docbook)
   #:use-module (gnu packages emacs)
+  #:use-module (gnu packages flex)
   #:use-module (gnu packages freedesktop)
   #:use-module (gnu packages gettext)
   #:use-module (gnu packages glib)
@@ -55,6 +57,7 @@ (define-module (gnu packages language)
   #:use-module (gnu packages python)
   #:use-module (gnu packages perl-check)
   #:use-module (gnu packages qt)
+  #:use-module (gnu packages readline)
   #:use-module (gnu packages ruby)
   #:use-module (gnu packages scheme)
   #:use-module (gnu packages sqlite)
@@ -1212,3 +1215,39 @@ (define-public hfst-ospell
 optimized-lookup format based spell checker library and a demonstrational
 implementation of command line based spell checker.")
     (license license:asl2.0)))
+
+(define-public foma
+  (let ((commit "fe40aceea1797642dd1cf0fa61fd024c7a7f7095")
+        (revision "0"))
+    (package
+      (name "foma")
+      (version (git-version "0.10.0" revision commit))
+      (source (origin
+                (method git-fetch)
+                (uri (git-reference
+                      (url "https://github.com/mhulden/foma")
+                      (commit commit)))
+                (file-name (git-file-name name version))
+                (sha256
+                 (base32
+                  "1y6yjc72wh6gqqip6jjnrzwv95inr1kncnnfd144vfnvv9bj3msx"))))
+      (build-system cmake-build-system)
+      (arguments
+       (list #:tests? #f ;no tests
+             #:phases #~(modify-phases %standard-phases
+                          (add-after 'unpack 'chdir
+                            (lambda _
+                              (chdir "foma") #t)))))
+      (inputs (list ncurses readline zlib))
+      (native-inputs (list bison flex pkg-config))
+      (home-page "https://fomafst.github.io/")
+      (synopsis "Finite-state compiler and C library")
+      (description
+       "Foma is a compiler, programming language, and C library for
+constructing finite-state automata and transducers for various uses.
+
+It has specific support for many natural language processing applications such
+as producing morphological analyzers.  Although NLP applications are probably
+the main use of foma, it is sufficiently generic to use for a large number of
+purposes.")
+      (license license:asl2.0))))
-- 
2.41.0

[bug#66049] [PATCH 3/4] gnu: Add libvoikko.

[Jussi Timperi]

* gnu/packages/language.scm (libvoikko): New variable.
---
 gnu/packages/language.scm | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/gnu/packages/language.scm b/gnu/packages/language.scm
index 625462fad0..7bfd67cea0 100644
--- a/gnu/packages/language.scm
+++ b/gnu/packages/language.scm
@@ -1251,3 +1251,39 @@ (define-public foma
 the main use of foma, it is sufficiently generic to use for a large number of
 purposes.")
       (license license:asl2.0))))
+
+(define-public libvoikko
+  (package
+    (name "libvoikko")
+    (version "4.3.2")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append
+                    "https://www.puimula.org/voikko-sources/libvoikko/"
+                    "libvoikko-" version ".tar.gz"))
+              (sha256
+               (base32
+                "18hdb7w25kc0ib6z0h0bia192igazknfyz6wmllbim1jmamc4mh1"))))
+    (build-system gnu-build-system)
+    (arguments
+     ;; Outdated tests
+     ;; see: https://github.com/voikko/corevoikko/issues/55
+     (list #:tests? #f))
+    (inputs (list hfst-ospell))
+    (native-inputs (list pkg-config python-minimal))
+    (native-search-paths
+     (list (search-path-specification
+            (variable "VOIKKO_DICTIONARY_PATH")
+            (files (list "lib/voikko")))))
+    (home-page "https://voikko.puimula.org")
+    (synopsis "Library of language tools")
+    (description
+     "Libvoikko provides spell checking, hyphenation, grammar checking and
+morphological analysis for Finnish language.  Spell checking is supported for
+other languages through @acronym{HFST, Helsinki Finite-State Technology}
+backend.
+
+Libvoikko aims to provide support for languages that are not well served by
+Hunspell or other existing free linguistic tools.")
+    ;; Library and default backends are under tri-license.
+    (license (list license:gpl2+ license:lgpl2.1+ license:mpl1.1))))
-- 
2.41.0

[bug#66049] [PATCH 4/4] gnu: Add voikko-fi.

[Jussi Timperi]

* gnu/packages/language.scm (voikko-fi): New variable.
---
 gnu/packages/language.scm | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/gnu/packages/language.scm b/gnu/packages/language.scm
index 7bfd67cea0..85aaa30a8e 100644
--- a/gnu/packages/language.scm
+++ b/gnu/packages/language.scm
@@ -1287,3 +1287,36 @@ (define-public libvoikko
 Hunspell or other existing free linguistic tools.")
     ;; Library and default backends are under tri-license.
     (license (list license:gpl2+ license:lgpl2.1+ license:mpl1.1))))
+
+(define-public voikko-fi
+  (package
+    (name "voikko-fi")
+    (version "2.5")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append
+                    "https://www.puimula.org/voikko-sources/voikko-fi/"
+                    "voikko-fi-" version ".tar.gz"))
+              (sha256
+               (base32
+                "1w90qqp00621hwbgqqxyag60xdjpmz5ba8xzawwif9i5ashb1j9v"))))
+    (build-system gnu-build-system)
+    (arguments
+     (list #:tests? #f ;no tests
+           #:make-flags #~(list (string-append "DESTDIR="
+                                               #$output "/lib/voikko"))
+           #:phases #~(modify-phases %standard-phases
+                        (delete 'configure)
+                        (replace 'install
+                          (lambda* (#:key make-flags #:allow-other-keys)
+                            (apply invoke "make" "vvfst-install" make-flags))))))
+    (native-inputs (list foma libvoikko python-minimal))
+    (home-page "https://voikko.puimula.org")
+    (synopsis "Finnish dictionary for Voikko")
+    (description
+     "Voikko-fi (previously know as Suomi-malaga) is a description of Finnish
+morphology written for libvoikko.
+
+The implementation uses unweighted @acronym{VFST, Varissuo Finite-State
+Transducer} format and provides format 5 Finnish dictionary for libvoikko.")
+    (license license:gpl2+)))
-- 
2.41.0

[bug#66049] [PATCH 0/4] Add Voikko.

[Saku Laesvuori via Guix-patches via]

[-- Attachment #1: Type: text/plain, Size: 375 bytes --]

> Hi Guix,
> 
> This series adds Voikko, a morphological analyzer, spelling and grammar checker,
> hyphenator and collection of related linguistic data. It is probably the most
> widely used libre spell checker for Finnish language.

Great! Seems to work well based on quick testing. Hopefully someone with
commit access finds the time to review and apply this soon.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#66049] [PATCH 0/4] Add Voikko.

[Bruno Victal]

Hi Jussi,

Jussi Timperi <jussi.timperi@iki.fi> writes:

> +       (list #:tests? #f ;no tests

I see a foma/tests directory. Looks like it's not being run
automatically as part of the build system (worth asking upstream to
integrate this into their CMake based system) but instead it's done by
manually invoking the run.sh script. You might have to patch this script
to have `^foma' replaced with (string-append #$output "/bin/foma") though.

> +             #:phases #~(modify-phases %standard-phases
> +                          (add-after 'unpack 'chdir
> +                            (lambda _
> +                              (chdir "foma") #t)))))

No need for a trailing #t.

-- 
Thanks,
Bruno.

[bug#66049] [PATCH v2 1/4] gnu: Add hfst-ospell.

[Jussi Timperi]

* gnu/packages/language.scm (hfst-ospell): New variable.
---
 gnu/packages/language.scm | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/gnu/packages/language.scm b/gnu/packages/language.scm
index faf3114b84..241b145440 100644
--- a/gnu/packages/language.scm
+++ b/gnu/packages/language.scm
@@ -8,6 +8,7 @@
 ;;; Copyright © 2022 Milran <milranmike@protonmail.com>
 ;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;; Copyright © 2023 gemmaro <gemmaro.dev@gmail.com>
+;;; Copyright © 2023 Jussi Timperi <jussi.timperi@iki.fi>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -29,6 +30,7 @@ (define-module (gnu packages language)
   #:use-module (gnu packages anthy)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages audio)
+  #:use-module (gnu packages backup)
   #:use-module (gnu packages base)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages dbm)
@@ -40,6 +42,7 @@ (define-module (gnu packages language)
   #:use-module (gnu packages gnome)
   #:use-module (gnu packages gtk)
   #:use-module (gnu packages ibus)
+  #:use-module (gnu packages icu4c)
   #:use-module (gnu packages java)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages llvm)
@@ -1179,3 +1182,33 @@ (define-public mecab-unidic
 from the database are used as entries (heading terms).")
     ;; triple-licensed (at the user’s choice)
     (license (list license:gpl2+ license:lgpl2.1 license:bsd-3))))
+
+(define-public hfst-ospell
+  (package
+    (name "hfst-ospell")
+    (version "0.5.3")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/hfst/hfst-ospell")
+                    (commit (string-append "v" version))))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "0n8zds3s3gj3fws6ghjr9cpgswasnjqr9rz4syxy2yq8n2fzb8fp"))))
+    (build-system gnu-build-system)
+    (arguments
+     (list
+      ;; Don't use obsolete XML libraries.
+      ;; see: https://github.com/hfst/hfst-ospell/issues/48
+      #:configure-flags #~'("--without-libxmlpp" "--without-tinyxml2"
+                            "--enable-zhfst")))
+    (inputs (list icu4c libarchive))
+    (native-inputs (list autoconf automake libtool pkg-config))
+    (home-page "https://github.com/hfst/hfst-ospell")
+    (synopsis "HFST spell checker library and command line tool")
+    (description
+     "Minimal @acronym{HFST, Helsinki Finite-State Technology}
+optimized-lookup format based spell checker library and a demonstrational
+implementation of command line based spell checker.")
+    (license license:asl2.0)))

base-commit: fafd3caef0d51811a5da81d6061789e2908b0dac
-- 
2.41.0

[bug#66049] [PATCH v2 2/4] gnu: Add foma.

[Jussi Timperi]

* gnu/packages/language.scm (foma): New variable.
---
 gnu/packages/language.scm | 54 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/gnu/packages/language.scm b/gnu/packages/language.scm
index 241b145440..f2dd1aba2f 100644
--- a/gnu/packages/language.scm
+++ b/gnu/packages/language.scm
@@ -32,10 +32,12 @@ (define-module (gnu packages language)
   #:use-module (gnu packages audio)
   #:use-module (gnu packages backup)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages bison)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages dbm)
   #:use-module (gnu packages docbook)
   #:use-module (gnu packages emacs)
+  #:use-module (gnu packages flex)
   #:use-module (gnu packages freedesktop)
   #:use-module (gnu packages gettext)
   #:use-module (gnu packages glib)
@@ -55,6 +57,7 @@ (define-module (gnu packages language)
   #:use-module (gnu packages python)
   #:use-module (gnu packages perl-check)
   #:use-module (gnu packages qt)
+  #:use-module (gnu packages readline)
   #:use-module (gnu packages ruby)
   #:use-module (gnu packages scheme)
   #:use-module (gnu packages sqlite)
@@ -1212,3 +1215,54 @@ (define-public hfst-ospell
 optimized-lookup format based spell checker library and a demonstrational
 implementation of command line based spell checker.")
     (license license:asl2.0)))
+
+(define-public foma
+  (let ((commit "fe40aceea1797642dd1cf0fa61fd024c7a7f7095")
+        (revision "0"))
+    (package
+      (name "foma")
+      (version (git-version "0.10.0" revision commit))
+      (source (origin
+                (method git-fetch)
+                (uri (git-reference
+                      (url "https://github.com/mhulden/foma")
+                      (commit commit)))
+                (file-name (git-file-name name version))
+                (sha256
+                 (base32
+                  "1y6yjc72wh6gqqip6jjnrzwv95inr1kncnnfd144vfnvv9bj3msx"))))
+      (build-system cmake-build-system)
+      (arguments
+       (list
+        #:phases
+        #~(modify-phases %standard-phases
+            (add-after 'unpack 'chdir
+              (lambda _
+                (chdir "foma")))
+            (replace 'check
+              (lambda* (#:key tests? out-of-source? #:allow-other-keys)
+                (if tests?
+                    (let* ((builddir (getcwd))
+                           (srcdir (if out-of-source?
+                                       (string-append builddir "/../foma")
+                                       builddir))
+                           (testdir (string-append srcdir "/tests")))
+                      (with-directory-excursion testdir
+                        (setenv "PATH" (string-append builddir
+                                                      ":"
+                                                      (getenv "PATH")))
+                        (invoke "./run.sh")))
+                    (format #t "test suite not run~%")))))))
+      (inputs (list ncurses readline zlib))
+      (native-inputs (list bison flex pkg-config))
+      (home-page "https://fomafst.github.io/")
+      (synopsis "Finite-state compiler and C library")
+      (description
+       "Foma is a compiler, programming language, and C library for
+constructing finite-state automata and transducers for various uses.
+
+It has specific support for many natural language processing applications such
+as producing morphological analyzers.  Although NLP applications are probably
+the main use of foma, it is sufficiently generic to use for a large number of
+purposes.")
+      (license license:asl2.0))))
-- 
2.41.0

[bug#66049] [PATCH v2 3/4] gnu: Add libvoikko.

[Jussi Timperi]

* gnu/packages/language.scm (libvoikko): New variable.
---
 gnu/packages/language.scm | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/gnu/packages/language.scm b/gnu/packages/language.scm
index f2dd1aba2f..76dfc47cbb 100644
--- a/gnu/packages/language.scm
+++ b/gnu/packages/language.scm
@@ -1266,3 +1266,39 @@ (define-public foma
 the main use of foma, it is sufficiently generic to use for a large number of
 purposes.")
       (license license:asl2.0))))
+
+(define-public libvoikko
+  (package
+    (name "libvoikko")
+    (version "4.3.2")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append
+                    "https://www.puimula.org/voikko-sources/libvoikko/"
+                    "libvoikko-" version ".tar.gz"))
+              (sha256
+               (base32
+                "18hdb7w25kc0ib6z0h0bia192igazknfyz6wmllbim1jmamc4mh1"))))
+    (build-system gnu-build-system)
+    (arguments
+     ;; Outdated tests
+     ;; see: https://github.com/voikko/corevoikko/issues/55
+     (list #:tests? #f))
+    (inputs (list hfst-ospell))
+    (native-inputs (list pkg-config python-minimal))
+    (native-search-paths
+     (list (search-path-specification
+            (variable "VOIKKO_DICTIONARY_PATH")
+            (files (list "lib/voikko")))))
+    (home-page "https://voikko.puimula.org")
+    (synopsis "Library of language tools")
+    (description
+     "Libvoikko provides spell checking, hyphenation, grammar checking and
+morphological analysis for Finnish language.  Spell checking is supported for
+other languages through @acronym{HFST, Helsinki Finite-State Technology}
+backend.
+
+Libvoikko aims to provide support for languages that are not well served by
+Hunspell or other existing free linguistic tools.")
+    ;; Library and default backends are under tri-license.
+    (license (list license:gpl2+ license:lgpl2.1+ license:mpl1.1))))
-- 
2.41.0

[bug#66049] [PATCH v2 4/4] gnu: Add voikko-fi.

[Jussi Timperi]

* gnu/packages/language.scm (voikko-fi): New variable.
---
 gnu/packages/language.scm | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/gnu/packages/language.scm b/gnu/packages/language.scm
index 76dfc47cbb..faf16ad137 100644
--- a/gnu/packages/language.scm
+++ b/gnu/packages/language.scm
@@ -1302,3 +1302,36 @@ (define-public libvoikko
 Hunspell or other existing free linguistic tools.")
     ;; Library and default backends are under tri-license.
     (license (list license:gpl2+ license:lgpl2.1+ license:mpl1.1))))
+
+(define-public voikko-fi
+  (package
+    (name "voikko-fi")
+    (version "2.5")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append
+                    "https://www.puimula.org/voikko-sources/voikko-fi/"
+                    "voikko-fi-" version ".tar.gz"))
+              (sha256
+               (base32
+                "1w90qqp00621hwbgqqxyag60xdjpmz5ba8xzawwif9i5ashb1j9v"))))
+    (build-system gnu-build-system)
+    (arguments
+     (list #:tests? #f ;no tests
+           #:make-flags #~(list (string-append "DESTDIR="
+                                               #$output "/lib/voikko"))
+           #:phases #~(modify-phases %standard-phases
+                        (delete 'configure)
+                        (replace 'install
+                          (lambda* (#:key make-flags #:allow-other-keys)
+                            (apply invoke "make" "vvfst-install" make-flags))))))
+    (native-inputs (list foma libvoikko python-minimal))
+    (home-page "https://voikko.puimula.org")
+    (synopsis "Finnish dictionary for Voikko")
+    (description
+     "Voikko-fi (previously know as Suomi-malaga) is a description of Finnish
+morphology written for libvoikko.
+
+The implementation uses unweighted @acronym{VFST, Varissuo Finite-State
+Transducer} format and provides format 5 Finnish dictionary for libvoikko.")
+    (license license:gpl2+)))
-- 
2.41.0

[bug#66049] [PATCH 0/4] Add Voikko.

[Jussi Timperi]

Hi Bruno,

Bruno Victal <mirai@makinata.eu> writes:

> I see a foma/tests directory. Looks like it's not being run
> automatically as part of the build system (worth asking upstream to
> integrate this into their CMake based system) but instead it's done by
> manually invoking the run.sh script. You might have to patch this script
> to have `^foma' replaced with (string-append #$output "/bin/foma") though.

What I did in V2 was prepending PATH with the directory with the newly
build 'foma' in it. Let me know if that's not OK, and I'll move the
phase around to have #$output populated before it is run.

I'll see about raising an issue upstream about integrating the tests.

> No need for a trailing #t.

Removed in V2.

Thanks,
--
Jussi