* [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files. @ 2023-02-27 20:55 jlicht 2023-02-27 22:43 ` Simon South 0 siblings, 1 reply; 10+ messages in thread From: jlicht @ 2023-02-27 20:55 UTC (permalink / raw) To: 61851; +Cc: Jelle Licht From: Jelle Licht <jlicht@fsfe.org> * gnu/packages/ocr.scm (tesseract-ocr-tessdata-fast)[source]: Add recursive? flag. Adjust hash accordingly. [arguments]<#:phases>: Remove unneeded workaround. --- gnu/packages/ocr.scm | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/gnu/packages/ocr.scm b/gnu/packages/ocr.scm index c1cd4f061e..e07d40bda4 100644 --- a/gnu/packages/ocr.scm +++ b/gnu/packages/ocr.scm @@ -82,18 +82,14 @@ (define-public tesseract-ocr-tessdata-fast (method git-fetch) (uri (git-reference (url "https://github.com/tesseract-ocr/tessdata_fast") + (recursive? #t) ; for tessconfigs (commit version))) (file-name (git-file-name name version)) (sha256 (base32 - "1m310cpb87xx8l8q7jy9fvzf6a0m8rm0dmjpbiwhc2mi6w4gn084")))) + "1hqdsy3zdy5b9l641fvhnawkw6wpb8nkvjql78q8g47js8109mhm")))) (build-system copy-build-system) - (arguments (list #:install-plan #~'(("." "share/tesseract-ocr/tessdata")) - #:phases #~(modify-phases %standard-phases - (add-after 'unpack 'delete-broken-links - (lambda _ - (delete-file "configs") - (delete-file "pdf.ttf")))))) + (arguments (list #:install-plan #~'(("." "share/tesseract-ocr/tessdata")))) (home-page "https://github.com/tesseract-ocr/tessdata_fast") (synopsis "Fast integer versions of trained LSTM models") (description "This repository contains fast integer versions of trained -- 2.39.1 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files. 2023-02-27 20:55 [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files jlicht @ 2023-02-27 22:43 ` Simon South 2023-02-27 22:48 ` [bug#61851] [PATCH] gnu: tesseract-ocr: Use standard TESSDATA_PREFIX Simon South 2023-02-28 0:31 ` [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files Jelle Licht 0 siblings, 2 replies; 10+ messages in thread From: Simon South @ 2023-02-27 22:43 UTC (permalink / raw) To: jlicht; +Cc: 61851 Jelle, Respectfully, and speaking only as an interested observer, I think this may not be the right fix. Guix's Tesseract is indeed missing its config files, causing (among other things) the examples in the online documentation[0] to not work, e.g.: ssouth@hamlet ~/tesseract-ocr-test [env]$ tesseract images/eurotext.png - -l eng hocr read_params_file: Can't open hocr The (quick) [brown] {fox} jumps! Over the $43,456.78 <lazy> #90 dog (...) But the root issue appears to be a misconfiguration of the TESSDATA_PREFIX search path in the tessdata-ocr package, which causes Tesseract's own config files to be installed in a folder other than the one it's configured to search. Fixing this places Tesseract's config files and the trained-data files together beneath /usr/share/tessdata, allowing Tesseract to work as expected: ssouth@hamlet ~/tesseract-ocr-test [env]$ tesseract images/eurotext.png - -l eng hocr <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" (...) This approach has the advantage of keeping the tesseract-ocr-tessdata-fast package "pure" and focused only on trained-data files, which will be important for the patch I'm working on that will split it into multiple packages, one for each language and script, to allow greater flexibility. I'll respond to this email with a draft (!) patch to tesseract-ocr that should achieve the same result as yours, making the config files available for use. Does this also fix the problem for you? If so, would you consider submitting this change instead? -- Simon South simon@simonsouth.net [0] https://tesseract-ocr.github.io/tessdoc/Command-Line-Usage.html ^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#61851] [PATCH] gnu: tesseract-ocr: Use standard TESSDATA_PREFIX. 2023-02-27 22:43 ` Simon South @ 2023-02-27 22:48 ` Simon South 2023-02-28 0:31 ` [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files Jelle Licht 1 sibling, 0 replies; 10+ messages in thread From: Simon South @ 2023-02-27 22:48 UTC (permalink / raw) To: jlicht; +Cc: 61851 --- gnu/packages/ocr.scm | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/gnu/packages/ocr.scm b/gnu/packages/ocr.scm index c1cd4f061e..fc069b83e3 100644 --- a/gnu/packages/ocr.scm +++ b/gnu/packages/ocr.scm @@ -88,7 +88,7 @@ (define-public tesseract-ocr-tessdata-fast (base32 "1m310cpb87xx8l8q7jy9fvzf6a0m8rm0dmjpbiwhc2mi6w4gn084")))) (build-system copy-build-system) - (arguments (list #:install-plan #~'(("." "share/tesseract-ocr/tessdata")) + (arguments (list #:install-plan #~'(("." "share/tessdata")) #:phases #~(modify-phases %standard-phases (add-after 'unpack 'delete-broken-links (lambda _ @@ -131,15 +131,6 @@ (define-public tesseract-ocr (substitute* "configure.ac" (("AC_SUBST\\(\\[XML_CATALOG_FILES])") "")))) - (add-after 'unpack 'adjust-TESSDATA_PREFIX-macro - (lambda _ - ;; Use a deeper TESSDATA_PREFIX hierarchy so that a more - ;; specific search-path than '/share' can be specified. The - ;; build system uses CPPFLAGS for itself, so we can't simply set - ;; a make flag. - (substitute* "Makefile.am" - (("-DTESSDATA_PREFIX='\"@datadir@\"'") - "-DTESSDATA_PREFIX='\"@datadir@/tesseract-ocr\"'")))) (add-after 'build 'build-training (lambda* (#:key parallel-build? #:allow-other-keys) (define n (if parallel-build? (number->string @@ -155,7 +146,7 @@ (define n (if parallel-build? (number->string ;; extended via TESSDATA_PREFIX. (lambda* (#:key native-inputs inputs #:allow-other-keys) (define eng.traineddata - "/share/tesseract-ocr/tessdata/eng.traineddata") + "/share/tessdata/eng.traineddata") (install-file (search-input-file (or native-inputs inputs) eng.traineddata) (dirname (string-append #$output @@ -183,7 +174,7 @@ (define eng.traineddata (list leptonica)) (native-search-paths (list (search-path-specification (variable "TESSDATA_PREFIX") - (files (list "share/tesseract-ocr/tessdata")) + (files (list "share/tessdata")) (separator #f)))) ;single value (home-page "https://github.com/tesseract-ocr/tesseract") (synopsis "Optical character recognition engine") -- 2.39.1 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files. 2023-02-27 22:43 ` Simon South 2023-02-27 22:48 ` [bug#61851] [PATCH] gnu: tesseract-ocr: Use standard TESSDATA_PREFIX Simon South @ 2023-02-28 0:31 ` Jelle Licht 2023-02-28 15:00 ` Simon South 1 sibling, 1 reply; 10+ messages in thread From: Jelle Licht @ 2023-02-28 0:31 UTC (permalink / raw) To: Simon South; +Cc: 61851, Maxim Cournoyer Hi Simon, Simon South <simon@simonsouth.net> writes: > Jelle, > > Respectfully, and speaking only as an interested observer, I think this > may not be the right fix. Cunningham's law strikes again :) [1]. > > Guix's Tesseract is indeed missing its config files, causing (among > other things) the examples in the online documentation[0] to not work, > e.g.: > > ssouth@hamlet ~/tesseract-ocr-test [env]$ tesseract images/eurotext.png - -l eng hocr > read_params_file: Can't open hocr > The (quick) [brown] {fox} jumps! > Over the $43,456.78 <lazy> #90 dog > (...) > > But the root issue appears to be a misconfiguration of the > TESSDATA_PREFIX search path in the tessdata-ocr package, which causes > Tesseract's own config files to be installed in a folder other than the > one it's configured to search. > > Fixing this places Tesseract's config files and the trained-data files > together beneath /usr/share/tessdata, allowing Tesseract to work as > expected: > > ssouth@hamlet ~/tesseract-ocr-test [env]$ tesseract images/eurotext.png - -l eng hocr > <?xml version="1.0" encoding="UTF-8"?> > <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" > (...) I will believe you without any doubt, but there's this spooky comment left in the tesseract-ocr 'adjust-TESSDATA_PREFIX-macro phase: --8<---------------cut here---------------start------------->8--- ;; Use a deeper TESSDATA_PREFIX hierarchy so that a more ;; specific search-path than '/share' can be specified. The ;; build system uses CPPFLAGS for itself, so we can't simply set ;; a make flag. --8<---------------cut here---------------end--------------->8--- This makes me believe the current situation was a deliberate choice, but I personally don't understand what the original problem was/is. > This approach has the advantage of keeping the > tesseract-ocr-tessdata-fast package "pure" and focused only on > trained-data files, which will be important for the patch I'm working on > that will split it into multiple packages, one for each language and > script, to allow greater flexibility. > > I'll respond to this email with a draft (!) patch to tesseract-ocr that > should achieve the same result as yours, making the config files > available for use. Does this also fix the problem for you? If so, > would you consider submitting this change instead? It seems to work for my stuff! I'm bringing Maxim to weigh in on this, as they are the (un?)lucky expert according to my git-foo. Thanks for paying attention! - Jelle [1] https://meta.wikimedia.org/wiki/Cunningham%27s_Law ^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files. 2023-02-28 0:31 ` [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files Jelle Licht @ 2023-02-28 15:00 ` Simon South 2023-02-28 15:35 ` Maxim Cournoyer 0 siblings, 1 reply; 10+ messages in thread From: Simon South @ 2023-02-28 15:00 UTC (permalink / raw) To: Jelle Licht; +Cc: 61851, Maxim Cournoyer Jelle Licht <jlicht@fsfe.org> writes: > Cunningham's law strikes again :) Ha, interesting. That one's new to me. > This makes me believe the current situation was a deliberate choice... Yes, it was, and I realize now I didn't provide much in the way of rationale in my previous email. So here's the background information for anyone interested: Tesseract normally expects to find its data files in /usr/share/tessdata and subfolders thereof. We'd like to use Guix's native-search-paths functionality to pull together data from (for instance) multiple language-specific data packages, and Tesseract conveniently honours a TESSDATA_PREFIX environment variable that specifies its data folder's location, so it seems we are all set. What should TESSDATA_PREFIX be set to? Tesseract's documentation[0] says TESSDATA_PREFIX environment variable should be set to the parent directory of “tessdata” directory. So "share" then, presumably, to have the data files located at "share/tessdata". The man page[1] seems to confirm this: To use a non-standard language pack named foo.traineddata, set the TESSDATA_PREFIX environment variable so the file can be found at TESSDATA_PREFIX/tessdata/foo.traineddata... This creates a problem, though, since defining a native-search-path of just "share" will pull in files from virtually every single Guix package. The solution then is to introduce an intermediate folder, "tesseract-ocr", that sidesteps this problem, and to configure Tesseract appropriately at build time so it installs its data files to "share/tesseract-ocr/tessdata" instead. This is why the existing code was written the way it was and what the comment you pointed out is referring to. However there's a problem with this, too: Patching Makefile.am the way the code does results in only some of Tesseract's data files being placed in "share/tesseract-ocr/tessdata"; you can see in the package output there is still a "share/tessdata" folder that contains Tesseract's config files. Since these aren't also placed beneath "share/tesseract-ocr/tessdata" Tesseract can't find them at runtime. The solution to this seems to be to remove this phase and instead use the "--datadir" configure flag to specify the desired data-folder path. Doing this results in all of Tesseract's data files being installed beneath "share/tesseract-ocr/tessdata" and the resulting package works as you'd expect. However the problem with this is... none of it is necessary in the first place! It turns out Tesseract's documentation is simply WRONG and the program actually expects TESSDATA_PREFIX to contain the complete path to the "tessdata" data folder, not the path of the folder directly above it. So Tesseract can be built as-is, the native-search-path can be safely defined as "share/tessdata", and everything just works. This is what the patch I passed on yesterday does. -- Simon South simon@simonsouth.net [0] https://tesseract-ocr.github.io/tessdoc/Command-Line-Usage.html#simplest-invocation-to-ocr-an-image [1] https://github.com/tesseract-ocr/tesseract/blob/main/doc/tesseract.1.asc ^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files. 2023-02-28 15:00 ` Simon South @ 2023-02-28 15:35 ` Maxim Cournoyer 2023-02-28 16:40 ` Simon South 0 siblings, 1 reply; 10+ messages in thread From: Maxim Cournoyer @ 2023-02-28 15:35 UTC (permalink / raw) To: Simon South; +Cc: Jelle Licht, 61851 Hi Simon, Simon South <simon@simonsouth.net> writes: > Jelle Licht <jlicht@fsfe.org> writes: >> Cunningham's law strikes again :) > > Ha, interesting. That one's new to me. > >> This makes me believe the current situation was a deliberate choice... > > Yes, it was, and I realize now I didn't provide much in the way of > rationale in my previous email. So here's the background information > for anyone interested: > > Tesseract normally expects to find its data files in /usr/share/tessdata > and subfolders thereof. We'd like to use Guix's native-search-paths > functionality to pull together data from (for instance) multiple > language-specific data packages, and Tesseract conveniently honours a > TESSDATA_PREFIX environment variable that specifies its data folder's > location, so it seems we are all set. > > What should TESSDATA_PREFIX be set to? Tesseract's documentation[0] > says > > TESSDATA_PREFIX environment variable should be set to the parent > directory of “tessdata” directory. > > So "share" then, presumably, to have the data files located at > "share/tessdata". The man page[1] seems to confirm this: > > To use a non-standard language pack named foo.traineddata, set the > TESSDATA_PREFIX environment variable so the file can be found at > TESSDATA_PREFIX/tessdata/foo.traineddata... > > This creates a problem, though, since defining a native-search-path of > just "share" will pull in files from virtually every single Guix > package. The solution then is to introduce an intermediate folder, > "tesseract-ocr", that sidesteps this problem, and to configure Tesseract > appropriately at build time so it installs its data files to > "share/tesseract-ocr/tessdata" instead. This is why the existing code > was written the way it was and what the comment you pointed out is > referring to. > > However there's a problem with this, too: Patching Makefile.am the way > the code does results in only some of Tesseract's data files being > placed in "share/tesseract-ocr/tessdata"; you can see in the package > output there is still a "share/tessdata" folder that contains > Tesseract's config files. Since these aren't also placed beneath > "share/tesseract-ocr/tessdata" Tesseract can't find them at runtime. > > The solution to this seems to be to remove this phase and instead use > the "--datadir" configure flag to specify the desired data-folder path. > Doing this results in all of Tesseract's data files being installed > beneath "share/tesseract-ocr/tessdata" and the resulting package works > as you'd expect. > > However the problem with this is... none of it is necessary in the first > place! It turns out Tesseract's documentation is simply WRONG and the > program actually expects TESSDATA_PREFIX to contain the complete path to > the "tessdata" data folder, not the path of the folder directly above > it. So Tesseract can be built as-is, the native-search-path can be > safely defined as "share/tessdata", and everything just works. > > This is what the patch I passed on yesterday does. Thanks for explaining, that makes sense! Would you be so kind as to open an issue with upstream about the misleading doc? That'd complete it and avoid any confusion in the future. -- Thanks, Maxim ^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files. 2023-02-28 15:35 ` Maxim Cournoyer @ 2023-02-28 16:40 ` Simon South 2023-02-28 21:41 ` Maxim Cournoyer 0 siblings, 1 reply; 10+ messages in thread From: Simon South @ 2023-02-28 16:40 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: Jelle Licht, 61851 Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > Would you be so kind as to open an issue with upstream about the > misleading doc? I would've submitted a patch already were the project not using GitHub. I don't have a GitHub account and don't intend to get one. Would anyone else be willing to be open an issue on this? -- Simon South simon@simonsouth.net ^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files. 2023-02-28 16:40 ` Simon South @ 2023-02-28 21:41 ` Maxim Cournoyer 2023-03-16 20:38 ` Jelle Licht 0 siblings, 1 reply; 10+ messages in thread From: Maxim Cournoyer @ 2023-02-28 21:41 UTC (permalink / raw) To: Simon South; +Cc: Jelle Licht, 61851 Hello, Simon South <simon@simonsouth.net> writes: > Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: >> Would you be so kind as to open an issue with upstream about the >> misleading doc? > > I would've submitted a patch already were the project not using GitHub. > I don't have a GitHub account and don't intend to get one. > > Would anyone else be willing to be open an issue on this? No problem; see: https://github.com/tesseract-ocr/tesseract/issues/4025. -- Thanks, Maxim ^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files. 2023-02-28 21:41 ` Maxim Cournoyer @ 2023-03-16 20:38 ` Jelle Licht 2023-03-21 3:13 ` bug#61851: " Maxim Cournoyer 0 siblings, 1 reply; 10+ messages in thread From: Jelle Licht @ 2023-03-16 20:38 UTC (permalink / raw) To: Maxim Cournoyer, Simon South; +Cc: 61851 Hey folks, Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > Hello, > > Simon South <simon@simonsouth.net> writes: > >> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: >>> Would you be so kind as to open an issue with upstream about the >>> misleading doc? >> >> I would've submitted a patch already were the project not using GitHub. >> I don't have a GitHub account and don't intend to get one. >> >> Would anyone else be willing to be open an issue on this? > > No problem; see: https://github.com/tesseract-ocr/tesseract/issues/4025. So it seems the issue was confirmed. In addition there seems to be some inconsistencies between build system with regards to how the data dir is interpreted by tesseract: https://github.com/tesseract-ocr/tesseract/issues/4026 I think it makes sense for us to apply [a version of] Simon's patch. QA also seems to show green lights, ignoring the unrelated recent openmpi-related failures. WDYT? - Jelle ^ permalink raw reply [flat|nested] 10+ messages in thread
* bug#61851: [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files. 2023-03-16 20:38 ` Jelle Licht @ 2023-03-21 3:13 ` Maxim Cournoyer 0 siblings, 0 replies; 10+ messages in thread From: Maxim Cournoyer @ 2023-03-21 3:13 UTC (permalink / raw) To: Jelle Licht; +Cc: 61851-done, Simon South Hello, Jelle Licht <jlicht@fsfe.org> writes: > Hey folks, > > Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > >> Hello, >> >> Simon South <simon@simonsouth.net> writes: >> >>> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: >>>> Would you be so kind as to open an issue with upstream about the >>>> misleading doc? >>> >>> I would've submitted a patch already were the project not using GitHub. >>> I don't have a GitHub account and don't intend to get one. >>> >>> Would anyone else be willing to be open an issue on this? >> >> No problem; see: https://github.com/tesseract-ocr/tesseract/issues/4025. > > So it seems the issue was confirmed. In addition there seems to be some > inconsistencies between build system with regards to how the data dir is > interpreted by tesseract: > > https://github.com/tesseract-ocr/tesseract/issues/4026 > > I think it makes sense for us to apply [a version of] Simon's patch. QA > also seems to show green lights, ignoring the unrelated recent > openmpi-related failures. > > WDYT? I've now applied it, after writing a proper change log commit message, and running the xvnc and lightdm system tests to get some confidence (they make use of tesseract-ocr). Thank you for looking into it! -- Thanks, Maxim ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2023-03-21 3:14 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-02-27 20:55 [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files jlicht 2023-02-27 22:43 ` Simon South 2023-02-27 22:48 ` [bug#61851] [PATCH] gnu: tesseract-ocr: Use standard TESSDATA_PREFIX Simon South 2023-02-28 0:31 ` [bug#61851] [PATCH] gnu: tesseract-ocr-tessdata-fast: Install tesseract config files Jelle Licht 2023-02-28 15:00 ` Simon South 2023-02-28 15:35 ` Maxim Cournoyer 2023-02-28 16:40 ` Simon South 2023-02-28 21:41 ` Maxim Cournoyer 2023-03-16 20:38 ` Jelle Licht 2023-03-21 3:13 ` bug#61851: " Maxim Cournoyer
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).