[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Dave Love]

This saves ~1GB.

* gnu/packages/simulation.scm (openfoam)[outputs]: Add debug.
[arguments]: Clean up .o and src after build.
---
 gnu/packages/simulation.scm | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/gnu/packages/simulation.scm b/gnu/packages/simulation.scm
index de07b6844..fef80a1ac 100644
--- a/gnu/packages/simulation.scm
+++ b/gnu/packages/simulation.scm
@@ -84,6 +84,10 @@
      `(("gzip" ,gzip)
        ("gnuplot" ,gnuplot)
        ("openmpi" ,openmpi)))
+    ;; FIXME: Also separate tutorials (80MB) and src (60MB); maybe also doc
+    ;; (8MB)
+    (outputs '("debug"                  ;~60MB
+               "out"))
     (arguments
      `( ;; Executable files and shared libraries are located in the 'platforms'
        ;; subdirectory.
@@ -171,6 +175,15 @@
                         (("lockDir=.*$")
                          "lockDir=$HOME/.$WM_PROJECT/.wmake\n"))
                       #t))
+                  (add-after 'build 'cleanup
+                    ;; Avoid lots of junk installed
+                    (lambda _
+                      (delete-file-recursively
+                       "platforms/linux64GccDPInt32Opt/src")
+                      (delete-file-recursively
+                       "platforms/linux64GccDPInt32OptSYSTEMOPENMPI/src")
+                      (zero?
+                       (system* "find" "-name" "*.o" "-delete"))))
                   (replace 'install
                     (lambda _
                       ;; use 'OpenFOAM-version' convention
-- 
2.11.0

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Ludovic Courtès]

Hi Dave,

Dave Love <fx@gnu.org> skribis:

> This saves ~1GB.
>
> * gnu/packages/simulation.scm (openfoam)[outputs]: Add debug.
> [arguments]: Clean up .o and src after build.

[...]

>  gnu/packages/simulation.scm | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/gnu/packages/simulation.scm b/gnu/packages/simulation.scm
> index de07b6844..fef80a1ac 100644
> --- a/gnu/packages/simulation.scm
> +++ b/gnu/packages/simulation.scm
> @@ -84,6 +84,10 @@
>       `(("gzip" ,gzip)
>         ("gnuplot" ,gnuplot)
>         ("openmpi" ,openmpi)))
> +    ;; FIXME: Also separate tutorials (80MB) and src (60MB); maybe also doc
> +    ;; (8MB)
> +    (outputs '("debug"                  ;~60MB
> +               "out"))

Normally the ‘strip’ phase would strip things.  I guess the problem here
is that libraries are not in lib/, so nothing gets stripped.  This would
be worked around by simply passing something like:

  #:strip-directories '("OpenFOAM-1.2.3/lib")

> +                  (add-after 'build 'cleanup
> +                    ;; Avoid lots of junk installed
> +                    (lambda _
> +                      (delete-file-recursively
> +                       "platforms/linux64GccDPInt32Opt/src")
> +                      (delete-file-recursively
> +                       "platforms/linux64GccDPInt32OptSYSTEMOPENMPI/src")
> +                      (zero?
> +                       (system* "find" "-name" "*.o" "-delete"))))

Rather:

  (for-each delete-file (find-files "." "\\.o$"))

Paul can you confirm that this is OK?

If it is, could you update the patch accordingly, Dave?

Thanks,
Ludo’.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Paul Garlick]

Hi Dave and Ludo,

Thank you Dave for your helpful suggestions on the OpenFOAM package
definition.

Firstly, on the question of adding a debug output, I have checked the
effect of the current '#:strip-directories' keyword definition.  In the
build log:

stripping binaries in "/gnu/store/4zqn4w0wlq0irdwh3dhrdbsr7i3f1dag-
openfoam-4.1/lib/OpenFOAM-4.1/platforms/linux64GccDPInt32Opt/bin" with
"strip" and flags ("--strip-debug" "--enable-deterministic-archives")
stripping binaries in "/gnu/store/4zqn4w0wlq0irdwh3dhrdbsr7i3f1dag-
openfoam-4.1/lib/OpenFOAM-4.1/platforms/linux64GccDPInt32Opt/lib" with
"strip" and flags ("--strip-debug" "--enable-deterministic-archives")

This suggests that the binaries in .../bin and .../lib are being
stripped.  However, if I check a randomly-selected executable in the
bin directory:

$ objdump --syms /home/paul/.guix-profile/lib/OpenFOAM-
4.1/platforms/linux64GccDPInt32Opt/bin/blockMesh | grep debug

0000000000000000       O *UND*	0000000000000000              _ZN
4Foam8fileName5debugE
0000000000000000       O *UND*	0000000000000000              _ZN
4Foam4word5debugE

The 'file' command also reports that the executables and shared objects
are 'not stripped'.  Does adding a debug output achieve the effect of
stripping the binaries? 

> 
> Normally the ‘strip’ phase would strip things.  I guess the problem
> here
> is that libraries are not in lib/, so nothing gets stripped.  This
> would
> be worked around by simply passing something like:
> 
>   #:strip-directories '("OpenFOAM-1.2.3/lib")

Would that not give a 'directory not found' message?  Currently,

#:strip-directories (list (string-append
                                  "lib/OpenFOAM-" ,version
                                  "/platforms/linux64GccDPInt32Opt/bin"
)
                                 (string-append
                                  "lib/OpenFOAM-" ,version
                                  "/platforms/linux64GccDPInt32Opt/lib"
))

> > 
> > +                  (add-after 'build 'cleanup
> > +                    ;; Avoid lots of junk installed
> > +                    (lambda _
> > +                      (delete-file-recursively
> > +                       "platforms/linux64GccDPInt32Opt/src")
> > +                      (delete-file-recursively
> > +                       "platforms/linux64GccDPInt32OptSYSTEMOPENMP
> > I/src")
> > +                      (zero?
> > +                       (system* "find" "-name" "*.o" "-delete"))))
> Rather:
> 
>   (for-each delete-file (find-files "." "\\.o$"))
> 
> Paul can you confirm that this is OK?
> 

Maybe.  We need to be careful that we not delete files which are needed
later on.  Typically, a user will copy part of the directory structure
to a subdirectory of $HOME and compile a new solver.  The OpenFOAM
'wmake' utility takes care of the dependencies and re-compiles object
files as needed.  

So, object files under 'platforms/linux64GccDPInt32Opt/src' should be
safe to delete.  However, this needs to be checked to make sure no
dependencies are deleted that cannot easily be re-compiled.  Have you
already checked this Dave by, for example, re-compiling a standard
solver?

Paul.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Ludovic Courtès]

Hi Paul,

Paul Garlick <pgarlick@tourbillion-technology.com> skribis:

> The 'file' command also reports that the executables and shared objects
> are 'not stripped'.

That’s because we use ‘--strip-debug’ and not ‘--strip-all’ (in some
cases, the latter breaks binaries in weird ways, hence the conservative
choice.)

However, you can see in the list of ELF sections reported by “objdump
-x” that there’s no .debug* section—i.e., binaries contain the symbol
table, but not DWARF debugging info, which is far larger.

>  Does adding a debug output achieve the effect of stripping the
> binaries? 

I don’t think it makes any difference.

>> Normally the ‘strip’ phase would strip things.  I guess the problem
>> here
>> is that libraries are not in lib/, so nothing gets stripped.  This
>> would
>> be worked around by simply passing something like:
>> 
>>   #:strip-directories '("OpenFOAM-1.2.3/lib")
>
> Would that not give a 'directory not found' message?  Currently,
>
> #:strip-directories (list (string-append
>                                   "lib/OpenFOAM-" ,version
>                                   "/platforms/linux64GccDPInt32Opt/bin"
> )
>                                  (string-append
>                                   "lib/OpenFOAM-" ,version
>                                   "/platforms/linux64GccDPInt32Opt/lib"
> ))

Oh sorry, I had forgotten we already had this.  This is perfect!

>> Rather:
>> 
>>   (for-each delete-file (find-files "." "\\.o$"))
>> 
>> Paul can you confirm that this is OK?
>> 
>
> Maybe.  We need to be careful that we not delete files which are needed
> later on.  Typically, a user will copy part of the directory structure
> to a subdirectory of $HOME and compile a new solver.  The OpenFOAM
> 'wmake' utility takes care of the dependencies and re-compiles object
> files as needed.  
>
> So, object files under 'platforms/linux64GccDPInt32Opt/src' should be
> safe to delete.  However, this needs to be checked to make sure no
> dependencies are deleted that cannot easily be re-compiled.  Have you
> already checked this Dave by, for example, re-compiling a standard
> solver?

I let Dave answer on this part.

Thanks,
Ludo’.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Dave Love]

Ludovic Courtès <ludo@gnu.org> writes:

> That’s because we use ‘--strip-debug’ and not ‘--strip-all’ (in some
> cases, the latter breaks binaries in weird ways, hence the conservative
> choice.)

Is that something Guix-specific?  As far as I know, with rpm and dpkg,
the binaries are always stripped, and I'm not aware of any problem with
that.

>>  Does adding a debug output achieve the effect of stripping the
>> binaries? 
>
> I don’t think it makes any difference.

Right.

Incidentally, you could save about half the size of the boost
contribution to closures like this by separating the headers and the
libraries, but there was some problem I don;t remember when I tried.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Ludovic Courtès]

Dave Love <fx@gnu.org> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> That’s because we use ‘--strip-debug’ and not ‘--strip-all’ (in some
>> cases, the latter breaks binaries in weird ways, hence the conservative
>> choice.)
>
> Is that something Guix-specific?  As far as I know, with rpm and dpkg,
> the binaries are always stripped, and I'm not aware of any problem with
> that.

I’m pretty sure I tried to default to “--strip-all” instead of
“--strip-debug” and that some packages had problems with that, I forgot
what it was.

Perhaps we should try to revisit this.

>>>  Does adding a debug output achieve the effect of stripping the
>>> binaries? 
>>
>> I don’t think it makes any difference.
>
> Right.
>
> Incidentally, you could save about half the size of the boost
> contribution to closures like this by separating the headers and the
> libraries, but there was some problem I don;t remember when I tried.

Too bad you don’t remember, it would be worth looking into that.

Ludo’.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Dave Love]

Ludovic Courtès <ludo@gnu.org> writes:

> Dave Love <fx@gnu.org> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> That’s because we use ‘--strip-debug’ and not ‘--strip-all’ (in some
>>> cases, the latter breaks binaries in weird ways, hence the conservative
>>> choice.)
>>
>> Is that something Guix-specific?  As far as I know, with rpm and dpkg,
>> the binaries are always stripped, and I'm not aware of any problem with
>> that.
>
> I’m pretty sure I tried to default to “--strip-all” instead of
> “--strip-debug” and that some packages had problems with that, I forgot
> what it was.
>
> Perhaps we should try to revisit this.

It seems worth trying.

By the way, I originally thought that debug info was left in the
binaries and a debug package separated it.  I think it's unfortunate not
to have debug info available (the GNU build default).  It presumably
should be available for something like openfoam, for people to build
parts of, anyhow.

>> Incidentally, you could save about half the size of the boost
>> contribution to closures like this by separating the headers and the
>> libraries, but there was some problem I don;t remember when I tried.
>
> Too bad you don’t remember, it would be worth looking into that.
>
> Ludo’.

It's easy (but not fast) to try again.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Ludovic Courtès]

Dave Love <fx@gnu.org> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Dave Love <fx@gnu.org> skribis:
>>
>>> Ludovic Courtès <ludo@gnu.org> writes:
>>>
>>>> That’s because we use ‘--strip-debug’ and not ‘--strip-all’ (in some
>>>> cases, the latter breaks binaries in weird ways, hence the conservative
>>>> choice.)
>>>
>>> Is that something Guix-specific?  As far as I know, with rpm and dpkg,
>>> the binaries are always stripped, and I'm not aware of any problem with
>>> that.
>>
>> I’m pretty sure I tried to default to “--strip-all” instead of
>> “--strip-debug” and that some packages had problems with that, I forgot
>> what it was.
>>
>> Perhaps we should try to revisit this.
>
> It seems worth trying.

I take it that you’re volunteering?  :-)

The change is a simple one-liner, but making sure that nothing breaks of
course takes more time (mostly CPU time!).

> By the way, I originally thought that debug info was left in the
> binaries and a debug package separated it.  I think it's unfortunate not
> to have debug info available (the GNU build default).  It presumably
> should be available for something like openfoam, for people to build
> parts of, anyhow.

Quoth
<https://www.gnu.org/software/guix/manual/html_node/Installing-Debugging-Files.html>:

     The ‘debug’ output mechanism in Guix is implemented by the
  ‘gnu-build-system’ (*note Build Systems::).  Currently, it is
  opt-in—debugging information is available only for the packages with
  definitions explicitly declaring a ‘debug’ output.  This may be changed
  to opt-out in the future if our build farm servers can handle the load.
  To check whether a package has a ‘debug’ output, use ‘guix package
  --list-available’ (*note Invoking guix package::).

It’s mostly a matter of disk space and bandwidth.

Ludo’.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Ludovic Courtès]

Hi,

ludo@gnu.org (Ludovic Courtès) skribis:

> Paul Garlick <pgarlick@tourbillion-technology.com> skribis:

[...]

>>> Rather:
>>> 
>>>   (for-each delete-file (find-files "." "\\.o$"))
>>> 
>>> Paul can you confirm that this is OK?
>>> 
>>
>> Maybe.  We need to be careful that we not delete files which are needed
>> later on.  Typically, a user will copy part of the directory structure
>> to a subdirectory of $HOME and compile a new solver.  The OpenFOAM
>> 'wmake' utility takes care of the dependencies and re-compiles object
>> files as needed.  
>>
>> So, object files under 'platforms/linux64GccDPInt32Opt/src' should be
>> safe to delete.  However, this needs to be checked to make sure no
>> dependencies are deleted that cannot easily be re-compiled.  Have you
>> already checked this Dave by, for example, re-compiling a standard
>> solver?
>
> I let Dave answer on this part.

I think we got sidetracked by the discussion about debugging info.

Paul, Dave: what do we do this patch?

  https://bugs.gnu.org/28593

TIA,
Ludo’.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Paul Garlick]

Hi Ludo and Dave,

> Paul, Dave: what do we do this patch?
> 
>   https://bugs.gnu.org/28593
> 

To recap on my comments on the patch:

i) a debug output would be fine; possibly with an extra comment for
OpenFOAM users to say what it can be used for

ii) deleting object files, to save disk space, under
'platforms/linux64GccDPInt32Opt/src' should also be fine, with the
proviso that the re-compilation process should be checked before
committing.  There is an example procedure, albeit for an earlier
version, given at https://openfoamwiki.net/index.php/How_to_add_tempera
ture_to_icoFoam .  A slightly modified procedure should work for
version 4.1.

Dave, can you try this on your patched version?  'wmake' should re-
generate the object files that are needed and produce a new executable.
 Check that there are no instances of errors such as 'file not found'

iii) substitute a more descriptive word than 'junk' for the object
files; 'intermediate' or some such

iv) the source files take up much less space than the object files.  I
wouldn't worry about deleting them.

Best regards,

Paul.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Dave Love]

Paul Garlick <pgarlick@tourbillion-technology.com> writes:

> ii) deleting object files, to save disk space, under
> 'platforms/linux64GccDPInt32Opt/src' should also be fine, with the
> proviso that the re-compilation process should be checked before
> committing.  There is an example procedure, albeit for an earlier
> version, given at https://openfoamwiki.net/index.php/How_to_add_tempera
> ture_to_icoFoam .  A slightly modified procedure should work for
> version 4.1.
>
> Dave, can you try this on your patched version?  'wmake' should re-
> generate the object files that are needed and produce a new executable.
>  Check that there are no instances of errors such as 'file not found'

Sorry, I haven't had a chance to check that yet, and I'm hampered
working on openfoam by a small-ish laptop with a slow disk.  I didn't
expect the dependency files to be necessary, as they aren't shipped in
my rpms for previous versions that people have been using, but I will do
the experiment when I can.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Ludovic Courtès]

Hello,

Dave Love <fx@gnu.org> skribis:

> Paul Garlick <pgarlick@tourbillion-technology.com> writes:
>
>> ii) deleting object files, to save disk space, under
>> 'platforms/linux64GccDPInt32Opt/src' should also be fine, with the
>> proviso that the re-compilation process should be checked before
>> committing.  There is an example procedure, albeit for an earlier
>> version, given at https://openfoamwiki.net/index.php/How_to_add_tempera
>> ture_to_icoFoam .  A slightly modified procedure should work for
>> version 4.1.
>>
>> Dave, can you try this on your patched version?  'wmake' should re-
>> generate the object files that are needed and produce a new executable.
>>  Check that there are no instances of errors such as 'file not found'
>
> Sorry, I haven't had a chance to check that yet, and I'm hampered
> working on openfoam by a small-ish laptop with a slow disk.  I didn't
> expect the dependency files to be necessary, as they aren't shipped in
> my rpms for previous versions that people have been using, but I will do
> the experiment when I can.

Paul seems to have access to a biffy build machine ;-), so maybe you can
update/adjust the patch, Paul?

Ludo’.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Paul Garlick]

Hi Dave and Ludo,

> > Sorry, I haven't had a chance to check that yet, and I'm hampered
> > working on openfoam by a small-ish laptop with a slow disk.  I
> > didn't
> > expect the dependency files to be necessary, as they aren't shipped
> > in
> > my rpms for previous versions that people have been using, but I
> > will do
> > the experiment when I can.

Great, thanks.  

One of the nice features of OpenFOAM is that the principles of
operation on laptops is just the same as that on supercomputers.

> Paul seems to have access to a biffy build machine ;-), so maybe you
> can
> update/adjust the patch, Paul?

Well, based on feedback on the existing package definition, I am
starting to think about how best to deal with multiple versions and
simplify the patch step.  There is a new version (5.0) that has been
released that I can work on.  

It is likely to take some time, although it should be quicker to
develop than the original definition.  So, if Dave can take charge of
this finishing patch (#28593) that would work best for me.

In the meantime, I have made an announcement of the package
availability on the CFD Online discusssion forum.  The link is:

https://www.cfd-online.com/Forums/openfoam-news-announcements-other/193
865-openfoam-4-1-packaged-gnu-guix.html

Best regards,

Paul.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Ludovic Courtès]

Hi Paul,

Paul Garlick <pgarlick@tourbillion-technology.com> skribis:

[...]

>> Paul seems to have access to a biffy build machine ;-), so maybe you
>> can
>> update/adjust the patch, Paul?
>
> Well, based on feedback on the existing package definition, I am
> starting to think about how best to deal with multiple versions and
> simplify the patch step.  There is a new version (5.0) that has been
> released that I can work on.  
>
> It is likely to take some time, although it should be quicker to
> develop than the original definition.  So, if Dave can take charge of
> this finishing patch (#28593) that would work best for me.

Sounds like a plan.

> In the meantime, I have made an announcement of the package
> availability on the CFD Online discusssion forum.  The link is:
>
> https://www.cfd-online.com/Forums/openfoam-news-announcements-other/193865-openfoam-4-1-packaged-gnu-guix.html

Nice!

It will be interesting to see to what extent it helps OpenFOAM users and
developers address their deployment issues.

Cheers,
Ludo’.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Dave Love]

Ludovic Courtès <ludo@gnu.org> writes:

>> https://www.cfd-online.com/Forums/openfoam-news-announcements-other/193865-openfoam-4-1-packaged-gnu-guix.html
>
> Nice!
>
> It will be interesting to see to what extent it helps OpenFOAM users and
> developers address their deployment issues.

In my experience of offering rpms and build advice, in the UK they're
remarkably resistant to making life easy, or at least their support
people are.  Actually, that's not unique to OF; sigh.

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Dave Love]

Paul Garlick <pgarlick@tourbillion-technology.com> writes:

> One of the nice features of OpenFOAM is that the principles of
> operation on laptops is just the same as that on supercomputers.

(Modulo MPI) but it's bad enough working with it on a decent login node!

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Dave Love]

[-- Attachment #1: Type: text/plain, Size: 898 bytes --]

Paul Garlick <pgarlick@tourbillion-technology.com> writes:

> ii) deleting object files, to save disk space, under
> 'platforms/linux64GccDPInt32Opt/src' should also be fine, with the
> proviso that the re-compilation process should be checked before
> committing.  There is an example procedure, albeit for an earlier
> version, given at https://openfoamwiki.net/index.php/How_to_add_tempera
> ture_to_icoFoam .  A slightly modified procedure should work for
> version 4.1.
>
> Dave, can you try this on your patched version?  'wmake' should re-
> generate the object files that are needed and produce a new executable.
>  Check that there are no instances of errors such as 'file not found'

I successfully rebuilt pisoFoam following the example in the current
documentation.  Is that good enough?

Here's a modified patch to change the comment and avoid an empty
directory.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-openfoam-Clean-up-to-reduce-closure.patch --]
[-- Type: text/x-diff, Size: 1747 bytes --]

From dc88db3e91c70da5e6e557ed5fdd528499cb1c65 Mon Sep 17 00:00:00 2001
From: Dave Love <fx@gnu.org>
Date: Sat, 21 Oct 2017 17:20:42 +0100
Subject: [PATCH] gnu: openfoam: Clean up to reduce closure.

This saves ~1GB.

* gnu/packages/simulation.scm (openfoam)[outputs]: Add debug.
[arguments]: Clean up .o and src after build.
---
 gnu/packages/simulation.scm | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/gnu/packages/simulation.scm b/gnu/packages/simulation.scm
index de07b6844..3e65d1687 100644
--- a/gnu/packages/simulation.scm
+++ b/gnu/packages/simulation.scm
@@ -84,6 +84,10 @@
      `(("gzip" ,gzip)
        ("gnuplot" ,gnuplot)
        ("openmpi" ,openmpi)))
+    ;; FIXME: Also separate tutorials (80MB) and src (60MB); maybe also doc
+    ;; (8MB)
+    (outputs '("debug"                  ;~60MB
+               "out"))
     (arguments
      `( ;; Executable files and shared libraries are located in the 'platforms'
        ;; subdirectory.
@@ -171,6 +175,15 @@
                         (("lockDir=.*$")
                          "lockDir=$HOME/.$WM_PROJECT/.wmake\n"))
                       #t))
+                  (add-after 'build 'cleanup
+                    ;; Avoid unncessary, voluminous object and dep files
+                    (lambda _
+                      (delete-file-recursively
+                       "platforms/linux64GccDPInt32Opt/src")
+                      (delete-file-recursively
+                       "platforms/linux64GccDPInt32OptSYSTEMOPENMPI")
+                      (zero?
+                       (system* "find" "-name" "*.o" "-delete"))))
                   (replace 'install
                     (lambda _
                       ;; use 'OpenFOAM-version' convention
-- 
2.11.0

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Paul Garlick]

On Sun, 2017-10-22 at 17:15 +0100, Dave Love wrote:
> 
> I successfully rebuilt pisoFoam following the example in the current
> documentation.  Is that good enough?


Yes, that is a good example case.  Thank you for doing the check.


> Here's a modified patch to change the comment and avoid an empty
> directory.

Fine, with a caveat on the FIXME comment:

i) 'wmake' will fail to find headers in subdirectories of 'src' if they
are not installed.  So, a re-compilation as you did above would not be
possible without installing the extra output.  This is perhaps
confusing for a new user, who may not know the details of the
dependency structure.

ii) OpenFOAM would be difficult to use without the 'tutorials'
directory, as these are often used as the starting point for new
development.  I think it is helpful for these files to be installed by
default.

My own preference would be to not fix the FIXME (or, in fact, to omit
the comment at this stage).

Best,

Paul.

bug#28593: [PATCH] gnu: openfoam: Clean up to reduce closure.

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 496 bytes --]

Dave Love <fx@gnu.org> skribis:

>>From dc88db3e91c70da5e6e557ed5fdd528499cb1c65 Mon Sep 17 00:00:00 2001
> From: Dave Love <fx@gnu.org>
> Date: Sat, 21 Oct 2017 17:20:42 +0100
> Subject: [PATCH] gnu: openfoam: Clean up to reduce closure.
>
> This saves ~1GB.
>
> * gnu/packages/simulation.scm (openfoam)[outputs]: Add debug.
> [arguments]: Clean up .o and src after build.

I removed the FIXME as suggested by Paul, made the changes below, and
committed.

Thank you,
Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1349 bytes --]

diff --git a/gnu/packages/simulation.scm b/gnu/packages/simulation.scm
index 3e65d1687..a5b661e34 100644
--- a/gnu/packages/simulation.scm
+++ b/gnu/packages/simulation.scm
@@ -84,8 +84,6 @@
      `(("gzip" ,gzip)
        ("gnuplot" ,gnuplot)
        ("openmpi" ,openmpi)))
-    ;; FIXME: Also separate tutorials (80MB) and src (60MB); maybe also doc
-    ;; (8MB)
     (outputs '("debug"                  ;~60MB
                "out"))
     (arguments
@@ -176,14 +174,14 @@
                          "lockDir=$HOME/.$WM_PROJECT/.wmake\n"))
                       #t))
                   (add-after 'build 'cleanup
-                    ;; Avoid unncessary, voluminous object and dep files
+                    ;; Avoid unncessary, voluminous object and dep files.
                     (lambda _
                       (delete-file-recursively
                        "platforms/linux64GccDPInt32Opt/src")
                       (delete-file-recursively
                        "platforms/linux64GccDPInt32OptSYSTEMOPENMPI")
-                      (zero?
-                       (system* "find" "-name" "*.o" "-delete"))))
+                      (for-each delete-file (find-files "." "\\.o$"))
+                      #t))
                   (replace 'install
                     (lambda _
                       ;; use 'OpenFOAM-version' convention

[bug#28593] [PATCH] gnu: openfoam: Clean up to reduce closure.

[Dave Love]

Paul Garlick <pgarlick@tourbillion-technology.com> writes:

> This suggests that the binaries in .../bin and .../lib are being
> stripped.  However, if I check a randomly-selected executable in the
> bin directory:
>
> $ objdump --syms /home/paul/.guix-profile/lib/OpenFOAM-
> 4.1/platforms/linux64GccDPInt32Opt/bin/blockMesh | grep debug
>
> 0000000000000000       O *UND*	0000000000000000              _ZN
> 4Foam8fileName5debugE
> 0000000000000000       O *UND*	0000000000000000              _ZN
> 4Foam4word5debugE
>
> The 'file' command also reports that the executables and shared objects
> are 'not stripped'.  Does adding a debug output achieve the effect of
> stripping the binaries? 

That was confusing me too, and I was going to ask about it...

>> > +                      (zero?
>> > +                       (system* "find" "-name" "*.o" "-delete"))))
>> Rather:
>> 
>>   (for-each delete-file (find-files "." "\\.o$"))

I don't understand the need to avoid system(*), but I assumed the
optimizations in find make it significantly faster then find-files -- is
that not true?  (Dealing with the file structure in openfoam is horribly
slow when I've done builds on the NFS filesystems.)

> Maybe.  We need to be careful that we not delete files which are needed
> later on.  Typically, a user will copy part of the directory structure
> to a subdirectory of $HOME and compile a new solver.  The OpenFOAM
> 'wmake' utility takes care of the dependencies and re-compiles object
> files as needed.  
>
> So, object files under 'platforms/linux64GccDPInt32Opt/src' should be
> safe to delete.  However, this needs to be checked to make sure no
> dependencies are deleted that cannot easily be re-compiled.  Have you
> already checked this Dave by, for example, re-compiling a standard
> solver?
>
> Paul.

Not in this case, but people have been using my rpm version in anger,
though it's an earlier version.  (I was looking at this partly to help
updating the rpm.)  I'd have expected to have to regenerate the
dependency files when modifying the source.

[I wish openfoam had a sane build system, even one that stopped on an
error, sigh.  Actually, it might be worth patching that, like the rpm --
I've several times missed components not being built -- assuming that
hasn't changed.]