Blender export backend missing

Blender export backend missing

[Théo Maxime Tyburn]

Hi,

I want to import and export mesh sequences to Alambic or USD files using blender. But these
import/export formats are not available in the import/export interface. Although the
version should be high enough, see
https://docs.blender.org/manual/en/latest/files/import_export/alembic.html

Is seems to be because the "WITH_ALEMBIC" option is not used as a
configure flag. I think
about trying to build it myself with this option. Is there any
particular reason this was not added? Can you forsee what should be done
extra to make it work (link another lib or something)? I noticed that other export
backends are missing, are they being kept out for the same reasons? 

Cheers,

Théo

Re: Blender export backend missing

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1124 bytes --]

Théo Maxime Tyburn schreef op do 19-05-2022 om 15:05 [+0200]:
> Is seems to be because the "WITH_ALEMBIC" option is not used as a
> configure flag. I think
> about trying to build it myself with this option. Is there any
> particular reason this was not added? Can you forsee what should be done
> extra to make it work (link another lib or something)? I noticed that other export
> backends are missing, are they being kept out for the same reasons? 

Possible reasons (I don't know which apply here and which are actually
true):

  * When blender was first packaged, that configuration flag did not
    exist.
  * Likewise, but the 'alembic' package was not yet packaged in Guix.
  * The original package author did not notice the WITH_ALEMBIC option.
  * The original package author noticed the option but didn't need
    'alembic' and kept things simple for now.
  * 'alembic' has a large closure size

You'll have to add alembic to the 'inputs' of blender and add
WITH_ALEMBIC to the #:configure-flags of 'blender'.  I don't expect
any problems but hard to tell in advance.

Greetings,
Maxime

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Blender export backend missing

[Théo Maxime Tyburn]

Maxime Devos <maximedevos@telenet.be> writes:

> Possible reasons (I don't know which apply here and which are actually
> true):
>
>   * When blender was first packaged, that configuration flag did not
>     exist.
>   * Likewise, but the 'alembic' package was not yet packaged in Guix.
>   * The original package author did not notice the WITH_ALEMBIC option.
>   * The original package author noticed the option but didn't need
>     'alembic' and kept things simple for now.
>   * 'alembic' has a large closure size

Makes sense.

> You'll have to add alembic to the 'inputs' of blender and add
> WITH_ALEMBIC to the #:configure-flags of 'blender'.  I don't expect
> any problems but hard to tell in advance.

I see, that’s encouraging, I’ll try it out!

Re: Blender export backend missing

[Théo Maxime Tyburn]

I added ("alembic" ,alembic) to the inputs
and "-DWITH_ALEMBIC=ON" in the configure-flags list.

Unfortunately it didn’t work directly. I get this error from ld
--8<---------------cut here---------------start------------->8---
ld: ../../lib/libbf_alembic.a(abc_writer_transform.cc.o): in function `blender::io::alembic::ABCTransformWriter::do_write(blender::io::HierarchyContext&)':
/tmp/guix-build-blender-3.0.1.drv-0/blender-3.0.1/source/blender/io/alembic/exporter/abc_writer_transform.cc:101: undefined reference to `Alembic::AbcGeom::v12::XformSample::setMatrix(Imath_2_5::Matrix44<double> const&)'
--8<---------------cut here---------------end--------------->8---

So I guess alembic is not accessible.

Any idea on how to go on from there?

Best,

Théo

Re: Blender export backend missing

[Ekaitz Zarraga]

Hi

------- Original Message -------
On Friday, May 20th, 2022 at 11:36 PM, Théo Maxime Tyburn <theo.tyburn@gmail.com> wrote:


> I added ("alembic" ,alembic) to the inputs
> and "-DWITH_ALEMBIC=ON" in the configure-flags list.
>
> Unfortunately it didn’t work directly. I get this error from ld
> --8<---------------cut here---------------start------------->8---
>
> ld: ../../lib/libbf_alembic.a(abc_writer_transform.cc.o): in function `blender::io::alembic::ABCTransformWriter::do_write(blender::io::HierarchyContext&)': /tmp/guix-build-blender-3.0.1.drv-0/blender-3.0.1/source/blender/io/alembic/exporter/abc_writer_transform.cc:101: undefined reference to` Alembic::AbcGeom::v12::XformSample::setMatrix(Imath_2_5::Matrix44<double> const&)'
>
> --8<---------------cut here---------------end--------------->8---
>
>
> So I guess alembic is not accessible.
>
> Any idea on how to go on from there?
>
> Best,
>
> Théo


Make sure the version of alembic we have packaged is the one Blender expects.

Sometimes the APIs change and that may led to that kind of errors.

Hope that helps,
Ekaitz

Re: Blender export backend missing

[Théo Maxime Tyburn]

Hi

Ekaitz Zarraga <ekaitz@elenq.tech> writes:

> Make sure the version of alembic we have packaged is the one Blender expects.
>
> Sometimes the APIs change and that may led to that kind of errors.

The version of alembic that blender 3.0.1 requires is the same we
have.

The problem might come from another version mismastch problem. Namely
with OpenEXR. OpenEXR
version 2.x provided Imath inside of its subdirectory Ilmbase, for
which we also have a separate package. From version 3.x of OpenEXR, Imath
became a separate package. The guix blender package still uses openexr
version 2.x. and its subpackage ilmbase. However guix package alembic uses the new imath. Blender
upstream actually also expect openexr 3.x and imath.

The quick and dirty fix is to just downgrade alembic. I just tried it
and it works. So at least we can work with alembic files in blender now.

A better fix would be to add openexr 3.x and imath
to blender package’s inputs and remove openexr-2 and ilmbase.

I tried that but it doesn’t work. The error is
--8<---------------cut here---------------start------------->8---
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
OPENEXR_HALF_LIBRARY (ADVANCED)
    linked by target "cycles_scene" in directory /tmp/guix-build-blender-3.0.1.drv-0/blender-3.0.1/intern/cycles/scene
OPENEXR_ILMIMF_LIBRARY (ADVANCED)
    linked by target "cycles_scene" in directory /tmp/guix-build-blender-3.0.1.drv-0/blender-3.0.1/intern/cycles/scene
--8<---------------cut here---------------end--------------->8---

It seems this is due to some error in the cmake of blender’s source. Some
other packge maintainers have come across the same issue aparently
https://devtalk.blender.org/t/building-blender-2-93-3-with-the-new-half-implementation-of-openexr-that-uses-half-of-imath/20074.
I guess it has not been fixed in upstream blender yet. There is a patch though
here
https://github.com/bartoszek/AUR-blender-2.83-git/blob/master/openexr3.patch

So this could be the better solution if want to rely on that patch. Do we?
I also have no idea how to apply a patch from another repo in a package
definition though. Any idea?

A general build / packaging question: For testing purposes I sometimes want to just try some command in the
build environment. Like applying this patch for example. Is there a
simple way to somehow enter the build env and do things from there
imperatively, just to quickly check some things?

Best,

Théo

Re: Blender export backend missing

[Ekaitz Zarraga]

Hi

------- Original Message -------
On Sunday, May 22nd, 2022 at 3:49 PM, Théo Maxime Tyburn <theo.tyburn@gmail.com> wrote:


> Hi
>
> Ekaitz Zarraga ekaitz@elenq.tech writes:
>
> > Make sure the version of alembic we have packaged is the one Blender expects.
> >
> > Sometimes the APIs change and that may led to that kind of errors.
>
>
> The version of alembic that blender 3.0.1 requires is the same we
> have.
>
> The problem might come from another version mismastch problem. Namely
> with OpenEXR. OpenEXR
> version 2.x provided Imath inside of its subdirectory Ilmbase, for
> which we also have a separate package. From version 3.x of OpenEXR, Imath
> became a separate package. The guix blender package still uses openexr
> version 2.x. and its subpackage ilmbase. However guix package alembic uses the new imath. Blender
> upstream actually also expect openexr 3.x and imath.
>
> The quick and dirty fix is to just downgrade alembic. I just tried it
> and it works. So at least we can work with alembic files in blender now.
>
> A better fix would be to add openexr 3.x and imath
> to blender package’s inputs and remove openexr-2 and ilmbase.
>
> I tried that but it doesn’t work. The error is
> --8<---------------cut here---------------start------------->8---
>
> CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
> Please set them or make sure they are set and tested correctly in the CMake files:
> OPENEXR_HALF_LIBRARY (ADVANCED)
> linked by target "cycles_scene" in directory /tmp/guix-build-blender-3.0.1.drv-0/blender-3.0.1/intern/cycles/scene
> OPENEXR_ILMIMF_LIBRARY (ADVANCED)
> linked by target "cycles_scene" in directory /tmp/guix-build-blender-3.0.1.drv-0/blender-3.0.1/intern/cycles/scene
> --8<---------------cut here---------------end--------------->8---
>
>
> It seems this is due to some error in the cmake of blender’s source. Some
> other packge maintainers have come across the same issue aparently
> https://devtalk.blender.org/t/building-blender-2-93-3-with-the-new-half-implementation-of-openexr-that-uses-half-of-imath/20074.
> I guess it has not been fixed in upstream blender yet. There is a patch though
> here
> https://github.com/bartoszek/AUR-blender-2.83-git/blob/master/openexr3.patch
>
> So this could be the better solution if want to rely on that patch. Do we?
> I also have no idea how to apply a patch from another repo in a package
> definition though. Any idea?
>
> A general build / packaging question: For testing purposes I sometimes want to just try some command in the
> build environment. Like applying this patch for example. Is there a
> simple way to somehow enter the build env and do things from there
> imperatively, just to quickly check some things?
>
> Best,
>
> Théo


You can try to build the package with `guix build -K` and then interact with the build process. The -K is going to keep the build directory and gives you the folder where it was building everything, then you can `cd` there and use the environment file that appears there to set up the same container where guix was building the package. You just need to load the file to your shell.

Also, you can add the patches directly with the guix package itself, look how `4store` does it: it adds a `patches` field in the `source` and those are applied when the code is downloaded.
If the patch you want to apply is small you can do it with regex substitutions too, but this looks better for your case.

I know this is a huge effort, I had to deal with similar things in Blender and Meshlab in the past, so I'm here to help!

Good job,
Ekaitz

Re: Blender export backend missing

[Théo Maxime Tyburn]

Hello,

I tried several things in the last days. Unfortunatly I still didn’t
manage to build Blender with the new OpenEXR IMath version. Long story
short: I think upgrading blender to 3.1.0 will solve the problem. But
this requires to have python 3.10. So I’ll wait until we have it and
come back to it later.

In the meantime we can leave openexr to the old version and just add
an alternative alembic package that is based on it. I used this approach in the last few days to work
with alembic files in blender and it just works fine. I’ll propose a
patch for this. I never did this for guix though, so I might still have
some questions :)

Ekaitz Zarraga <ekaitz@elenq.tech> writes:
[...]
> I know this is a huge effort, I had to deal with similar things in
> Blender and Meshlab in the past, so I'm here to help!
>
> Good job,
> Ekaitz

Thank you for encouraging me! It is indeed a messy problem. Let’s see
how this works out with the newer version of python and blender.

Here is the story of my experimentations with gorry details:

The patch I mentioned in my previous message is actually for a much older version, so it can’t
even apply. There is a simple way to avoid the error occuring at config
time though: if you set "-DWITH_CYCLES=OFF" then the config succedes. So
it seems the problem comes from Cycles. This is a problem on its
own. Not sure what to do about it. It looks more like an upstream
problem.

Anyway, even with that problem put aside, there is another one. The package half is not found at
compile time. In earlier versions of OpenEXR it was part of IlmBase, now
it is in IMath. The trick for blender based on ilmbase was to add
(string-append (assoc-ref inputs "ilmbase") "/include/OpenEXR/")
to C_INCLUDE_PATH and CPLUS_INCLUDE_PATH so that half can be found.
So now that half is in IMath I changed it to
(string-append (assoc-ref inputs "imath") "/include/IMath/")
unfortunatly this didn’t work: half is still not found.

I am confused because I tried a small experiement with a simple c code
and it worked when setting C_INCLUDE_PATH. So this compiles:
--8<---------------cut here---------------start------------->8---
C_INCLUDE_PATH=/gnu/store/pxf5591fpradfjjqyx7j269xilwndcmk-imath-3.1.3/include/Imath/ gcc use-half.c
--8<---------------cut here---------------end--------------->8---
where use-half.c contains
--8<---------------cut here---------------start------------->8---
#include <half.h>

void main() {
  printf("hello");
}
--8<---------------cut here---------------end--------------->8---
So I’m not sure why this doesn’t work in the context of blender being
  built. Any idea?
  
By the way this seems to be something that should be set by the ilmbase and
imath packages, right? Or is it just a problem of blender not specifying
half is to be found in IMath. For example this compiles:
--8<---------------cut here---------------start------------->8---
gcc use-Imath-half.c
--8<---------------cut here---------------end--------------->8---
where use-Imath-half.c contains
--8<---------------cut here---------------start------------->8---
#include <Imath/half.h>

void main() {
  printf("hello");
}
--8<---------------cut here---------------end--------------->8---
So we could actually try to just change all #include <half.h> with
  #include <Imath/half.h> in the blender source code. How can one
  achieve that?

But even if this works, we still have the problem that Cycles need to
have the old version of OpenEXR.

As long as this problem with configuring Cycles with newer versions of
OpenEXR and Imath fails I see no way around.

Happy hacking,

Théo