Re: Have GPGPU support in guix?

[Fis Trivial <ybbs.daans@hotmail.com>]

Ludovic Courtès writes:

> Hi,
>
> Fis Trivial <ybbs.daans@hotmail.com> skribis:
>
>>>> However, I wanted to manage all of these with guix so that we can have a
>>>> unified dependency tree. Currently there are a few options for OpenCL
>>>> runtime. Namely, beignet, neo from Intel, ROCm stack from AMD, the POCL
>>>> project and the implementation in mesa.
>>>>
>>>> I tried to package beignet (an old OpenCL runtime from Intel) but it
>>>> wasn't successful due to failed tests (it failed from recognizing
>>>> device).
>>>> https://github.com/trivialfis/guixpkgs/blob/master/opencl.scm
>>>
>>> Regarding failed tests, I think that tests that expect GPU devices to be
>>> able are likely to fail: it could be that the relevant /dev nodes are
>>> inaccessible to the build users, or, more importantly, it could be that
>>> we’re building on a GPU-less machine such as our build farm.  So these
>>> tests would have to be skipped.
>>
>> I think we might need a better scheme other than skipping tests to
>> verify the usability of these packages. If we skip tests for OpenCL
>> runtime, then all the other packages (I promise it will be A LOT) that
>> depend on it will need to skip their tests, since they won't run without
>> a proper runtime, resulting in a whole tree of untested packages. And
>> these thing break easily, due to the fact that they usually rely on
>> specific version of Clang and LLVM, and many of them still in heavy
>> development stage.
>
> Are you sure?  What I would expect is that packages would be able to use
> the OpenCL run-time, they’d even be able to run OpenCL code as part of
> their tests, only it would run on the main CPU instead of on GPUs, as if
> the machine had no usable GPUs.
>
> Does that make sense?
>
> Thanks,
> Ludo’.

In most of the time, we have two way to run these packages on CPU, For
the first one:

Most of the packages have a CPU backend and an OpenCL backend,
implementing same algorithms twice, so when OpenCL device is not
available, they will run on their CPU backend. And the normal testing
scheme for these packages is doing the same tests for each backend, but
do them independently. So, if we were to put these packages in Guix, we
could run the tests for CPU backend.

For the second one:

OpenCL implementations (the compiler) have a fall back mode, which means
they can treat CPU as the device and compile OpenCL code to run it on
CPU. I think this is what you meant by the line "run OpenCL code as part
of their tests". This feature is provided by the OpenCL compiler, which
we currently can't test.

I'm still trying to package a basic OpenCL stack. And I did success in
run some very primitive tests from myself outside of the store. But
still, there are a lots of stuffs I need to learn.