* Pre-built binaries vs. performance
@ 2018-01-31 13:39 Ludovic Courtès
2018-01-31 17:59 ` Amirouche Boubekki
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: Ludovic Courtès @ 2018-01-31 13:39 UTC (permalink / raw)
To: guix-devel
Hello Guix!
This post is a followup to our previous discussions on how to handle
architecture-specific optimizations:
https://guix-hpc.bordeaux.inria.fr/blog/2018/01/pre-built-binaries-vs-performance/
Comments welcome!
Ludo’.
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: Pre-built binaries vs. performance 2018-01-31 13:39 Pre-built binaries vs. performance Ludovic Courtès @ 2018-01-31 17:59 ` Amirouche Boubekki 2018-01-31 18:21 ` Alex Vong 2018-02-01 15:49 ` Fis Trivial 2 siblings, 0 replies; 9+ messages in thread From: Amirouche Boubekki @ 2018-01-31 17:59 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 590 bytes --] Le 31 janv. 2018 3:08 PM, "Ludovic Courtès" <ludovic.courtes@inria.fr> a écrit : Hello Guix! This post is a followup to our previous discussions on how to handle architecture-specific optimizations: https://guix-hpc.bordeaux.inria.fr/blog/2018/01/pre-built- binaries-vs-performance/ Comments welcome! Ludo’. Tx for the update. Optimised binaries are not only useful for HPC. I stayed with GNU/Linux also because it delivered good performance for my old hardware. Even if I am not helping, I am happy to know that it's a matter that is known to guixers. [-- Attachment #2: Type: text/html, Size: 1235 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Pre-built binaries vs. performance 2018-01-31 13:39 Pre-built binaries vs. performance Ludovic Courtès 2018-01-31 17:59 ` Amirouche Boubekki @ 2018-01-31 18:21 ` Alex Vong 2018-02-01 15:49 ` Fis Trivial 2 siblings, 0 replies; 9+ messages in thread From: Alex Vong @ 2018-01-31 18:21 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel ludovic.courtes@inria.fr (Ludovic Courtès) writes: > Hello Guix! > > This post is a followup to our previous discussions on how to handle > architecture-specific optimizations: > > https://guix-hpc.bordeaux.inria.fr/blog/2018/01/pre-built-binaries-vs-performance/ > > Comments welcome! > > Ludo’. I encountered this problem several years ago when packaging for Debian. The program (a single binary) could be compiled with 3 different sets of options: generic, AVX or AVX2. The goal was to package it so that users didn't have to specify which version to install. The hack I came up with was to compile all 3 variants and wrote a shell wrapper to run self-tests for all 3 variants. If the self-test for the AVX2 variant works, then pick it. If not, try AVX. If not, fall back to generic. This way was quite crude. Now I think we can make use of /proc/cpuinfo. Will this method work? An obvious disadvantage is that more disk space will be needed. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Pre-built binaries vs. performance 2018-01-31 13:39 Pre-built binaries vs. performance Ludovic Courtès 2018-01-31 17:59 ` Amirouche Boubekki 2018-01-31 18:21 ` Alex Vong @ 2018-02-01 15:49 ` Fis Trivial 2018-02-01 16:54 ` Ludovic Courtès 2 siblings, 1 reply; 9+ messages in thread From: Fis Trivial @ 2018-02-01 15:49 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès writes: > Hello Guix! > > This post is a followup to our previous discussions on how to handle > architecture-specific optimizations: > > https://guix-hpc.bordeaux.inria.fr/blog/2018/01/pre-built-binaries-vs-performance/ > > Comments welcome! > > Ludo’. Instructions support seems like just another kind of runtime dependency, can we reconsider this: https://lists.gnu.org/archive/html/guix-devel/2015-01/msg00166.html Thanks for your time. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Pre-built binaries vs. performance 2018-02-01 15:49 ` Fis Trivial @ 2018-02-01 16:54 ` Ludovic Courtès 2018-02-01 17:10 ` Fis Trivial 0 siblings, 1 reply; 9+ messages in thread From: Ludovic Courtès @ 2018-02-01 16:54 UTC (permalink / raw) To: Fis Trivial; +Cc: guix-devel Hello, Fis Trivial <ybbs.daans@hotmail.com> skribis: > Ludovic Courtès writes: > >> Hello Guix! >> >> This post is a followup to our previous discussions on how to handle >> architecture-specific optimizations: >> >> https://guix-hpc.bordeaux.inria.fr/blog/2018/01/pre-built-binaries-vs-performance/ >> >> Comments welcome! >> >> Ludo’. > > Instructions support seems like just another kind of runtime dependency, > can we reconsider this: > https://lists.gnu.org/archive/html/guix-devel/2015-01/msg00166.html What do you mean? The thread you’re referring to doesn’t discuss architecture-specific optimizations. Ludo’. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Pre-built binaries vs. performance 2018-02-01 16:54 ` Ludovic Courtès @ 2018-02-01 17:10 ` Fis Trivial 2018-02-01 17:16 ` Fis Trivial ` (2 more replies) 0 siblings, 3 replies; 9+ messages in thread From: Fis Trivial @ 2018-02-01 17:10 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès writes: > Hello, > > Fis Trivial <ybbs.daans@hotmail.com> skribis: > >> Ludovic Courtès writes: >> >>> Hello Guix! >>> >>> This post is a followup to our previous discussions on how to handle >>> architecture-specific optimizations: >>> >>> https://guix-hpc.bordeaux.inria.fr/blog/2018/01/pre-built-binaries-vs-performance/ >>> >>> Comments welcome! >>> >>> Ludo’. >> >> Instructions support seems like just another kind of runtime dependency, >> can we reconsider this: >> https://lists.gnu.org/archive/html/guix-devel/2015-01/msg00166.html > > What do you mean? The thread you’re referring to doesn’t discuss > architecture-specific optimizations. > > Ludo’. It's not architecture specific. But we can treat architecture as one of the runtime dependency, right? For example, openBlas depends on SSE instructions set, but, optionally it can depends on AVX2. (Not a perfect example since openblas can optimize itself at runtime, but hopefully you can understand the idea) So, we can build packages in a generic way, and then provide optional dependencies. In this case, we treat SSE as default dependency and AVX2 as an optional dependency, let users specify whether they want it or not. AVX2 need not to be actual packages, it's just an abstraction. It's just a thought that we can abstract architecture as part of the dependency graph. Thanks for you time. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Pre-built binaries vs. performance 2018-02-01 17:10 ` Fis Trivial @ 2018-02-01 17:16 ` Fis Trivial 2018-02-01 17:48 ` Ludovic Courtès 2018-02-05 9:22 ` Ludovic Courtès 2 siblings, 0 replies; 9+ messages in thread From: Fis Trivial @ 2018-02-01 17:16 UTC (permalink / raw) To: Fis Trivial; +Cc: guix-devel, Ludovic Courtès Fis Trivial writes: > Ludovic Courtès writes: > >> Hello, >> >> Fis Trivial <ybbs.daans@hotmail.com> skribis: >> >>> Ludovic Courtès writes: >>> >>>> Hello Guix! >>>> >>>> This post is a followup to our previous discussions on how to handle >>>> architecture-specific optimizations: >>>> >>>> https://guix-hpc.bordeaux.inria.fr/blog/2018/01/pre-built-binaries-vs-performance/ >>>> >>>> Comments welcome! >>>> >>>> Ludo’. >>> >>> Instructions support seems like just another kind of runtime dependency, >>> can we reconsider this: >>> https://lists.gnu.org/archive/html/guix-devel/2015-01/msg00166.html >> >> What do you mean? The thread you’re referring to doesn’t discuss >> architecture-specific optimizations. >> >> Ludo’. > > It's not architecture specific. But we can treat architecture as one of > the runtime dependency, right? For example, openBlas depends on SSE > instructions set, but, optionally it can depends on AVX2. (Not a perfect > example since openblas can optimize itself at runtime, but hopefully you > can understand the idea) > > So, we can build packages in a generic way, and then provide optional > dependencies. > In this case, we treat SSE as default dependency and AVX2 as an optional > dependency, let users specify whether they want it or not. > AVX2 need not to be actual packages, it's just an abstraction. > > It's just a thought that we can abstract architecture as part of the > dependency graph. > > Thanks for you time. Replying to myself, if this is doable, it will not only benefit performance, but also all the other packages who have optional language bindings, optional GUI backends or any other optional runtime dependencies/feature. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Pre-built binaries vs. performance 2018-02-01 17:10 ` Fis Trivial 2018-02-01 17:16 ` Fis Trivial @ 2018-02-01 17:48 ` Ludovic Courtès 2018-02-05 9:22 ` Ludovic Courtès 2 siblings, 0 replies; 9+ messages in thread From: Ludovic Courtès @ 2018-02-01 17:48 UTC (permalink / raw) To: Fis Trivial; +Cc: guix-devel Fis Trivial <ybbs.daans@hotmail.com> skribis: > So, we can build packages in a generic way, and then provide optional > dependencies. > In this case, we treat SSE as default dependency and AVX2 as an optional > dependency, let users specify whether they want it or not. > AVX2 need not to be actual packages, it's just an abstraction. > > It's just a thought that we can abstract architecture as part of the > dependency graph. We do that in the form of the “system type”—“x86_64-linux”, etc. That captures what’s otherwise not captured in the graph: the architecture and the kernel. Someone proposed having, say, “x86_64avx2-linux”, which is similar in spirit to what you’re proposing, I think. However that’d be inconvenient because every single package in the graph would have to be rebuilt compared to “x86_64-linux”. Also, I don’t see how to make that an _optional_ dependency. Thanks, Ludo’. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Pre-built binaries vs. performance 2018-02-01 17:10 ` Fis Trivial 2018-02-01 17:16 ` Fis Trivial 2018-02-01 17:48 ` Ludovic Courtès @ 2018-02-05 9:22 ` Ludovic Courtès 2 siblings, 0 replies; 9+ messages in thread From: Ludovic Courtès @ 2018-02-05 9:22 UTC (permalink / raw) To: Fis Trivial; +Cc: guix-devel Heya, Fis Trivial <ybbs.daans@hotmail.com> skribis: >>> Instructions support seems like just another kind of runtime dependency, >>> can we reconsider this: >>> https://lists.gnu.org/archive/html/guix-devel/2015-01/msg00166.html >> >> What do you mean? The thread you’re referring to doesn’t discuss >> architecture-specific optimizations. >> >> Ludo’. > > It's not architecture specific. But we can treat architecture as one of > the runtime dependency, right? For example, openBlas depends on SSE > instructions set, but, optionally it can depends on AVX2. (Not a perfect > example since openblas can optimize itself at runtime, but hopefully you > can understand the idea) > > So, we can build packages in a generic way, and then provide optional > dependencies. > In this case, we treat SSE as default dependency and AVX2 as an optional > dependency, let users specify whether they want it or not. > AVX2 need not to be actual packages, it's just an abstraction. > > It's just a thought that we can abstract architecture as part of the > dependency graph. Yeah, I see what you mean. It would be nice, but it seems quite far stretched from the current design; I wouldn’t know how to do that. Ludo’. ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2018-02-05 9:22 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2018-01-31 13:39 Pre-built binaries vs. performance Ludovic Courtès 2018-01-31 17:59 ` Amirouche Boubekki 2018-01-31 18:21 ` Alex Vong 2018-02-01 15:49 ` Fis Trivial 2018-02-01 16:54 ` Ludovic Courtès 2018-02-01 17:10 ` Fis Trivial 2018-02-01 17:16 ` Fis Trivial 2018-02-01 17:48 ` Ludovic Courtès 2018-02-05 9:22 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).