Re: Creating subtitles for the Guix Days videos!

[Tanguy LE CARROUR <tanguy@bioneland.org>]

[-- Attachment #1: Type: text/plain, Size: 671 bytes --]

Hi Julien,


Quoting Julien Lepiller (2022-03-01 15:36:19)
> I'm looking for volunteers to create English subtitles for the Guix Days
> talks. […] Please send me the subtitles once
> they are completed, I'll add them with the videos.

It's my first time, so thank you for your indulgence! :-)

I'm attaching my humble contribution:

- `.txt` the transcription ;
- `.ass` the file created by Aegisub ; and
- `.sub` an attempt to export it to sub format.

I have to admit that is pretty bad "punctuation-wise", because I was not
sure were the sentences started and ended. Sorry!

Just let me know if I have to fix anything.

Regards,

-- 
Tanguy

[-- Attachment #2: guix-days-2022-modernizing-python-build-system.mkv.txt --]
[-- Type: text/plain, Size: 11693 bytes --]

hello everyone
nice to have you here at Guix Days 2022
I would like to talk about Python build system
and why we need to modernize it
python-build-system is the Guix component
that turns the source distribution of any Python package out there
into an installable image
so, basically, a directory under the GNU store
let's have a look at tomli a quite simple Python package
which has no external dependencies
so, if we import it using `guix import`, here
add a few imports to the resulting file
and then try to build it
it should work out of the box, right?
The problem is, it does not!
and instead we see at the bottom
an error message saying "no setup.py found"
and, indeed, if we look at the source distribution
there is no `setup.py` to be found anywhere
So, what is going on?
Fortunately, this package is already available in Guix
as `python-tomli`
so we can have a look at its definition
to see how it is currently built
let's just do that
looking at the build system's arguments
we see the phases `build` here and `install` here
which are usually provided by Python build system
replaced with custom code
I'm only showing the interesting parts here
the actual commands are actually much longer
first the build phase
uses a Python module called `build`
to build the wheel, as we can see here
the wheel is basically a distribution format in the Python world
then in the install phase
we simply use a well known tool called Pip
to install the wheel that we just built
into the output which would be somewhere around the GNU store
so how does the build module knows what to do
what to build?
it follows PEP 517
PEPs are kind of the RFCs of the Python world
and this PEP basically splits building wheels into two parts
a frontend and a backend
the frontend is the user facing part
for example the `build` we just saw here
this is the user facing part of the build process
and then a backend
the frontend is supposed to read a file called `pyproject.toml`
this is what we are seeing here
and in that TOML file, a section called `build-system`
this one here
declares which backend will actually build our wheel
and, in this case, another package called `flit_core`
its requirements a build time dependency of tomli
and its module `flit_core.buildapi`
is providing us with the build entrypoint.
The file also contains standardize metadata
and tool related configuration data
which I'm not showing here
A PEP 517* compatible build backend
provides a standard function entrypoint
called `build_wheel`
in the module I just referenced here in the top
and, if we call it, it will just do its magic
and it will produce a wheel file
and its first argument it's the wheel directory
that wheel is basically a zip file
with a predefined structure
that we can extract into our store
and we are almost done
and this is what Pip does in the install phase here
and that's basically the entire build process
as specified by PEP 517
there is no `setup.py` involved any more
we don't have to call it
we don't have to create it as a package provider
so the reason why the error message I showed, showed up earlier
will keep on poping up more and more
is simple: we are late!
we are really really late, actually
because PEP 517 was originally created in 2015
and that it gained provisional acceptance in 2017
and just last year,
after being basically being the *de facto* successor of `setup.py` for some time
it has been finalized and fully accepted
and more importantly, flit which you remember from the previous slide
is also able to create source distributions
and upload them to PyPI
Python public package repository basically
so far, it has been generating a `setup.py`
and does nobody really noticed
but since version 3.5, which was released in November 2021
flit stop doing that by default
and thus we are seeing more and more packages without `setup.py`
in their source distributions
and so we are basically unable to build this projects right now
or this Python modules
a look at the Guix's repository in late January
and back then only 11 packages actually used Pip
or PyPA build as we've seen
but I think our ecosystem is quite old
about half the packages not being the latest versions available upstream
according to `guix refresh`
so it's possible that more packages actually require support for this `pyproject.toml`
and we simply have not updated them yet for whatever reason
maybe because it's too difficult or nobody poped to do it yet
they are also more issues with our current Python build system
for instance `setup.py` test ???? has been deprecated for quite some time
since 2019 actually
and the Python build system's default check phase relies on that
even though it doesn't work for a lot of packages right now
and thus, almost every package in our repository
needs to replace that check phase with a call to Pytest
which is the *de facto* standard right now for executing tests
???? tox does something similar but not quite the same
another long standing issue is wrapping of Python path
or Guix Python path now
because it includes native inputs
and thus propagates way too many packages
and I'm sure we can find more issues with the current Python build system
thus, for almost a year now, I've been working on a modernized Python build system
that addresses some of this issues
unfortunately I haven't had much time lately to improve it for others
I hope to pick it up again
and get it into shape
and get it to master at some point
you can read about it here
in Guix issue #46848
or you can just pull a branch called `wip-python-pep517`
it's also build by the CI by the way
so you don't have to do the whole world rebuild by yourself
and it's using exactly the method I described above
the part written in Guile ends up reading the project's `pyproject.toml`
and then calls the build system's entrypoint `build_wheel` as we can see here
and if there's no `pyproject.toml` it's also not a problem actually
because setuptools, which is still provided by default,
and still included in the build system by default,
because setuptools still provides a PEP 517 build backend
which simply uses the existing `setup.py` behind the curtains
so we don't have to, like provide extra logic for that
they were actually other options for this build phase
in how to do it basically
for example, we could also use the build module
that we have seen above
and some packages use currently
however that raises questions about bootstraping
how do you build the build package if it essentially requires itself
because the build module is part of the Python build system
and of of it's dependency
tomli for example is one of the dependency of build
and what do we do with tomli
like how do we install it without having build
so this is tricky, and I tried it in the beginning
and it works
but we need to do ???? manual copying
and that's not really pretty to be honest
we could also use Pip
which is usually distributed with Python
also our python package bundles Pip and setuptools
this has caused a few issues in the past
because we cannot update either of them
for provide alternative newer versions
without having a full world rebuild basically
and also Pip being a massive project
it bundles quite a few of it's dependencies
and I feel "we" as a Guix project
should try to untangle that mess whenever we can
and in this case, we actually can
so, in conclusion, I think running this 4 lines of Python code
that you can see here
is a much simpler solution
that trying to bootstrap the build module,
tomli and all the modules required to get a Python based builder basically
we can just do it in 4 lines of code
there are a few changes for packagers
which aime to reduce the need for custom phases
first, it is possible to override the build backend detected from the `pyproject.toml`
especially early adopters of PEP 517
use modules which have been renamed in the meantime
I think both Poetry and Flit did that at some point
the split their build logic into a separate project
and renamed the backend
so, for some packages, it's necessary to override this detected build backend
and you can do that simply by specifying the `#:build-backend` option
and also if the detection is wrong at some point
we could override it without causing too much damage
secondly, we have `#:configure-flags`
they had existed before, but where applied only during install
but not during the build phase
which, I believe, is a ????, we'll see
and this are arguments to `setup.py`
but their syntax depends on the build backend unfortunately
sometimes you can pass the argument directly
sometimes you need to pass them as an option to another option
like we can see here
that's the setuptools specific way to do it
so you have an option
and then you have to pass the actual options as an argument to that option
and, in theory, they could also look completly different
like no relationship to any command line flags
it's just legacy code right now
and it's not pretty
and it's one of the things that have been criticized about PEP 517
this configure flags are not really specified how they are supposed to look
but this solution exists, so we have to swim with the flow, kind of
and, finally, they are `#:test-flags`
my proposal was to auto detect test binary based on the package's native inputs
so if pytest is available, it will just use pytest
if nose is available, it will use nose
and so on
now, test flags override the default arguments passed to this test command
and, hopefully, this can reduce the number of custom phases required
to exclude tests mostly
in this case, we are simply don't want to run the `test_legacy_callbacks` function
and so we can just exclude it with the flags
and the package will build just fine, probably, hopefully
so, where do we go from here?
the first step would be to finalize the work on the Python build system
and get the branch ready into a state where it can be merged
unfortunately, right now, quite a few packages fail to build there
because their previously disabled test suite fails
not necessarily disabled, but it was never ran
because the test runner was buggy and the `setup.py test` command simply did nothing
and didn't detect edge cases
we could also go a different route and add a new build system
without touching the old one
and just slowly migrate every Python package we have over
instead of doing the big switch at once
that was also one of the suggestions that was certainly possible
I haven't explored that it would works
as I mentioned earlier, currently the native inputs end up in wrapped binaries
and I think this is one thing we should address while doing the whole world rebuild anyway
but I haven't done any work on that yet, unfortunately
and finally, the pypi import needs support to read `pyproject.toml`
not so much for the runtime dependencies,
this is fine, we have another metadata file which contains the runtime dependencies
but for build time and test dependencies
because they are not included in the wheel file,
there's a file called metadata, and it doesn't have this data
so right now, if you import a `pyproject.toml` based project
you will lack the build dependencies basically
for example flit will not be in the native input right now
and that's it for me for now
I hope to answer some of your questions during the live Q and A
and in the PDF version of this slides, you'll find some clickable links
to different talks and to the standards I have referenced
so, I hope I'll see you soon!

[-- Attachment #3: guix-days-2022-modernizing-python-build-system.mkv.sub --]
[-- Type: text/plain, Size: 14898 bytes --]

{29}{43}hello everyone
{44}{109}nice to have you here at Guix Days 2022
{110}{161}I would like to talk about Python build system
{162}{202}and why we need to modernize it
{203}{251}python-build-system is the Guix component
{252}{318}that turns the source distribution of any Python package out there
{319}{347}into an installable image
{348}{405}so, basically, a directory under the GNU store
{406}{492}let's have a look at tomli a quite simple Python package
{493}{526}which has no external dependencies
{527}{608}so, if we import it using `guix import`, here
{609}{668}add a few imports to the resulting file
{669}{697}and then try to build it
{698}{753}it should work out of the box, right?
{754}{782}The problem is, it does not!
{783}{840}and instead we see at the bottom
{841}{927}an error message saying "no setup.py found"
{928}{985}and, indeed, if we look at the source distribution
{986}{1043}there is no `setup.py` to be found anywhere
{1044}{1074}So, what is going on?
{1075}{1142}Fortunately, this package is already available in Guix
{1143}{1171}as `python-tomli`
{1172}{1209}so we can have a look at its definition
{1210}{1239}to see how it is currently built
{1240}{1268}let's just do that
{1269}{1327}looking at the build system's arguments
{1328}{1395}we see the phases `build` here and `install` here
{1421}{1486}which are usually provided by Python build system
{1487}{1531}replaced with custom code
{1532}{1582}I'm only showing the interesting parts here
{1583}{1634}the actual commands are actually much longer
{1635}{1681}first the build phase
{1682}{1739}uses a Python module called `build`
{1740}{1801}to build the wheel, as we can see here
{1802}{1926}the wheel is basically a distribution format in the Python world
{1927}{1995}then in the install phase
{1996}{2052}we simply use a well known tool called Pip
{2053}{2116}to install the wheel that we just built
{2117}{2203}into the output which would be somewhere around the GNU store
{2204}{2261}so how does the build module knows what to do
{2262}{2291}what to build?
{2292}{2355}it follows PEP 517
{2356}{2406}PEPs are kind of the RFCs of the Python world
{2407}{2522}and this PEP basically splits building wheels into two parts
{2523}{2551}a frontend and a backend
{2552}{2585}the frontend is the user facing part
{2586}{2638}for example the `build` we just saw here
{2639}{2696}this is the user facing part of the build process
{2697}{2725}and then a backend
{2726}{2870}the frontend is supposed to read a file called `pyproject.toml`
{2871}{2899}this is what we are seeing here
{2900}{2993}and in that TOML file, a section called `build-system`
{2994}{3015}this one here
{3016}{3075}declares which backend will actually build our wheel
{3076}{3160}and, in this case, another package called `flit_core`
{3161}{3247}its requirements a build time dependency of tomli
{3248}{3305}and its module `flit_core.buildapi`
{3306}{3361}is providing us with the build entrypoints.
{3362}{3421}The file also contains standardize metadata
{3422}{3460}and tool related configuration data
{3461}{3489}which I'm not showing here
{3490}{3595}A PEP 517* compatible build backend
{3596}{3653}provides a standard function entrypoint
{3654}{3682}called `build_wheel`
{3683}{3740}in the module I just referenced here in the top
{3741}{3802}and, if we call it, it will just do its magic
{3803}{3847}and it will produce a wheel file
{3848}{3911}and its first argument it's the wheel directory
{3912}{3968}that wheel is basically a zip file
{3969}{3990}with a predefined structure
{3991}{4032}that we can extract into our store
{4033}{4081}and we are almost done
{4082}{4146}and this is what Pip does in the install phase here
{4234}{4291}and that's basically the entire build process
{4292}{4326}as specified by PEP 517
{4327}{4378}there is no `setup.py` involved any more
{4379}{4387}we don't have to call it
{4388}{4465}we don't have to create it as a package provider
{4491}{4575}so the reason why the error message I showed, showed up earlier
{4576}{4610}will keep on poping up more and more
{4611}{4645}is pretty simple we are late!
{4646}{4697}we are really really late, actually
{4698}{4784}because PEP 517 was originally created in 2015
{4785}{4869}and that it gained provisional acceptance in 2017
{4870}{4898}and just last year,
{4899}{4987}after being basically being the *de facto* successor of `setup.py` for some time
{4988}{5045}it has been finalized and fully accepted
{5046}{5137}and more importantly, flit which you remember from the previous slide
{5138}{5197}is also able to create source distributions
{5198}{5233}and upload them to PyPI
{5234}{5297}Python's public package repository basically
{5298}{5363}so far, it has been generating a `setup.py`
{5364}{5393}and does nobody really noticed
{5394}{5500}but since version 3.5, which was released in November 2021
{5501}{5561}flit stop doing that by default
{5562}{5627}and thus we are seeing more and more packages without `setup.py`
{5628}{5656}in their source distributions
{5657}{5734}and so we are basically unable to build this projects right now
{5735}{5763}or this Python modules
{5764}{5830}a look at the Guix's repository in late January
{5831}{5896}and back then only 11 packages actually used Pip
{5897}{5939}or PyPA build as we've seen
{5940}{5996}but I think our ecosystem is quite old
{5997}{6060}with about half the packages not being the latest versions available upstream
{6061}{6089}according to `guix refresh`
{6090}{6193}so it's possible that more packages actually require support for this `pyproject.toml`
{6194}{6263}and we simply have not updated them yet for whatever reason
{6264}{6321}maybe because it's too difficult or nobody poped to do it yet
{6351}{6437}they are also more issues with our current Python build system
{6438}{6546}for instance `setup.py` test ???? has been deprecated for quite some time
{6547}{6589}since 2019 actually
{6590}{6675}and the default Python build system's default check phase relies on that
{6676}{6727}even though it doesn't work for a lot of packages right now
{6728}{6791}and thus, almost every package in our repository
{6792}{6872}needs to replace that check phase with a call to Pytest
{6873}{6930}which is the *de facto* standard right now for executing tests
{6931}{7021}???? tox does something similar but not really quite the same
{7022}{7104}another long standing issue is wrapping of Python path
{7105}{7133}or Guix Python path now
{7134}{7178}because it includes native inputs
{7179}{7222}and thus propagates way too many packages
{7223}{7307}and I'm sure we can find more issues with the current Python build system
{7308}{7481}thus, for almost a year now, I've been working on a modernized Python build system
{7482}{7513}that addresses some of this issues
{7514}{7588}unfortunately I haven't had much time lately to improve it for other
{7589}{7615}I hope to pick it up again
{7616}{7628}and get it into shape
{7629}{7657}and get it to master at some point
{7658}{7686}you can read about it here
{7687}{7771}in Guix issue #46848
{7772}{7858}or you can just pull a branch called `wip-python-pep517`
{7859}{7904}it's also build by the CI by the way
{7905}{7966}so you don't have to do the whole world rebuild by yourself
{7967}{8032}and it's using exactly the method I described above
{8034}{8148}the part written in Guile ends up reading the project's `pyproject.toml`
{8149}{8264}and then calls the build system's entrypoint `build_wheel` as we can see here
{8265}{8351}and if there's no `pyproject.toml` it's also not a problem actually
{8352}{8438}because setuptools, which is still provided by default,
{8439}{8473}and still included in the build system by default,
{8474}{8564}because setuptools provides a PEP 517 build backend
{8591}{8673}which simply uses the existing `setup.py` behind the curtains
{8674}{8757}so we don't have to, like provide extra logic for that
{8787}{8847}they were actually other options for this build phase
{8848}{8876}in how to do it basically
{8903}{8943}for example, we could also use the build module
{8944}{8963}that we have seen above
{8964}{9016}and some packages use currently
{9017}{9063}however that raises questions about bootstraping, like
{9064}{9134}how do you build the build package if it essentially requires itself
{9135}{9184}because the build module is part of the Python build system
{9185}{9221}and of of it's dependency
{9222}{9279}tomli for example is one of the dependency of build
{9280}{9337}and what do we do with tomli
{9338}{9367}like how do we install it without having build
{9368}{9434}so this is tricky, and I tried it in the beginning
{9435}{9457}and it works
{9458}{9521}but we need to do like some weird manual copying
{9522}{9569}and that's not really pretty to be honest
{9570}{9598}we could also use Pip
{9599}{9656}which is usually distributed with Python
{9657}{9721}also our python package bundles Pip and setuptools
{9722}{9772}this has caused a few issues in the past
{9773}{9801}because we cannot update either of them
{9802}{9853}for provide alternative newer versions
{9854}{9917}without having a full world rebuild basically
{9918}{9959}and also Pip being a massive project
{9960}{10004}it bundles quite a few of it's dependencies
{10005}{10033}and I feel "we" as a Guix project
{10034}{10091}should try to untangle that mess whenever we can
{10092}{10120}and in this case, we actually can
{10121}{10207}so, in conclusion, I think running this 4 lines of Python code
{10208}{10236}that you can see here
{10262}{10294}is a much simpler solution
{10295}{10351}than trying to bootstrap the build module,
{10352}{10468}and tomli and all the modules required to get a Python based builder basically
{10469}{10501}we can just do it in 4 lines of code
{10556}{10606}there are a few changes for packagers
{10607}{10654}which aime to reduce the need for custom phases
{10655}{10787}first, it is possible to override the build backend detected from the `pyproject.toml`
{10788}{10848}especially early adopters of PEP 517
{10849}{10905}use modules which have been renamed in the meantime
{10933}{10990}I think both Poetry and Flit did that at some point
{10991}{11048}and they split their build logic into a separate project
{11049}{11077}and renamed the backend
{11078}{11164}so, for some packages, it's necessary to override this detected build backend
{11165}{11222}and you can do that simply by specifying the `#:build-backend` option
{11223}{11280}and also if the detection is wrong at some point
{11281}{11338}we can just override it without causing too much damage
{11345}{11416}secondly, we have `#:configure-flags`
{11425}{11483}they have existed before, but where applied only during install
{11484}{11512}but not during the build phase
{11513}{11552}which, I believe, is a ????, we'll see
{11553}{11628}and this are kind of arguments to `setup.py`
{11629}{11676}but their syntax depends on the build backend unfortunately
{11677}{11738}so sometimes you can pass the arguments directly
{11739}{11788}sometimes you need to pass them as an option to another option
{11789}{11813}like we can see here
{11814}{11870}I think that's the setuptools specific way to do it
{11871}{11906}so you have an option
{11907}{11976}and then you have to pass the actual option as an argument to that option
{11977}{12063}and, in theory, they could also look completly different
{12064}{12120}like with no relationship to any command line flag
{12121}{12181}it's just legacy code/stuff right now
{12182}{12208}and it's not pretty
{12209}{12271}and it's one of the things that have been criticized about PEP 517
{12272}{12353}this configure flags are not really specified how they are supposed to look
{12354}{12440}but this solution exists, so we have to swim with the flow, kind of
{12444}{12500}and, finally, they are `#:test-flags`
{12501}{12614}my proposal was to auto detect test binary based on the package's native inputs
{12615}{12673}so if pytest is available, it will just use pytest
{12674}{12723}if nose is available, it will use nose
{12724}{12737}and so on
{12738}{12826}now, test flags override the default arguments passed to this test command
{12827}{12892}and, hopefully, this can reduce the number of custom phases required
{12893}{12925}to exclude tests mostly
{12926}{13015}in this case, we are simply don't want to run the `test_legacy_callbacks` function
{13016}{13064}and so we can just exclude it with the flags
{13092}{13157}and the package will build just fine, probably, hopefully
{13158}{13220}so, where do we go from here?
{13221}{13310}the first step would be to finalize the work on the Python build system
{13311}{13404}and get the branch ready into a state where it can be merged
{13405}{13484}unfortunately, right now, quite a few packages fail to build there
{13485}{13571}because their previously disabled test suite fails
{13572}{13612}not necessarily disabled, but it was never ran
{13613}{13713}because the test runner was buggy and the `setup.py test` command simply did nothing
{13714}{13745}and didn't detect edge cases
{13746}{13816}we could also go a different route and add a new build system
{13817}{13841}without touching the old one
{13842}{13913}and just slowly migrate every Python package we have over
{13914}{13945}instead of doing the big switch at once
{13949}{13988}that was also one of the suggestions that was certainly possible
{13989}{14027}I have explored that it would work
{14028}{14154}as I mentioned earlier, currently the native inputs end up in wrapped binaries
{14155}{14280}and I think this is one thing we should address too while doing the whole world rebuild anyway
{14281}{14354}but I haven't done any work on that yet, unfortunately
{14355}{14444}and finally, the pypi import needs support to read `pyproject.toml`
{14445}{14499}not so much for the runtime dependencies,
{14500}{14582}this is fine, we have another metadata file which contains the runtime dependencies
{14583}{14615}but for build time and test dependencies
{14616}{14668}because they are not included in the wheel file,
{14669}{14754}there's a file called metadata, and it doesn't have this data
{14755}{14870}so right now, if you import a `pyproject.toml` based project
{14871}{14934}you will lack the build dependencies basically
{14935}{15020}for example flit will not be in the native inpust right now
{15021}{15091}and that's it for me for now
{15092}{15166}I hope to answer some of your questions during the live Q and A
{15167}{15253}and in the PDF version of this slides, you'll find some clickable links bellow
{15254}{15340}to different talks and to the standards I have referenced
{15341}{15369}so, I hope I'll see you soon!

[-- Attachment #4: guix-days-2022-modernizing-python-build-system.mkv.ass --]
[-- Type: text/plain, Size: 25221 bytes --]

[Script Info]
; Script generated by Aegisub 3.2.2
; http://www.aegisub.org/
Title: Default Aegisub file
ScriptType: v4.00+
WrapStyle: 0
ScaledBorderAndShadow: yes
YCbCr Matrix: TV.601
PlayResX: 1916
PlayResY: 1076

[Aegisub Project Garbage]
Export Encoding: Unicode (UTF-8)
Audio File: guix-days-2022-modernizing-python-build-system.mkv
Video File: guix-days-2022-modernizing-python-build-system.mkv
Video AR Mode: 4
Video AR Value: 1.780669
Video Zoom Percent: 0.250000
Active Line: 248
Video Position: 15341

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Default,Arial,20,&H00FFFFFF,&H000000FF,&H00000000,&H00000000,0,0,0,0,100,100,0,0,1,2,2,2,10,10,10,1

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:01.64,0:00:02.60,Default,,0,0,0,,hello everyone
Dialogue: 0,0:00:02.60,0:00:06.92,Default,,0,0,0,,nice to have you here at Guix Days 2022
Dialogue: 0,0:00:06.92,0:00:10.64,Default,,0,0,0,,I would like to talk about Python build system
Dialogue: 0,0:00:10.64,0:00:13.32,Default,,0,0,0,,and why we need to modernize it
Dialogue: 0,0:00:13.32,0:00:16.76,Default,,0,0,0,,python-build-system is the Guix component
Dialogue: 0,0:00:16.76,0:00:21.38,Default,,0,0,0,,that turns the source distribution of any Python package out there
Dialogue: 0,0:00:21.38,0:00:23.38,Default,,0,0,0,,into an installable image
Dialogue: 0,0:00:23.38,0:00:27.78,Default,,0,0,0,,so, basically, a directory under the GNU store
Dialogue: 0,0:00:27.78,0:00:33.14,Default,,0,0,0,,let's have a look at tomli a quite simple Python package
Dialogue: 0,0:00:33.14,0:00:36.08,Default,,0,0,0,,which has no external dependencies
Dialogue: 0,0:00:36.08,0:00:41.12,Default,,0,0,0,,so, if we import it using `guix import`, here
Dialogue: 0,0:00:41.12,0:00:45.94,Default,,0,0,0,,add a few imports to the resulting file
Dialogue: 0,0:00:45.94,0:00:47.94,Default,,0,0,0,,and then try to build it
Dialogue: 0,0:00:47.94,0:00:51.12,Default,,0,0,0,,it should work out of the box, right?
Dialogue: 0,0:00:51.12,0:00:53.84,Default,,0,0,0,,The problem is, it does not!
Dialogue: 0,0:00:53.84,0:00:57.38,Default,,0,0,0,,and instead we see at the bottom
Dialogue: 0,0:00:57.38,0:01:03.20,Default,,0,0,0,,an error message saying "no setup.py found"
Dialogue: 0,0:01:03.20,0:01:07.64,Default,,0,0,0,,and, indeed, if we look at the source distribution
Dialogue: 0,0:01:07.64,0:01:11.32,Default,,0,0,0,,there is no `setup.py` to be found anywhere
Dialogue: 0,0:01:11.32,0:01:13.86,Default,,0,0,0,,So, what is going on?
Dialogue: 0,0:01:13.86,0:01:18.26,Default,,0,0,0,,Fortunately, this package is already available in Guix
Dialogue: 0,0:01:18.26,0:01:20.26,Default,,0,0,0,,as `python-tomli`
Dialogue: 0,0:01:20.26,0:01:22.64,Default,,0,0,0,,so we can have a look at its definition
Dialogue: 0,0:01:22.64,0:01:24.64,Default,,0,0,0,,to see how it is currently built
Dialogue: 0,0:01:24.64,0:01:26.64,Default,,0,0,0,,let's just do that
Dialogue: 0,0:01:26.64,0:01:30.68,Default,,0,0,0,,looking at the build system's arguments
Dialogue: 0,0:01:30.68,0:01:35.92,Default,,0,0,0,,we see the phases `build` here and `install` here
Dialogue: 0,0:01:37.36,0:01:42.04,Default,,0,0,0,,which are usually provided by Python build system
Dialogue: 0,0:01:42.04,0:01:44.66,Default,,0,0,0,,replaced with custom code
Dialogue: 0,0:01:44.66,0:01:48.36,Default,,0,0,0,,I'm only showing the interesting parts here
Dialogue: 0,0:01:48.36,0:01:52.10,Default,,0,0,0,,the actual commands are actually much longer
Dialogue: 0,0:01:52.10,0:01:55.50,Default,,0,0,0,,first the build phase
Dialogue: 0,0:01:55.50,0:01:59.60,Default,,0,0,0,,uses a Python module called `build`
Dialogue: 0,0:01:59.60,0:02:03.80,Default,,0,0,0,,to build the wheel, as we can see here
Dialogue: 0,0:02:03.80,0:02:12.14,Default,,0,0,0,,the wheel is basically a distribution format in the Python world
Dialogue: 0,0:02:12.14,0:02:16.54,Default,,0,0,0,,then in the install phase
Dialogue: 0,0:02:16.54,0:02:20.50,Default,,0,0,0,,we simply use a well known tool called Pip
Dialogue: 0,0:02:20.50,0:02:25.06,Default,,0,0,0,,to install the wheel that we just built
Dialogue: 0,0:02:25.06,0:02:31.04,Default,,0,0,0,,into the output which would be somewhere around the GNU store
Dialogue: 0,0:02:31.04,0:02:35.58,Default,,0,0,0,,so how does the build module knows what to do
Dialogue: 0,0:02:35.58,0:02:37.58,Default,,0,0,0,,what to build?
Dialogue: 0,0:02:37.58,0:02:41.84,Default,,0,0,0,,it follows PEP 517
Dialogue: 0,0:02:41.84,0:02:45.48,Default,,0,0,0,,PEPs are kind of the RFCs of the Python world
Dialogue: 0,0:02:45.48,0:02:52.78,Default,,0,0,0,,and this PEP basically splits building wheels into two parts
Dialogue: 0,0:02:52.78,0:02:54.78,Default,,0,0,0,,a frontend and a backend
Dialogue: 0,0:02:54.78,0:02:57.72,Default,,0,0,0,,the frontend is the user facing part
Dialogue: 0,0:02:57.72,0:03:00.94,Default,,0,0,0,,for example the `build` we just saw here
Dialogue: 0,0:03:00.94,0:03:04.78,Default,,0,0,0,,this is the user facing part of the build process
Dialogue: 0,0:03:04.78,0:03:06.78,Default,,0,0,0,,and then a backend
Dialogue: 0,0:03:06.78,0:03:16.84,Default,,0,0,0,,the frontend is supposed to read a file called `pyproject.toml`
Dialogue: 0,0:03:16.84,0:03:18.84,Default,,0,0,0,,this is what we are seeing here
Dialogue: 0,0:03:18.84,0:03:25.70,Default,,0,0,0,,and in that TOML file, a section called `build-system`
Dialogue: 0,0:03:25.70,0:03:27.30,Default,,0,0,0,,this one here
Dialogue: 0,0:03:27.30,0:03:31.48,Default,,0,0,0,,declares which backend will actually build our wheel
Dialogue: 0,0:03:31.48,0:03:36.98,Default,,0,0,0,,and, in this case, another package called `flit_core`
Dialogue: 0,0:03:36.98,0:03:42.68,Default,,0,0,0,,its requirements a build time dependency of tomli
Dialogue: 0,0:03:42.68,0:03:46.96,Default,,0,0,0,,and its module `flit_core.buildapi`
Dialogue: 0,0:03:46.96,0:03:50.46,Default,,0,0,0,,is providing us with the build entrypoints.
Dialogue: 0,0:03:50.46,0:03:55.12,Default,,0,0,0,,The file also contains standardize metadata
Dialogue: 0,0:03:55.12,0:03:57.74,Default,,0,0,0,,and tool related configuration data
Dialogue: 0,0:03:57.74,0:03:59.74,Default,,0,0,0,,which I'm not showing here
Dialogue: 0,0:03:59.74,0:04:06.74,Default,,0,0,0,,A PEP 517* compatible build backend
Dialogue: 0,0:04:06.74,0:04:10.56,Default,,0,0,0,,provides a standard function entrypoint
Dialogue: 0,0:04:10.56,0:04:12.56,Default,,0,0,0,,called `build_wheel`
Dialogue: 0,0:04:12.56,0:04:17.00,Default,,0,0,0,,in the module I just referenced here in the top
Dialogue: 0,0:04:17.00,0:04:21.42,Default,,0,0,0,,and, if we call it, it will just do its magic
Dialogue: 0,0:04:21.42,0:04:24.06,Default,,0,0,0,,and it will produce a wheel file
Dialogue: 0,0:04:24.06,0:04:28.30,Default,,0,0,0,,and its first argument it's the wheel directory
Dialogue: 0,0:04:28.30,0:04:32.22,Default,,0,0,0,,that wheel is basically a zip file
Dialogue: 0,0:04:32.22,0:04:33.94,Default,,0,0,0,,with a predefined structure
Dialogue: 0,0:04:33.94,0:04:37.30,Default,,0,0,0,,that we can extract into our store
Dialogue: 0,0:04:37.30,0:04:40.12,Default,,0,0,0,,and we are almost done
Dialogue: 0,0:04:40.12,0:04:44.76,Default,,0,0,0,,and this is what Pip does in the install phase here
Dialogue: 0,0:04:50.36,0:04:54.48,Default,,0,0,0,,and that's basically the entire build process
Dialogue: 0,0:04:54.48,0:04:57.42,Default,,0,0,0,,as specified by PEP 517
Dialogue: 0,0:04:57.42,0:05:00.50,Default,,0,0,0,,there is no `setup.py` involved any more
Dialogue: 0,0:05:00.50,0:05:01.50,Default,,0,0,0,,we don't have to call it
Dialogue: 0,0:05:01.50,0:05:06.78,Default,,0,0,0,,we don't have to create it as a package provider
Dialogue: 0,0:05:08.16,0:05:14.08,Default,,0,0,0,,so the reason why the error message I showed, showed up earlier
Dialogue: 0,0:05:14.08,0:05:16.46,Default,,0,0,0,,will keep on poping up more and more
Dialogue: 0,0:05:16.52,0:05:19.34,Default,,0,0,0,,is pretty simple we are late!
Dialogue: 0,0:05:19.34,0:05:22.46,Default,,0,0,0,,we are really really late, actually
Dialogue: 0,0:05:22.46,0:05:28.72,Default,,0,0,0,,because PEP 517 was originally created in 2015
Dialogue: 0,0:05:28.72,0:05:34.14,Default,,0,0,0,,and that it gained provisional acceptance in 2017
Dialogue: 0,0:05:34.14,0:05:36.14,Default,,0,0,0,,and just last year,
Dialogue: 0,0:05:36.14,0:05:42.74,Default,,0,0,0,,after being basically being the *de facto* successor of `setup.py` for some time
Dialogue: 0,0:05:42.74,0:05:46.32,Default,,0,0,0,,it has been finalized and fully accepted
Dialogue: 0,0:05:46.32,0:05:53.18,Default,,0,0,0,,and more importantly, flit which you remember from the previous slide
Dialogue: 0,0:05:53.18,0:05:57.26,Default,,0,0,0,,is also able to create source distributions
Dialogue: 0,0:05:57.26,0:05:59.54,Default,,0,0,0,,and upload them to PyPI
Dialogue: 0,0:05:59.54,0:06:03.80,Default,,0,0,0,,Python's public package repository basically
Dialogue: 0,0:06:03.80,0:06:08.08,Default,,0,0,0,,so far, it has been generating a `setup.py`
Dialogue: 0,0:06:08.08,0:06:10.88,Default,,0,0,0,,and does nobody really noticed
Dialogue: 0,0:06:10.88,0:06:17.74,Default,,0,0,0,,but since version 3.5, which was released in November 2021
Dialogue: 0,0:06:17.74,0:06:21.88,Default,,0,0,0,,flit stop doing that by default
Dialogue: 0,0:06:21.88,0:06:26.98,Default,,0,0,0,,and thus we are seeing more and more packages without `setup.py`
Dialogue: 0,0:06:26.98,0:06:28.98,Default,,0,0,0,,in their source distributions
Dialogue: 0,0:06:28.98,0:06:33.78,Default,,0,0,0,,and so we are basically unable to build this projects right now
Dialogue: 0,0:06:33.78,0:06:35.78,Default,,0,0,0,,or this Python modules
Dialogue: 0,0:06:35.78,0:06:40.98,Default,,0,0,0,,a look at the Guix's repository in late January
Dialogue: 0,0:06:40.98,0:06:45.26,Default,,0,0,0,,and back then only 11 packages actually used Pip
Dialogue: 0,0:06:45.26,0:06:47.84,Default,,0,0,0,,or PyPA build as we've seen
Dialogue: 0,0:06:47.84,0:06:51.80,Default,,0,0,0,,but I think our ecosystem is quite old
Dialogue: 0,0:06:51.80,0:06:56.68,Default,,0,0,0,,with about half the packages not being the latest versions available upstream
Dialogue: 0,0:06:56.68,0:06:58.68,Default,,0,0,0,,according to `guix refresh`
Dialogue: 0,0:06:58.68,0:07:05.56,Default,,0,0,0,,so it's possible that more packages actually require support for this `pyproject.toml`
Dialogue: 0,0:07:05.56,0:07:10.40,Default,,0,0,0,,and we simply have not updated them yet for whatever reason
Dialogue: 0,0:07:10.40,0:07:14.82,Default,,0,0,0,,maybe because it's too difficult or nobody poped to do it yet
Dialogue: 0,0:07:16.82,0:07:22.16,Default,,0,0,0,,they are also more issues with our current Python build system
Dialogue: 0,0:07:22.16,0:07:29.74,Default,,0,0,0,,for instance `setup.py` test ???? has been deprecated for quite some time
Dialogue: 0,0:07:29.74,0:07:33.14,Default,,0,0,0,,since 2019 actually
Dialogue: 0,0:07:33.14,0:07:39.12,Default,,0,0,0,,and the default Python build system's default check phase relies on that
Dialogue: 0,0:07:39.12,0:07:42.72,Default,,0,0,0,,even though it doesn't work for a lot of packages right now
Dialogue: 0,0:07:42.72,0:07:47.12,Default,,0,0,0,,and thus, almost every package in our repository
Dialogue: 0,0:07:47.12,0:07:52.12,Default,,0,0,0,,needs to replace that check phase with a call to Pytest
Dialogue: 0,0:07:52.12,0:07:56.40,Default,,0,0,0,,which is the *de facto* standard right now for executing tests
Dialogue: 0,0:07:56.40,0:08:03.02,Default,,0,0,0,,???? tox does something similar but not really quite the same
Dialogue: 0,0:08:03.02,0:08:08.68,Default,,0,0,0,,another long standing issue is wrapping of Python path
Dialogue: 0,0:08:08.68,0:08:10.68,Default,,0,0,0,,or Guix Python path now
Dialogue: 0,0:08:10.68,0:08:13.52,Default,,0,0,0,,because it includes native inputs
Dialogue: 0,0:08:13.52,0:08:16.94,Default,,0,0,0,,and thus propagates way too many packages
Dialogue: 0,0:08:16.94,0:08:22.30,Default,,0,0,0,,and I'm sure we can find more issues with the current Python build system
Dialogue: 0,0:08:22.30,0:08:34.40,Default,,0,0,0,,thus, for almost a year now, I've been working on a modernized Python build system
Dialogue: 0,0:08:34.40,0:08:37.00,Default,,0,0,0,,that addresses some of this issues
Dialogue: 0,0:08:37.00,0:08:41.68,Default,,0,0,0,,unfortunately I haven't had much time lately to improve it for other
Dialogue: 0,0:08:41.68,0:08:43.58,Default,,0,0,0,,I hope to pick it up again
Dialogue: 0,0:08:43.58,0:08:44.96,Default,,0,0,0,,and get it into shape
Dialogue: 0,0:08:44.96,0:08:46.96,Default,,0,0,0,,and get it to master at some point
Dialogue: 0,0:08:46.96,0:08:48.96,Default,,0,0,0,,you can read about it here
Dialogue: 0,0:08:48.96,0:08:54.48,Default,,0,0,0,,in Guix issue #46848
Dialogue: 0,0:08:54.48,0:09:00.88,Default,,0,0,0,,or you can just pull a branch called `wip-python-pep517`
Dialogue: 0,0:09:00.88,0:09:03.56,Default,,0,0,0,,it's also build by the CI by the way
Dialogue: 0,0:09:03.56,0:09:07.70,Default,,0,0,0,,so you don't have to do the whole world rebuild by yourself
Dialogue: 0,0:09:07.70,0:09:12.62,Default,,0,0,0,,and it's using exactly the method I described above
Dialogue: 0,0:09:12.92,0:09:20.66,Default,,0,0,0,,the part written in Guile ends up reading the project's `pyproject.toml`
Dialogue: 0,0:09:20.66,0:09:28.58,Default,,0,0,0,,and then calls the build system's entrypoint `build_wheel` as we can see here
Dialogue: 0,0:09:28.58,0:09:34.84,Default,,0,0,0,,and if there's no `pyproject.toml` it's also not a problem actually
Dialogue: 0,0:09:34.84,0:09:40.10,Default,,0,0,0,,because setuptools, which is still provided by default,
Dialogue: 0,0:09:40.10,0:09:43.10,Default,,0,0,0,,and still included in the build system by default,
Dialogue: 0,0:09:43.10,0:09:49.26,Default,,0,0,0,,because setuptools provides a PEP 517 build backend
Dialogue: 0,0:09:51.16,0:09:57.00,Default,,0,0,0,,which simply uses the existing `setup.py` behind the curtains
Dialogue: 0,0:09:57.00,0:10:02.68,Default,,0,0,0,,so we don't have to, like provide extra logic for that
Dialogue: 0,0:10:04.30,0:10:09.00,Default,,0,0,0,,they were actually other options for this build phase
Dialogue: 0,0:10:09.00,0:10:11.00,Default,,0,0,0,,in how to do it basically
Dialogue: 0,0:10:12.14,0:10:15.36,Default,,0,0,0,,for example, we could also use the build module
Dialogue: 0,0:10:15.36,0:10:16.98,Default,,0,0,0,,that we have seen above
Dialogue: 0,0:10:16.98,0:10:19.96,Default,,0,0,0,,and some packages use currently
Dialogue: 0,0:10:19.96,0:10:23.50,Default,,0,0,0,,however that raises questions about bootstraping, like
Dialogue: 0,0:10:23.50,0:10:28.10,Default,,0,0,0,,how do you build the build package if it essentially requires itself
Dialogue: 0,0:10:28.10,0:10:31.70,Default,,0,0,0,,because the build module is part of the Python build system
Dialogue: 0,0:10:31.70,0:10:34.74,Default,,0,0,0,,and of of it's dependency
Dialogue: 0,0:10:34.74,0:10:38.76,Default,,0,0,0,,tomli for example is one of the dependency of build
Dialogue: 0,0:10:38.76,0:10:42.10,Default,,0,0,0,,and what do we do with tomli
Dialogue: 0,0:10:42.10,0:10:44.90,Default,,0,0,0,,like how do we install it without having build
Dialogue: 0,0:10:44.90,0:10:49.26,Default,,0,0,0,,so this is tricky, and I tried it in the beginning
Dialogue: 0,0:10:49.26,0:10:51.02,Default,,0,0,0,,and it works
Dialogue: 0,0:10:51.02,0:10:55.28,Default,,0,0,0,,but we need to do like some weird manual copying
Dialogue: 0,0:10:55.28,0:10:58.46,Default,,0,0,0,,and that's not really pretty to be honest
Dialogue: 0,0:10:58.46,0:11:00.78,Default,,0,0,0,,we could also use Pip
Dialogue: 0,0:11:00.78,0:11:04.12,Default,,0,0,0,,which is usually distributed with Python
Dialogue: 0,0:11:04.12,0:11:09.16,Default,,0,0,0,,also our python package bundles Pip and setuptools
Dialogue: 0,0:11:09.16,0:11:12.36,Default,,0,0,0,,this has caused a few issues in the past
Dialogue: 0,0:11:12.36,0:11:14.70,Default,,0,0,0,,because we cannot update either of them
Dialogue: 0,0:11:14.70,0:11:17.78,Default,,0,0,0,,for provide alternative newer versions
Dialogue: 0,0:11:17.78,0:11:22.18,Default,,0,0,0,,without having a full world rebuild basically
Dialogue: 0,0:11:22.18,0:11:25.40,Default,,0,0,0,,and also Pip being a massive project
Dialogue: 0,0:11:25.40,0:11:28.38,Default,,0,0,0,,it bundles quite a few of it's dependencies
Dialogue: 0,0:11:28.38,0:11:30.72,Default,,0,0,0,,and I feel "we" as a Guix project
Dialogue: 0,0:11:30.72,0:11:34.66,Default,,0,0,0,,should try to untangle that mess whenever we can
Dialogue: 0,0:11:34.66,0:11:36.66,Default,,0,0,0,,and in this case, we actually can
Dialogue: 0,0:11:36.66,0:11:42.22,Default,,0,0,0,,so, in conclusion, I think running this 4 lines of Python code
Dialogue: 0,0:11:42.22,0:11:44.22,Default,,0,0,0,,that you can see here
Dialogue: 0,0:11:45.88,0:11:48.82,Default,,0,0,0,,is a much simpler solution
Dialogue: 0,0:11:48.82,0:11:51.98,Default,,0,0,0,,than trying to bootstrap the build module,
Dialogue: 0,0:11:51.98,0:12:00.24,Default,,0,0,0,,and tomli and all the modules required to get a Python based builder basically
Dialogue: 0,0:12:00.24,0:12:03.02,Default,,0,0,0,,we can just do it in 4 lines of code
Dialogue: 0,0:12:06.52,0:12:09.76,Default,,0,0,0,,there are a few changes for packagers
Dialogue: 0,0:12:09.76,0:12:13.34,Default,,0,0,0,,which aime to reduce the need for custom phases
Dialogue: 0,0:12:13.34,0:12:22.66,Default,,0,0,0,,first, it is possible to override the build backend detected from the `pyproject.toml`
Dialogue: 0,0:12:22.66,0:12:27.00,Default,,0,0,0,,especially early adopters of PEP 517
Dialogue: 0,0:12:27.00,0:12:30.94,Default,,0,0,0,,use modules which have been renamed in the meantime
Dialogue: 0,0:12:32.66,0:12:36.08,Default,,0,0,0,,I think both Poetry and Flit did that at some point
Dialogue: 0,0:12:36.08,0:12:40.62,Default,,0,0,0,,and they split their build logic into a separate project
Dialogue: 0,0:12:40.62,0:12:42.62,Default,,0,0,0,,and renamed the backend
Dialogue: 0,0:12:42.62,0:12:48.72,Default,,0,0,0,,so, for some packages, it's necessary to override this detected build backend
Dialogue: 0,0:12:48.72,0:12:52.60,Default,,0,0,0,,and you can do that simply by specifying the `#:build-backend` option
Dialogue: 0,0:12:52.60,0:12:56.30,Default,,0,0,0,,and also if the detection is wrong at some point
Dialogue: 0,0:12:56.30,0:13:00.16,Default,,0,0,0,,we can just override it without causing too much damage
Dialogue: 0,0:13:01.10,0:13:05.68,Default,,0,0,0,,secondly, we have `#:configure-flags`
Dialogue: 0,0:13:05.98,0:13:10.46,Default,,0,0,0,,they have existed before, but where applied only during install
Dialogue: 0,0:13:10.46,0:13:12.14,Default,,0,0,0,,but not during the build phase
Dialogue: 0,0:13:12.14,0:13:15.30,Default,,0,0,0,,which, I believe, is a ????, we'll see
Dialogue: 0,0:13:15.30,0:13:20.06,Default,,0,0,0,,and this are kind of arguments to `setup.py`
Dialogue: 0,0:13:20.14,0:13:23.64,Default,,0,0,0,,but their syntax depends on the build backend unfortunately
Dialogue: 0,0:13:23.64,0:13:27.80,Default,,0,0,0,,so sometimes you can pass the arguments directly
Dialogue: 0,0:13:27.80,0:13:31.46,Default,,0,0,0,,sometimes you need to pass them as an option to another option
Dialogue: 0,0:13:31.46,0:13:33.32,Default,,0,0,0,,like we can see here
Dialogue: 0,0:13:33.32,0:13:37.28,Default,,0,0,0,,I think that's the setuptools specific way to do it
Dialogue: 0,0:13:37.28,0:13:39.54,Default,,0,0,0,,so you have an option
Dialogue: 0,0:13:39.54,0:13:44.48,Default,,0,0,0,,and then you have to pass the actual option as an argument to that option
Dialogue: 0,0:13:44.48,0:13:50.60,Default,,0,0,0,,and, in theory, they could also look completly different
Dialogue: 0,0:13:50.60,0:13:53.98,Default,,0,0,0,,like with no relationship to any command line flag
Dialogue: 0,0:13:53.98,0:13:58.94,Default,,0,0,0,,it's just legacy code/stuff right now
Dialogue: 0,0:13:58.94,0:14:00.50,Default,,0,0,0,,and it's not pretty
Dialogue: 0,0:14:00.50,0:14:05.06,Default,,0,0,0,,and it's one of the things that have been criticized about PEP 517
Dialogue: 0,0:14:05.06,0:14:10.78,Default,,0,0,0,,this configure flags are not really specified how they are supposed to look
Dialogue: 0,0:14:10.78,0:14:16.18,Default,,0,0,0,,but this solution exists, so we have to swim with the flow, kind of
Dialogue: 0,0:14:17.00,0:14:20.96,Default,,0,0,0,,and, finally, they are `#:test-flags`
Dialogue: 0,0:14:20.96,0:14:28.48,Default,,0,0,0,,my proposal was to auto detect test binary based on the package's native inputs
Dialogue: 0,0:14:28.48,0:14:32.92,Default,,0,0,0,,so if pytest is available, it will just use pytest
Dialogue: 0,0:14:32.92,0:14:35.74,Default,,0,0,0,,if nose is available, it will use nose
Dialogue: 0,0:14:35.74,0:14:37.16,Default,,0,0,0,,and so on
Dialogue: 0,0:14:37.16,0:14:43.22,Default,,0,0,0,,now, test flags override the default arguments passed to this test command
Dialogue: 0,0:14:43.22,0:14:47.54,Default,,0,0,0,,and, hopefully, this can reduce the number of custom phases required
Dialogue: 0,0:14:47.54,0:14:49.70,Default,,0,0,0,,to exclude tests mostly
Dialogue: 0,0:14:49.70,0:14:55.82,Default,,0,0,0,,in this case, we are simply don't want to run the `test_legacy_callbacks` function
Dialogue: 0,0:14:55.82,0:14:59.46,Default,,0,0,0,,and so we can just exclude it with the flags
Dialogue: 0,0:15:01.38,0:15:05.72,Default,,0,0,0,,and the package will build just fine, probably, hopefully
Dialogue: 0,0:15:05.72,0:15:09.92,Default,,0,0,0,,so, where do we go from here?
Dialogue: 0,0:15:09.92,0:15:16.08,Default,,0,0,0,,the first step would be to finalize the work on the Python build system
Dialogue: 0,0:15:16.08,0:15:23.16,Default,,0,0,0,,and get the branch ready into a state where it can be merged
Dialogue: 0,0:15:23.16,0:15:28.50,Default,,0,0,0,,unfortunately, right now, quite a few packages fail to build there
Dialogue: 0,0:15:28.50,0:15:34.04,Default,,0,0,0,,because their previously disabled test suite fails
Dialogue: 0,0:15:34.04,0:15:37.36,Default,,0,0,0,,not necessarily disabled, but it was never ran
Dialogue: 0,0:15:37.36,0:15:43.92,Default,,0,0,0,,because the test runner was buggy and the `setup.py test` command simply did nothing
Dialogue: 0,0:15:43.92,0:15:46.24,Default,,0,0,0,,and didn't detect edge cases
Dialogue: 0,0:15:46.24,0:15:51.38,Default,,0,0,0,,we could also go a different route and add a new build system
Dialogue: 0,0:15:51.38,0:15:53.22,Default,,0,0,0,,without touching the old one
Dialogue: 0,0:15:53.22,0:15:57.80,Default,,0,0,0,,and just slowly migrate every Python package we have over
Dialogue: 0,0:15:57.80,0:15:59.90,Default,,0,0,0,,instead of doing the big switch at once
Dialogue: 0,0:16:00.02,0:16:03.30,Default,,0,0,0,,that was also one of the suggestions that was certainly possible
Dialogue: 0,0:16:03.30,0:16:05.76,Default,,0,0,0,,I have explored that it would work
Dialogue: 0,0:16:05.76,0:16:14.98,Default,,0,0,0,,as I mentioned earlier, currently the native inputs end up in wrapped binaries
Dialogue: 0,0:16:14.98,0:16:23.38,Default,,0,0,0,,and I think this is one thing we should address too while doing the whole world rebuild anyway
Dialogue: 0,0:16:23.38,0:16:28.36,Default,,0,0,0,,but I haven't done any work on that yet, unfortunately
Dialogue: 0,0:16:28.36,0:16:34.98,Default,,0,0,0,,and finally, the pypi import needs support to read `pyproject.toml`
Dialogue: 0,0:16:34.98,0:16:38.74,Default,,0,0,0,,not so much for the runtime dependencies,
Dialogue: 0,0:16:38.74,0:16:43.88,Default,,0,0,0,,this is fine, we have another metadata file which contains the runtime dependencies
Dialogue: 0,0:16:43.88,0:16:46.76,Default,,0,0,0,,but for build time and test dependencies
Dialogue: 0,0:16:46.76,0:16:49.84,Default,,0,0,0,,because they are not included in the wheel file,
Dialogue: 0,0:16:49.84,0:16:55.80,Default,,0,0,0,,there's a file called metadata, and it doesn't have this data
Dialogue: 0,0:16:55.80,0:17:03.78,Default,,0,0,0,,so right now, if you import a `pyproject.toml` based project
Dialogue: 0,0:17:03.78,0:17:08.70,Default,,0,0,0,,you will lack the build dependencies basically
Dialogue: 0,0:17:08.70,0:17:13.98,Default,,0,0,0,,for example flit will not be in the native inpust right now
Dialogue: 0,0:17:13.98,0:17:19.36,Default,,0,0,0,,and that's it for me for now
Dialogue: 0,0:17:19.36,0:17:24.60,Default,,0,0,0,,I hope to answer some of your questions during the live Q and A
Dialogue: 0,0:17:24.60,0:17:30.38,Default,,0,0,0,,and in the PDF version of this slides, you'll find some clickable links bellow
Dialogue: 0,0:17:30.38,0:17:36.34,Default,,0,0,0,,to different talks and to the standards I have referenced
Dialogue: 0,0:17:36.34,0:17:38.34,Default,,0,0,0,,so, I hope I'll see you soon!