From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kyle Meyer Subject: Re: Preparing for a new release In-Reply-To: References: <87h801p818.fsf@elephly.net> <87h800u84z.fsf@kyleam.com> <87a75spx3i.fsf@elephly.net> <877e0vq5iy.fsf@elephly.net> Date: Mon, 10 Feb 2020 01:25:44 +0000 Message-ID: <87tv3z2pyf.fsf@kyleam.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable To: zimoun , Ricardo Wurmus Cc: gwl-devel@gnu.org List-ID: zimoun writes: > On Sun, 9 Feb 2020 at 14:01, Ricardo Wurmus wrote: > >> * inputs and outputs are not validated >> >> When a process declares that it produces an output, but then doesn=E2= =80=99t do >> that, the next process will fail with a nasty error message. This is >> especially nasty when using containerization as the error is about >> failing to map the input into the container. >> >> Processes should automatically validate their inputs and outputs. >> Since inputs and outputs could technically be something other than >> files I=E2=80=99m not sure exactly how to do this. > > From my understanding, Snakemake uses only files as inputs/outputs. > But I do not know what happens if 'rule 1' claims to output 'file' and > 'rule 2' says its input is 'file' and then 'rule 1' never produces > this file 'file'; the 'rule 2' is never processed, I guess. Hum? need > to check... In that case, an exception is raised: Waiting at most 5 seconds for missing files. MissingOutputException in line 2 of /tmp/snakemake-hAlqJ/Snakefile: Missing files after 5 seconds: foo This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait. Shutting down, this might take some time.