https://livesys.se/tags/snakemake/ Recent content in Snakemake on Living Systems_ Hugo en-us Mon, 26 Oct 2015 21:23:00 +0100 https://livesys.se/posts/dynamic-workflow-scheduling/ Mon, 26 Oct 2015 21:23:00 +0100 https://livesys.se/posts/dynamic-workflow-scheduling/ <p><p class="image"> <img src="scheduling_unsplash-1.jpg" alt="" /> </p> </p> <p><em>Photo credits: <a href="https://unsplash.com/whale" target="_blank" rel="noopener">Matthew Smith</a> / <a href="https://unsplash.com/photos/OiiThC8Wf68" target="_blank" rel="noopener">Unsplash</a> </em></p> <p>In <a href="https://jcheminf.springeropen.com/articles/10.1186/s13321-016-0179-6" target="_blank" rel="noopener">our work on automating machine learning computations in cheminformatics with scientific workflow tools</a> , we have came to realize something; <em>Dynamic scheduling in scientific workflow tools is very important and sometimes badly needed.</em></p> <p>What I mean is that <em>new tasks should be able to be scheduled during the execution of a workflow, not just in its scheduling phase.</em></p> <p>What is striking is that far from all workflow tools allow this. Many tools completely separate the execution into two stages:</p>