Scipipe on Living Systems_

SciPipe used at NASA Glenn Research Center

Sat, 13 Apr 2024 12:00:00 +0200

I was happy to see the publication finally going online , of work done at NASA Glenn Research Center , where SciPipe has been used to process and track provenance of the analyses, “Modeling the impact of thoracic pressure on intracranial pressure”. I’ve known the work existed for a couple of years, after getting some extraordinarily useful contributions from Drayton fixing some bugs I’m not sure I’d ever find otherwise, but cool to now also see it published! Also a big kudos for acknowledging the tool in the paper. Not all that common to do, but a gesture that is deeply appreciated.

SciPipe paper published in GigaScience

Sat, 27 Apr 2019 14:48:00 +0200

We just wanted to share that the paper on our Go-based workflow library, SciPipe, was just published in GigaScience:

Abstract

Background

The complex nature of biological data has driven the development of specialized software tools. Scientific workflow management systems simplify the assembly of such tools into pipelines, assist with job automation, and aid reproducibility of analyses. Many contemporary workflow tools are specialized or not designed for highly complex workflows, such as with nested loops, dynamic scheduling, and parametrization, which is common in, e.g., machine learning.

Preprint on SciPipe - Go-based scientific workflow library

Thu, 02 Aug 2018 01:01:00 +0200

A pre-print for our Go-based workflow libarary SciPipe , is out, with the title SciPipe - A workflow library for agile development of complex and dynamic bioinformatics pipelines , co-authored by me and colleagues at pharmb.io : Martin Dahlö , Jonathan Alvarsson and Ola Spjuth . Access it here .

It has been more than three years since the first commit on the SciPipe Git repository in March, 2015, and development has been going in various degrees of intensity during these years, often besides other duties at pharmb.io and NBIS , and often at a lower pace than I might have wished. On the other hand, this might also have helped to let design ideas mature well before implementing them.

What is a scientific (batch) workflow?

Thu, 07 Dec 2017 00:57:00 +0100

Workflows and DAGs - Confusion about the concepts

Jörgen Brandt tweeted a comment that got me thinking again on something I’ve pondered a lot lately:

“A workflow is a DAG.” is really a weak definition. That’s like saying “A love letter is a sequence of characters.” representation ≠ meaning

– @joergenbr

Jörgen makes a good point. A Directed Acyclic Graph (DAG) does not by any means capture the full semantic content included in a computational workflow. I think Werner Gitt’s universal information model is highly relevant here, suggesting that information comes in at least five abstraction layers: statistics (signals, number of symbols), syntax (set of symbols, grammar), semantics (meaning), pragmatics (action), apobetics (purpose, result). A DAG seems to cover the syntax and semantics layers, leaving out three layers out of five.

First production run with SciPipe - A Go-based scientific workflow tool

Thu, 28 Sep 2017 19:32:00 +0200

Today marked the day when we ran the very first production workflow with SciPipe , the Go -based scientific workflow tool we’ve been working on over the last couple of years. Yay! :)

This is how it looked (no fancy GUI or such yet, sorry):

The first result we got in this very very first job was a list of counts of ligands (chemical compounds) in the ExcapeDB dataset (download here ) interacting with the 44 protein/gene targets identified by Bowes et al as a good baseline set for identifying hazardous side-effects effects in the body (that is, any chemical compounds binding these proteins, will never become an approved drug).