Welcome to Bpipe

Bpipe provides a platform for running big bioinformatics jobs that consist of a series of processing stages - known as 'pipelines'.

Bpipe has been published in Bioinformatics! If you use Bpipe, please cite:

Sadedin S, Pope B & Oshlack A, Bpipe: A Tool for Running and Managing Bioinformatics Pipelines, Bioinformatics

Why Bpipe?

Many people working with bioinformatics data end up running jobs as shell scripts. While this makes running them easy it has a lot of limitations. For example, when scripts fail half way through it is often hard to tell where, or why they failed, and even harder to restart the job from the point of failure. There is no automatic log of the commands executed or automatic capture of console output to ensure it is possible to later on see what happened. Sometimes jobs fail half way through, leaving half created files that can get confused with good files. Modifying the pipeline is also time consuming and error prone - adding or removing a stage causes modifications in multiple places, meaning that it is all too easy to have a change of file name cause later commands to fail or worse, run on incorrect data. Bpipe tries to solve all these problems (and more!) while departing as little as possible from the simplicity of the shell script. In fact, your Bpipe scripts will often look very similar to the original shell scripts you might have started with.

By turning your shell scripts into Bpipe scripts, here are some of the features you can get:

Take a look at the Overview to see Bpipe in action, work through the Basic Tutorial for simple first steps, see a step by step example of a realistic pipeline made using Bpipe, or take a look at the Reference to see all the documentation.