Bpipe Version 0.9.9.3
Summary
Bpipe 0.9.9.3 contains a large number of bug fixes that affect specific scenarios where unexpected behavior was being encountered. It also includes some major enhancements including the ability to "pause" and resume a pipeline without stopping in-progress jobs, and also support for pre-allocating resources when running pipelines in environments where resources can take time become available after they are requested. The new 'bpipe jobs' command is also worth a look, especially the 'bpipe jobs -watch' option.
Source code: https://github.com/ssadedin/bpipe/releases/tag/0.9.9.3
Highlights
Enhancements
-
Support for preallocated resources for commands and allocated compute resources that persist between pipeline invocations. See the documentation for more details.
-
Support for using Groovy Grab/Grape to add libraries to pipeline (see documentation)
-
Improved output to 'bpipe query'
-
Greatly enhanced, real time display for 'bpipe jobs'
-
More informative display from 'bpipe status'
-
Generic custom options for queuing system jobs (see #193)
-
Improved behavior of stop and status when no pipeline running
-
Support to 'pause' a running pipeline via 'bpipe pause'
-
Tools can be located relative to pipeline for purpose of version probing
-
Experimental support for inline python, sqlite and groovy commands
-
Support to remake specific files via bpipe remake
-
Ability to split genomes into arbitrary sized parts specified in bases
-
Support for syntax of form ${output.withFlag("--someFlag")}
Bugfixes
-
Fix torque executor printing spurious errors when stopping jobs
-
fix spurious error produced by torque module request
-
Fix #209 - torque mem param variable missing in some situations
-
Fix sbatch/qsub failures not recognised correctly, producing misleading errors
-
Fix: files in output dirs not cleaned up if command fails
-
Fix #199: Stop command not halting jobs on some clusters
-
Fix timeout not treated as failure under SLURM
-
Fix split pattern on symbolic links in directory not resolving
-
Fix $outputs.ext (multiple outputs based on extension) broken
-
Fix genomes lacking 'chr' prefix not supported by 'chr' command