Changes for Bpipe Version 0.9.9.8
Download: bpipe-0.9.9.8.tar.gz
-
Feature: options statement for easy parsing of script specific command line arguments
-
Add workaround to fix possible failure on old pipelines checking file existence
-
BUG: Fix spurious error printed if broken symbolic link present
-
BUG: prevent agent launched commands hanging due to if bpipe prompts for information
-
Feature: add fileSystemSyncTimeMs which allows for customizable delay in file system sync before failing pipeline on missing output
-
BUG: fix regression - transform using regex does not substitute groups
-
eg: transform('(.*)_R1.fastq') to("$1.txt") { ... }
-
Many fixes / optimisations to GCloud usage:
-
make workingDirectory specified for stage automatically placed within mounted storage
-
inherit region from main config inside filesystems, so it does not need to be double specified
-
support preemptible flag passed to google cloud instances
-
add sha1 hash of pipeline path / host to instanceid to make it unique b/w pipelines
-
fix region configured for cloud executor overriding region specified with -L
-
search for root input files in all filesystems, not only those configured as storages for stages
-
fix zone flag missing from gcloud commands, don't die mounting storage if instance not created
-
support using split / partitioned regions in cloud provider storage
-
BUG: Don't start executor pools when user is only testing the pipeline
-
BUG: ensure that executors clean up even if they error in stop command
-
Fix table output format so it shows correctly in GitHub markdown
-
Better behavior for bpipe test with send, and send for missing outputs
-
BUG: input patterns with dir in the pattern causing error / exception
-
Feature: support for reply -to in JMS bpipe agent
-
If the reply-to or JMSReplyTo header is set, the bpipe agent will now send a message containing the original command and pipeline success / fail to specified reply-to address (allows eg: camel inOut routing)
-
BUG: Fix inverted logic causing incorrect detection of file existence when file is out of sync, causing file systems with high latency to fail sporadically
-
Better tolerance of file system sync issues for files referenced using file(...)
-
BUG:
from
sometimes failing to ensure specified input is resolved in preference to other available inputs preceding it in the pipeline -
BUG: from sometimes selecting incorrect input in complex scenarios
-
BUG: fix file system watchers still created even if poller watcher specified, hence still resulting in file watcher exhaustion errors
-
BUG: -d option not respected in some situations, incorrect output dir may be used if default output already referenced
-
Feature: Support for Groovy 2.5.6 via optional gradle.properties setting
-
Feature: support for template dirs created by bpipe agent with incrementing values. Directories containing {inc} will be named with auto-incrementing counter.
-
Feature: 'terminateOnError' flag in send attributes causes notification failure as to be treated as error
-
BUG: improve unfriendly / incorrect behavior when duplicate mappings exist in transform
-
BUG: checks sometimes re-executing when not necessary
-
Feature:
bpipe touch
command updates files from current run of pipeline to new timestamps so they do not re-run onbpipe run
.