nf-core/configs: azurebatch
Azure BATCH Cloud Profile
nf-core/configs: azurebatch Configuration
To be used with the azurebatch profile by specifying the -profile azurebatch when running nf-core pipelines.
Custom queue and storage need to be supplied with params.az_location, params.batch_name, params.batch_key, params.storage_name, params.sas_token.
Required Parameters
--storage_name
Name of Azure blob storage account.
--storage_key
Access key to Azure blob storage account. --storage_key or --storage_sas are required, but not both.
--storage_sas
SAS for access to Azure blob storage account (see relevant permissions on Nextflow documentation). --storage_key or --storage_sas are required, but not both.
—storage_duration
The duration of the shared access signature token created by Nextflow when the --storage_sas option is not specified (default: 48h). Make sure to increase this for large and expensive workflows!
--az_location
The Azure Batch region where the computation is executed in VMs. Default (westus2).
--batch_name
The Azure Batch account name.
--batch_key
The Azure Batch account key.
-w
The Azure Blob container to be used as Nextflow work directory (-w az://work).
--vm_type
VM size to use with Nextflow autopool or when creating a worker pool in Azure Batch. Make sure your Azure account has sufficient quota. Defaults to Standard_D8s_v3. See Azure VM Size documentation for more information.
--autopoolmode
Whether to use Nextflow autopool mode which creates an autoscaling pool for running Nextflow jobs. Defaults to true.
--allowpoolcreation
Allow Nextflow to create a pool for running Nextflow jobs. Defaults to true.
--deletejobs
Allow Nextflow to delete pools after completion. Defaults to true.
--az_worker_pool
Select an existing pool by name to run Nextflow jobs on. Defaults to auto (matching --autopoolmode).
--acr_registry
URL to Azure container registry for private docker images.
--acr_username
Username to access private Azure container registry.
--acr_password
Password to access private Azure container registry.
Azure Batch Setup
Please refer to the Nextflow documentation which describe how to setup the Azure Batch environment.
Config file
//Nextflow config file for running on Azure batchparams { config_profile_description = 'Azure BATCH Cloud Profile' config_profile_contact = 'Venkat Malladi (@vsmalladi) & Adam Talbot (@adamrtalbot)' config_profile_url = 'https://azure.microsoft.com/services/batch/'
// Storage storage_name = null storage_key = null storage_sas = null storage_duration = '48h'
// Batch az_location = "westus2" batch_name = null batch_key = null
vm_type = "Standard_E*d_v5" autopoolmode = true allowpoolcreation = true deletejobs = true deletepools = true az_worker_pool = "auto"
// ACR acr_registry = null acr_username = null acr_password = null}
process { executor = "azurebatch"}
azure { process { queue = params.az_worker_pool machineType = params.vm_type } storage { accountName = params.storage_name accountKey = params.storage_key sasToken = params.storage_sas tokenDuration = params.storage_duration } batch { location = params.az_location accountName = params.batch_name accountKey = params.batch_key tokenDuration = "24h" autoPoolMode = params.autopoolmode allowPoolCreation = params.allowpoolcreation deleteJobsOnCompletion = params.deletejobs deletePoolsOnCompletion = params.deletepools pools { auto { autoScale = true vmCount = 1 maxVmCount = 12 } } } registry { server = params.acr_registry userName = params.acr_username password = params.acr_password }}