Groovy syntax

Key Points

Getting Started with Nextflow	A workflow is a sequence of tasks that process a set of data. A workflow management system (WfMS) is a computational platform that provides an infrastructure for the set-up, execution and monitoring of workflows. Nextflow is a workflow management system that comprises both a runtime environment and a domain specific language (DSL). Nextflow scripts comprise of channels for controlling inputs and outputs, and processes for defining workflow tasks. You run a Nextflow script using the `nextflow run` command.
Nextflow scripting	Nextflow is a Domain Specific Language (DSL) implemented on top of the Groovy programming language. To define a variable, assign a value to it e.g., `a = 1` . Comments use the same syntax as in the C-family programming languages: `//` or multiline `/* /`. Multiple values can be stored in lists [value1, value2, value3, …] or maps [chromosome: 1, start :1]. Lists are indexed and sliced with square brackets (e.g., list[0] and list[2..9]) String interpolation (variable interpolation, variable substitution, or variable expansion) is the process of evaluating a string literal containing one or more placeholders, yielding a result in which the placeholders are replaced with their corresponding values. A closure is an expression (block of code) encased in `{}` e.g. `{ it it }`.
Workflow parameterization	Pipeline parameters are specified by prepending the prefix `params` to a variable name, separated by dot character. To specify a pipeline parameter on the command line for a Nextflow run use `--variable_name` syntax. You can add parameters to a JSON or YAML formatted file and pass them to the script using option `-params-file`.
Channels	Channels must be used to import data into Nextflow. Nextflow has two different kinds of channels: queue channels and value channels. Data in value channels can be used multiple times in workflow. Data in queue channels are consumed when they are used by a process or an operator. Channel factory methods, such as `Channel.of`, are used to create channels. Channel factory methods have optional parameters e.g., `checkIfExists`, that can be used to alter the creation and behaviour of a channel.
Processes	A Nextflow process is an independent step in a workflow Processes contain up to five definition blocks including: directives, inputs, outputs, when clause and finally a script block. The script block contains the commands you would like to run. A process should have a script but the other four blocks are optional Inputs are defined in the input block with a type qualifier and a name.
Processes Part 2	Outputs to a process are defined using the output blocks. You can group input and output data from a process using the tuple qualifer. The execution of a process can be controlled using the `when` declaration and conditional statements. Files produced within a process and defined as `output` can be saved to a directory using the `publishDir` directive.
Workflow	A Nextflow workflow is defined by invoking `processes` inside the `workflow` scope. A process is invoked like a function inside the `workflow` scope passing any required input parameters as arguments. e.g. `INDEX(transcriptome_ch)`. Process outputs can be accessed using the `out` attribute for the respective `process`. Multiple outputs from a single process can be accessed using the `[]` or output name.
Operators	Nextflow operators are methods that allow you to modify, set or view channels. Operators can be separated into the several groups; filtering, transforming, splitting, combining, forking and Maths operators To use an operator use the dot notation after the Channel object e.g. `my_ch.view()`. You can parse text items emitted by a channel, that are formatted using the CSV format, using the `splitCsv` operator.
Nextflow configuration	Nextflow configuration can be managed using a Nextflow configuration file. Nextflow configuration files are plain text files containing a set of properties. You can define process specific settings, such as cpus and memory, within the `process` scope. You can assign different resources to different processes using the process selectors `withName` or `withLabel`. You can define a profile for different configurations using the `profiles` scope. These profiles can be selected when launching a pipeline execution by using the `-profile` command-line option Nextflow configuration settings are evaluated in the order they are read-in. Workflow configuration settings can be inspected using `nextflow config <script> [options]`.
Simple RNA-Seq pipeline	Nextflow can combine tasks (processes) and manage data flows using channels into a single pipeline/workflow. A Workflow can be parameterised using `params`. These values of the parameters can be captured in a log file using `log.info` Nextflow can handle a workflow’s software requirements using several technologies including the `conda` package and enviroment manager. Workflow steps are connected via their `inputs` and `outputs` using `Channels`. Intermediate pipeline results can be transformed using Channel `operators` such as `combine`. Nextflow can execute an action when the pipeline completes the execution using the `workflow.onComplete` event handler to print a confirmation message. Nextflow is able to produce multiple reports and charts providing several runtime metrics and execution information using the command line options `-with-report`, `-with-trace`, `-with-timeline` and produce a graph using `-with-dag`.
Modules	A module file is a Nextflow script containing one or more `process` definitions that can be imported from another Nextflow script. To import a module into a workflow use the `include` keyword. A module script can define one or more parameters using the same syntax of a Nextflow workflow script. The module inherits the parameters defined before the include statement, therefore any further parameter set later is ignored.
Sub-workflows	Nextflow allows to define reusable sub-workflows. Sub-workflow allows the definition of workflow processes that can be included from any other script and invoked as a custom function within the new workflow scope. This enables reuse of workflow components The `entry` option of the nextflow `run` command specifies the workflow name to be executed
Reporting	Nextflow can produce a custom execution report with run information using the `log` command. You can generate a report using the `-t` option specifying a template file.
Workflow caching and checkpointing	Nextflow automatically keeping track of all the processes executed in your pipeline via caching and checkpointing. You can restart a Nextflow workflow using new data using new data skipping steps that have been successfully executed. Nextflow stores intermediate data in a working directory.
Deploying nf-core pipelines	nf-core is a community-led project to develop a set of best-practice pipelines built using the Nextflow workflow management system. nf-core tools is a suite of helper tools that aims to help people run and develop pipelines. nf-core pipelines can found using the nf-core helper tool `--list` option or from the nf-core website. nf-core pipelines can be run using `nextflow run nf-core/<pipeline>` syntax, or launched and parameters configured using the nf-core helper tool launch option. nf-core pipelines can be configured by modifying nextflow config files and/or adding command line parameters.

Here are some fundamental concepts of the Groovy language.

Single line comments (lines not meant to be executed) start with //. Multi-line comments are nested between /* and */ tags.

  // This is a single line comment. Everything after the // is ignored.
  /*
      Comments can also
      span multiple
      lines.
  */

Variables are assigned using = and can have any value. Variables used inside a double quoted string are prefixed with a $ to denote the variable should be interpolated (replace the variable with the variable’s value). A variable is otherwise just used by it’s name.

  myvar = 1                                     // Integer
  myvar = -3.1499392                            // Floating point number
  myvar = false                                 // Boolean
  myvar = "Hello world"                         // String
  myvar = new java.util.Date()                  // Object - Abstract data structure
  message = "The file $file cannot be found!"   // Variable used inside a string.
  println message                               // Print variable

Lists (also known as arrays) are defined using the square bracket [] notation.

  emptylist = []                      // an empty list
  // Lists can contain duplicates, and the values can be of any type.
  mixedList = [1, 1, 'string', true, null, 5 as byte]
  mixedList.add("new value")          // adds "new value" to the end of mixedList
  println mixedList.size()            // prints the size
  println mixedList[0]                // prints 1

Maps (also known as associative arrays) are defined using the [:] literal. They associate a unique string with a value, and are commonly referred to as key-value pairs.

  emptyMap = [:]                      // an empty map
  mymap = [ name : "Steve", age: 43, likes: ['walks','cooking','coding']]
  mymap.age = 44                      // values can be accessed using map property notation.
  println mymap['name']               // alternatively values can be accessed using quoted keys.

Groovy supports common control structures such as if/else tests, switch/case tests, for loops, and while loops.

  // if / else test
  x = false
  if ( !x ) {
      return 1
  } else {
      return 0
  }

  // switch / case test
  switch (x) {
      case "found foo":
          result = "Got foo"
          // continues to following cases

      case Number:
          result = "Number"
          break          // stops at this case

      default:
          result = "default"
  }

  // for loop (other variants exist)
  String message = ''
  for (int i = 0; i < 5; i++) {
      message += 'Hi '
  }

  // while loop
  y = 5
  while ( y-- > 0 ){
      println "Are we there yet?"
  }
  println "We've arrived!"

Closures are open, anonymous, blocks of code that can take arguments, return a value and be assigned to a variable. A closure definition follows the syntax { [closureParameters ->] statements }. A closure is evaluated when it is called, returning the result of the last statement in the closure.
```
  // Find all elements > 1
  above_one = [1, 2, 3].findAll { it -> it > 1 }

  // Closure in a string
  message = "The file ${ file.getName() } cannot be found!"
```

Groovy is very syntax-rich and supports many more operations. A full description of Groovy semantics can be found in the Groovy Documentation.

Glossary

Term	Description
Dataflow programming	dataflow programming is a programming paradigm that models a program as a directed graph of the data flowing between operation
Domain Specific Language	A domain-specific language (DSL) is a computer language specialized to a particular application domain. Nextflow is a language for computational workflows
Tuple	An ordered, immutable list of elements. A tuple can be seen as an ordered collection of objects of different types. These objects do not necessarily relate to each other in any way, but collectively they will have some meaning.

FIXME

Introduction to Bioinformatics workflows with Nextflow and nf-core: Groovy syntax

Key Points

Groovy syntax

Glossary