|
| 1 | +<!-- title=More syntax sugar for Nextflow developer! |
| 2 | +date=2020-11-02 |
| 3 | +type=post |
| 4 | +tags=nextflow,dsl |
| 5 | +status=published |
| 6 | +author=Paolo Di Tommaso |
| 7 | +icon=paolo.jpg |
| 8 | +~~~~~~ --> |
| 9 | + |
| 10 | +# More syntax sugar for Nextflow developer |
| 11 | + |
| 12 | +Latest Nextflow version 2020.10.0 is the first stable release running on Groovy 3. |
| 13 | + |
| 14 | +The first benefit of this chnage is that now Nextflow can be compiled and run on any modern Java virtual machine, |
| 15 | +from Java 8 up to latest Java 15! |
| 16 | + |
| 17 | +Along with this, the new Groovy runtime brings a bunch of syntax enhancements that can be useful |
| 18 | +in the everyday life of pipeline developers. Let's see them more in detail. |
| 19 | + |
| 20 | + |
| 21 | +## Improved not operator |
| 22 | + |
| 23 | +The `!` (not) operator can now prefix the `in` and `instaceof` keyword. |
| 24 | +This makes a bit more concise writing some conditional expression, for example the following snippet: |
| 25 | + |
| 26 | +``` |
| 27 | +if( !(x in list) ) { |
| 28 | + // .. |
| 29 | +} |
| 30 | +else if( !(x instanceof String) ) { |
| 31 | + // .. |
| 32 | +} |
| 33 | +``` |
| 34 | + |
| 35 | +could be replaced by the following: |
| 36 | + |
| 37 | +``` |
| 38 | +list = [10,20,30] |
| 39 | +
|
| 40 | +if( x !in list ) { |
| 41 | + // .. |
| 42 | +} |
| 43 | +else if( x !instanceof String ) { |
| 44 | + // .. |
| 45 | +} |
| 46 | +``` |
| 47 | + |
| 48 | +Again, this is a small syntax change which only makes the code a bit more |
| 49 | +readable. |
| 50 | + |
| 51 | + |
| 52 | +## Elvis assignment operator |
| 53 | + |
| 54 | +The elvis assigment operator `?=` allows the assignment of a value only if it was |
| 55 | +not previouvsly assigned (or it evaluets to `null`). Consider the following example: |
| 56 | + |
| 57 | +``` |
| 58 | +def opts = [foo: 1] |
| 59 | +
|
| 60 | +opts.foo ?= 10 |
| 61 | +opts.bar ?= 20 |
| 62 | +
|
| 63 | +assert opts.foo == 1 |
| 64 | +assert opts.bar == 20 |
| 65 | +``` |
| 66 | + |
| 67 | +In this snippet the assignment `opts.foo ?= 10` is ignored because the dictionary `opts` already |
| 68 | +contains a value for the `foo` attribute, while the following is assigned as expected. |
| 69 | + |
| 70 | +In other words this is a shortcut for the following idiom: |
| 71 | + |
| 72 | +``` |
| 73 | +if( some_variable != null ) { |
| 74 | + some_variable = 'Hello' |
| 75 | +} |
| 76 | +``` |
| 77 | + |
| 78 | +### Java style lambda expression |
| 79 | + |
| 80 | +Groovy 3 supports the syntax for Java lambda expression. If you don't know what is a Java lamda expression |
| 81 | +don't worry, it's a concept very similar to a Groovy closure, thought with slight differences |
| 82 | +both in the syntax and the semantic (i na few words a Groovy closure can modify a variable in the outside scope, |
| 83 | +while a Java lamda cannot). |
| 84 | + |
| 85 | +In terms of syntax a Groovy closure is defined as: |
| 86 | + |
| 87 | +``` |
| 88 | +{ it -> SOME_EXPRESSION_HERE } |
| 89 | +``` |
| 90 | + |
| 91 | +While Java lamba looks like: |
| 92 | + |
| 93 | +``` |
| 94 | +it -> { SOME_EXPRESSION_HERE } |
| 95 | +``` |
| 96 | + |
| 97 | +which can be simplified to the following form when the expression is a single statement: |
| 98 | + |
| 99 | +``` |
| 100 | +it -> SOME_EXPRESSION_HERE |
| 101 | +``` |
| 102 | + |
| 103 | +The good news is that the two syntax are interoperable in many cases and we can use the *lamda* |
| 104 | +syntax to get rid-off of the curly bracket parentheses used by the Groovy notation and make our Nextflow |
| 105 | +script more readable. |
| 106 | + |
| 107 | +For example the following Nextlow idiom: |
| 108 | + |
| 109 | +``` |
| 110 | +Channel |
| 111 | + .of( 1,2,3 ) |
| 112 | + .map { it * it +1 } |
| 113 | + .view { "the value is $it" } |
| 114 | +``` |
| 115 | + |
| 116 | +Can be rewritting using the labda syntax as: |
| 117 | + |
| 118 | +``` |
| 119 | +Channel |
| 120 | + .of( 1,2,3 ) |
| 121 | + .map( it -> it * it +1 ) |
| 122 | + .view( it -> "the value is $it" ) |
| 123 | +``` |
| 124 | + |
| 125 | +Which is a bit more consistent. Note however that the `it ->` implicit argument is now mandatory (while using the closure syntax it could be omitted) and when the operator argument is not *single* value, the lambda requires the |
| 126 | +round parentheses to define the argument e.g. |
| 127 | + |
| 128 | +``` |
| 129 | +Channel |
| 130 | + .of( 1,2,3 ) |
| 131 | + .map( it -> tuple(it * it, it+1) ) |
| 132 | + .view( (a,b) -> "the values are $a and $b" ) |
| 133 | +``` |
| 134 | + |
| 135 | + |
| 136 | +### Fully support Java streams API |
| 137 | + |
| 138 | +Java since version 8 provide a [stream library](https://winterbe.com/posts/2014/07/31/java8-stream-tutorial-examples/) that is very powerful and implements some concepts and operators similar to Nextflow channels. |
| 139 | + |
| 140 | +The main differences between the two is the Nextflow channels and the corresponding operators are *non-blocking* |
| 141 | +i.e. their evaluation is performed asynchronously without blocking your program execution, while Java streams are |
| 142 | +executed in a synchronus manner (at least by default). |
| 143 | + |
| 144 | +A Java stream looks like the following: |
| 145 | + |
| 146 | +``` |
| 147 | +assert (1..10).stream() |
| 148 | + .filter(e -> e % 2 == 0) |
| 149 | + .map(e -> e * 2) |
| 150 | + .toList() == [4, 8, 12, 16, 20] |
| 151 | +
|
| 152 | +``` |
| 153 | + |
| 154 | +Note, in the above example |
| 155 | +[filter](https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#filter-java.util.function.Predicate-), |
| 156 | +[map](https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#map-java.util.function.Function-) and |
| 157 | +[toList](https://docs.oracle.com/javase/8/docs/api/java/util/stream/Collectors.html#toList--) |
| 158 | +methods are Java stream operator not the |
| 159 | +[Nextflow](https://www.nextflow.io/docs/latest/operator.html#filter) |
| 160 | +[homonymous](https://www.nextflow.io/docs/latest/operator.html#map) |
| 161 | +[ones](https://www.nextflow.io/docs/latest/operator.html#tolist). |
| 162 | + |
| 163 | + |
| 164 | +### Java style method reference |
| 165 | + |
| 166 | +The new runtime allows also the use of the `::` operator to reference an object method. |
| 167 | +This can be useful to pass a method as an argument to a Nextflow operator in a similar manner |
| 168 | +how it was already possible using a closure. For example: |
| 169 | + |
| 170 | +``` |
| 171 | +Channel |
| 172 | + .of( 'a', 'b', 'c') |
| 173 | + .view( String::toUpperCase ) |
| 174 | + ``` |
| 175 | + |
| 176 | +The above prints: |
| 177 | + |
| 178 | +``` |
| 179 | + A |
| 180 | + B |
| 181 | + C |
| 182 | +``` |
| 183 | + |
| 184 | +Because to [view](https://www.nextflow.io/docs/latest/operator.html#filter) operator applied |
| 185 | +the method [toUpperCase](https://docs.oracle.com/javase/8/docs/api/java/lang/String.html#toUpperCase--) |
| 186 | +to each element emitted by the channel. |
| 187 | + |
| 188 | + |
| 189 | +### Conclusion |
| 190 | + |
| 191 | +The new Groovy runtime brings a lot syntax sugar for Nextflow pipelines and allow the use of modern Java |
| 192 | +runtime which deliver better performances and resources usage. |
| 193 | + |
| 194 | +The ones listed above are only some of them which may be usefull to everyday Nextflow developers. |
| 195 | +If you are curious to learn more about all the changes in the new Groovy parser you can find a |
| 196 | +[this link](https://groovy-lang.org/releasenotes/groovy-3.0.html). |
| 197 | + |
| 198 | +Finally, a big thanks to the Groovy community for the big effort of developing and maintaining this |
| 199 | +awesome programming environment. |
| 200 | + |
0 commit comments