Jekyll2023-07-29T12:59:31+00:00https://nikgrozev.com/feed.xmlNikolay GrozevStaying on top of it.Nikolay Grozevnikolay.grozev@gmail.comSimple and Unopinionated Java API2023-07-28T01:22:09+00:002023-07-28T01:22:09+00:00https://nikgrozev.com/2023/07/28/java-api-starter<h1 id="table-of-contents">Table of Contents</h1>
<ul>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#set_up">Set up</a></li>
<li><a href="#javalin">Javalin</a></li>
<li><a href="#testing">Testing</a></li>
<li><a href="#externalisation">Externalisation</a></li>
<li><a href="#spotless">Code Formatting with Spotless</a></li>
<li><a href="#spotbugs">Code Analysis with Spotbugs</a></li>
<li><a href="#sonarqube">SonarQube</a></li>
<li><a href="#dependency_analysis">Dependency Vulnerability Analysis</a></li>
<li><a href="#docker_image">Docker Image</a></li>
<li><a href="#docker_image_vulnerability_analysis">Docker Image: Vulnerability Analysis</a></li>
</ul>
<div id="introduction" />
<h1 id="introduction">Introduction</h1>
<p>I often have to build a small Java-based API or microservice. Using <a href="http://jakarta.ee/">Jakarta EE</a> or <a href="https://spring.io/">Spring</a>
sometimes feels like an overkill, especially when I work with teams which are not
experienced with Java.</p>
<p>In this post, I’ll explain how to build an unopionated, production ready API
which isn’t coupled with complex frameworks or requiring understanding of the Java
enterprise ecosystem. We’ll cover how to do automate testing, formatting, code analysis,
and vulnerability checks with <a href="https://gradle.org/">Gradle</a> commands.</p>
<p>The API starter <a href="https://github.com/nikolayg/java-api-starter">code is on GitHub</a> and in the rest of this post we’ll break it down.</p>
<div id="set_up" />
<h1 id="set-up">Set up</h1>
<p>Let’s start by setting up a local dev environment. I like <a href="https://sdkman.io/">SDKMAN</a>, which is a utility for installing and managing different versions of Java and related tools:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="c"># Install SDKMan, if you don't have it</span>
curl <span class="nt">-s</span> <span class="s2">"https://get.sdkman.io"</span> | bash
<span class="c"># View all available Java distributions:</span>
sdk list java
<span class="c"># Install the latest Java 17 Temurin distro</span>
<span class="c"># (Temurin used to be called AdoptOpenJDK)</span>
sdk <span class="nb">install </span>java 17.0.8-tem
</pre></td></tr></tbody></table></code></pre></div></div>
<p>If you cloned the <a href="https://github.com/nikolayg/java-api-starter">GitHub project</a>, then that’s all you need.
If you want to follow along, then you’ll need <code class="language-plaintext highlighter-rouge">gradle</code> to initialise the project:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="c"># View available Gradle distributions:</span>
sdk list gradle
<span class="c"># Install a gradle version from the above list</span>
sdk <span class="nb">install </span>gradle 8.2.1
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now let’s initialise a project with <code class="language-plaintext highlighter-rouge">gradle init</code>. This will open a terminal “wizard”,
and we’ll have to select the following:</p>
<div class="language-md highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
</pre></td><td class="rouge-code"><pre><span class="gt">> gradle init</span>
Select type of project to generate:
1: basic
2: application
Enter selection (default: basic) [1..4] 2
Select implementation language:
1: C++
...
3: Java
Enter selection (default: Java) [1..6] 3
Split functionality across multiple subprojects?:
1: no - only one application project
2: yes - application and library projects
Enter selection (default: no - only one application project) [1..2] 1
Select build script DSL:
1: Groovy
2: Kotlin
Enter selection (default: Groovy) [1..2] 1
Select test framework:
1: JUnit 4
...
4: JUnit Jupiter
Enter selection (default: JUnit Jupiter) [1..4] 4
Project name (default: java-api-starter):
Source package (default: java.api.starter): com.nikgrozev
Enter target version of Java (min. 7) (default: 17): 17
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The above creates a single project
<a href="https://docs.gradle.org/current/userguide/application_plugin.html">gradle application</a>
which can run a main method and packages your code in an archive. If you’re new to gradle,
you can have a look at the <a href="/2017/02/10/gradle-quickstart/">Gradle Quickstart</a> tutorial.
From now on, we’ll use the <a href="https://docs.gradle.org/current/userguide/gradle_wrapper.html">Gradle Wrapper</a>
script <code class="language-plaintext highlighter-rouge">./gradlew</code> to ensure consistent builds across environments.</p>
<p>Open the new project in you favourite IDE. At this point, we have a functioning Java app,
with a main method <code class="language-plaintext highlighter-rouge">App.java</code> and a unit test in <code class="language-plaintext highlighter-rouge">AppTest.java</code>. We can already build, test,
run, and check the docs:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
</pre></td><td class="rouge-code"><pre><span class="c"># Clean up, compile and rebuild</span>
./gradlew clean build
<span class="c"># Run the JUnit5 tests</span>
./gradlew <span class="nb">test</span>
<span class="c"># Run the app locally</span>
./gradlew run
<span class="c"># Checkout all gradle tasks or get help for a task</span>
./gradlew task
./gradlew <span class="nb">help</span> <span class="nt">--task</span> <span class="nb">test</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Next, let’s build a single executable uber jar file, which we can distribute.
<a href="https://www.baeldung.com/gradle-fat-jar">One approach</a> is to modify the
<code class="language-plaintext highlighter-rouge">jar</code> task in <code class="language-plaintext highlighter-rouge">build.gradle</code>:</p>
<div class="language-gradle highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre><span class="c1">// Based on https://www.baeldung.com/gradle-fat-jar</span>
<span class="n">jar</span> <span class="o">{</span>
<span class="n">duplicatesStrategy</span> <span class="o">=</span> <span class="n">DuplicatesStrategy</span><span class="o">.</span><span class="na">EXCLUDE</span>
<span class="n">manifest</span> <span class="o">{</span>
<span class="n">attributes</span> <span class="s2">"Main-Class"</span><span class="o">:</span> <span class="n">application</span><span class="o">.</span><span class="na">mainClass</span>
<span class="o">}</span>
<span class="n">from</span> <span class="o">{</span>
<span class="n">configurations</span><span class="o">.</span><span class="na">runtimeClasspath</span><span class="o">.</span><span class="na">collect</span> <span class="o">{</span> <span class="n">it</span><span class="o">.</span><span class="na">isDirectory</span><span class="o">()</span> <span class="o">?</span> <span class="n">it</span> <span class="o">:</span> <span class="n">zipTree</span><span class="o">(</span><span class="n">it</span><span class="o">)</span> <span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now we can build and run the executable jar file:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="c"># Clean up, compile and rebuild</span>
./gradlew clean build
<span class="c"># The above creates an executable uber jar file</span>
java <span class="nt">-jar</span> ./app/build/libs/app.jar
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="javalin" />
<h1 id="javalin">Javalin</h1>
<p><a href="https://javalin.io/">Javalin</a> is a lightweight and unopiniated web server built on top of
<a href="https://eclipse.dev/jetty/">Jetty</a>. It’s easy to get started with and comes with convenient testing
utilities. Let’s add the following to <code class="language-plaintext highlighter-rouge">build.gradle</code> and rebuild:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre><span class="n">dependencies</span> <span class="o">{</span>
<span class="c1">//...</span>
<span class="c1">// Javalin Web Server</span>
<span class="n">implementation</span> <span class="s2">"io.javalin:javalin:5.6.1"</span>
<span class="c1">// Allows Javalin to parse and create JSON</span>
<span class="n">implementation</span> <span class="s1">'com.fasterxml.jackson.core:jackson-databind:2.15.0'</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Javalin is quite <a href="https://javalin.io/documentation#getting-started">easy to start with</a>.
We just need to create a <code class="language-plaintext highlighter-rouge">Javalin</code> instance and add routes. I particularly like
the <a href="https://javalin.io/documentation#crudhandler">CRUD handler</a>, which allows you to create
REST-ful endpoints and organise them in a separate class:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
</pre></td><td class="rouge-code"><pre><span class="kd">public</span> <span class="kd">class</span> <span class="nc">App</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">Javalin</span> <span class="nf">makeJavalinServer</span><span class="o">()</span> <span class="o">{</span>
<span class="kt">var</span> <span class="n">app</span> <span class="o">=</span> <span class="nc">Javalin</span><span class="o">.</span><span class="na">create</span><span class="o">();</span>
<span class="n">app</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"/api/health"</span><span class="o">,</span> <span class="n">ctx</span> <span class="o">-></span> <span class="n">ctx</span><span class="o">.</span><span class="na">result</span><span class="o">(</span><span class="s">"Healthy"</span><span class="o">));</span>
<span class="n">app</span><span class="o">.</span><span class="na">routes</span><span class="o">(()</span> <span class="o">-></span> <span class="n">crud</span><span class="o">(</span><span class="s">"/api/items/{id}"</span><span class="o">,</span>
<span class="c1">// ItemController is our implementation of Javalin's CrudHandler</span>
<span class="k">new</span> <span class="nf">ItemController</span><span class="o">())</span>
<span class="o">);</span>
<span class="k">return</span> <span class="n">app</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">main</span><span class="o">(</span><span class="nc">String</span><span class="o">[]</span> <span class="n">args</span><span class="o">)</span> <span class="o">{</span>
<span class="n">app</span><span class="o">.</span><span class="na">start</span><span class="o">(</span><span class="mi">9090</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>For a full example, see the <a href="https://github.com/nikolayg/java-api-starter">GitHub project</a>.
With the above, we have a fully functional web server:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre><span class="c"># Run the app locally</span>
<span class="c"># Checkout http://localhost:9090/api/health</span>
./gradlew run
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Beyond the basisc, Javalin offers many more advanced functionalities like
<a href="https://javalin.io/documentation#before-handlers">before and after handlers</a>,
supports
<a href="https://javalin.io/documentation#websockets">web sockets</a>, <a href="https://javalin.io/documentation#validation">validation</a> and more.</p>
<div id="testing" />
<h1 id="testing">Testing</h1>
<p>Javalin <a href="https://javalin.io/tutorials/testing">supports two types of tests</a>:
unit tests for each handler and end-to-end tests. Let’s start by adding the following
dependencies:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="n">dependencies</span> <span class="o">{</span>
<span class="c1">// Mockito for unit testing Javalin</span>
<span class="n">testImplementation</span> <span class="s1">'org.mockito:mockito-inline:5.2.0'</span>
<span class="c1">// Javalin test utility</span>
<span class="n">testImplementation</span> <span class="s1">'io.javalin:javalin-testtools:5.6.1'</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>For unit testing our handlers, we’ll use <a href="https://site.mockito.org/">Mockito</a> to mock all external
dependencies, as in the following example:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
</pre></td><td class="rouge-code"><pre><span class="kd">public</span> <span class="kd">class</span> <span class="nc">ItemControllerTest</span> <span class="o">{</span>
<span class="kd">private</span> <span class="kd">final</span> <span class="nc">Context</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">mock</span><span class="o">(</span><span class="nc">Context</span><span class="o">.</span><span class="na">class</span><span class="o">);</span>
<span class="kd">private</span> <span class="nc">ItemController</span> <span class="n">controller</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ItemController</span><span class="o">();</span>
<span class="nd">@BeforeEach</span>
<span class="kt">void</span> <span class="nf">setUp</span><span class="o">()</span> <span class="o">{</span>
<span class="n">controller</span><span class="o">.</span><span class="na">setItems</span><span class="o">(</span><span class="nc">List</span><span class="o">.</span><span class="na">of</span><span class="o">());</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kt">void</span> <span class="nf">testCreate</span><span class="o">()</span> <span class="o">{</span>
<span class="kt">var</span> <span class="n">newItem</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Item</span><span class="o">(</span><span class="s">"id1"</span><span class="o">,</span> <span class="s">"item1"</span><span class="o">);</span>
<span class="n">when</span><span class="o">(</span><span class="n">ctx</span><span class="o">.</span><span class="na">bodyAsClass</span><span class="o">(</span><span class="nc">Item</span><span class="o">.</span><span class="na">class</span><span class="o">)).</span><span class="na">thenReturn</span><span class="o">(</span><span class="n">newItem</span><span class="o">);</span>
<span class="n">controller</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="n">ctx</span><span class="o">);</span>
<span class="n">verify</span><span class="o">(</span><span class="n">ctx</span><span class="o">).</span><span class="na">json</span><span class="o">(</span><span class="n">newItem</span><span class="o">,</span> <span class="nc">Item</span><span class="o">.</span><span class="na">class</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="n">controller</span><span class="o">.</span><span class="na">getItems</span><span class="o">().</span><span class="na">size</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="s">"id1"</span><span class="o">,</span> <span class="n">controller</span><span class="o">.</span><span class="na">getItems</span><span class="o">().</span><span class="na">get</span><span class="o">(</span><span class="mi">0</span><span class="o">).</span><span class="na">getId</span><span class="o">());</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>This is great, but I prefer to mock as little as possible. Mocking excessively makes
tests unreleastic. Fortunately,
<a href="https://mvnrepository.com/artifact/io.javalin/javalin-testtools">Javalin Test Tools</a>
allows us to test an actual instance of a Javalin webserver and make realistic API calls:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
</pre></td><td class="rouge-code"><pre><span class="kd">public</span> <span class="kd">class</span> <span class="nc">AppTest</span> <span class="o">{</span>
<span class="kd">private</span> <span class="nc">Javalin</span> <span class="n">app</span><span class="o">;</span>
<span class="nd">@BeforeEach</span>
<span class="kt">void</span> <span class="nf">setUp</span><span class="o">()</span> <span class="kd">throws</span> <span class="nc">Exception</span> <span class="o">{</span>
<span class="n">app</span> <span class="o">=</span> <span class="nc">App</span><span class="o">.</span><span class="na">makeJavalinServer</span><span class="o">();</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kt">void</span> <span class="nf">testHealthEndpoint</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">JavalinTest</span><span class="o">.</span><span class="na">test</span><span class="o">(</span>
<span class="n">app</span><span class="o">,</span>
<span class="o">(</span><span class="n">server</span><span class="o">,</span> <span class="n">client</span><span class="o">)</span> <span class="o">-></span> <span class="o">{</span>
<span class="kt">var</span> <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"/api/health"</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">200</span><span class="o">,</span> <span class="n">response</span><span class="o">.</span><span class="na">code</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="s">"Healthy"</span><span class="o">,</span> <span class="n">response</span><span class="o">.</span><span class="na">body</span><span class="o">().</span><span class="na">string</span><span class="o">());</span>
<span class="o">});</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Once again, the full code with the unit tests is available on <a href="https://github.com/nikolayg/java-api-starter">GitHub</a>.</p>
<div id="externalisation" />
<h1 id="externalisation">Externalisation</h1>
<p><a href="https://github.com/cdimascio/dotenv-java">Dotenv</a> is a Java alternative of the popular npm package
<a href="https://www.npmjs.com/package/dotenv">dotenv</a>. It allows us to keep all dev env variables
in a local file <code class="language-plaintext highlighter-rouge">./app/.env</code> and load them from the app. In production, you wouldn’t have a <code class="language-plaintext highlighter-rouge">.env</code>
and then the library would load actual environment variables instead. Let’s add it as a dependency:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre><span class="n">dependencies</span> <span class="o">{</span>
<span class="c1">// To read vars from .env file or environment</span>
<span class="n">implementation</span> <span class="s1">'io.github.cdimascio:dotenv-java:3.0.0'</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now we can use it in our code:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="kd">public</span> <span class="kd">class</span> <span class="nc">App</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">main</span><span class="o">(</span><span class="nc">String</span><span class="o">[]</span> <span class="n">args</span><span class="o">)</span> <span class="o">{</span>
<span class="c1">// load env vars via https://github.com/cdimascio/dotenv-java</span>
<span class="kt">var</span> <span class="n">dotenv</span> <span class="o">=</span> <span class="nc">Dotenv</span><span class="o">.</span><span class="na">configure</span><span class="o">().</span><span class="na">ignoreIfMissing</span><span class="o">().</span><span class="na">systemProperties</span><span class="o">().</span><span class="na">load</span><span class="o">();</span>
<span class="kt">var</span> <span class="n">app</span> <span class="o">=</span> <span class="n">makeJavalinServer</span><span class="o">();</span>
<span class="n">app</span><span class="o">.</span><span class="na">start</span><span class="o">(</span><span class="nc">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">dotenv</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"PORT"</span><span class="o">,</span> <span class="s">"7070"</span><span class="o">)));</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="spotless" />
<h1 id="code-formatting-with-spotless">Code Formatting with Spotless</h1>
<p><a href="https://github.com/diffplug/spotless">Spotless</a> is a popular
linter for Java and it comes with a handy <a href="https://github.com/diffplug/spotless">Gradle Plugin</a>.
Let’s add it to our <code class="language-plaintext highlighter-rouge">build.gradle</code>:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
</pre></td><td class="rouge-code"><pre><span class="n">plugins</span> <span class="o">{</span>
<span class="c1">// ...</span>
<span class="n">id</span> <span class="s2">"com.diffplug.spotless"</span> <span class="n">version</span> <span class="s2">"6.20.0"</span>
<span class="o">}</span>
<span class="c1">// Based on https://github.com/diffplug/spotless/tree/main/plugin-gradle</span>
<span class="n">spotless</span> <span class="o">{</span>
<span class="n">format</span> <span class="s1">'misc'</span><span class="o">,</span> <span class="o">{</span>
<span class="c1">// define the files to apply `misc` to</span>
<span class="n">target</span> <span class="s1">'*.gradle'</span><span class="o">,</span> <span class="s1">'*.md'</span><span class="o">,</span> <span class="s1">'.gitignore'</span>
<span class="c1">// define the steps to apply to those files</span>
<span class="n">trimTrailingWhitespace</span><span class="o">()</span>
<span class="c1">// or tabs/spaces. Takes an integer argument if you don't like 4</span>
<span class="n">indentWithSpaces</span><span class="o">()</span>
<span class="n">endWithNewline</span><span class="o">()</span>
<span class="o">}</span>
<span class="n">java</span> <span class="o">{</span>
<span class="c1">// apply a specific flavor of google-java-format</span>
<span class="n">googleJavaFormat</span><span class="o">(</span><span class="s1">'1.17.0'</span><span class="o">).</span><span class="na">aosp</span><span class="o">().</span><span class="na">reflowLongStrings</span><span class="o">()</span>
<span class="c1">// fix formatting of type annotations</span>
<span class="n">formatAnnotations</span><span class="o">()</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>In the above, we used out-of-the-box formatter from Google. Have a look at the
<a href="https://github.com/diffplug/spotless">documentation</a> to see all configuration options.</p>
<p>Now we can format the code or check if it complies with the format:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="c"># Check for violations (useful in CI)</span>
./gradlew spotlessJavaCheck
<span class="c"># Format the code with Spotless</span>
./gradlew spotlessJavaApply
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="spotbugs" />
<h1 id="code-analysis-with-spotbugs">Code Analysis with Spotbugs</h1>
<p><a href="https://spotbugs.github.io/">Spotbugs</a> is a great tool for code analysis and
it has a <a href="https://github.com/spotbugs/spotbugs-gradle-plugin">Gradle Plugin</a> as well.
Let’s configure it with the default:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
</pre></td><td class="rouge-code"><pre><span class="n">plugins</span> <span class="o">{</span>
<span class="c1">// ....</span>
<span class="n">id</span> <span class="s2">"com.github.spotbugs"</span> <span class="n">version</span> <span class="s2">"5.0.14"</span>
<span class="o">}</span>
<span class="c1">// Extensions - https://spotbugs-gradle-plugin.netlify.app/com/github/spotbugs/snom/spotbugsextension</span>
<span class="n">spotbugs</span> <span class="o">{</span>
<span class="n">ignoreFailures</span> <span class="o">=</span> <span class="kc">false</span>
<span class="n">showStackTraces</span> <span class="o">=</span> <span class="kc">true</span>
<span class="n">showProgress</span> <span class="o">=</span> <span class="kc">true</span>
<span class="n">effort</span> <span class="o">=</span> <span class="s1">'default'</span>
<span class="n">reportLevel</span> <span class="o">=</span> <span class="s1">'default'</span>
<span class="n">reportsDir</span> <span class="o">=</span> <span class="n">file</span><span class="o">(</span><span class="s2">"$buildDir/reports/spotbugs"</span><span class="o">)</span>
<span class="o">}</span>
<span class="n">spotbugsMain</span> <span class="o">{</span>
<span class="n">reports</span> <span class="o">{</span>
<span class="n">html</span> <span class="o">{</span>
<span class="n">required</span> <span class="o">=</span> <span class="kc">true</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="n">spotbugsTest</span> <span class="o">{</span>
<span class="n">reports</span> <span class="o">{</span>
<span class="n">html</span> <span class="o">{</span>
<span class="n">required</span> <span class="o">=</span> <span class="kc">true</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now we can analyse our code and generate HTML reports for every violation:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre><span class="c"># Analyse the code for issues with SpotBugs</span>
<span class="c"># HTML reports are generated in ./app/build/reports/spotbugs</span>
./gradlew spotbugsMain spotbugsTest
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Spotless is very configurable, and you can check their documentation about the various options.</p>
<div id="sonar1ube" />
<h1 id="sonarqube">SonarQube</h1>
<p><a href="https://docs.sonarsource.com/sonarqube/latest/">SonarQube</a> is a popular code analysis tool.
It usually runs in a central server within an organisation. Developers and CI tools
can submit code for analysis, which is then validated against a predefined
<a href="https://docs.sonarsource.com/sonarqube/latest/user-guide/quality-gates/">Quality Gate</a>.</p>
<p>Let’s configure a <a href="https://docs.sonarsource.com/sonarqube/latest/">Gradle Plugin</a> to push
code to a SonarQube instance. Before that, we’ll need to congirue test coverage reports,
so that SonaQube can inspect that too (SonarQube won’t run your tests).
We’ll use the popular <a href="https://docs.gradle.org/current/userguide/jacoco_plugin.html">JaCoCo</a>
code coverate tool:
:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
</pre></td><td class="rouge-code"><pre><span class="n">plugins</span> <span class="o">{</span>
<span class="c1">// ...</span>
<span class="n">id</span> <span class="s2">"org.sonarqube"</span> <span class="n">version</span> <span class="s2">"4.3.0.3225"</span>
<span class="n">id</span> <span class="s1">'jacoco'</span>
<span class="o">}</span>
<span class="c1">// https://docs.gradle.org/current/userguide/jacoco_plugin.html</span>
<span class="n">test</span> <span class="o">{</span>
<span class="c1">// report is always generated after tests run</span>
<span class="n">finalizedBy</span> <span class="n">jacocoTestReport</span>
<span class="o">}</span>
<span class="n">jacocoTestReport</span> <span class="o">{</span>
<span class="c1">// tests are required to run before generating the report</span>
<span class="c1">// By default, reports go into app/build/reports/jacoco</span>
<span class="n">dependsOn</span> <span class="n">test</span>
<span class="n">reports</span> <span class="o">{</span>
<span class="n">xml</span><span class="o">.</span><span class="na">required</span> <span class="o">=</span> <span class="kc">true</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="c1">// https://docs.sonarsource.com/sonarqube/latest/analyzing-source-code/scanners/sonarscanner-for-gradle/</span>
<span class="n">sonar</span> <span class="o">{</span>
<span class="n">properties</span> <span class="o">{</span>
<span class="n">property</span> <span class="s2">"sonar.projectKey"</span><span class="o">,</span> <span class="n">rootProject</span><span class="o">.</span><span class="na">name</span>
<span class="n">property</span> <span class="s2">"sonar.projectName"</span><span class="o">,</span> <span class="n">rootProject</span><span class="o">.</span><span class="na">name</span>
<span class="n">property</span> <span class="s2">"sonar.host.url"</span><span class="o">,</span> <span class="s2">"???"</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>To run a SonaQube analsis, please change the SonaQube url in the above config.
You’ll also need an access token:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="c"># Push the code for analsys to a SonarQube instance</span>
<span class="c"># Note 1: you need to edit "build.gradle" and add your "sonar.host.url"</span>
<span class="c"># Note 2: you need an env var SONAR_TOKEN to authenticate</span>
<span class="nb">export </span><span class="nv">SONAR_TOKEN</span><span class="o">=</span>???
./gradlew sonar
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="dependency_analysis" />
<h1 id="dependency-vulnerability-analysis">Dependency Vulnerability Analysis</h1>
<p><a href="https://owasp.org/www-project-dependency-check/">Dependency vulnerability analysis</a>
is essential to maintain a healthy and secure application.
We’ll use the <a href="https://plugins.gradle.org/plugin/org.owasp.dependencycheck">OWASP Gradle plugin</a>
to inspect our code - let’s configure it:</p>
<div class="language-gradle highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
</pre></td><td class="rouge-code"><pre><span class="n">plugins</span> <span class="o">{</span>
<span class="c1">// ...</span>
<span class="n">id</span> <span class="s2">"org.owasp.dependencycheck"</span> <span class="n">version</span> <span class="s2">"8.1.2"</span>
<span class="o">}</span>
<span class="c1">// Results will be in app/build/reports/dependency-check-report.html</span>
<span class="n">dependencyCheck</span> <span class="o">{</span>
<span class="c1">// Low(0.1-3.9), Med(4.0-6.9), High(7.0-8.9), Critical(9.0-10.0)</span>
<span class="n">failBuildOnCVSS</span> <span class="o">=</span> <span class="mi">7</span>
<span class="c1">// Based on https://github.com/dependency-check/dependency-check-gradle/issues/22#issuecomment-575568801</span>
<span class="kt">def</span> <span class="n">skipConfigurationPatterns</span> <span class="o">=</span> <span class="o">[</span>
<span class="s2">"test.*"</span><span class="o">,</span>
<span class="s2">"spotbugs.*"</span><span class="o">,</span>
<span class="s2">"spotless.*"</span><span class="o">,</span>
<span class="s2">"sonar.*"</span><span class="o">,</span>
<span class="s2">"jacoco.*"</span>
<span class="o">]</span>
<span class="k">allprojects</span> <span class="o">{</span>
<span class="n">configurations</span><span class="o">.</span><span class="na">all</span> <span class="o">{</span> <span class="k">configuration</span> <span class="o">-></span>
<span class="k">if</span> <span class="o">(</span><span class="k">configuration</span><span class="o">.</span><span class="na">name</span> <span class="k">in</span> <span class="n">skipConfigurations</span><span class="o">)</span> <span class="o">{</span>
<span class="k">return</span>
<span class="o">}</span>
<span class="n">skipConfigurationPatterns</span><span class="o">.</span><span class="na">each</span> <span class="o">{</span> <span class="n">pattern</span> <span class="o">-></span>
<span class="k">if</span> <span class="o">(</span><span class="k">configuration</span><span class="o">.</span><span class="na">name</span><span class="o">.</span><span class="na">matches</span><span class="o">(</span><span class="n">pattern</span><span class="o">))</span> <span class="o">{</span>
<span class="n">skipConfigurations</span> <span class="o"><<</span> <span class="k">configuration</span><span class="o">.</span><span class="na">name</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>In the above configuration, we’ve ignored medium and lower severity dependencies (<code class="language-plaintext highlighter-rouge">failBuildOnCVSS = 7</code>).
We are also ignoring test and tool dependencies (e.g. of <code class="language-plaintext highlighter-rouge">spotbugs</code> and <code class="language-plaintext highlighter-rouge">sonar</code>) as we will not package
them into our build. Now we can run a check, which will also generate an HTML report:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre><span class="c"># Check the dependencies for known high or ciritcal vulnerabilities</span>
<span class="c"># Results will be in app/build/reports/dependency-check-report.html</span>
./gradlew dependencyCheckAnalyze
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="docker_image" />
<h1 id="docker-image">Docker Image</h1>
<p>Finally, let’s package our application in a Docker image. We will use a
<a href="https://docs.docker.com/build/building/multi-stage/">two-stage</a> Dockerfile.
The first stage will build the executable jar file, and the second will run it.</p>
<p>I’ve chosen to use the <a href="https://hub.docker.com/_/eclipse-temurin">Eclipse Temurin</a>
official Docker image a base image. If you want to minimise the resulting docker image
you may use its Alpine variant or another base image. Furthermore,
the <a href="https://hub.docker.com/_/eclipse-temurin">Eclipse Temurin</a> page documents
how to use <a href="https://www.baeldung.com/jlink">jlink</a> to create a smaller and custom JRE,
which would result in an even smaller image. For simplicity, let’s just use the
out of the box image:</p>
<div class="language-Dockerfile highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
</pre></td><td class="rouge-code"><pre><span class="k">FROM</span><span class="s"> eclipse-temurin:17 as builder</span>
<span class="c"># Set the working directory inside the container</span>
<span class="k">WORKDIR</span><span class="s"> /app</span>
<span class="c"># Copy the code after installing dependencies - Note! .dockerignore</span>
<span class="k">COPY</span><span class="s"> . .</span>
<span class="c"># Run the Gradle build to compile the app</span>
<span class="k">RUN </span>./gradlew clean build
<span class="c"># Stage 2: Build the actual container from the builder's output</span>
<span class="k">FROM</span><span class="s"> eclipse-temurin:17</span>
<span class="c"># The port we're listening on</span>
<span class="c"># Copy build bundle from the builder container</span>
<span class="k">COPY</span><span class="s"> --from=builder /app/app/build/libs/app.jar /app/</span>
<span class="c"># Make sure we don't run as root</span>
<span class="k">RUN </span><span class="nb">chown</span> <span class="nt">-R</span> nobody /app
<span class="k">USER</span><span class="s"> nobody</span>
<span class="k">CMD</span><span class="s"> ["java", "-jar", "/app/app.jar"]</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>It’s a good practice to add a <code class="language-plaintext highlighter-rouge">.dockerignore</code> file to your code base. This will speed
up the build. I have added a
<a href="https://github.com/nikolayg/java-api-starter/blob/main/.dockerignore">sample</a> one in the GitHub project.</p>
<p>Let’s proceed to build and tag our docker image:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>docker build <span class="nb">.</span> <span class="nt">-t</span> java-api-starter
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now we can start the image locally. It’s often convenient to
pass all required env variables in a file:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="c"># Replace 9090 with the PORT from .env</span>
<span class="c"># Visit http://localhost:9090/api/health</span>
docker run <span class="se">\</span>
<span class="nt">-p</span> 9090:9090 <span class="se">\</span>
<span class="nt">--env-file</span><span class="o">=</span>./app/.env <span class="se">\</span>
java-api-starter
</pre></td></tr></tbody></table></code></pre></div></div>
<p>To debug the image locally, we can open a shell “into” it:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre><span class="c"># Replace 9090 with the PORT from .env</span>
<span class="c"># Will open a shell</span>
docker run <span class="se">\</span>
<span class="nt">-p</span> 9090:9090 <span class="se">\</span>
<span class="nt">--env-file</span><span class="o">=</span>./app/.env <span class="se">\</span>
<span class="nt">-it</span> java-api-starter <span class="se">\</span>
/bin/sh
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="docker_image_vulnerability_analysis" />
<h1 id="docker-image-vulnerability-analysis">Docker Image: Vulnerability Analysis</h1>
<p>Many CI/CD tools scan images for known vulnerabilities. I often find it useful
to scan the image locally, so I can quickly experiment with different approaches
to mitigate vulnerabilities. I find <a href="https://rancherdesktop.io/">Rancher Desktop</a>
very useful. It allows me to pick a local docker image and scan it in the UI.</p>
<p>Let’s start by selecting an image to scan:</p>
<figure>
<img style="max-width: 700px;" src="/images/blog/java-api-starter/choose-image.png" alt="Choose a docker image." />
<figcaption>
Select a local docker image to scan.
</figcaption>
</figure>
<p>And then we can preview all known vulnerabilities and their severity:</p>
<figure>
<img style="max-width: 700px;" src="/images/blog/java-api-starter/image-analysis.png" alt="Docker image analysis." />
<figcaption>
Preview vulnerabilities.
</figcaption>
</figure>Nikolay Grozevnikolay.grozev@gmail.comThis post summarises how to build a simple and unopinionated Java API with Javalin. It demonstrates how to test it, analyse code and dependncies, package it, externalise, and run it.Solving Progress Indication Issues in Legacy Web Applications2023-07-14T01:22:09+00:002023-07-14T01:22:09+00:00https://nikgrozev.com/2023/07/14/solving_progress_indication_issues_in_legacy_web_applications<h1 id="introduction">Introduction</h1>
<p>Recently, I was working on a large legacy web application which didn’t have adequate progress feedback and action confirmation.
Users would often click a button, wait for a few seconds and click it again, because they didn’t see any indication
that it was clicked. Double clicks resulted in repeated records in the database.</p>
<p>I was tasked to mitigate this across the application, with as little changes to existing screens and logic as possible.
Hence, it had to be done globally with JavaScript by dynamically decorating HTML nodes.</p>
<p>For the impatient, here’s a <a href="https://jsfiddle.net/nikgrozev/5x0vd2c9/251/">JSFiddle working example</a> of the approach. Let’s break it down.</p>
<h1 id="dynamic-styling-of-buttons-on-click">Dynamic Styling of Buttons on Click</h1>
<p>Let’s start by creating a function, which given a button, adds a style whenever it’s clicked.
We’ll mark every button with a data attribute <code class="language-plaintext highlighter-rouge">data-button-registered</code>
so that we know it’s been processed and we don’t process it again.
We’ll assume that the class <code class="language-plaintext highlighter-rouge">button-clicked</code> adds a visual
effect to a clicked button:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
</pre></td><td class="rouge-code"><pre><span class="kd">const</span> <span class="nx">clickNotificationTimeout</span> <span class="o">=</span> <span class="mi">500</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">registerNewButton</span> <span class="o">=</span> <span class="p">(</span><span class="nx">button</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="c1">// Skip if already processed</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">button</span> <span class="o">||</span> <span class="nx">button</span><span class="p">.</span><span class="nx">getAttribute</span><span class="p">(</span><span class="dl">"</span><span class="s2">data-button-registered</span><span class="dl">"</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">true</span><span class="dl">"</span><span class="p">))</span> <span class="p">{</span>
<span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Callback to enable, disable after timeout</span>
<span class="kd">const</span> <span class="nx">onBtnClick</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">button</span><span class="p">.</span><span class="nx">classList</span><span class="p">.</span><span class="nx">add</span><span class="p">(</span><span class="dl">"</span><span class="s2">button-clicked</span><span class="dl">"</span><span class="p">);</span>
<span class="nx">setTimeout</span><span class="p">(()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">button</span><span class="p">.</span><span class="nx">classList</span><span class="p">.</span><span class="nx">remove</span><span class="p">(</span><span class="dl">"</span><span class="s2">button-clicked</span><span class="dl">"</span><span class="p">);</span>
<span class="p">},</span> <span class="nx">clickNotificationTimeout</span><span class="p">);</span>
<span class="p">};</span>
<span class="c1">// Add the listener - do not run the listener immediately as a courtesy to subsequent events</span>
<span class="nx">button</span><span class="p">.</span><span class="nx">addEventListener</span><span class="p">(</span><span class="dl">"</span><span class="s2">click</span><span class="dl">"</span><span class="p">,</span> <span class="p">()</span> <span class="o">=></span> <span class="nx">setTimeout</span><span class="p">(</span><span class="nx">onBtnClick</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="p">{</span>
<span class="na">capture</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span>
<span class="p">});</span>
<span class="c1">// mark it as processed</span>
<span class="nx">button</span><span class="p">.</span><span class="nx">setAttribute</span><span class="p">(</span><span class="dl">"</span><span class="s2">data-button-registered</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">true</span><span class="dl">"</span><span class="p">);</span>
<span class="p">};</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Note that we added <code class="language-plaintext highlighter-rouge">capture: true</code> to the event listener so that it runs before other non-capture events
and hence there’s less chance it’s precented
(see the <a href="https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener#syntax">options documentation</a>).
Also, we’re wrapping up the event handler in a <code class="language-plaintext highlighter-rouge">setTimeout</code> so that we don’t block subsequent events.</p>
<h1 id="register-all-buttons-dynamically">Register All Buttons Dynamically</h1>
<p>Now that we have a function to process all buttons, we’ll need to call it for every button in the app.
However, our app’s using many AJAX calls, which dynamically add/remove HTML content, including buttons.
Furthermore, DOM elementes are added dynamically with JavaScript.
Hence, we need to listen to the page loading event
<a href="https://developer.mozilla.org/en-US/docs/Web/API/Window/DOMContentLoaded_event">DOMContentLoaded</a>
and any subsequent DOM change via a <a href="https://developer.mozilla.org/en-US/docs/Web/API/MutationObserver">MutationObserver</a>.
The observer provides the DOM difference on every change, which we would need to process:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
</pre></td><td class="rouge-code"><pre><span class="c1">// Get all buttons on the page after load:</span>
<span class="nb">window</span><span class="p">.</span><span class="nx">addEventListener</span><span class="p">(</span><span class="dl">"</span><span class="s2">DOMContentLoaded</span><span class="dl">"</span><span class="p">,</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">buttons</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">getElementsByTagName</span><span class="p">(</span><span class="dl">"</span><span class="s2">button</span><span class="dl">"</span><span class="p">);</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">let</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">buttons</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">registerNewButton</span><span class="p">(</span><span class="nx">buttons</span><span class="p">[</span><span class="nx">i</span><span class="p">]);</span>
<span class="p">}</span>
<span class="p">});</span>
<span class="c1">// Traverse a node recursively to get all the buttons</span>
<span class="kd">const</span> <span class="nx">getButtonChildren</span> <span class="o">=</span> <span class="p">(</span><span class="nx">node</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">node</span> <span class="o">||</span> <span class="o">!</span><span class="nx">node</span><span class="p">.</span><span class="nx">tagName</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[];</span>
<span class="p">}</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">node</span><span class="p">.</span><span class="nx">tagName</span><span class="p">.</span><span class="nx">toLowerCase</span><span class="p">()</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">button</span><span class="dl">"</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[</span><span class="nx">node</span><span class="p">];</span>
<span class="p">}</span>
<span class="kd">const</span> <span class="nx">result</span> <span class="o">=</span> <span class="p">[];</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">let</span> <span class="nx">child</span> <span class="k">of</span> <span class="nb">Array</span><span class="p">.</span><span class="k">from</span><span class="p">(</span><span class="nx">node</span><span class="p">.</span><span class="nx">childNodes</span><span class="p">))</span> <span class="p">{</span>
<span class="nx">result</span><span class="p">.</span><span class="nx">push</span><span class="p">(...</span><span class="nx">getButtonChildren</span><span class="p">(</span><span class="nx">child</span><span class="p">));</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nx">result</span><span class="p">;</span>
<span class="p">};</span>
<span class="c1">// Observe the DOM for changes</span>
<span class="kd">const</span> <span class="nx">domObserver</span> <span class="o">=</span> <span class="k">new</span> <span class="nb">window</span><span class="p">.</span><span class="nx">MutationObserver</span><span class="p">(</span><span class="kd">function</span> <span class="p">(</span><span class="nx">mutations</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">mutations</span><span class="p">.</span><span class="nx">forEach</span><span class="p">(</span><span class="kd">function</span> <span class="p">(</span><span class="nx">mutation</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// For every change - get the new buttons</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">let</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">mutation</span><span class="p">.</span><span class="nx">addedNodes</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">buttons</span> <span class="o">=</span> <span class="nx">getButtonChildren</span><span class="p">(</span><span class="nx">mutation</span><span class="p">.</span><span class="nx">addedNodes</span><span class="p">[</span><span class="nx">i</span><span class="p">]);</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">const</span> <span class="nx">button</span> <span class="k">of</span> <span class="nx">buttons</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">registerNewButton</span><span class="p">(</span><span class="nx">button</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">});</span>
<span class="p">});</span>
<span class="nx">domObserver</span><span class="p">.</span><span class="nx">observe</span><span class="p">(</span><span class="nb">document</span><span class="p">,</span> <span class="p">{</span>
<span class="na">childList</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span>
<span class="na">subtree</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span>
<span class="p">});</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now the user will see a visual effect on every button click! We can easily extend that to add visual
effects on other HTML element types too.</p>
<h1 id="progress-indicator">Progress Indicator</h1>
<p>Although the user can see a visual indication that a button is clicked, they can still double click it by mistake.
Hence, we need a progress indicator layer which blocks user input while an operation is in progress.
The <a href="https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest">XMLHttpRequest</a> and
<a href="https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API">fetch</a> APIs are the main ways to make AJAX and API requests
from the browser. We can overwrite their definitions so that we execute code before and and after every request, which
opens or closes a page spinner.</p>
<p>Another considerations is that HTTP requests can overlap, so we’ll need to take care of concurrency. We’ll keep
track of how many API requests are active, and we’ll keep the spinner open if there’s at least one.</p>
<p>Lastly, we also need to show a spinner when navigating to another page for which we’ll use the
<a href="https://stackoverflow.com/questions/10616456/how-to-display-a-loading-dialog-while-navigating-between-pages">beforeunload event</a>:</p>
<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
</pre></td><td class="rouge-code"><pre><span class="kd">let</span> <span class="nx">concurrentAPIOperations</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">startSpinner</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">concurrentAPIOperations</span><span class="o">++</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">concurrentAPIOperations</span> <span class="o">===</span> <span class="mi">1</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// In the HTML - we'll need a new HTML element which represents the spinner overlay</span>
<span class="c1">// The following code assumes hidden and running-spinner are the appropriate CSS classe</span>
<span class="kd">const</span> <span class="nx">spinner</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">getElementById</span><span class="p">(</span><span class="dl">"</span><span class="s2">spinner-id</span><span class="dl">"</span><span class="p">);</span>
<span class="nx">spinner</span><span class="p">.</span><span class="nx">classList</span><span class="p">.</span><span class="nx">remove</span><span class="p">(</span><span class="dl">"</span><span class="s2">hidden</span><span class="dl">"</span><span class="p">);</span>
<span class="nx">spinner</span><span class="p">.</span><span class="nx">classList</span><span class="p">.</span><span class="nx">add</span><span class="p">(</span><span class="dl">"</span><span class="s2">running-spinner</span><span class="dl">"</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="kd">const</span> <span class="nx">stopSpinner</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">concurrentAPIOperations</span> <span class="o">=</span> <span class="nb">Math</span><span class="p">.</span><span class="nx">max</span><span class="p">(</span><span class="nx">concurrentAPIOperations</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">concurrentAPIOperations</span> <span class="o">===</span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// In the HTML - we'll need a new HTML element which represents the spinner overlay</span>
<span class="c1">// The following code assumes hidden and running-spinner are the appropriate CSS classe</span>
<span class="kd">const</span> <span class="nx">spinner</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">getElementById</span><span class="p">(</span><span class="dl">"</span><span class="s2">spinner-id</span><span class="dl">"</span><span class="p">);</span>
<span class="nx">spinner</span><span class="p">.</span><span class="nx">classList</span><span class="p">.</span><span class="nx">add</span><span class="p">(</span><span class="dl">"</span><span class="s2">hidden</span><span class="dl">"</span><span class="p">);</span>
<span class="nx">spinner</span><span class="p">.</span><span class="nx">classList</span><span class="p">.</span><span class="nx">remove</span><span class="p">(</span><span class="dl">"</span><span class="s2">running-spinner</span><span class="dl">"</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="c1">// https://stackoverflow.com/questions/5202296/add-a-hook-to-all-ajax-requests-on-a-page</span>
<span class="kd">const</span> <span class="nx">_open</span> <span class="o">=</span> <span class="nx">XMLHttpRequest</span><span class="p">.</span><span class="nx">prototype</span><span class="p">.</span><span class="nx">open</span><span class="p">;</span>
<span class="nx">XMLHttpRequest</span><span class="p">.</span><span class="nx">prototype</span><span class="p">.</span><span class="nx">open</span> <span class="o">=</span> <span class="kd">function</span> <span class="p">()</span> <span class="p">{</span>
<span class="nx">startSpinner</span><span class="p">();</span>
<span class="k">this</span><span class="p">.</span><span class="nx">addEventListener</span><span class="p">(</span><span class="dl">"</span><span class="s2">loadend</span><span class="dl">"</span><span class="p">,</span> <span class="nx">stopSpinner</span><span class="p">);</span>
<span class="nx">_open</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span><span class="k">this</span><span class="p">,</span> <span class="nx">arguments</span><span class="p">);</span>
<span class="p">};</span>
<span class="c1">// https://stackoverflow.com/questions/63122604/how-can-i-execute-a-function-every-time-fetch-is-used-in-javascript</span>
<span class="kd">const</span> <span class="nx">_fetch</span> <span class="o">=</span> <span class="nb">window</span><span class="p">.</span><span class="nx">fetch</span><span class="p">;</span>
<span class="nb">window</span><span class="p">.</span><span class="nx">fetch</span> <span class="o">=</span> <span class="kd">function</span> <span class="p">(...</span><span class="nx">args</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">startSpinner</span><span class="p">();</span>
<span class="k">return</span> <span class="nb">Promise</span><span class="p">.</span><span class="nx">resolve</span><span class="p">(</span><span class="nx">_fetch</span><span class="p">.</span><span class="nx">apply</span><span class="p">(</span><span class="nb">window</span><span class="p">,</span> <span class="nx">args</span><span class="p">)).</span><span class="k">finally</span><span class="p">(</span><span class="nx">stopSpinner</span><span class="p">);</span>
<span class="p">};</span>
<span class="c1">// /https://stackoverflow.com/questions/10616456/how-to-display-a-loading-dialog-while-navigating-between-pages</span>
<span class="nb">window</span><span class="p">.</span><span class="nx">addEventListener</span><span class="p">(</span><span class="dl">"</span><span class="s2">beforeunload</span><span class="dl">"</span><span class="p">,</span> <span class="nx">startSpinner</span><span class="p">);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The full example is on <a href="https://jsfiddle.net/nikgrozev/5x0vd2c9/253/">JSFiddle</a>.</p>
<h1 id="conclusion-and-catches">Conclusion and Catches</h1>
<p>This approach works pretty well for us and we didn’t have to modify the existing code.
We only had to add a small JavaScript snippet together with the required HTML and CSS.
However, there are a few things to look out for.</p>
<p>Firstly, if the app has <code class="language-plaintext highlighter-rouge">capture</code> click event handlers which stop the event propagation, then
our click events may not be triggered. Search in the codebase to see if that happens.
If there are Ajax/API calls that happen on blur, they may prevent subsequent click events, due to
the extra logic for starting/stopping the spinner - see here for <a href="https://erikmartinjordan.com/onblur-prevents-onclick-react">more details</a>.
<a href="https://erikmartinjordan.com/onblur-prevents-onclick-react">There are workarounds</a> like opening the
spinner overlay with a delay or using <code class="language-plaintext highlighter-rouge">onMouseDown</code> instead of <code class="language-plaintext highlighter-rouge">click</code>.</p>Nikolay Grozevnikolay.grozev@gmail.comThis post summarises an approach to add progress indication for a legacy web app. This is done globally, without having to rewrite or retest the application.New Features in Java 9 and Later Versions2021-12-04T01:22:09+00:002021-12-04T01:22:09+00:00https://nikgrozev.com/2021/12/04/java-9-and-afterwards<h1 id="table-of-contents">Table of Contents</h1>
<ul>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#set-up">Set Up</a></li>
<li><a href="#jshell">JShell</a></li>
<li><a href="#run-java-files">Run Java files</a></li>
<li><a href="#var-local-inference">Var - Local Variable Type Inference</a></li>
<li><a href="#unmodifiable-collections">Unmodifiable Collections - New Factory and Utility Methods</a></li>
<li><a href="#streams-new-methods">Streams - New Methods</a></li>
<li><a href="#optional-new-methods">Optional - New Methods</a></li>
<li><a href="#string-new-methods">String - New Methods</a></li>
<li><a href="#files-new-methods">Files - New Methods</a></li>
<li><a href="#switch-expressions">Switch Expressions</a></li>
<li><a href="#pattern-matching-instanceof">Pattern Matching For InstanceOf</a></li>
<li><a href="#text-blocks">Text Blocks (Multiline Strings)</a></li>
<li><a href="#interfaces">New Syntax for Interfaces</a></li>
<li><a href="#records">Records</a></li>
<li><a href="#sealed-classes">Sealed Classes</a></li>
<li><a href="#garbage-collection">Garbage Collection Algorithms</a></li>
<li><a href="#java-modules">Java Modules (Overview)</a></li>
<li><a href="#resources">Resources</a></li>
</ul>
<div id="introduction" />
<h1 id="introduction">Introduction</h1>
<p>Java has been often referred to as a slow moving
language and platform. It took nearly 5 years for Java 7
to come out (released in 2011)
then 3 more for Java 8 (2014) and then 3 more for Java 9 (2017).</p>
<p>Since then, there’s been a new release every 6 months!
There’re many projects
<a href="https://www.marcobehler.com/guides/a-guide-to-java-versions-and-features#_why_are_companies_still_stuck_with_java_8">stuck with Java 8</a>.
It turns out, developers are
having a hard time keeping up to speed and upgrading their code bases.</p>
<p>In this post, I’ll summarise the new features in Java 9 and later versions.
At the time of writing, the latest is Java 17 and I will keep this
article up to date as new versions are released.
If you need to catch up with Java 8, check out
<a href="/2014/03/21/java-8-in-a-nutshell/">Java 8 in a Nutshell</a>.</p>
<p>To keep this short, I’ll only cover the most prominent features from
a developer’s perspective. I will omit or just give a high level overview
of less commonly used features.</p>
<div id="set-up" />
<h1 id="set-up">Set Up</h1>
<p><a href="http://sdkman.io/">SDKMAN</a> is a utility for installing Java and related tools. It’s similar to
Node’s <a href="https://github.com/creationix/nvm">NVM</a> and Python’s <a href="https://github.com/yyuu/pyenv">PyEnv</a>. SDKMAN can
install and switch between multiple versions of Java, Scala, Maven, Gradle, etc.</p>
<p>To install on Linux or Mac:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>curl <span class="nt">-s</span> <span class="s2">"https://get.sdkman.io"</span> | bash
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Then we can install Java 17:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="c"># View all available Java distributions:</span>
sdk list java
<span class="c"># Install the latest Java 17 Temurin distro</span>
<span class="c"># (Temurin used to be called AdoptOpenJDK)</span>
sdk <span class="nb">install </span>java 17.0.1-tem
</pre></td></tr></tbody></table></code></pre></div></div>
<p>To follow along, you can start a new Java project. In this example, I’ll use Gradle:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
</pre></td><td class="rouge-code"><pre><span class="c"># Check out what gradle versions are available</span>
sdk list gradle
<span class="c"># Get the latest one (7.3 or later for Java 17):</span>
sdk <span class="nb">install </span>gradle 7.3
<span class="c"># Make a project</span>
<span class="nb">mkdir </span>javaplayground <span class="o">&&</span> <span class="nb">cd </span>javaplayground
<span class="c"># Set up an "application" project using the wizard</span>
gradle init
<span class="c"># Make sure it runs:</span>
./gradlew run
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="jshell" />
<h1 id="jshell">JShell</h1>
<p>Java 9 introduced a <a href="https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop">REPL</a>,
which is great for quick experimentation.</p>
<p>It has <code class="language-plaintext highlighter-rouge">tab</code> key autocompletion, allows you to look up JavaDoc, and displays user friendly errors.
Lastly, it imports many packages by default (including <code class="language-plaintext highlighter-rouge">java.util.*</code>), so you won’t have to do it yourself.</p>
<p>To start it, use the <code class="language-plaintext highlighter-rouge">jshell</code> command:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
</pre></td><td class="rouge-code"><pre><span class="c"># Start the JShell the REPL</span>
<span class="o">></span> jshell
<span class="c"># Check out which imports are included by default</span>
jshell> /imports
<span class="c"># Define variables, print something, etc</span>
jshell> String hello <span class="o">=</span> <span class="s2">"hello"</span>
jshell> String world <span class="o">=</span> <span class="s2">"world"</span>
jshell> System.out.println<span class="o">(</span>hello + <span class="s2">" "</span> + world<span class="o">)</span>
<span class="c"># No need to import List from java.util</span>
jshell> List<String> list <span class="o">=</span> List.of<span class="o">(</span><span class="s2">"a"</span>, <span class="s2">"b"</span><span class="o">)</span>
<span class="c"># Tab auto completion - will list all System methods</span>
jshell> System.<tab>
<span class="c"># Multiple <tabs> shows javadoc and scrolls through it</span>
jshell> System<tab><tab>
<span class="c"># Quit</span>
jshell> /exit
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="run-java-files" />
<h1 id="run-java-files">Run Java files</h1>
<p>As of Java 10, you can run java source files (with main methods)
without compiling them:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>java ./app/src/main/java/com/nikgrozev/App.java
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="var-local-inference" />
<h1 id="var---local-variable-type-inference">Var - Local Variable Type Inference</h1>
<p>Local variables can now be defined with the <code class="language-plaintext highlighter-rouge">var</code> keyword
without explicitly specifying their type. The compiler
automatically infers their types:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="c1">// Before, we had to write List<String> list = ...</span>
<span class="kt">var</span> <span class="n">list</span> <span class="o">=</span> <span class="nc">List</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"1"</span><span class="o">,</span> <span class="s">"2"</span><span class="o">,</span> <span class="s">"3"</span><span class="o">);</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">var</span> <span class="n">e</span> <span class="o">:</span> <span class="n">list</span><span class="o">)</span> <span class="o">{</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">e</span><span class="o">);</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="unmodifiable-collections" />
<h1 id="unmodifiable-collections---new-factory-and-utility-methods">Unmodifiable Collections - New Factory and Utility Methods</h1>
<p>The <code class="language-plaintext highlighter-rouge">List</code>, <code class="language-plaintext highlighter-rouge">Set</code>, and <code class="language-plaintext highlighter-rouge">Map</code> interfaces
have a new factory method <code class="language-plaintext highlighter-rouge">of</code> to quickly instantiate
immutable collections.</p>
<p>Another new collection factory method is <code class="language-plaintext highlighter-rouge">copyOf</code>, which
creates an immutable copy of its argument.</p>
<p>Finally, new collectors have been introduced, which convert a
stream to an unmodifiable collections</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
</pre></td><td class="rouge-code"><pre><span class="c1">// New Helper factory methods</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">sampleList</span> <span class="o">=</span> <span class="nc">List</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"a"</span><span class="o">,</span> <span class="s">"b"</span><span class="o">,</span> <span class="s">"c"</span><span class="o">);</span>
<span class="nc">Set</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">sampleSet</span> <span class="o">=</span> <span class="nc">Set</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"a"</span><span class="o">,</span> <span class="s">"b"</span><span class="o">,</span> <span class="s">"c"</span><span class="o">);</span>
<span class="nc">Map</span><span class="o"><</span><span class="nc">String</span><span class="o">,</span> <span class="nc">String</span><span class="o">></span> <span class="n">sampleMap</span> <span class="o">=</span> <span class="nc">Map</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"a"</span><span class="o">,</span> <span class="s">"a-Value"</span><span class="o">,</span> <span class="s">"b"</span><span class="o">,</span> <span class="s">"b-Value"</span><span class="o">);</span>
<span class="c1">// Immutable collection - will throw error on modification</span>
<span class="c1">// sampleList.set(0, "will be error");</span>
<span class="c1">// Prints out:</span>
<span class="c1">// class java.util.ImmutableCollections$ListN,</span>
<span class="c1">// class java.util.ImmutableCollections$SetN,</span>
<span class="c1">// class java.util.ImmutableCollections$MapN</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">printf</span><span class="o">(</span><span class="s">"%s,\n%s,\n%s\n"</span><span class="o">,</span>
<span class="n">sampleList</span><span class="o">.</span><span class="na">getClass</span><span class="o">(),</span>
<span class="n">sampleSet</span><span class="o">.</span><span class="na">getClass</span><span class="o">(),</span>
<span class="n">sampleMap</span><span class="o">.</span><span class="na">getClass</span><span class="o">());</span>
<span class="c1">// List, Set, Map have a new method "copyOf" to create an immutable copy</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">mutableList</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><>(</span><span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="s">"1"</span><span class="o">,</span> <span class="s">"2"</span><span class="o">,</span> <span class="s">"3"</span><span class="o">));</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">immutableList</span> <span class="o">=</span> <span class="nc">List</span><span class="o">.</span><span class="na">copyOf</span><span class="o">(</span><span class="n">mutableList</span><span class="o">);</span>
<span class="c1">// prints java.util.ImmutableCollections$ListN</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">immutableList</span><span class="o">.</span><span class="na">getClass</span><span class="o">());</span>
<span class="c1">// New collectors - toUnmodifiableXXX convert a stream to immutable collection</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">immutableListCollected</span> <span class="o">=</span> <span class="n">mutableList</span><span class="o">.</span>
<span class="nf">stream</span><span class="o">().</span>
<span class="n">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">toUnmodifiableList</span><span class="o">());</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="streams-new-methods" />
<h1 id="streams---new-methods">Streams - New Methods</h1>
<p>Java 8 introduced the <code class="language-plaintext highlighter-rouge">Stream.iterate</code> method, which creates
and infinite stream.
It has 2 parameters - a <code class="language-plaintext highlighter-rouge">seed</code> value and a generator function <code class="language-plaintext highlighter-rouge">f</code>.
The <code class="language-plaintext highlighter-rouge">iterate</code> method creates a stream, which dynamically generates the sequence of values:</p>
<p><code class="language-plaintext highlighter-rouge">seed, f(seed), ..., f(f(...f(seed)...))</code></p>
<p>For example the following, creates a stream of all integers, starting from 0 and incrementing by 1:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre><span class="nc">Stream</span><span class="o"><</span><span class="nc">Integer</span><span class="o">></span> <span class="n">infiniteN</span> <span class="o">=</span> <span class="nc">Stream</span><span class="o">.</span><span class="na">iterate</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="n">i</span> <span class="o">-></span> <span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="o">);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Java 9 introduced a new version of the <code class="language-plaintext highlighter-rouge">iterate</code> method,
which also takes a predicate. It dynamically generates values
until it reaches one that violates the predicate:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="c1">// Stream.iterate - typically used to create ranges</span>
<span class="nc">Stream</span><span class="o"><</span><span class="nc">Integer</span><span class="o">></span> <span class="n">sampleStream</span> <span class="o">=</span> <span class="nc">Stream</span><span class="o">.</span><span class="na">iterate</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="n">i</span> <span class="o">-></span> <span class="n">i</span> <span class="o"><</span> <span class="mi">10</span><span class="o">,</span> <span class="n">i</span> <span class="o">-></span> <span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="o">);</span>
<span class="c1">// Prints 0, 1, 2, 3, 4, 5, 6, 7, 8, 9</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">prinln</span><span class="o">(</span><span class="n">sampleStream</span><span class="o">.</span><span class="na">toArray</span><span class="o">());</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Java 9 also introduced the <code class="language-plaintext highlighter-rouge">dropWhile</code> and <code class="language-plaintext highlighter-rouge">takeWhile</code> stream
methods, which skip or select the initial stream
values based on a predicate:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="c1">// Stream 0 to 9</span>
<span class="nc">Stream</span><span class="o"><</span><span class="nc">Integer</span><span class="o">></span> <span class="n">sampleStream</span> <span class="o">=</span> <span class="nc">Stream</span><span class="o">.</span><span class="na">iterate</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="n">i</span> <span class="o">-></span> <span class="n">i</span> <span class="o"><</span> <span class="mi">10</span><span class="o">,</span> <span class="n">i</span> <span class="o">-></span> <span class="n">i</span> <span class="o">+</span> <span class="mi">1</span><span class="o">);</span>
<span class="nc">Stream</span><span class="o"><</span><span class="nc">Integer</span><span class="o">></span> <span class="n">selectedPart</span> <span class="o">=</span> <span class="n">sampleStream</span><span class="o">.</span>
<span class="nf">dropWhile</span><span class="o">(</span><span class="n">i</span> <span class="o">-></span> <span class="n">i</span> <span class="o"><</span> <span class="mi">2</span><span class="o">).</span>
<span class="n">takeWhile</span><span class="o">(</span><span class="n">i</span> <span class="o">-></span> <span class="n">i</span> <span class="o"><</span> <span class="mi">5</span><span class="o">);</span>
<span class="c1">// Prints Range [2, 3, 4]</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">printf</span><span class="o">(</span><span class="s">"Range %s\n"</span><span class="o">,</span> <span class="n">selectedPart</span><span class="o">.</span><span class="na">toList</span><span class="o">().</span><span class="na">toString</span><span class="o">());</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="optional-new-methods" />
<h1 id="optional---new-methods">Optional - New Methods</h1>
<p>Java 8 inroduced the <code class="language-plaintext highlighter-rouge">Optional</code> class as container which
either has a single value or none.
Recently, it got two new methods: <code class="language-plaintext highlighter-rouge">ifPresentOrElse</code> and <code class="language-plaintext highlighter-rouge">orElseThrow</code>:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
</pre></td><td class="rouge-code"><pre><span class="c1">// Optional has a new method - ifPresentOrElse</span>
<span class="nc">Optional</span><span class="o"><</span><span class="nc">Integer</span><span class="o">></span> <span class="n">sampleOption</span> <span class="o">=</span> <span class="nc">Optional</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="mi">123</span><span class="o">);</span>
<span class="n">sampleOption</span><span class="o">.</span><span class="na">ifPresentOrElse</span><span class="o">(</span>
<span class="o">(</span><span class="n">v</span><span class="o">)</span> <span class="o">-></span> <span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"Has value: "</span> <span class="o">+</span> <span class="n">v</span><span class="o">),</span>
<span class="o">()</span> <span class="o">-></span> <span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"Has NO value"</span><span class="o">));</span>
<span class="nc">Optional</span><span class="o"><</span><span class="nc">Object</span><span class="o">></span> <span class="n">opt</span> <span class="o">=</span> <span class="nc">Optional</span><span class="o">.</span><span class="na">empty</span><span class="o">();</span>
<span class="k">try</span> <span class="o">{</span>
<span class="c1">// same as opt.get(), but throws if missing</span>
<span class="kt">var</span> <span class="n">value</span> <span class="o">=</span> <span class="n">opt</span><span class="o">.</span><span class="na">orElseThrow</span><span class="o">();</span>
<span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">NoSuchElementException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"Option was empty"</span><span class="o">);</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="string-new-methods" />
<h1 id="string---new-methods">String - New Methods</h1>
<p><code class="language-plaintext highlighter-rouge">String</code> got a few convenient methods:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
</pre></td><td class="rouge-code"><pre><span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">""</span><span class="o">.</span><span class="na">isBlank</span><span class="o">());</span> <span class="c1">// true</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">" "</span><span class="o">.</span><span class="na">isBlank</span><span class="o">());</span> <span class="c1">// true</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">" \n "</span><span class="o">.</span><span class="na">isBlank</span><span class="o">());</span> <span class="c1">// true</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">" s "</span><span class="o">.</span><span class="na">isBlank</span><span class="o">());</span> <span class="c1">// false</span>
<span class="c1">// unicode strip</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">" x "</span><span class="o">.</span><span class="na">strip</span><span class="o">());</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">" x "</span><span class="o">.</span><span class="na">stripLeading</span><span class="o">());</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">" x "</span><span class="o">.</span><span class="na">stripTrailing</span><span class="o">());</span>
<span class="c1">// Prints " x x x "</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">" x "</span><span class="o">.</span><span class="na">repeat</span><span class="o">(</span><span class="mi">3</span><span class="o">));</span>
<span class="c1">// Make a stream of lines!</span>
<span class="s">"\ntest\ntest2\n\nstest"</span><span class="o">.</span>
<span class="n">lines</span><span class="o">().</span>
<span class="n">forEachOrdered</span><span class="o">((</span><span class="kt">var</span> <span class="n">s</span><span class="o">)</span> <span class="o">-></span> <span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">">> "</span> <span class="o">+</span> <span class="n">s</span><span class="o">));</span>
<span class="c1">// Manage indentation:</span>
<span class="kt">var</span> <span class="n">multiline</span> <span class="o">=</span> <span class="s">" line1\n line2"</span><span class="o">;</span>
<span class="c1">// Adds 2 spaces at each line's start. Normalises new lines (\n)</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">multiline</span><span class="o">.</span><span class="na">indent</span><span class="o">(</span><span class="mi">2</span><span class="o">));</span>
<span class="c1">// Removes up to 3 spaces on each line's start and normalises new</span>
<span class="c1">// lines symbol. Can be less if a line has fewer lead spaces</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">multiline</span><span class="o">.</span><span class="na">indent</span><span class="o">(-</span><span class="mi">2</span><span class="o">));</span>
<span class="c1">// Normalises new lines (\n) - nothing else changes</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">multiline</span><span class="o">.</span><span class="na">indent</span><span class="o">(</span><span class="mi">0</span><span class="o">));</span>
<span class="c1">// Removes the common indentation - e.g. if all lines have</span>
<span class="c1">// between 2 and 5 leading spaces it will remove 2.</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">multiline</span><span class="o">.</span><span class="na">stripIndent</span><span class="o">());</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="files-new-methods" />
<h1 id="files---new-methods">Files - New Methods</h1>
<p>With the new <code class="language-plaintext highlighter-rouge">Files</code>, developers can read, write, compare files more easily:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre><span class="c1">// Creating and reading text files with a single line!</span>
<span class="nc">Path</span> <span class="n">path</span> <span class="o">=</span>
<span class="nc">Files</span><span class="o">.</span><span class="na">writeString</span><span class="o">(</span><span class="nc">Files</span><span class="o">.</span><span class="na">createTempFile</span><span class="o">(</span><span class="s">"test"</span><span class="o">,</span> <span class="s">".txt"</span><span class="o">),</span> <span class="s">"Demo"</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">path</span><span class="o">);</span>
<span class="nc">String</span> <span class="n">s</span> <span class="o">=</span> <span class="nc">Files</span><span class="o">.</span><span class="na">readString</span><span class="o">(</span><span class="n">path</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">s</span><span class="o">);</span> <span class="c1">//Prints "Demo"</span>
<span class="c1">// "mismatch" compares two files efficiently - i.e. first by size,</span>
<span class="c1">// then by content. Returns -1 if they're equal</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="nc">Files</span><span class="o">.</span><span class="na">mismatch</span><span class="o">(</span><span class="n">path</span><span class="o">,</span> <span class="n">path</span><span class="o">));</span> <span class="c1">// prints -1</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="switch-expressions" />
<h1 id="switch-expressions">Switch Expressions</h1>
<p>This new syntax avoids the pitfalls of
<a href="https://docs.oracle.com/javase/tutorial/java/nutsandbolts/switch.html">switch fall through</a>:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre><span class="kt">var</span> <span class="n">text</span> <span class="o">=</span> <span class="s">"A"</span><span class="o">;</span>
<span class="kt">var</span> <span class="n">index</span> <span class="o">=</span> <span class="k">switch</span> <span class="o">(</span><span class="n">text</span><span class="o">)</span> <span class="o">{</span>
<span class="k">case</span> <span class="s">"A"</span> <span class="o">-></span> <span class="mi">1</span><span class="o">;</span>
<span class="k">case</span> <span class="s">"B"</span> <span class="o">-></span> <span class="mi">2</span><span class="o">;</span>
<span class="k">default</span> <span class="o">-></span> <span class="k">throw</span> <span class="k">new</span> <span class="nc">IllegalArgumentException</span><span class="o">(</span><span class="s">"Unknown letter"</span><span class="o">);</span>
<span class="o">};</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">index</span><span class="o">);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="pattern-matching-instanceof" />
<h1 id="pattern-matching-for-instanceof">Pattern Matching For InstanceOf</h1>
<p>Type guards with <code class="language-plaintext highlighter-rouge">instanceOf</code> are common in Java code.
In older versions, you’d still need to explicitly cast within the
guarded code, but now there’s a shortcut:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
</pre></td><td class="rouge-code"><pre><span class="nc">Object</span> <span class="n">o</span> <span class="o">=</span> <span class="s">"some text"</span><span class="o">;</span>
<span class="c1">// In older versions we had to cast expicitly</span>
<span class="k">if</span> <span class="o">(</span><span class="n">o</span> <span class="k">instanceof</span> <span class="nc">String</span><span class="o">)</span> <span class="o">{</span>
<span class="c1">// Need to cast, although we're type guarding</span>
<span class="nc">String</span> <span class="n">oAsString</span> <span class="o">=</span> <span class="o">(</span><span class="nc">String</span><span class="o">)</span> <span class="n">o</span><span class="o">;</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">oAsString</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// New feature - type guard and cast together</span>
<span class="k">if</span> <span class="o">(</span><span class="n">o</span> <span class="k">instanceof</span> <span class="nc">String</span> <span class="n">oString</span><span class="o">)</span> <span class="o">{</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">oString</span><span class="o">);</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="text-blocks" />
<h1 id="text-blocks-multiline-strings">Text Blocks (Multiline Strings)</h1>
<p>Text Blocks (a.k.a. multiline strings) surrounded by
triple double quotes are a godsend for everyone writing SQL:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="kt">var</span> <span class="n">sql</span> <span class="o">=</span> <span class="s">"""
SELECT * FROM
MY_TABLE
WHERE X > 1
"""</span><span class="o">;</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The lines’ common indentation and trailing spaces are stripped.
To add leading indentation either use the <code class="language-plaintext highlighter-rouge">indent</code> method
or move the closing quotes at the beginning of the line:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="kt">var</span> <span class="n">sqlExplicitlyIndented</span> <span class="o">=</span> <span class="s">"""
SELECT * FROM
MY_TABLE
"""</span><span class="o">.</span><span class="na">indent</span><span class="o">(</span><span class="mi">2</span><span class="o">);</span>
<span class="kt">var</span> <span class="n">sqlOriginalIndentation</span> <span class="o">=</span> <span class="s">"""
SELECT * FROM
MY_TABLE
"""</span><span class="o">;</span> <span class="c1">// <----- closing bracket is at the line's beginning</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Each line’s trailing spaces will be ignored.</p>
<div id="interfaces" />
<h1 id="new-syntax-for-interfaces">New Syntax for Interfaces</h1>
<p>Prior to Java 8, interfaces could only defined static constants
and abstract methods. Since Java 8, interfaces can also
define public overrideable methods with default implementation.
Java 9 allows interfaces to have private methods as well:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
</pre></td><td class="rouge-code"><pre><span class="kd">interface</span> <span class="nc">IStudent</span> <span class="o">{</span>
<span class="c1">// Abstract method - all non-abstract subclasses must implement it</span>
<span class="kd">public</span> <span class="kt">double</span> <span class="nf">getGPAGrade</span><span class="o">();</span>
<span class="c1">// New In Java 8: Public static method</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kt">double</span> <span class="nf">getMaxGPAGrade</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="mi">7</span><span class="o">;</span>
<span class="o">}</span>
<span class="c1">// New in Java 8: Default method - no need to implement, but can override</span>
<span class="k">default</span> <span class="kt">double</span> <span class="nf">getPercentageGrade</span><span class="o">()</span> <span class="o">{</span>
<span class="n">validateGPA</span><span class="o">();</span> <span class="c1">// Call the private method</span>
<span class="k">return</span> <span class="o">(</span><span class="n">getGPAGrade</span><span class="o">()</span> <span class="o">/</span> <span class="n">getMaxGPAGrade</span><span class="o">())</span> <span class="o">*</span> <span class="mi">100</span><span class="o">;</span>
<span class="o">}</span>
<span class="c1">// New in Java 9: private interface methods</span>
<span class="kd">private</span> <span class="kt">void</span> <span class="nf">validateGPA</span><span class="o">()</span> <span class="o">{</span>
<span class="n">validateGPAScore</span><span class="o">(</span><span class="k">this</span><span class="o">.</span><span class="na">getGPAGrade</span><span class="o">());</span>
<span class="o">}</span>
<span class="c1">// New in Java 9: private static methods</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">validateGPAScore</span><span class="o">(</span><span class="kt">double</span> <span class="n">gpa</span><span class="o">){</span>
<span class="k">if</span> <span class="o">(</span><span class="n">gpa</span> <span class="o"><</span> <span class="mi">0</span> <span class="o">||</span> <span class="n">gpa</span> <span class="o">></span> <span class="n">getMaxGPAGrade</span><span class="o">())</span> <span class="o">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">IllegalArgumentException</span><span class="o">(</span><span class="s">"Invalid GPA"</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="kd">class</span> <span class="nc">Student</span> <span class="kd">implements</span> <span class="nc">IStudent</span> <span class="o">{</span>
<span class="kt">double</span> <span class="n">gpaGrade</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
<span class="kd">public</span> <span class="nf">Student</span><span class="o">(</span><span class="kt">double</span> <span class="n">gpaGrade</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">gpaGrade</span> <span class="o">=</span> <span class="n">gpaGrade</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kt">double</span> <span class="nf">getGPAGrade</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">gpaGrade</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="records" />
<h1 id="records">Records</h1>
<p>Consider all the boilerplate needed for
a simple data-only class - get methods, <code class="language-plaintext highlighter-rouge">equals</code>, <code class="language-plaintext highlighter-rouge">hashCode</code>, <code class="language-plaintext highlighter-rouge">toString</code>, etc.
Developers often use their IDEs to auto generate all of these.</p>
<p>As of Java 17, there’s a shortcut for creating <em>immutable data-only classes</em> via the
<code class="language-plaintext highlighter-rouge">record</code> keyword. Records are final and can not be extended.
They get automatic accessor methods, constructor, and the
aforementioned <code class="language-plaintext highlighter-rouge">equals</code>, <code class="language-plaintext highlighter-rouge">hashCode</code>, <code class="language-plaintext highlighter-rouge">toString</code>:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
</pre></td><td class="rouge-code"><pre><span class="c1">// Can not be extended</span>
<span class="n">record</span> <span class="nf">Name</span><span class="o">(</span><span class="nc">String</span> <span class="n">first</span><span class="o">,</span> <span class="nc">String</span> <span class="n">last</span><span class="o">)</span> <span class="o">{};</span>
<span class="c1">// Default constructor</span>
<span class="kt">var</span> <span class="n">name</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Name</span><span class="o">(</span><span class="s">"john"</span><span class="o">,</span> <span class="s">"doe"</span><span class="o">);</span>
<span class="c1">// equals, hashCode, toString for free</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">name</span><span class="o">);</span> <span class="c1">// prints Name[first=john, last=doe]</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">name</span><span class="o">.</span><span class="na">equals</span><span class="o">(</span><span class="k">new</span> <span class="nc">Name</span><span class="o">(</span><span class="s">"john"</span><span class="o">,</span> <span class="s">"doe"</span><span class="o">)));</span>
<span class="c1">// default accessors</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">name</span><span class="o">.</span><span class="na">first</span><span class="o">()</span> <span class="o">+</span> <span class="s">" "</span> <span class="o">+</span> <span class="n">name</span><span class="o">.</span><span class="na">last</span><span class="o">());</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Optionally, you can implement additional constructors and methods:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
</pre></td><td class="rouge-code"><pre><span class="n">record</span> <span class="nf">ComplexName</span><span class="o">(</span><span class="nc">String</span> <span class="n">first</span><span class="o">,</span> <span class="nc">String</span> <span class="n">last</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nf">ComplexName</span><span class="o">(</span><span class="nc">String</span> <span class="n">fullName</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">(</span><span class="n">fullName</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">"\s+"</span><span class="o">)[</span><span class="mi">0</span><span class="o">],</span> <span class="n">fullName</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">"\s+"</span><span class="o">)[</span><span class="mi">1</span><span class="o">]);</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="nc">String</span> <span class="nf">funnyPrint</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="s">":) first="</span> <span class="o">+</span> <span class="n">first</span><span class="o">()</span> <span class="o">+</span> <span class="s">", last="</span> <span class="o">+</span> <span class="n">last</span><span class="o">();</span>
<span class="o">}</span>
<span class="o">};</span>
<span class="c1">// Default constructor</span>
<span class="kt">var</span> <span class="n">complexName1</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ComplexName</span><span class="o">(</span><span class="s">"john"</span><span class="o">,</span> <span class="s">"doe"</span><span class="o">);</span>
<span class="c1">// Custom constructor</span>
<span class="kt">var</span> <span class="n">complexName2</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ComplexName</span><span class="o">(</span><span class="s">"john doe"</span><span class="o">);</span>
<span class="c1">// Call the custom method</span>
<span class="n">complexName1</span><span class="o">.</span><span class="na">funnyPrint</span><span class="o">();</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Developers must be careful and ensure that all record’s member variables are
immutable. Otherwise, the record won’t be really immutable:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre><span class="n">record</span> <span class="nf">BadRecord</span><span class="o">(</span><span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">list</span><span class="o">){};</span>
<span class="c1">// Member variable isn't immutable ...</span>
<span class="kt">var</span> <span class="n">bad</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">BadRecord</span><span class="o">(</span><span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><>(</span><span class="nc">List</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"1"</span><span class="o">,</span> <span class="s">"2"</span><span class="o">,</span> <span class="s">"3"</span><span class="o">)));</span>
<span class="c1">// We can modify the record</span>
<span class="n">bad</span><span class="o">.</span><span class="na">list</span><span class="o">().</span><span class="na">remove</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="sealed-classes" />
<h1 id="sealed-classes">Sealed classes</h1>
<p>Sealed classes are a new feature which allows you to control how a class hierarchy is extended.</p>
<p>Previously, a class could be either <code class="language-plaintext highlighter-rouge">final</code> or freely extendable.
What if you need to have your own class hierarchy, but want to prevent client programmers
(e.g. library users) extending it or part of it?</p>
<p>For example, you can create a class <code class="language-plaintext highlighter-rouge">ImmutableSet</code> and a subclass <code class="language-plaintext highlighter-rouge">ImmutableOrderedSet</code>.
You don’t want client programmers extending <code class="language-plaintext highlighter-rouge">ImmutableSet</code> because they can make it mutable.
However, it can’t be <code class="language-plaintext highlighter-rouge">final</code>, because <code class="language-plaintext highlighter-rouge">ImmutableOrderedSet</code> needs to inherit from it.</p>
<p>Now you can do this by making it a <code class="language-plaintext highlighter-rouge">sealed</code> class that
<code class="language-plaintext highlighter-rouge">permits</code> only <code class="language-plaintext highlighter-rouge">ImmutableOrderedSet</code> to extend it.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre><span class="c1">// Not final - can be extended by one class only (in this example)</span>
<span class="n">sealed</span> <span class="kd">class</span> <span class="nc">ImmutableSet</span> <span class="n">permits</span> <span class="nc">ImmutableOrderedSet</span> <span class="o">{</span> <span class="cm">/*...*/</span> <span class="o">}</span>
<span class="kd">final</span> <span class="kd">class</span> <span class="nc">ImmutableOrderedSet</span> <span class="kd">extends</span> <span class="nc">ImmutableSet</span> <span class="o">{</span> <span class="cm">/*...*/</span> <span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Java requires that a permitted subclass must be marked as either:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">final</code> - can’t be extended;</li>
<li><code class="language-plaintext highlighter-rouge">sealed</code> - the subclass is itself sealed and permits limited extension;</li>
<li><code class="language-plaintext highlighter-rouge">non-sealed</code> - the subclass can be freely extended.</li>
</ul>
<p>The following example depicts how to use sealed classes:</p>
<figure>
<img src="/images/blog/java-9-to-now/sealed-classes.png" alt="Sealed classes demonstration" />
<figcaption>Example sealed class hierarchy.</figcaption>
</figure>
<p>And here is a complete code of the example:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
</pre></td><td class="rouge-code"><pre><span class="c1">// A sealed class can only be extended from the classes it permits</span>
<span class="n">sealed</span> <span class="kd">abstract</span> <span class="kd">class</span> <span class="nc">Loan</span> <span class="n">permits</span> <span class="nc">Mortgage</span><span class="o">,</span> <span class="nc">CarLoan</span><span class="o">,</span> <span class="nc">PersonalLoan</span> <span class="o">{</span>
<span class="kd">protected</span> <span class="kt">long</span> <span class="n">annualIncome</span><span class="o">;</span>
<span class="kd">abstract</span> <span class="kt">boolean</span> <span class="nf">isApproved</span><span class="o">();</span>
<span class="o">}</span>
<span class="c1">// A final sub class</span>
<span class="kd">final</span> <span class="kd">class</span> <span class="nc">Mortgage</span> <span class="kd">extends</span> <span class="nc">Loan</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kt">long</span> <span class="n">housePrice</span><span class="o">;</span>
<span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">isApproved</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">housePrice</span> <span class="o">/</span> <span class="o">(</span><span class="kt">float</span><span class="o">)</span><span class="n">annualIncome</span> <span class="o"><</span> <span class="mi">10</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="c1">// A sealed subclass - allows one more sub-sub-class</span>
<span class="n">sealed</span> <span class="kd">class</span> <span class="nc">CarLoan</span> <span class="kd">extends</span> <span class="nc">Loan</span> <span class="n">permits</span> <span class="nc">TaxiCarLoan</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kt">long</span> <span class="n">carValue</span><span class="o">;</span>
<span class="kd">public</span> <span class="kt">long</span> <span class="n">carMaintenancePerYear</span><span class="o">;</span>
<span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">isApproved</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">carValue</span> <span class="o">/</span> <span class="n">annualIncome</span> <span class="o"><</span> <span class="mi">2</span> <span class="o">&&</span>
<span class="n">carMaintenancePerYear</span> <span class="o"><</span> <span class="mf">0.1</span> <span class="o">*</span> <span class="n">annualIncome</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="kd">final</span> <span class="kd">class</span> <span class="nc">TaxiCarLoan</span> <span class="kd">extends</span> <span class="nc">CarLoan</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kt">long</span> <span class="n">taxiRegoFee</span><span class="o">;</span>
<span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">isApproved</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="kd">super</span><span class="o">.</span><span class="na">isApproved</span><span class="o">()</span> <span class="o">&&</span> <span class="n">taxiRegoFee</span> <span class="o"><</span> <span class="mi">1000</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="c1">// non-sealed class - can be freely extended</span>
<span class="n">non</span><span class="o">-</span><span class="n">sealed</span> <span class="kd">class</span> <span class="nc">PersonalLoan</span> <span class="kd">extends</span> <span class="nc">Loan</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kt">long</span> <span class="n">loanValue</span><span class="o">;</span>
<span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">isApproved</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">loanValue</span> <span class="o">/</span> <span class="o">(</span><span class="kt">float</span><span class="o">)</span> <span class="n">annualIncome</span> <span class="o"><</span> <span class="mf">0.1</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="c1">// External code can extend PersonalLoan without restrictions</span>
<span class="kd">class</span> <span class="nc">LoanSharkProduct</span> <span class="kd">extends</span> <span class="nc">PersonalLoan</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">isApproved</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="kc">true</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="garbage-collection" />
<h1 id="garbage-collection-algorithms">Garbage Collection Algorithms</h1>
<p>Java is one of the few platforms to offer developers fine grained
control over which Garbage Collection (GC) algorithm is used and its
parameters. In recent years, Java changed the default GC algorithm to
<a href="https://www.redhat.com/en/blog/part-1-introduction-g1-garbage-collector">G1</a>
and introduced the <a href="https://hub.packtpub.com/getting-started-with-z-garbage-collectorzgc-in-java-11-tutorial/">ZGC</a>
and <a href="https://wiki.openjdk.java.net/display/shenandoah/Main">Shenandoah</a>
algorithms for low latency applications. In this section, I’ll summarise
the main GC algorithms (old and new) and how they compare.</p>
<p><strong>Serial GC</strong> is the simplest GC algorithm, tailored
for single threaded applications.
It works on a single thread and stops the app while running.
It uses a mark-compact collection method, which clears the heap of
dereferenced objects and then compacts it into a contiguous memory space.</p>
<!-- Hence, allocating new memory blocks is faster. -->
<p><strong>Parallel GC</strong> was the default before Java 9. It’s a <em>generational</em> algorithm,
meaning that it divides the heap into sections called “generations”.
The idea is that new objects are more likely to require collection
than older objects. Hence, younger generations are cleaned more often
and the surviving objects are gradually moved to the older generations.
Although the algorithm is called Parallel, it doesn’t run
in parallel with the application. It pauses the app, just like <strong>Serial GC</strong>.
However, the garbage collection itself should be much faster since it uses
multiple threads.</p>
<p>The <strong>CMS (Concurrent Mark Sweep)</strong> algorithm is now deprecated. It’s similar
to the <strong>Parallel GC</strong>, but it attempts not to pause the application. It consumes
more CPU than other algorithms as it needs to keep track of which heap areas
the application threads are using. It may still pause the application threads
while cleaning the oldest heap generations.</p>
<p>As mentioned, <strong>Garbage-first (G1)</strong> is now the default. It divides the
heap into equal sized regions - usually a few megabytes each.
The regions are then classified into a few generations (youngest to oldest).
New objects are allocated in the youngest generation regions and
over time they’re either garbage collected or moved to older regions.
G1 uses multiple threads to scan the regions and collects from the regions
with the most dead objects. Because it works on selected regions, G1
minimises the amount of time the app is paused.</p>
<p>The <strong>Epsilon Garbage Collector</strong> doesn’t actually collect any garbage.
Thus, apps can only allocate memory on the heap, but it is never deleted.
It’s used for low latency apps where developers are well aware
of the memory consumption. It’s also useful for short lived jobs
where the upper limit of allocated memory is small.</p>
<p>The <strong>Z Garbage Collector (ZGC)</strong> runs an analysis of the heap known as
marking. For each object reference, ZGC stores the object state
(e.g. ready for collection, or being relocated) in
unused bits of the reference itself - a technique known as
colouring.</p>
<p>ZGC uses <code class="language-plaintext highlighter-rouge">load barriers</code> which are callbacks executed by the JVM every time
a thread loads a reference. This allows ZGC to inspect the reference’s state
flags, and if it’s to be relocated, it can return a different
reference to the calling thread.
Hence, ZGC can move around objects and compact the heap in parallel with
the app without stopping it and using additional data structures.</p>
<p>In general, ZGC pauses the app very rarely (e.g. scanning for root references)
and runs in parallel with it.</p>
<p>The <strong>Shenandoah GC</strong> algorithm also runs in parallel with the app,
compacts the heap in real time, and minimises app pauses.
Unlike ZGC, it uses additional data
structures to keep track of objects’ state and hence its
memory and CPU footprints are much higher.</p>
<div id="java-modules" />
<h1 id="java-modules-overview">Java Modules (Overview)</h1>
<p>Java modules are new in Java 9.
A <code class="language-plaintext highlighter-rouge">module</code> is a collection of java packages
and can be distributed as a jar file. A module declares its dependencies
on other modules and exposes some of its packages via a manifest file.
Only the exposed packages can be used by other modules.</p>
<p>Most importantly, the JDK itself has been broken into modules. Hence, you can
cherry pick which part of the JDK you need to package with your app.
This leads to smaller and more secure executables and runtime environments
(e.g. Docker images).</p>
<p>Unfortunately, some popular libraries haven’t migrated to modules yet.
Also, there is no interoperability with <a href="https://www.osgi.org/">OSGI</a>
which was the de facto standard for app modularisation before Java 9.</p>
<p>For more in-depth discussion of the syntax of modules, please
check out this overview of <a href="http://tutorials.jenkov.com/java/modules.html">Java Modules</a>.</p>
<div id="resources" />
<h1 id="resources">Resources</h1>
<ul>
<li><a href="https://www.journaldev.com/13121/java-9-features-with-examples">Java 9 - 17 Features with Examples</a></li>
<li><a href="https://www.marcobehler.com/guides/a-guide-to-java-versions-and-features">Java Versions and Features</a></li>
<li><a href="https://medium.com/@hasithalgamge/seven-types-of-java-garbage-collectors-6297a1418e82">Seven Types of Java Garbage Collectors</a></li>
<li><a href="https://www.baeldung.com/jvm-garbage-collectors">JVM Garbage Collectors</a></li>
<li><a href="https://hub.packtpub.com/getting-started-with-z-garbage-collectorzgc-in-java-11-tutorial/">Getting started with Z Garbage Collector (ZGC) in Java 11</a></li>
<li><a href="https://www.freecodecamp.org/news/garbage-collection-in-java-what-is-gc-and-how-it-works-in-the-jvm/">Garbage Collection in Java – What is GC and How it Works in the JVM</a></li>
<li><a href="http://tutorials.jenkov.com/java/modules.html">Java Modules</a></li>
<li><a href="https://blogs.oracle.com/javamagazine/post/java-sealed-classes-fight-ambiguity">Fight ambiguity and improve your code with Java 17’s sealed classes</a></li>
</ul>Nikolay Grozevnikolay.grozev@gmail.comThis post summarises the new features in Java after version 9. It covers the most prominent features from a developer's perspective. It either omits or gives a high level overview of less commonly used features.How To Start A Debug Container in Kubernetes2020-07-29T05:22:09+00:002020-07-29T05:22:09+00:00https://nikgrozev.com/2020/07/29/debug-container-in-kubernetes<h1 id="background">Background</h1>
<p>On a recent project, I’ve been troubleshooting some connectivity issues in a Kubernetes cluster.
The pods were failing to talk
to external on-prem systems and I had to prepare a
<a href="https://stackoverflow.com/help/minimal-reproducible-example">Minimal, Reproducible Example (MRE)</a> for
the network administrators.</p>
<p>In this post, I’ll demonstrate how to
start a temporary debug container in a K8S cluster
and open up a terminal sessions into it. This will allow us
to test various network issues without leaving any permanent
pods behind.</p>
<h1 id="the-test-pod--container">The Test Pod & Container</h1>
<p>Let’s start by creating a YAML file which defines the test pod:</p>
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">testcurl</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">curl</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">curlimages/curl</span>
<span class="na">command</span><span class="pi">:</span> <span class="pi">[</span> <span class="s2">"</span><span class="s">sleep"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">600"</span> <span class="pi">]</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>In the above, we’re defining a new pod with a single container
based on the <code class="language-plaintext highlighter-rouge">curlimages/curl</code> image. This is a minimalistic
image (about 11MB) which includes <code class="language-plaintext highlighter-rouge">curl</code> - you can obviously choose to use a bigger image with more networking tools.</p>
<p><strong>The container will complete in 10 minutes and the pod will
be die/exit.</strong> If you need more time for your troubleshooting
please increase the sleep interval in the above config.</p>
<p>Let’s create the pod in the cluster:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>kubectl apply <span class="nt">-f</span> pod.yaml
</pre></td></tr></tbody></table></code></pre></div></div>
<p>After the pod starts, you should be able to open a terminal
into the container and execute <code class="language-plaintext highlighter-rouge">curl</code> commands:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="c"># Will open up a terminal session into the container</span>
kubectl <span class="nb">exec</span> <span class="nt">-it</span> testcurl <span class="nt">--</span> sh
<span class="c"># We can now curl external addresses or internal services:</span>
<span class="o">></span> curl http://example.com/
<span class="o">></span> curl myservice/health
</pre></td></tr></tbody></table></code></pre></div></div>Nikolay Grozevnikolay.grozev@gmail.comThis post demonstrates how to start a temporary debug container in a K8S cluster and open up a terminal sessions into it. This will allow us to test various network issues without leaving any permanent pods behind.Improve Your TypeScript With Static Analysis2020-03-22T05:22:09+00:002020-03-22T05:22:09+00:00https://nikgrozev.com/2020/03/22/improve-your-typescript-with-static-analysis<h1 id="table-of-contents">Table of Contents</h1>
<ul>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#eslint">ESLint and Friends</a></li>
<li><a href="#jest">Jest and Code Coverage</a></li>
<li><a href="#jest-cra">Jest and Create React App (CRA)</a></li>
<li><a href="#dep-audit">Dependency Audit / Supply Chain Analysis</a></li>
<li><a href="#sonar-qube">Local SonarQube</a></li>
<li><a href="#sonar-analysis">Run SonarQube Analysis</a></li>
<li><a href="#husky">Git Hooks with Husky</a></li>
</ul>
<div id="introduction" />
<h1 id="introduction">Introduction</h1>
<p>Recently, I’ve been working on a number of TypeScript projects.
This includes both back-end Node JS APIs and front-end React user intefaces.
I wanted to automate the code quality assessment as much as possible,
but it turned out this is not trivial.</p>
<p>In this post, I’ll describe a set of tools for static analysis and automated audits.
I’ll demonstrate how to configure and execute them locally so you can get quick
feedback during development. The same scripts and commands can be used in any CI/CD tool.</p>
<p>Most of these tools and configurations are not specific to TypeScript
and can be adapted to plain JavaScript. However, if you’re looking at automated
code quality tooling for large code bases
<a href="https://www.youtube.com/watch?v=wYgSiFaYSSo">then TypeScript is your friend</a>.</p>
<p>There are many code snippets in this post. At any point, you can refer to
<a href="https://github.com/nikolayg/secure-typescript-boilerplate">the GitHub repo</a>
which has all these tools and configurations ready to go.
Just clone and play with it!</p>
<div id="eslint" />
<h1 id="eslint-and-friends">ESLint and Friends</h1>
<p><a href="https://eslint.org/">ESLint</a> has replaced <a href="https://palantir.github.io/tslint/">TSLint</a>
as the go-to static analsys tool for TypeScript. ESLint has a
plugin architecture, where additional rules and language support can be
installed and configured.
You can configure ESLint’s plugins and rules
via a <code class="language-plaintext highlighter-rouge">.eslintrc.js</code> file in the root of your project.</p>
<p>We’ll need the following ESLint plugins and auxiliary packages:</p>
<ul>
<li><a href="https://www.npmjs.com/package/@typescript-eslint/parser">@typescript-eslint/parser</a> - ESLint parser for TypeScript;</li>
<li><a href="https://www.npmjs.com/package/@typescript-eslint/eslint-plugin">@typescript-eslint/eslint-plugin</a> - ESLint rules for TypeScript;</li>
<li><a href="https://www.npmjs.com/package/eslint-plugin-import">eslint-plugin-import</a> - rules ensuring the proper use of <code class="language-plaintext highlighter-rouge">import</code> statements;</li>
<li><a href="https://www.npmjs.com/package/eslint-plugin-sonarjs">eslint-plugin-sonarjs</a> - code quality ESLint rules from <a href="https://www.sonarsource.com/">Sonar Source</a>;</li>
</ul>
<p><a href="https://prettier.io/">Prettier</a> is a popular code formatting tool. If you want to integrate it with ESLint, we’ll need the following.</p>
<ul>
<li><a href="https://www.npmjs.com/package/prettier">prettier</a> - the <a href="https://prettier.io/">Prettier</a> tool itself;</li>
<li><a href="https://www.npmjs.com/package/eslint-config-prettier">eslint-config-prettier</a> - rules for the <a href="https://prettier.io/">Prettier</a> code formatter;</li>
<li><a href="https://www.npmjs.com/package/eslint-plugin-prettier">eslint-plugin-prettier</a> - turns off ESLint rules which might conflict with Prettier;</li>
<li><a href="https://www.npmjs.com/package/prettier-eslint">prettier-eslint</a> - allows ESLint to
<a href="https://eslint.org/docs/user-guide/command-line-interface#fix">auto-fix</a> formatting issues
in your code;</li>
</ul>
<p>If you’re using React, you’ll also need:</p>
<ul>
<li><a href="https://www.npmjs.com/package/eslint-plugin-react">eslint-plugin-react</a> - rules for React;</li>
<li><a href="https://www.npmjs.com/package/eslint-plugin-react-hooks">eslint-plugin-react-hooks</a> - enforces the <a href="https://reactjs.org/docs/hooks-rules.html">React hook rules</a>;</li>
</ul>
<p>Let’s install all of these via <code class="language-plaintext highlighter-rouge">npm</code> or <code class="language-plaintext highlighter-rouge">yarn</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
</pre></td><td class="rouge-code"><pre><span class="c"># Core TypeScript ESLint support</span>
npm <span class="nb">install</span> <span class="nt">--save-dev</span> @typescript-eslint/parser
npm <span class="nb">install</span> <span class="nt">--save-dev</span> @typescript-eslint/eslint-plugin
<span class="c"># Extra code quality rules</span>
npm <span class="nb">install</span> <span class="nt">--save-dev</span> eslint-plugin-import
npm <span class="nb">install</span> <span class="nt">--save-dev</span> eslint-plugin-sonarjs
<span class="c"># Prettier and its ESLint integration</span>
npm <span class="nb">install</span> <span class="nt">--save-dev</span> prettier
npm <span class="nb">install</span> <span class="nt">--save-dev</span> eslint-config-prettier
npm <span class="nb">install</span> <span class="nt">--save-dev</span> eslint-plugin-prettier
npm <span class="nb">install</span> <span class="nt">--save-dev</span> prettier-eslint
<span class="c"># React rules for ESLint (Optional)</span>
npm <span class="nb">install</span> <span class="nt">--save-dev</span> eslint-plugin-react eslint-plugin-react-hooks
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now let’s create the <code class="language-plaintext highlighter-rouge">.eslintrc.js</code> configuration file.
<a href="https://github.com/nikolayg/secure-typescript-boilerplate/blob/master/.eslintrc.js">Here is the full file</a>, below is just a highlight of the important sections:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
</pre></td><td class="rouge-code"><pre><span class="c1">// `.eslintrc.js</span>
<span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">parser</span><span class="p">:</span> <span class="dl">'</span><span class="s1">@typescript-eslint/parser</span><span class="dl">'</span><span class="p">,</span> <span class="c1">// Parse TypeScript</span>
<span class="na">parserOptions</span><span class="p">:</span> <span class="p">{</span>
<span class="na">project</span><span class="p">:</span> <span class="dl">'</span><span class="s1">./tsconfig.json</span><span class="dl">'</span><span class="p">,</span>
<span class="na">jsx</span><span class="p">:</span> <span class="kc">false</span> <span class="c1">// True for React</span>
<span class="p">},</span>
<span class="na">rules</span><span class="p">:</span> <span class="p">{</span>
<span class="cm">/* disable or configure individual rules */</span>
<span class="cm">/* Will need the following for React hooks: */</span>
<span class="c1">// "react-hooks/rules-of-hooks": "error",</span>
<span class="c1">// "react-hooks/exhaustive-deps": "warn"</span>
<span class="p">},</span>
<span class="c1">// Use the rules from these plugins </span>
<span class="na">extends</span><span class="p">:</span> <span class="p">[</span>
<span class="dl">'</span><span class="s1">plugin:@typescript-eslint/recommended</span><span class="dl">'</span><span class="p">,</span>
<span class="dl">'</span><span class="s1">prettier/@typescript-eslint</span><span class="dl">'</span><span class="p">,</span>
<span class="dl">'</span><span class="s1">prettier</span><span class="dl">'</span><span class="p">,</span>
<span class="dl">'</span><span class="s1">plugin:prettier/recommended</span><span class="dl">'</span><span class="p">,</span>
<span class="dl">'</span><span class="s1">plugin:sonarjs/recommended</span><span class="dl">'</span><span class="p">,</span>
<span class="c1">// 'plugin:react/recommended' // If we need React</span>
<span class="p">]</span>
<span class="p">};</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>You can configure the behaviour or Prettier with a <a href="https://github.com/nikolayg/secure-typescript-boilerplate/blob/master/.prettierrc">.prettierrc</a> and <a href="https://github.com/nikolayg/secure-typescript-boilerplate/blob/master/.prettierignore">.prettierignore</a>
files in the root of the project.</p>
<p>Finally, let’s add two additional commands to our <code class="language-plaintext highlighter-rouge">package.json</code>. They’ll allow us
to run <code class="language-plaintext highlighter-rouge">ESLint</code> and automatically fix formatting issues:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre>{
...
"scripts": {
...
"lint": "eslint './src/**/*.{tsx,ts}'",
"lint-fix": "eslint './src/**/*.{tsx,ts}' --fix",
}
}
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="jest" />
<h1 id="jest-and-code-coverage">Jest and Code Coverage</h1>
<p><a href="https://jestjs.io/">Jest</a> has emerged as the most popular JavaScript testing framework.
To make it work with TypeScript we’ll need a helper module called <a href="https://www.npmjs.com/package/ts-jest">ts-jest</a>.
It dynamically compiles the TypeScript code.
We also need <code class="language-plaintext highlighter-rouge">jest</code> to generate a test coverage report. We’ll feed this report
into <a href="https://www.sonarqube.org/">SonarQube</a> later on for further analysis. There’s a handy module for this called <a href="https://www.npmjs.com/package/jest-sonar-reporter">jest-sonar-reporter</a>.</p>
<p>We can install them and <code class="language-plaintext highlighter-rouge">jest</code> itself via <code class="language-plaintext highlighter-rouge">npm</code> or <code class="language-plaintext highlighter-rouge">yarn</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre>npm <span class="nb">install</span> <span class="nt">--save-dev</span> jest @types/jest
npm <span class="nb">install</span> <span class="nt">--save-dev</span> ts-jest jest-sonar-reporter
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Jest can be configured via a file called <code class="language-plaintext highlighter-rouge">jest.config.js</code> in the project root folder.
We’ll use it to transform all test files matching the
<a href="https://www.reactnative.guide/7-testing/7.1-jest-setup.html">Jest naming convention</a>
with <code class="language-plaintext highlighter-rouge">ts-jest</code> and generate reports
via <code class="language-plaintext highlighter-rouge">jest-sonar-reporter</code>.
<a href="https://github.com/nikolayg/secure-typescript-boilerplate/blob/master/jest.config.js">Here is</a> the full file:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
</pre></td><td class="rouge-code"><pre><span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">coverageDirectory</span><span class="p">:</span> <span class="dl">'</span><span class="s1">./coverage</span><span class="dl">'</span><span class="p">,</span>
<span class="na">collectCoverageFrom</span><span class="p">:</span> <span class="p">[</span><span class="dl">'</span><span class="s1">src/**/*.ts</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">src/**/*.tsx</span><span class="dl">'</span><span class="p">],</span>
<span class="na">testEnvironment</span><span class="p">:</span> <span class="dl">'</span><span class="s1">node</span><span class="dl">'</span><span class="p">,</span>
<span class="na">modulePaths</span><span class="p">:</span> <span class="p">[</span><span class="dl">'</span><span class="s1"><rootDir>/src</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">node_modules</span><span class="dl">'</span><span class="p">],</span>
<span class="na">roots</span><span class="p">:</span> <span class="p">[</span><span class="dl">'</span><span class="s1">src</span><span class="dl">'</span><span class="p">],</span>
<span class="na">transform</span><span class="p">:</span> <span class="p">{</span>
<span class="dl">'</span><span class="s1">^.+</span><span class="se">\\</span><span class="s1">.tsx?$</span><span class="dl">'</span><span class="p">:</span> <span class="dl">'</span><span class="s1">ts-jest</span><span class="dl">'</span>
<span class="p">},</span>
<span class="na">testRegex</span><span class="p">:</span> <span class="dl">'</span><span class="s1">(/__tests__/.*|</span><span class="se">\\</span><span class="s1">.(test|spec))</span><span class="se">\\</span><span class="s1">.(ts|tsx)$</span><span class="dl">'</span><span class="p">,</span>
<span class="na">coverageReporters</span><span class="p">:</span> <span class="p">[</span><span class="dl">'</span><span class="s1">json</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">lcov</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">text</span><span class="dl">'</span><span class="p">],</span>
<span class="na">coveragePathIgnorePatterns</span><span class="p">:</span> <span class="p">[</span><span class="dl">'</span><span class="s1">.*/src/.*</span><span class="se">\\</span><span class="s1">.d</span><span class="se">\\</span><span class="s1">.ts</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">.*/src/testUtil/.*</span><span class="dl">'</span><span class="p">],</span>
<span class="na">testResultsProcessor</span><span class="p">:</span> <span class="dl">'</span><span class="s1">jest-sonar-reporter</span><span class="dl">'</span>
<span class="c1">// Use the below to set coverage goals.</span>
<span class="c1">// "npm test" will fail if these metrics are violated</span>
<span class="na">coverageThreshold</span><span class="p">:</span> <span class="p">{</span>
<span class="na">global</span><span class="p">:</span> <span class="p">{</span>
<span class="na">statements</span><span class="p">:</span> <span class="mi">80</span><span class="p">,</span>
<span class="na">branches</span><span class="p">:</span> <span class="mi">80</span><span class="p">,</span>
<span class="na">functions</span><span class="p">:</span> <span class="mi">80</span><span class="p">,</span>
<span class="na">lines</span><span class="p">:</span> <span class="mi">80</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">};</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now, we need to tell <code class="language-plaintext highlighter-rouge">jest-sonar-reporter</code> where to put the coverage output.
Edit <code class="language-plaintext highlighter-rouge">package.json</code> and add the following at the end:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre><span class="p">{</span><span class="w">
</span><span class="err">...</span><span class="w">
</span><span class="nl">"jestSonar"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"reportPath"</span><span class="p">:</span><span class="w"> </span><span class="s2">"coverage"</span><span class="p">,</span><span class="w">
</span><span class="nl">"reportFile"</span><span class="p">:</span><span class="w"> </span><span class="s2">"test-reporter.xml"</span><span class="p">,</span><span class="w">
</span><span class="nl">"indent"</span><span class="p">:</span><span class="w"> </span><span class="mi">4</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div></div>
<p>As a last step, we need to ensure we generate the coverage reports during testing.
Add the following to your scripts in <code class="language-plaintext highlighter-rouge">package.json</code>:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre><span class="nl">"test"</span><span class="p">:</span><span class="w"> </span><span class="s2">"jest --forceExit --detectOpenHandles --coverage"</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div></div>
<p>If you run <code class="language-plaintext highlighter-rouge">npm test</code> you should see a new folder <code class="language-plaintext highlighter-rouge">./coverage</code> with the code coverage reports.</p>
<div id="jest-cra" />
<h1 id="jest-and-create-react-app-cra">Jest and Create React App (CRA)</h1>
<p>Many React projects use the <a href="https://create-react-app.dev/">Create React App</a> code
generator. CRA uses Jest internally but hides many of its properties.
Also, it doesn’t pick up the configuration in <code class="language-plaintext highlighter-rouge">jest.config.js</code>.</p>
<p>Fortunately, you can still configure the coverage output directly in the
<code class="language-plaintext highlighter-rouge">package.json</code> script:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="p">{</span><span class="w">
</span><span class="err">...</span><span class="w">
</span><span class="nl">"scripts"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"test"</span><span class="p">:</span><span class="w"> </span><span class="s2">"CI=true react-scripts test --silent --env=jsdom --coverage --testResultsProcessor jest-sonar-reporter"</span><span class="p">,</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div></div>
<p>You will still need to install <code class="language-plaintext highlighter-rouge">jest-sonar-reporter</code> and add the <code class="language-plaintext highlighter-rouge">"jestSonar"</code>
configuration as in the previous section.</p>
<div id="dep-audit" />
<h1 id="dependency-audit--supply-chain-analysis">Dependency Audit / Supply Chain Analysis</h1>
<p>A large JavaScript project can have hundreds of direct dependencies.
Each of them can have many dependencies on its own.
A security vulnerability in any of them can become a vulnerability for
the entire project. This is known as a <a href="https://devops.com/software-supply-chain-attacks-how-to-disrupt-attackers/">Supply Chain Attack</a>.</p>
<p>To address this, both <code class="language-plaintext highlighter-rouge">npm</code> and <code class="language-plaintext highlighter-rouge">yarn</code> introduced a new command called <code class="language-plaintext highlighter-rouge">audit</code>.
It checks whether your dependencies have known vulnerabilities and provides a report.
Unfortunately, working directly with <code class="language-plaintext highlighter-rouge">npm audit</code> can be tricky. It provides
limited options to filter threats by severity and to white list modules.
It can generate a lot of noise.</p>
<p>There are a few auxiliary tools which “wrap” the audit command
and give developers much more control. A popular option is
<a href="https://www.npmjs.com/package/audit-ci">audit-ci</a>.
It automatically detects whether you’re using <code class="language-plaintext highlighter-rouge">npm</code> or <code class="language-plaintext highlighter-rouge">yarn</code> and gives you plenty
of configuration options. Let’s install it:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>npm <span class="nb">install</span> <span class="nt">--save-dev</span> audit-ci
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now, let’s create a file <code class="language-plaintext highlighter-rouge">audit-ci.json</code> with its config:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="p">{</span><span class="w">
</span><span class="nl">"high"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span><span class="w">
</span><span class="nl">"package-manager"</span><span class="p">:</span><span class="w"> </span><span class="s2">"auto"</span><span class="p">,</span><span class="w">
</span><span class="nl">"report-type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"summary"</span><span class="p">,</span><span class="w">
</span><span class="nl">"path-whitelist"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="s2">"1217|sonarqube-scanner>download>decompress"</span><span class="p">,</span><span class="w">
</span><span class="s2">"1217|sonarqube-verify>sonarqube-scanner>download>decompress"</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div></div>
<p>In the above, we asked <code class="language-plaintext highlighter-rouge">audit-ci</code> to fail only if there’re threats classified as
<code class="language-plaintext highlighter-rouge">high</code> or more critical. We also white listed a couple of threats from development
dependencies so they will not cause failure.</p>
<p>Lastly, let’s configure a script for <code class="language-plaintext highlighter-rouge">audit-ci</code> in <code class="language-plaintext highlighter-rouge">package.json</code>:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="p">{</span><span class="w">
</span><span class="err">...</span><span class="w">
</span><span class="nl">"scripts"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"audit-dependencies"</span><span class="p">:</span><span class="w"> </span><span class="s2">"audit-ci --config audit-ci.json"</span><span class="p">,</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div></div>
<p>Now, you should be able to run <code class="language-plaintext highlighter-rouge">npm run audit-dependencies</code> and analyze your
supply chain for vulnerabilities.</p>
<div id="sonar-qube" />
<h1 id="local-sonarqube">Local SonarQube</h1>
<p><a href="https://www.sonarqube.org/">SonarQube</a> is a popular tool for static source
code analysis. It supports many languages including TypeScript.
Typically, a company would have a SonarQube instance which analyses
all of its projects. The CI/CD pipeline would push your code to the
SonarQube instance during each build.</p>
<p>I often find it convenient to run SonarQube locally so I can quickly analyse
my code before integrating it. This is quite easy with a
<a href="https://dev.to/rafaeldias97/nodejs-express-docker-jest-sonarqube-45me">Docker container</a>.</p>
<p>Let’s create a <code class="language-plaintext highlighter-rouge">docker-compose.sonar.yml</code> file:</p>
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre><span class="na">version</span><span class="pi">:</span> <span class="s1">'</span><span class="s">3'</span>
<span class="na">services</span><span class="pi">:</span>
<span class="na">sonarqube</span><span class="pi">:</span>
<span class="na">container_name</span><span class="pi">:</span> <span class="s">sonarqube</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">sonarqube:latest</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">9000:9000'</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">9092:9092'</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Then we can start/stop SonarQube with:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="c"># Start it ...</span>
docker-compose <span class="nt">-f</span> docker-compose.sonar.yml up <span class="nt">-d</span>
<span class="c"># Stop it ...</span>
docker-compose <span class="nt">-f</span> docker-compose.sonar.yml down
</pre></td></tr></tbody></table></code></pre></div></div>
<p>After starting it, wait for Sonar to load on
<a href="http://localhost:9000">http://localhost:9000</a> (it can take 1-2mins).
Navigate to <a href="http://localhost:9000/dashboard">http://localhost:9000/dashboard</a>
and use <code class="language-plaintext highlighter-rouge">admin</code>/<code class="language-plaintext highlighter-rouge">admin</code> to login.</p>
<p>Note that the local SonarQube instance uses in-memory data storage.
Any changes you make will be wiped out on restart.</p>
<div id="sonar-analysis" />
<h1 id="run-sonarqube-analysis">Run SonarQube Analysis</h1>
<p>Before you analyse a project in SonarQube, you need to create a config file
called <code class="language-plaintext highlighter-rouge">sonar-project.properties</code> in the root folder:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
</pre></td><td class="rouge-code"><pre>sonar.projectKey=secure-typescript-boilerplate
sonar.projectName=secure-typescript-boilerplate
sonar.projectVersion=1.0
sonar.language=ts
sonar.sources=src
sonar.sourceEncoding=UTF-8
sonar.exclusions=src/**/*.test.ts
sonar.test.inclusions=src/**/*.test.ts
sonar.coverage.exclusions=src/**/*.test.ts,src/**/*.mock.ts,node_modules/*,coverage/lcov-report/*
sonar.typescript.lcov.reportPaths=coverage/lcov.info
sonar.testExecutionReportPaths=coverage/test-reporter.xml
</pre></td></tr></tbody></table></code></pre></div></div>
<p>This file defines the project name, key, and version. It also specifies the programming
language, code location, and the code coverage report.</p>
<p>To run the SonarQube analysis we will need an auxiliary module
called <a href="https://www.npmjs.com/package/sonarqube-scanner">sonarqube-scanner</a>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>npm <span class="nb">install</span> <span class="nt">--save-dev</span> sonarqube-scanner
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The module expects to find a file called <code class="language-plaintext highlighter-rouge">sonar-project.js</code> in the project root.
Let’s create it:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
</pre></td><td class="rouge-code"><pre><span class="kd">const</span> <span class="nx">sonarqubeScanner</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">sonarqube-scanner</span><span class="dl">'</span><span class="p">);</span>
<span class="nx">sonarqubeScanner</span><span class="p">(</span>
<span class="p">{</span>
<span class="na">serverUrl</span><span class="p">:</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">SONAR_SERVER</span> <span class="o">||</span> <span class="dl">'</span><span class="s1">http://localhost:9000</span><span class="dl">'</span><span class="p">,</span>
<span class="na">token</span><span class="p">:</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">SONAR_TOKEN</span> <span class="o">||</span> <span class="dl">''</span><span class="p">,</span>
<span class="na">options</span><span class="p">:</span> <span class="p">{}</span>
<span class="p">},</span>
<span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">>> Sonar analysis is done!</span><span class="dl">'</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now we can run <code class="language-plaintext highlighter-rouge">sonarqube-scanner</code> with <code class="language-plaintext highlighter-rouge">node sonar-project.js</code> and this will submit
our code sonar server.
By default, the local server is used, but this can be overriden via environment
variables. Go on and test it to make sure it works.</p>
<p>The <code class="language-plaintext highlighter-rouge">sonarqube-scanner</code> module has one shortcoming. It starts the SonarQube analysis
asynchronously and it doesn’t wait for it to complete. This is problematic
when you want to integrate the code scan into a CI/CD pipeline. You may
need to wait for the analysis to complete and either fail/proceed based on the
result.</p>
<p>Thus, we’ll use another module called
<a href="https://www.npmjs.com/package/sonarqube-verify">sonarqube-verify</a>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>npm <span class="nb">install</span> <span class="nt">--save-dev</span> sonarqube-verify
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">sonarqube-verify</code> library is a wrapper of <code class="language-plaintext highlighter-rouge">sonarqube-scanner</code>.
It starts the code analsys and then checks its progress every few seconds.
On completion, it either suceeds or fails based on the analysis result.</p>
<p>Let’s create the script to run it:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="p">{</span><span class="w">
</span><span class="err">...</span><span class="w">
</span><span class="nl">"scripts"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"sonar"</span><span class="p">:</span><span class="w"> </span><span class="s2">"sonarqube-verify"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div></div>
<p>Now we can run the analysis:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre><span class="c"># Generate the test coverage report</span>
npm <span class="nb">test</span>
<span class="c"># Run the analysis (remember to start SonarQube)</span>
npm run sonar
<span class="c"># Visit http://localhost:9000/projects to see result </span>
<span class="c"># login with admin/admin</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Check out the documentation of <a href="https://www.npmjs.com/package/sonarqube-verify">sonarqube-verify</a>
to see how to push to a remote server via env variables.</p>
<div id="husky" />
<h1 id="git-hooks-with-husky">Git Hooks with Husky</h1>
<p><a href="https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks">Git hooks</a> are a
great way to ensure code quality before your code hits the CI/CD pipeline.
They can significantly speed up the developer feedback loop and can
enforce discipline in a team. <a href="https://www.npmjs.com/package/husky">Husky</a>
is a popular Node JS library for custom hooks. Let’s install it:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>npm <span class="nb">install</span> <span class="nt">--save-dev</span> husky
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now we need to create a <code class="language-plaintext highlighter-rouge">.huskyrc</code> file which defines the hook:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre><span class="nl">"hooks"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"pre-push"</span><span class="p">:</span><span class="w"> </span><span class="s2">"npm build && npm test && npm run lint && npm run audit-dependencies"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div></div>
<p>This ensures that the code builds, the tests pass, no ESLint errors are present,
and there’re no known vulnerabilities in the dependencies.
Depending on the project size these commands can take a while.
I prefer to use <code class="language-plaintext highlighter-rouge">pre-push</code> instead of <code class="language-plaintext highlighter-rouge">pre-commit</code> so that local commits
are quick, but everyone is forced to clean up their code before they integrate.
Every team has its own philosophy - use whatever works for you!</p>
<p>From now on, on every <code class="language-plaintext highlighter-rouge">git push</code> the above command will run. If it fails,
no code will be pushed to the remote repository.</p>Nikolay Grozevnikolay.grozev@gmail.comThis article describes a set of tools for static analysis and automated audits of TypeScript code.Switch Between Multiple Kubernetes Clusters With Ease2019-10-03T05:22:09+00:002019-10-03T05:22:09+00:00https://nikgrozev.com/2019/10/03/switch-between-multiple-kubernetes-clusters-with-ease<h1 id="background">Background</h1>
<p>I often need to switch between Kubernetes clusters for different projects.
Managing the <code class="language-plaintext highlighter-rouge">kubectl</code> configuration for each of them and their files can be cumbersome.</p>
<p>By default, the <code class="language-plaintext highlighter-rouge">kubectl</code> command line client uses the <code class="language-plaintext highlighter-rouge">~/.kube/config</code> file to store
the Kubernetes endpoint and credentials. If you use
<a href="https://minikube.sigs.k8s.io/">minikube</a>
or <a href="https://www.docker.com/products/docker-desktop">Docker Desktop</a>’s local Kubernetes,
you should be able to see their configurations in that file.</p>
<p>When you work with a cloud based Kubernetes instance, the cloud console will provide
the configuration as a <code class="language-plaintext highlighter-rouge">yml</code> file. You then need to specify the file
as the value of the environment variable <code class="language-plaintext highlighter-rouge">KUBECONFIG</code>, which is used by <code class="language-plaintext highlighter-rouge">kubectl</code>.
This can quickly become cumbersome and hard to maintain.</p>
<h1 id="multiple-contexts">Multiple Contexts</h1>
<p>To solve this problem, I’ve started storing all cloud cluster configuration <code class="language-plaintext highlighter-rouge">yml</code> files
in a new folder <code class="language-plaintext highlighter-rouge">~/.kube/custom-contexts/</code>. Usually, I’d make a folder for each cloud
Kube instance, which contains the <code class="language-plaintext highlighter-rouge">yml</code> config and other auxiliary files (e.g. <code class="language-plaintext highlighter-rouge">pem</code> certificates).</p>
<p>Here’s the content of my <code class="language-plaintext highlighter-rouge">~/.kube</code> folder:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre>~/.kube/
├── cache
│ └── ...
├── config
└── custom-contexts
└── NikTest-Kube-Config
├── ca-hou02-NikTest.pem
└── kube-config-hou02-NikTest.yml
</pre></td></tr></tbody></table></code></pre></div></div>
<p>It has a single Kubernetes configuration in the <code class="language-plaintext highlighter-rouge">NikTest-Kube-Config</code> folder which
contains the <code class="language-plaintext highlighter-rouge">yml</code> config and <code class="language-plaintext highlighter-rouge">pem</code> certificate downloaded from the IBM Cloud console.</p>
<p>Now we need to load all this configuration in the <code class="language-plaintext highlighter-rouge">KUBECONFIG</code> environment variable
so that <code class="language-plaintext highlighter-rouge">kubectl</code> can switch between them. The following script does the trick:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
</pre></td><td class="rouge-code"><pre><span class="c"># Set the default kube context if present</span>
<span class="nv">DEFAULT_KUBE_CONTEXTS</span><span class="o">=</span><span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/.kube/config"</span>
<span class="k">if </span><span class="nb">test</span> <span class="nt">-f</span> <span class="s2">"</span><span class="k">${</span><span class="nv">DEFAULT_KUBE_CONTEXTS</span><span class="k">}</span><span class="s2">"</span>
<span class="k">then
</span><span class="nb">export </span><span class="nv">KUBECONFIG</span><span class="o">=</span><span class="s2">"</span><span class="nv">$DEFAULT_KUBE_CONTEXTS</span><span class="s2">"</span>
<span class="k">fi</span>
<span class="c"># Additional contexts should be in ~/.kube/custom-contexts/ </span>
<span class="nv">CUSTOM_KUBE_CONTEXTS</span><span class="o">=</span><span class="s2">"</span><span class="nv">$HOME</span><span class="s2">/.kube/custom-contexts"</span>
<span class="nb">mkdir</span> <span class="nt">-p</span> <span class="s2">"</span><span class="k">${</span><span class="nv">CUSTOM_KUBE_CONTEXTS</span><span class="k">}</span><span class="s2">"</span>
<span class="nv">OIFS</span><span class="o">=</span><span class="s2">"</span><span class="nv">$IFS</span><span class="s2">"</span>
<span class="nv">IFS</span><span class="o">=</span><span class="s1">$'</span><span class="se">\n</span><span class="s1">'</span>
<span class="k">for </span>contextFile <span class="k">in</span> <span class="sb">`</span>find <span class="s2">"</span><span class="k">${</span><span class="nv">CUSTOM_KUBE_CONTEXTS</span><span class="k">}</span><span class="s2">"</span> <span class="nt">-type</span> f <span class="nt">-name</span> <span class="s2">"*.yml"</span><span class="sb">`</span>
<span class="k">do
</span><span class="nb">export </span><span class="nv">KUBECONFIG</span><span class="o">=</span><span class="s2">"</span><span class="nv">$contextFile</span><span class="s2">:</span><span class="nv">$KUBECONFIG</span><span class="s2">"</span>
<span class="k">done
</span><span class="nv">IFS</span><span class="o">=</span><span class="s2">"</span><span class="nv">$OIFS</span><span class="s2">"</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>I put the above code snippet in my <code class="language-plaintext highlighter-rouge">~/.bashrc</code> so that it loads on every new terminal session.
This way, if I add or delete configuration files from <code class="language-plaintext highlighter-rouge">~/.kube/custom-contexts</code>, this will be
reflected in every subsequent terminal session.</p>
<h1 id="switching-between-contexts">Switching Between Contexts</h1>
<p>Now we can list all preconfigured contexts and see which one is active:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre>kubectl config get-contexts
<span class="c"># CURRENT NAME CLUSTER AUTHINFO</span>
<span class="c"># NikTest NikTest <****> </span>
<span class="c"># docker-desktop docker-desktop docker-desktop </span>
<span class="c"># * minikube minikube minikube </span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>We can also get the name of the active context/cluster:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre><span class="c"># Prints the current config nema - e.g. minikube</span>
kubectl config current-context
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Finally, we can switch between the predefined contexts:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre><span class="c"># Switch to a context/cluster</span>
kubectl config use-context NikTest
</pre></td></tr></tbody></table></code></pre></div></div>Nikolay Grozevnikolay.grozev@gmail.comThis article describes how to work with multiple Kubernetes clusters in a structured and convenient way ...React’s useCallback and useMemo Hooks By Example2019-04-07T05:22:09+00:002019-04-07T05:22:09+00:00https://nikgrozev.com/2019/04/07/reacts-usecallback-and-usememo-hooks-by-example<h1 id="introduction">Introduction</h1>
<p>I recently started learning about the <a href="https://reactjs.org/docs/hooks-reference.html">React hooks</a> API and I was
amazed by how expressive it is. Hooks allow me to rewrite tens of lines of boilerplate code with
just a few lines. Unfortunately, this convenience comes at a cost.
I found that some more advanced hooks like <code class="language-plaintext highlighter-rouge">useCallback</code> and <code class="language-plaintext highlighter-rouge">useMemo</code>
are hard to learn and appear counter-intuitive at first.</p>
<p>In this article, I’ll demonstrate with a few simple examples why we need these hooks
and when and how to use them. This is not an introduction to hooks, and you must be familiar
with the <a href="https://reactjs.org/docs/hooks-reference.html#usestate">useState</a> hook to follow.</p>
<h1 id="the-problem">The Problem</h1>
<p>Before we start, let’s introduce a helper button component. We’ll use
<a href="https://reactjs.org/docs/react-api.html#reactmemo">React.memo</a>
to turn it into a memoized component. This will force React to
never re-render it, unless some of its properties change.
We’ll also add a random colour as its background
so we can track when it re-rerenders:</p>
<div class="language-tsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nx">React</span><span class="p">,</span> <span class="p">{</span> <span class="nx">useState</span><span class="p">,</span> <span class="nx">useCallback</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">react</span><span class="dl">'</span><span class="p">;</span>
<span class="c1">// Generates random colours any time it's called</span>
<span class="kd">const</span> <span class="nx">randomColour</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="dl">'</span><span class="s1">#</span><span class="dl">'</span><span class="o">+</span><span class="p">(</span><span class="nb">Math</span><span class="p">.</span><span class="nx">random</span><span class="p">()</span><span class="o">*</span><span class="mh">0xFFFFFF</span><span class="o"><<</span><span class="mi">0</span><span class="p">).</span><span class="nx">toString</span><span class="p">(</span><span class="mi">16</span><span class="p">);</span>
<span class="c1">// The type of the props</span>
<span class="kd">type</span> <span class="nx">ButtonProps</span> <span class="o">=</span> <span class="nx">React</span><span class="p">.</span><span class="nx">ButtonHTMLAttributes</span><span class="o"><</span><span class="nx">HTMLButtonElement</span><span class="o">></span><span class="p">;</span>
<span class="c1">// A memoized button with a random background colour</span>
<span class="kd">const</span> <span class="nx">Button</span> <span class="o">=</span> <span class="nx">React</span><span class="p">.</span><span class="nx">memo</span><span class="p">((</span><span class="nx">props</span><span class="p">:</span> <span class="nx">ButtonProps</span><span class="p">)</span> <span class="o">=></span>
<span class="p"><</span><span class="nt">button</span> <span class="na">onClick</span><span class="p">=</span><span class="si">{</span><span class="nx">props</span><span class="p">.</span><span class="nx">onClick</span><span class="si">}</span> <span class="na">style</span><span class="p">=</span><span class="si">{</span><span class="p">{</span><span class="na">background</span><span class="p">:</span> <span class="nx">randomColour</span><span class="p">()}</span><span class="si">}</span><span class="p">></span>
<span class="si">{</span><span class="nx">props</span><span class="p">.</span><span class="nx">children</span><span class="si">}</span>
<span class="p"></</span><span class="nt">button</span><span class="p">></span>
<span class="p">)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now let’s look at the following simple app. It displays 2 numbers - a
counter <code class="language-plaintext highlighter-rouge">c</code> and a <code class="language-plaintext highlighter-rouge">delta</code>. One button allows the user to increment <code class="language-plaintext highlighter-rouge">delta</code> by 1.
A second button, allows the user to increment the counter by adding <code class="language-plaintext highlighter-rouge">delta</code> to it.
We’ll create 2 functions <code class="language-plaintext highlighter-rouge">increment</code> and <code class="language-plaintext highlighter-rouge">incrementDelta</code> and assign them to the buttons’ on-click event handlers.
Let’s also keep track of how many such functions are created while the user clicks the buttons:</p>
<div class="language-tsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nx">React</span><span class="p">,</span> <span class="p">{</span> <span class="nx">useState</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">react</span><span class="dl">'</span><span class="p">;</span>
<span class="c1">// Keeps track of all created functions during the app's life </span>
<span class="kd">const</span> <span class="nx">functions</span><span class="p">:</span> <span class="nb">Set</span><span class="o"><</span><span class="kr">any</span><span class="o">></span> <span class="o">=</span> <span class="k">new</span> <span class="nb">Set</span><span class="p">();</span>
<span class="kd">const</span> <span class="nx">App</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="p">[</span><span class="nx">delta</span><span class="p">,</span> <span class="nx">setDelta</span><span class="p">]</span> <span class="o">=</span> <span class="nx">useState</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
<span class="kd">const</span> <span class="p">[</span><span class="nx">c</span><span class="p">,</span> <span class="nx">setC</span><span class="p">]</span> <span class="o">=</span> <span class="nx">useState</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">incrementDelta</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="nx">setDelta</span><span class="p">(</span><span class="nx">delta</span> <span class="o">=></span> <span class="nx">delta</span> <span class="o">+</span> <span class="mi">1</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">increment</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="nx">setC</span><span class="p">(</span><span class="nx">c</span> <span class="o">=></span> <span class="nx">c</span> <span class="o">+</span> <span class="nx">delta</span><span class="p">);</span>
<span class="c1">// Register the functions so we can count them</span>
<span class="nx">functions</span><span class="p">.</span><span class="nx">add</span><span class="p">(</span><span class="nx">incrementDelta</span><span class="p">);</span>
<span class="nx">functions</span><span class="p">.</span><span class="nx">add</span><span class="p">(</span><span class="nx">increment</span><span class="p">);</span>
<span class="k">return</span> <span class="p">(<</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span><span class="p">></span> Delta is <span class="si">{</span><span class="nx">delta</span><span class="si">}</span> <span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span><span class="p">></span> Counter is <span class="si">{</span><span class="nx">c</span><span class="si">}</span> <span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nt">br</span><span class="p">/></span>
<span class="p"><</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nc">Button</span> <span class="na">onClick</span><span class="p">=</span><span class="si">{</span><span class="nx">incrementDelta</span><span class="si">}</span><span class="p">></span>Increment Delta<span class="p"></</span><span class="nc">Button</span><span class="p">></span>
<span class="p"><</span><span class="nc">Button</span> <span class="na">onClick</span><span class="p">=</span><span class="si">{</span><span class="nx">increment</span><span class="si">}</span><span class="p">></span>Increment Counter<span class="p"></</span><span class="nc">Button</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nt">br</span><span class="p">/></span>
<span class="p"><</span><span class="nt">div</span><span class="p">></span> Newly Created Functions: <span class="si">{</span><span class="nx">functions</span><span class="p">.</span><span class="nx">size</span> <span class="o">-</span> <span class="mi">2</span><span class="si">}</span> <span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">>)</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>When we run the app and start clicking the buttons we observe something interesting.
For every click of a button there are 2 newly created functions!
Futhermore, both buttons re-render on every change!</p>
<figure>
<img style="background: white; padding: 0.5rem;" src="/images/blog/React useCallback and useMemo Hooks By Example/without-use-callback.png" alt="Without useCallback" />
<figcaption>
For every re-render of the component, 2 new functions are created.
Every change causes both buttoms to re-render.
</figcaption>
</figure>
<p>In other words, at every re-render we’re creating 2 new functions.
If we increment <code class="language-plaintext highlighter-rouge">c</code>, why do we need to recreate the <code class="language-plaintext highlighter-rouge">incrementDelta</code> function?
This is not just about memory - it causes child components to re-render
unnecessarily.
This can quickly become a performance issue.</p>
<p>One solution would be to move the two functions outside of the the <code class="language-plaintext highlighter-rouge">App</code> functional component.
Unfortunately, this wouldn’t work because they use the state variables from <code class="language-plaintext highlighter-rouge">App</code>’s scope.</p>
<h1 id="naive-solution---why-dependencies-matter">Naive solution - Why dependencies matter</h1>
<p>This is where the <a href="https://reactjs.org/docs/hooks-reference.html#usecallback">useCallback</a> hook comes in.
It takes as an arguement a function and returns a cached/memoized version of it.
It also takes a second parameter which will cover later. Let’s rewrite with <code class="language-plaintext highlighter-rouge">useCallBack</code>:</p>
<div class="language-tsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
</pre></td><td class="rouge-code"><pre><span class="kd">const</span> <span class="nx">App</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="p">[</span><span class="nx">delta</span><span class="p">,</span> <span class="nx">setDelta</span><span class="p">]</span> <span class="o">=</span> <span class="nx">useState</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
<span class="kd">const</span> <span class="p">[</span><span class="nx">c</span><span class="p">,</span> <span class="nx">setC</span><span class="p">]</span> <span class="o">=</span> <span class="nx">useState</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
<span class="c1">// No dependencies (i.e. []) for now</span>
<span class="kd">const</span> <span class="nx">incrementDelta</span> <span class="o">=</span> <span class="nx">useCallback</span><span class="p">(()</span> <span class="o">=></span> <span class="nx">setDelta</span><span class="p">(</span><span class="nx">delta</span> <span class="o">=></span> <span class="nx">delta</span> <span class="o">+</span> <span class="mi">1</span><span class="p">),</span> <span class="p">[]);</span>
<span class="kd">const</span> <span class="nx">increment</span> <span class="o">=</span> <span class="nx">useCallback</span><span class="p">(()</span> <span class="o">=></span> <span class="nx">setC</span><span class="p">(</span><span class="nx">c</span> <span class="o">=></span> <span class="nx">c</span> <span class="o">+</span> <span class="nx">delta</span><span class="p">),</span> <span class="p">[]);</span>
<span class="c1">// Register the functions so we can count them</span>
<span class="nx">functions</span><span class="p">.</span><span class="nx">add</span><span class="p">(</span><span class="nx">incrementDelta</span><span class="p">);</span>
<span class="nx">functions</span><span class="p">.</span><span class="nx">add</span><span class="p">(</span><span class="nx">increment</span><span class="p">);</span>
<span class="k">return</span> <span class="p">(<</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span><span class="p">></span> Delta is <span class="si">{</span><span class="nx">delta</span><span class="si">}</span> <span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span><span class="p">></span> Counter is <span class="si">{</span><span class="nx">c</span><span class="si">}</span> <span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nt">br</span><span class="p">/></span>
<span class="p"><</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nc">Button</span> <span class="na">onClick</span><span class="p">=</span><span class="si">{</span><span class="nx">incrementDelta</span><span class="si">}</span><span class="p">></span>Increment Delta<span class="p"></</span><span class="nc">Button</span><span class="p">></span>
<span class="p"><</span><span class="nc">Button</span> <span class="na">onClick</span><span class="p">=</span><span class="si">{</span><span class="nx">increment</span><span class="si">}</span><span class="p">></span>Increment Counter<span class="p"></</span><span class="nc">Button</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nt">br</span><span class="p">/></span>
<span class="p"><</span><span class="nt">div</span><span class="p">></span> Newly Created Functions: <span class="si">{</span><span class="nx">functions</span><span class="p">.</span><span class="nx">size</span> <span class="o">-</span> <span class="mi">2</span><span class="si">}</span> <span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">>)</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>This prevents the instantiation of new functions and unnecessary re-renders.
However, when we re-run the app, we notice that we’ve introduced a bug. If we
increment <code class="language-plaintext highlighter-rouge">detla</code> to 2, and then try to increment the counter, its value increases
by 1, not by 2:</p>
<figure>
<img style="background: white; padding: 0.5rem;" src="/images/blog/React useCallback and useMemo Hooks By Example/without-dependencies.png" alt="Without dependencies" />
<figcaption>
No new functions are created regardless of delta's state changes. During the
initial rendering, `useCallback` created a single cached version of the
"increment" function, which encapsulate the detla state value
and reused it on every re-render.
</figcaption>
</figure>
<p>This is because at the point of instantiation of the <code class="language-plaintext highlighter-rouge">increment</code> function, the value
of <code class="language-plaintext highlighter-rouge">delta</code> was 1, and this was captured in the function’s scope. Since we’re caching
the <code class="language-plaintext highlighter-rouge">increment</code> instance, it’s never recreated and it uses its original scope with
<code class="language-plaintext highlighter-rouge">detla = 1</code>.</p>
<p>The <code class="language-plaintext highlighter-rouge">useCallback</code> hook has created a single cached version of <code class="language-plaintext highlighter-rouge">increment</code>, which
encapsulates the <strong>initial value</strong> of <code class="language-plaintext highlighter-rouge">delta</code>. When <code class="language-plaintext highlighter-rouge">App</code> re-renders with different values for
<code class="language-plaintext highlighter-rouge">delta</code>, <code class="language-plaintext highlighter-rouge">useCallback</code> returns the previous version of the <code class="language-plaintext highlighter-rouge">increment</code> function
which keeps the old value of <code class="language-plaintext highlighter-rouge">delta</code> from the first rendering.</p>
<p>We need to tell <code class="language-plaintext highlighter-rouge">useCallback</code> to create new cached version of <code class="language-plaintext highlighter-rouge">increment</code>
for every change of <code class="language-plaintext highlighter-rouge">delta</code>.</p>
<h1 id="dependencies">Dependencies</h1>
<p>This is where the second arguement of <code class="language-plaintext highlighter-rouge">useCallback</code> comes in. It is an array of values,
which represents the <strong>dependencies</strong> of the cache. On any two subsequent re-renders,
<code class="language-plaintext highlighter-rouge">useCallback</code> will return the same cached function instance if the values of the dependencies are equal.</p>
<p>We can use dependencies to solve the previous bug:</p>
<div class="language-tsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre> <span class="kd">const</span> <span class="nx">incrementDelta</span> <span class="o">=</span> <span class="nx">useCallback</span><span class="p">(()</span> <span class="o">=></span> <span class="nx">setDelta</span><span class="p">(</span><span class="nx">delta</span> <span class="o">=></span> <span class="nx">delta</span> <span class="o">+</span> <span class="mi">1</span><span class="p">),</span> <span class="p">[]);</span>
<span class="c1">// Recreate increment on every change of delta!</span>
<span class="kd">const</span> <span class="nx">increment</span> <span class="o">=</span> <span class="nx">useCallback</span><span class="p">(()</span> <span class="o">=></span> <span class="nx">setC</span><span class="p">(</span><span class="nx">c</span> <span class="o">=></span> <span class="nx">c</span> <span class="o">+</span> <span class="nx">delta</span><span class="p">),</span> <span class="p">[</span><span class="nx">delta</span><span class="p">]);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now we can see that a new <code class="language-plaintext highlighter-rouge">increment</code> function is created only when <code class="language-plaintext highlighter-rouge">delta</code> changes.
Therefore, the counter button only re-renders when <code class="language-plaintext highlighter-rouge">delta</code> changes, because
a new instance of the <code class="language-plaintext highlighter-rouge">onClick</code> property is added.
In other words,
<strong>we only create a new callback, if the part of the closure it uses (i.e. the dependencies)
has changed since the previous rendering</strong>.</p>
<figure>
<img style="background: white; padding: 0.5rem;" src="/images/blog/React useCallback and useMemo Hooks By Example/with-dependencies.png" alt="With dependencies" />
<figcaption>
A new `increment` function is created on every change of `delta`.
Only the function whose dependencies change is recreated.
</figcaption>
</figure>
<p>A really useful feature of <code class="language-plaintext highlighter-rouge">useCallback</code> is that it returns the <strong>same function instance</strong>
if the depencies don’t change. Hence we can use it in the dependecy lists of other hooks.
For example, let’s create a cached/memoized function which increments both numbers:</p>
<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre><span class="kd">const</span> <span class="nx">incrementDelta</span> <span class="o">=</span> <span class="nx">useCallback</span><span class="p">(()</span> <span class="o">=></span> <span class="nx">setDelta</span><span class="p">(</span><span class="nx">delta</span> <span class="o">=></span> <span class="nx">delta</span> <span class="o">+</span> <span class="mi">1</span><span class="p">),</span> <span class="p">[]);</span>
<span class="kd">const</span> <span class="nx">increment</span> <span class="o">=</span> <span class="nx">useCallback</span><span class="p">(()</span> <span class="o">=></span> <span class="nx">setC</span><span class="p">(</span><span class="nx">c</span> <span class="o">=></span> <span class="nx">c</span> <span class="o">+</span> <span class="nx">delta</span><span class="p">),</span> <span class="p">[</span><span class="nx">delta</span><span class="p">]);</span>
<span class="c1">// Can depend on [delta] instead, but it would be brittle</span>
<span class="kd">const</span> <span class="nx">incrementBoth</span> <span class="o">=</span> <span class="nx">useCallback</span><span class="p">(()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">incrementDelta</span><span class="p">();</span>
<span class="nx">increment</span><span class="p">();</span>
<span class="p">},</span> <span class="p">[</span><span class="nx">increment</span><span class="p">,</span> <span class="nx">incrementDelta</span><span class="p">]);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The new <code class="language-plaintext highlighter-rouge">incrementBoth</code> function transitively depends on <code class="language-plaintext highlighter-rouge">delta</code>.
We could write <code class="language-plaintext highlighter-rouge">useCallback(... ,[delta])</code> and that would work.
However, this is a very brittle approach! If we changed the dependencies
of <code class="language-plaintext highlighter-rouge">increment</code> or <code class="language-plaintext highlighter-rouge">incrementDelta</code>, we would have to remember to change the
dependencies of <code class="language-plaintext highlighter-rouge">incrementBoth</code>.</p>
<p>Since the references of <code class="language-plaintext highlighter-rouge">increment</code> and <code class="language-plaintext highlighter-rouge">incrementDelta</code> won’t change unless their dependencies change,
we could use them instead. Transitive dependencies can be ignored!
This makes for a straightforward rule:</p>
<blockquote>
<p>Each function declared within a functional component’s scope must be memoized/cached with <code class="language-plaintext highlighter-rouge">useCallback</code>.
If it references functions or other variables from the component scope
it should list them in its dependency list.</p>
</blockquote>
<p>This rule can be enforced by a
<a href="https://www.npmjs.com/package/eslint-plugin-react-hooks">linter</a> which checks that
your <code class="language-plaintext highlighter-rouge">useCallback</code> cache dependenices are consistent.</p>
<h1 id="two-similar-hooks---usecallback-and-usememo">Two similar hooks - useCallback and useMemo</h1>
<p>React introduces another similar hook called <a href="https://reactjs.org/docs/hooks-reference.html#usememo">useMemo</a>.
It has similar signature, but works differently.
Unlike <code class="language-plaintext highlighter-rouge">useCallback</code>, which caches the provided function instance, <code class="language-plaintext highlighter-rouge">useMemo</code> invokes
the provided function and caches its result.</p>
<p>In other words <code class="language-plaintext highlighter-rouge">useMemo</code> caches a computed value. This is usefull when the computation
requires significant resources and we don’t want to repeat it on every re-render, as in this example:</p>
<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="kd">const</span> <span class="p">[</span><span class="nx">c</span><span class="p">,</span> <span class="nx">setC</span><span class="p">]</span> <span class="o">=</span> <span class="nx">useState</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
<span class="c1">// This value will not be recomputed between re-renders</span>
<span class="c1">// unless the value of c changes</span>
<span class="kd">const</span> <span class="nx">sinOfC</span><span class="p">:</span> <span class="kr">number</span> <span class="o">=</span> <span class="nx">useMemo</span><span class="p">(()</span> <span class="o">=></span> <span class="nb">Math</span><span class="p">.</span><span class="nx">sin</span><span class="p">(</span><span class="nx">c</span><span class="p">)</span> <span class="p">,</span> <span class="p">[</span><span class="nx">c</span><span class="p">])</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Just as with <code class="language-plaintext highlighter-rouge">useCallback</code>, the values returned by <code class="language-plaintext highlighter-rouge">useMemo</code> can be used as other hooks’ dependencies.</p>
<p>As an interesting aside, <code class="language-plaintext highlighter-rouge">useMemo</code> can cache a function value too.
In other words, it is a generalised version of <code class="language-plaintext highlighter-rouge">useCallback</code> and can replace it
as in the following example</p>
<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="c1">// Some function ...</span>
<span class="kd">const</span> <span class="nx">f</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span> <span class="p">...</span> <span class="p">}</span>
<span class="c1">// The following are functionally equivalent</span>
<span class="kd">const</span> <span class="nx">callbackF</span> <span class="o">=</span> <span class="nx">useCallback</span><span class="p">(</span><span class="nx">f</span><span class="p">,</span> <span class="p">[])</span>
<span class="kd">const</span> <span class="nx">callbackF</span> <span class="o">=</span> <span class="nx">useMemo</span><span class="p">(()</span> <span class="o">=></span> <span class="nx">f</span><span class="p">,</span> <span class="p">[])</span>
</pre></td></tr></tbody></table></code></pre></div></div>Nikolay Grozevnikolay.grozev@gmail.comThis article demonstrates with a few simple examples why we need the useCallback and useMemo hooks and when and how to use them.Python API With Flask and Flask RESTPlus2018-10-12T05:22:09+00:002018-10-12T05:22:09+00:00https://nikgrozev.com/2018/10/12/python-api-with-flask-and-flask-restplus<h1 id="table-of-contents">Table of Contents</h1>
<ul>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#pyenv">Pyenv</a></li>
<li><a href="#pyenv-project-configuration">Pyenv Project Configuration</a></li>
<li><a href="#pipenv">Pipenv</a></li>
<li><a href="#pythonpath">PYTHONPATH</a></li>
<li><a href="#environments-and-externalisation">Environments and Externalisation</a></li>
<li><a href="#flask-and-flask-restplus">Flask and Flask-RESTPlus</a></li>
<li><a href="#the-server">The Server</a></li>
<li><a href="#models">Models</a></li>
<li><a href="#resources">Resources</a></li>
<li><a href="#application-entry-point">Application Entry Point</a></li>
<li><a href="#unit-tests">Unit Tests</a></li>
</ul>
<div id="introduction" />
<h1 id="introduction">Introduction</h1>
<p>I have been considering Python as a language for API development.
The Python ecosystem is very mature when it comes to machine learning and
statistics libraries. On some projects, it may be
useful to write parts of the backend in Python, so we can use libraries like
<a href="http://www.numpy.org/">NumPy</a>, <a href="https://pandas.pydata.org/">Pandas</a>, and <a href="https://www.scipy.org/">SciPy</a>.</p>
<p>This article will describe how to get started with the python ecosystem, so
we can write APIs. We’ll leave out application specific aspects like authentication
and database access and will focus on the basics. As an example, we’ll build a
simple REST-ful API. You can find the full code <a href="https://github.com/nikolayg/sample-python-api">here</a>.</p>
<p>The following has been tested on Ubuntu and OS X.</p>
<div id="pyenv" />
<h1 id="pyenv">Pyenv</h1>
<p>Most modern programming languages have tools that allow us to quickly install
and switch between runtime versions. Similar to Node’s <a href="https://github.com/creationix/nvm">NVM</a> and Java’s <a href="https://sdkman.io/">SDKMan</a>, Python has <a href="https://github.com/pyenv/pyenv">Pyenv</a>.</p>
<p>If you don’t already have it, you can follow the
<a href="https://github.com/pyenv/pyenv#installation">Official installation instructions</a>.
On OS X, you can <code class="language-plaintext highlighter-rouge">brew</code> it:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>brew <span class="nb">install </span>pyenv
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Once you have pyenv, you can install and configure any python version:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre><span class="c"># Install the desired python version - e.g. 3.4.3</span>
pyenv <span class="nb">install </span>3.4.3
<span class="c"># Set it up as a global version - pyenv will reconfigure your PATH accordingly</span>
pyenv global 3.4.3
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="pyenv-project-configuration" />
<h1 id="pyenv-project-configuration">Pyenv Project Configuration</h1>
<p>After installing Pyenv, let’s create a folder for our sample API:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre><span class="nb">mkdir </span>sample-python-api <span class="o">&&</span> <span class="nb">cd </span>sample-python-api
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Whenever you enter a folder, Pyenv looks for a special file called <code class="language-plaintext highlighter-rouge">.python-version</code>.
It specifies the Python version for the project. If it exists,
Pyenv will automatically switch to the respective version.</p>
<p>Let’s create the file:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre><span class="c"># Define project specific Python versions</span>
<span class="nb">echo</span> <span class="s2">"3.6.0"</span> <span class="o">></span> .python-version
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now we can check that Pyenv detects the file and install the
specified Python version.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="c"># Should print the version from ".python-version".</span>
pyenv <span class="nb">local</span>
<span class="c"># Installs the version from ".python-version" if not installed </span>
<span class="c"># Can take some time.</span>
pyenv <span class="nb">install</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="pipenv" />
<h1 id="pipenv">Pipenv</h1>
<p><a href="https://github.com/pypa/pip">PIP</a> is Python’s most popular package manager.
<a href="https://pipenv.readthedocs.io/en/latest/">Pipenv</a> is a new package and
environment manager which uses <code class="language-plaintext highlighter-rouge">PIP</code> under the hood. It provides more
advanced features like version locking and dependency isolation between projects.</p>
<p>After installing <code class="language-plaintext highlighter-rouge">pyenv</code>, you should be able to add <code class="language-plaintext highlighter-rouge">pipenv</code> like this:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>pip <span class="nb">install</span> <span class="nt">--user</span> pipenv
</pre></td></tr></tbody></table></code></pre></div></div>
<p>If that doesn’t work for you check the <a href="https://pipenv.readthedocs.io/en/latest/install/#installing-pipenv">Pipenv’s installation documentation</a>
for alternative installation instructions for your platform.</p>
<p>Now we can install a dependency/library for our sample API:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>pipenv <span class="nb">install </span>flask
</pre></td></tr></tbody></table></code></pre></div></div>
<p>This will create the <code class="language-plaintext highlighter-rouge">Pipfile</code> and <code class="language-plaintext highlighter-rouge">Pipfile.lock</code> files in the project directory.
The first one defines the dependencies and the second specifies
their exact installed versions.
Whenever we add more dependencies, they’ll be added to these two files.</p>
<p><code class="language-plaintext highlighter-rouge">Pipenv</code> has the concept of a shell/environment which encapsulates the python process.
We can start a <code class="language-plaintext highlighter-rouge">pipenv</code> shell via:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre><span class="c"># Initialises environment variables </span>
<span class="c"># and loads dependencies</span>
pipenv shell
</pre></td></tr></tbody></table></code></pre></div></div>
<p>This will initialise common environment variables in the current shell.
As a result, when you run <code class="language-plaintext highlighter-rouge">python</code> the shell will start the REPL interpreter
with all <code class="language-plaintext highlighter-rouge">pipenv</code> libraries loaded at your disposal:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre><span class="c"># Initialises common environment variables and dependencies</span>
pipenv shell
<span class="c"># Start the shell</span>
python
<span class="c"># We have all dependencies loaded, so we can import them</span>
<span class="o">></span> from flask import Flask
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="pythonpath" />
<h1 id="pythonpath">PYTHONPATH</h1>
<p><code class="language-plaintext highlighter-rouge">PYTHONPATH</code> is a special environment variable which tells the Python
interpreter where to search for modules. Let’s create two
source folders <code class="language-plaintext highlighter-rouge">src</code> and <code class="language-plaintext highlighter-rouge">tests</code>. By now your project should look like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre>sample-python-api/
│
├──── .python-version
├──── Pipfile
├──── Pipfile.lock
├──── src/
└──── tests/
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now we’ll need to tell Pipenv the right <code class="language-plaintext highlighter-rouge">PYTHONPATH</code> so the Python interpreter
can include the files from <code class="language-plaintext highlighter-rouge">src</code> and <code class="language-plaintext highlighter-rouge">tests</code>.</p>
<p>Whenever Pipenv starts a shell, it looks
for a special file called <code class="language-plaintext highlighter-rouge">.env</code> and loads all environment
variable from there.
Let’s create a <code class="language-plaintext highlighter-rouge">.env</code> file with the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>PYTHONPATH="src:tests"
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now restart the Pipenv shell for the change to take effect.</p>
<div id="environments-and-externalisation" />
<h1 id="environments-and-externalisation">Environments and Externalisation</h1>
<p>In Node JS, developers use the special environment variable <code class="language-plaintext highlighter-rouge">NODE_ENV</code>
to specify whether the code is running in <em>production</em> or <em>development</em> mode.
Depending on the “mode”, different configuration can be used.</p>
<p>To follow this pattern in Python we can set the <code class="language-plaintext highlighter-rouge">PYTHON_ENV</code> environment variable.
For example:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre><span class="c"># Will run in production mode.</span>
<span class="nv">PYTHON_ENV</span><span class="o">=</span>production python <my_script.py>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now let’s create a file <code class="language-plaintext highlighter-rouge">src/environment/instance.py</code>, which will exemplify how to
load different configurations depending on the environment:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
</pre></td><td class="rouge-code"><pre><span class="kn">import</span> <span class="nn">os</span>
<span class="c1"># Load the development "mode". Use "developmen" if not specified
</span><span class="n">env</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">environ</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"PYTHON_ENV"</span><span class="p">,</span> <span class="s">"development"</span><span class="p">)</span>
<span class="c1"># Configuration for each environment
# Alternatively use "python-dotenv"
</span><span class="n">all_environments</span> <span class="o">=</span> <span class="p">{</span>
<span class="s">"development"</span><span class="p">:</span> <span class="p">{</span> <span class="s">"port"</span><span class="p">:</span> <span class="mi">5000</span><span class="p">,</span> <span class="s">"debug"</span><span class="p">:</span> <span class="bp">True</span><span class="p">,</span> <span class="s">"swagger-url"</span><span class="p">:</span> <span class="s">"/api/swagger"</span> <span class="p">},</span>
<span class="s">"production"</span><span class="p">:</span> <span class="p">{</span> <span class="s">"port"</span><span class="p">:</span> <span class="mi">8080</span><span class="p">,</span> <span class="s">"debug"</span><span class="p">:</span> <span class="bp">False</span><span class="p">,</span> <span class="s">"swagger-url"</span><span class="p">:</span> <span class="bp">None</span> <span class="p">}</span>
<span class="p">}</span>
<span class="c1"># The config for the current environment
</span><span class="n">environment_config</span> <span class="o">=</span> <span class="n">all_environments</span><span class="p">[</span><span class="n">env</span><span class="p">]</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Our other modules can import <code class="language-plaintext highlighter-rouge">environment_config</code> to access environment specific
configuration. Note that the values in the above configuration are specific to
<code class="language-plaintext highlighter-rouge">Flask</code>, which we will discuss next.</p>
<div id="flask-and-flask-restplus" />
<h1 id="flask-and-flask-restplus">Flask and Flask-RESTPlus</h1>
<p><a href="http://flask.pocoo.org/">Flask</a> is a lightweight web server and framework.
We can create a Web API directly with <code class="language-plaintext highlighter-rouge">flask</code>. However, the
<a href="https://flask-restplus.readthedocs.io/en/stable/">Flask-RESTPlus</a>
extension makes it much easier to get started. It also automatically configures
a <a href="https://swagger.io/tools/swagger-ui/">Swagger UI</a> endpoint for the API.</p>
<p>If you have been following so far, we already installed <code class="language-plaintext highlighter-rouge">flask</code> by:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>pipenv <span class="nb">install </span>flask
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now let’s add <code class="language-plaintext highlighter-rouge">flask-restplus</code> too:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>pipenv <span class="nb">install </span>flask-restplus
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="the-server" />
<h1 id="the-server">The Server</h1>
<p>Let’s start by creating a flask server in a file <code class="language-plaintext highlighter-rouge">./src/server/instance.py</code>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
</pre></td><td class="rouge-code"><pre><span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span>
<span class="kn">from</span> <span class="nn">flask_restplus</span> <span class="kn">import</span> <span class="n">Api</span><span class="p">,</span> <span class="n">Resource</span><span class="p">,</span> <span class="n">fields</span>
<span class="kn">from</span> <span class="nn">environment.instance</span> <span class="kn">import</span> <span class="n">environment_config</span>
<span class="k">class</span> <span class="nc">Server</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="p">.</span><span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="n">__name__</span><span class="p">)</span>
<span class="bp">self</span><span class="p">.</span><span class="n">api</span> <span class="o">=</span> <span class="n">Api</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">app</span><span class="p">,</span>
<span class="n">version</span><span class="o">=</span><span class="s">'1.0'</span><span class="p">,</span>
<span class="n">title</span><span class="o">=</span><span class="s">'Sample Book API'</span><span class="p">,</span>
<span class="n">description</span><span class="o">=</span><span class="s">'A simple Book API'</span><span class="p">,</span>
<span class="n">doc</span> <span class="o">=</span> <span class="n">environment_config</span><span class="p">[</span><span class="s">"swagger-url"</span><span class="p">]</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="p">.</span><span class="n">app</span><span class="p">.</span><span class="n">run</span><span class="p">(</span>
<span class="n">debug</span> <span class="o">=</span> <span class="n">environment_config</span><span class="p">[</span><span class="s">"debug"</span><span class="p">],</span>
<span class="n">port</span> <span class="o">=</span> <span class="n">environment_config</span><span class="p">[</span><span class="s">"port"</span><span class="p">]</span>
<span class="p">)</span>
<span class="n">server</span> <span class="o">=</span> <span class="n">Server</span><span class="p">()</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The above code defines a wrapper class <code class="language-plaintext highlighter-rouge">Server</code> which aggregates the flask
and the flask-restplus server instances called <code class="language-plaintext highlighter-rouge">app</code> and <code class="language-plaintext highlighter-rouge">api</code>.
We use the environment configuration to parameterise <code class="language-plaintext highlighter-rouge">api</code>.
The <code class="language-plaintext highlighter-rouge">environment_config["swagger-url"]</code> parameter defines the URL path
of the Swagger UI for the project. If it’s <code class="language-plaintext highlighter-rouge">None</code>, then the server won’t start
a Swagger UI.</p>
<p>The <code class="language-plaintext highlighter-rouge">start</code> method (which also uses environment parameters) initiates the server.
The <code class="language-plaintext highlighter-rouge">server</code> object represents the global (ala singleton) instance of the web server.</p>
<div id="models" />
<h1 id="models">Models</h1>
<p>The <code class="language-plaintext highlighter-rouge">flask-restplus</code> library introduces 2 main abstractions: <em>models</em>
and <em>resources</em>. A model defines the structure/schema of the
payload of a request or response. This includes the list of all fields
and their types. I keep all my models in a separate folder <code class="language-plaintext highlighter-rouge">./src/models</code>.
Let’s create a new model <code class="language-plaintext highlighter-rouge">./src/models/book.py</code>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre><span class="kn">from</span> <span class="nn">flask_restplus</span> <span class="kn">import</span> <span class="n">fields</span>
<span class="kn">from</span> <span class="nn">server.instance</span> <span class="kn">import</span> <span class="n">server</span>
<span class="n">book</span> <span class="o">=</span> <span class="n">server</span><span class="p">.</span><span class="n">api</span><span class="p">.</span><span class="n">model</span><span class="p">(</span><span class="s">'Book'</span><span class="p">,</span> <span class="p">{</span>
<span class="s">'id'</span><span class="p">:</span> <span class="n">fields</span><span class="p">.</span><span class="n">Integer</span><span class="p">(</span><span class="n">description</span><span class="o">=</span><span class="s">'Id'</span><span class="p">),</span>
<span class="s">'title'</span><span class="p">:</span> <span class="n">fields</span><span class="p">.</span><span class="n">String</span><span class="p">(</span><span class="n">required</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span> <span class="n">min_length</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">max_length</span><span class="o">=</span><span class="mi">200</span><span class="p">,</span> <span class="n">description</span><span class="o">=</span><span class="s">'Book title'</span><span class="p">)</span>
<span class="p">})</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="resources" />
<h1 id="resources">Resources</h1>
<p>A <code class="language-plaintext highlighter-rouge">resource</code> is a class whose methods are mapped to an API/URL endpoint.
We use <code class="language-plaintext highlighter-rouge">flask-restplus</code> annotations to define the
URL pattern for every such class.
For every resources class, the method whose names match the HTTP
methods (e.g. <code class="language-plaintext highlighter-rouge">get</code>, <code class="language-plaintext highlighter-rouge">put</code>) will handle the matching HTTP calls.</p>
<p>By using the <code class="language-plaintext highlighter-rouge">expect</code> annotation, for every HTTP method
we can specify the expected <em>model</em> of the payload body.
Similarly, by using the <code class="language-plaintext highlighter-rouge">marshal</code> annotations, we can define
the respective response payload <em>model</em>.</p>
<p>I keep my resources in the <code class="language-plaintext highlighter-rouge">./src/resources</code> folder. Let’s
create a simple resource <code class="language-plaintext highlighter-rouge">./src/resources/book.py</code>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
</pre></td><td class="rouge-code"><pre><span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span>
<span class="kn">from</span> <span class="nn">flask_restplus</span> <span class="kn">import</span> <span class="n">Api</span><span class="p">,</span> <span class="n">Resource</span><span class="p">,</span> <span class="n">fields</span>
<span class="kn">from</span> <span class="nn">server.instance</span> <span class="kn">import</span> <span class="n">server</span>
<span class="kn">from</span> <span class="nn">models.book</span> <span class="kn">import</span> <span class="n">book</span>
<span class="n">app</span><span class="p">,</span> <span class="n">api</span> <span class="o">=</span> <span class="n">server</span><span class="p">.</span><span class="n">app</span><span class="p">,</span> <span class="n">server</span><span class="p">.</span><span class="n">api</span>
<span class="c1"># Let's just keep them in memory
</span><span class="n">books_db</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span><span class="s">"id"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="s">"title"</span><span class="p">:</span> <span class="s">"War and Peace"</span><span class="p">},</span>
<span class="p">{</span><span class="s">"id"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="s">"title"</span><span class="p">:</span> <span class="s">"Python for Dummies"</span><span class="p">},</span>
<span class="p">]</span>
<span class="c1"># This class will handle GET and POST to /books
</span><span class="o">@</span><span class="n">api</span><span class="p">.</span><span class="n">route</span><span class="p">(</span><span class="s">'/books'</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">BookList</span><span class="p">(</span><span class="n">Resource</span><span class="p">):</span>
<span class="o">@</span><span class="n">api</span><span class="p">.</span><span class="n">marshal_list_with</span><span class="p">(</span><span class="n">book</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="n">books_db</span>
<span class="c1"># Ask flask_restplus to validate the incoming payload
</span> <span class="o">@</span><span class="n">api</span><span class="p">.</span><span class="n">expect</span><span class="p">(</span><span class="n">book</span><span class="p">,</span> <span class="n">validate</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="o">@</span><span class="n">api</span><span class="p">.</span><span class="n">marshal_with</span><span class="p">(</span><span class="n">book</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">post</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="c1"># Generate new Id
</span> <span class="n">api</span><span class="p">.</span><span class="n">payload</span><span class="p">[</span><span class="s">"id"</span><span class="p">]</span> <span class="o">=</span> <span class="n">books_db</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">][</span><span class="s">"id"</span><span class="p">]</span> <span class="o">+</span> <span class="mi">1</span> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">books_db</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span> <span class="k">else</span> <span class="mi">0</span>
<span class="n">books_db</span><span class="p">.</span><span class="n">append</span><span class="p">(</span><span class="n">api</span><span class="p">.</span><span class="n">payload</span><span class="p">)</span>
<span class="k">return</span> <span class="n">api</span><span class="p">.</span><span class="n">payload</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The above example handles HTTP <code class="language-plaintext highlighter-rouge">GET</code> and <code class="language-plaintext highlighter-rouge">POST</code> to the <code class="language-plaintext highlighter-rouge">/books</code> endpoint.
Based on the <code class="language-plaintext highlighter-rouge">expect</code> and <code class="language-plaintext highlighter-rouge">marshal</code> annotations, <code class="language-plaintext highlighter-rouge">flask-restplus</code> will
automatically convert the JSON payloads to dictionaries and vice versa.</p>
<p>Now let’s implement individual book retrieval and update. In the same
file we can write another resource class:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
</pre></td><td class="rouge-code"><pre><span class="c1"># Handles GET and PUT to /books/:id
# The path parameter will be supplied as a parameter to every method
</span><span class="o">@</span><span class="n">api</span><span class="p">.</span><span class="n">route</span><span class="p">(</span><span class="s">'/books/<int:id>'</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Book</span><span class="p">(</span><span class="n">Resource</span><span class="p">):</span>
<span class="c1"># Utility method
</span> <span class="k">def</span> <span class="nf">find_one</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="nb">id</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">next</span><span class="p">((</span><span class="n">b</span> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">books_db</span> <span class="k">if</span> <span class="n">b</span><span class="p">[</span><span class="s">"id"</span><span class="p">]</span> <span class="o">==</span> <span class="nb">id</span><span class="p">),</span> <span class="bp">None</span><span class="p">)</span>
<span class="o">@</span><span class="n">api</span><span class="p">.</span><span class="n">marshal_with</span><span class="p">(</span><span class="n">book</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="nb">id</span><span class="p">):</span>
<span class="n">match</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">find_one</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
<span class="k">return</span> <span class="n">match</span> <span class="k">if</span> <span class="n">match</span> <span class="k">else</span> <span class="p">(</span><span class="s">"Not found"</span><span class="p">,</span> <span class="mi">404</span><span class="p">)</span>
<span class="o">@</span><span class="n">api</span><span class="p">.</span><span class="n">marshal_with</span><span class="p">(</span><span class="n">book</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">delete</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="nb">id</span><span class="p">):</span>
<span class="k">global</span> <span class="n">books_db</span>
<span class="n">match</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">find_one</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
<span class="n">books_db</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">b</span><span class="p">:</span> <span class="n">b</span><span class="p">[</span><span class="s">"id"</span><span class="p">]</span> <span class="o">!=</span> <span class="nb">id</span><span class="p">,</span> <span class="n">books_db</span><span class="p">))</span>
<span class="k">return</span> <span class="n">match</span>
<span class="c1"># Ask flask_restplus to validate the incoming payload
</span> <span class="o">@</span><span class="n">api</span><span class="p">.</span><span class="n">expect</span><span class="p">(</span><span class="n">book</span><span class="p">,</span> <span class="n">validate</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="o">@</span><span class="n">api</span><span class="p">.</span><span class="n">marshal_with</span><span class="p">(</span><span class="n">book</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">put</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="nb">id</span><span class="p">):</span>
<span class="n">match</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">find_one</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
<span class="k">if</span> <span class="n">match</span> <span class="o">!=</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">match</span><span class="p">.</span><span class="n">update</span><span class="p">(</span><span class="n">api</span><span class="p">.</span><span class="n">payload</span><span class="p">)</span>
<span class="n">match</span><span class="p">[</span><span class="s">"id"</span><span class="p">]</span> <span class="o">=</span> <span class="nb">id</span>
<span class="k">return</span> <span class="n">match</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="application-entry-point" />
<h1 id="application-entry-point">Application Entry Point</h1>
<p>We now have all necessary components to start the API.
If you have followed along, your code structure should look like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
</pre></td><td class="rouge-code"><pre>sample-python-api/
│
├── .python-version
├── Pipfile
├── Pipfile.lock
└── src
├── environment
│ └── instance.py
├── models
│ └── book.py
├── resources
│ └── book.py
└── server
└── instance.py
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Let’s create an entry point in the <code class="language-plaintext highlighter-rouge">./src/main.py</code> file:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="kn">from</span> <span class="nn">server.instance</span> <span class="kn">import</span> <span class="n">server</span>
<span class="kn">import</span> <span class="nn">sys</span><span class="p">,</span> <span class="n">os</span>
<span class="c1"># Need to import all resources
# so that they register with the server
</span><span class="kn">from</span> <span class="nn">resources.book</span> <span class="kn">import</span> <span class="o">*</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">'__main__'</span><span class="p">:</span>
<span class="n">server</span><span class="p">.</span><span class="n">run</span><span class="p">()</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>To start the server in development mode:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre><span class="c"># If you haven't already, then start a pipenv shell</span>
pipenv shell
<span class="nv">PYTHON_ENV</span><span class="o">=</span>development python src/main.py
</pre></td></tr></tbody></table></code></pre></div></div>
<p>This will start the server at the default port 5000. You can access the Swagger UI
on <a href="http://localhost:5000/api/swagger">http://localhost:5000/api/swagger</a>:</p>
<figure>
<img src="/images/blog/Python-flask-rest-api/swagger-ui.png" alt="Swagger UI" />
<figcaption>Swagger UI on [http://localhost:5000/api/swagger](http://localhost:5000/api/swagger)</figcaption>
</figure>
<p>To start the server in production mode:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre><span class="c"># If you haven't already, then start a pipenv shell</span>
pipenv shell
<span class="nv">PYTHON_ENV</span><span class="o">=</span>production python src/main.py
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Because of our environment configuration, you won’t see the Swagger UI.
You can still make API calls:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>curl http://localhost:8080/books
</pre></td></tr></tbody></table></code></pre></div></div>
<p><strong>Note that despite running our code in production mode/environment,
we are still using flask’s embedded development server</strong>. It
is not optimised for large workloads.</p>
<figure>
<img src="/images/blog/Python-flask-rest-api/flask-server-warning.png" alt="Flask Warning" />
<figcaption>Flask Warning</figcaption>
</figure>
<p>Refer to the <a href="http://flask.pocoo.org/docs/0.12/deploying/">official flask documentation</a> about suitable production
deployment options.</p>
<div id="unit-tests" />
<h1 id="unit-tests">Unit Tests</h1>
<p>We’re going to use the <a href="https://docs.pytest.org/en/latest/">pytest</a> library
for unit testing. Let’s install it</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>pipenv <span class="nb">install </span>pytest
</pre></td></tr></tbody></table></code></pre></div></div>
<p>We’re also going to use <code class="language-plaintext highlighter-rouge">pytest-flask</code>, which is a <code class="language-plaintext highlighter-rouge">pytest</code> plug-in
for testing <code class="language-plaintext highlighter-rouge">flask</code> apps:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre>pipenv <span class="nb">install </span>pytest-flask
</pre></td></tr></tbody></table></code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">Pytest</code> is quite different from other popular unit testing libraries
like <code class="language-plaintext highlighter-rouge">junit</code> or <code class="language-plaintext highlighter-rouge">jest</code>. It introduces the concept of
<em>fixtures</em>. In the simplest case, a fixture is just a named function,
which constructs a test object (e.g. a mock database connection).
Whenever a test function declares a formal parameter whose name
coincides with the name of the fixture, <code class="language-plaintext highlighter-rouge">pytest</code> will invoke
the corresponding fixture function and pass its result to the test.</p>
<p>When <code class="language-plaintext highlighter-rouge">pytest</code> starts, it looks for a special file called <code class="language-plaintext highlighter-rouge">./conftest.py</code>
and runs it before all tests. This is the usual place to define
global fixtures.
Let’s define a fixture for our server in <code class="language-plaintext highlighter-rouge">./conftest.py</code>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="kn">import</span> <span class="nn">pytest</span>
<span class="kn">from</span> <span class="nn">server.instance</span> <span class="kn">import</span> <span class="n">server</span>
<span class="c1"># Creates a fixture whose name is "app"
# and returns our flask server instance
</span><span class="o">@</span><span class="n">pytest</span><span class="p">.</span><span class="n">fixture</span>
<span class="k">def</span> <span class="nf">app</span><span class="p">():</span>
<span class="n">app</span> <span class="o">=</span> <span class="n">server</span><span class="p">.</span><span class="n">app</span>
<span class="k">return</span> <span class="n">app</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The above code defines a global fixture for our flask instance.
This is where the <code class="language-plaintext highlighter-rouge">pytest-flask</code> plug-in kicks in. Given the <code class="language-plaintext highlighter-rouge">app</code>
fixture, it implicitly and “magically” creates the <code class="language-plaintext highlighter-rouge">client</code> fixture,
which allows us to execute test API calls.</p>
<p>Let’s demonstrate this by creating a new test file <code class="language-plaintext highlighter-rouge">./test/resources/test_book.py</code>.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
</pre></td><td class="rouge-code"><pre><span class="c1"># Import the resource/controllers we're testing
</span><span class="kn">from</span> <span class="nn">resources.book</span> <span class="kn">import</span> <span class="o">*</span>
<span class="c1"># client is a fixture, injected by the `pytest-flask` plugin
</span><span class="k">def</span> <span class="nf">test_get_book</span><span class="p">(</span><span class="n">client</span><span class="p">):</span>
<span class="c1"># Make a tes call to /books/1
</span> <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"/books/1"</span><span class="p">)</span>
<span class="c1"># Validate the response
</span> <span class="k">assert</span> <span class="n">response</span><span class="p">.</span><span class="n">status_code</span> <span class="o">==</span> <span class="mi">200</span>
<span class="k">assert</span> <span class="n">response</span><span class="p">.</span><span class="n">json</span> <span class="o">==</span> <span class="p">{</span>
<span class="s">"id"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s">"title"</span><span class="p">:</span> <span class="s">"Python for Dummies"</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Pytest expects all unit test files
and functions to start with <code class="language-plaintext highlighter-rouge">test_</code> prefix. When adding more tests,
we’ll need to ensure we follow this naming convention.</p>
<p>To run the unit tests:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre><span class="c"># If you haven't already, then start a pipenv shell</span>
pipenv shell
python <span class="nt">-m</span> pytest
</pre></td></tr></tbody></table></code></pre></div></div>
<p>This should give us the following outcome:</p>
<figure>
<img src="/images/blog/Python-flask-rest-api/pytest-results.png" alt="Pytest Results" />
<figcaption>Pytest Results</figcaption>
</figure>Nikolay Grozevnikolay.grozev@gmail.comThis article will describe how to get started with the python ecosystem, so we can write APIs and test them. We'll leave out application specific aspects like authentication and database access and will focus on the basics. As an example, we'll build a simple REST-ful API...DIY Type Safe AOP in TypeScript2018-09-18T05:22:09+00:002018-09-18T05:22:09+00:00https://nikgrozev.com/2018/09/18/diy-type-safe-aop-in-typescript<div id="introduction" />
<h1 id="introduction">Introduction</h1>
<p>Recently, I have been working on a few TypeScript projects where I had to implement audit logging
for many opeartions. For example, I had to log every request to an external API and the received response or error message.
I also had to audit log every attempt to access externally hosted resources (e.g. SMTP or WebDAV servers).</p>
<p>I had to wrap every function invocation, which needed to be audited, with the same logging logic.
For simplicity, let’s assume the audit logging had to be done via <code class="language-plaintext highlighter-rouge">console.log</code>.
For just 2 API calls I had to write code like this:</p>
<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
</pre></td><td class="rouge-code"><pre><span class="k">try</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Calling API 1 </span><span class="p">${</span><span class="nx">param1</span><span class="p">}</span><span class="s2">, </span><span class="p">${</span><span class="nx">param2</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">resultCall1</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">apiCall_1</span><span class="p">(</span><span class="nx">param1</span><span class="p">,</span> <span class="nx">param2</span><span class="p">);</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Call to API 1 Succeeded with result: </span><span class="p">${</span><span class="nx">resultCall1</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="p">}</span> <span class="k">catch</span> <span class="p">(</span><span class="nx">e</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Call to API 1 Failed with error: </span><span class="p">${</span><span class="nx">e</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="k">throw</span> <span class="nx">e</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// ....</span>
<span class="k">try</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Calling API 2 </span><span class="p">${</span><span class="nx">param1</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">resultCall2</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">apiCall_2</span><span class="p">(</span><span class="nx">param1</span><span class="p">);</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Call to API 2 Succeeded with result: </span><span class="p">${</span><span class="nx">resultCall2</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="p">}</span> <span class="k">catch</span> <span class="p">(</span><span class="nx">e</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Call to API 2 Failed with error: </span><span class="p">${</span><span class="nx">e</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="k">throw</span> <span class="nx">e</span><span class="p">;</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>This is obviously a very brittle and verbose solution. Moreover, I had to mix essential buisiness logic (the API calls)
with <a href="https://en.wikipedia.org/wiki/Aspect_(computer_programming)">cross-cutting aspects</a> (i.e. the logging).
Unfortunately, I couldn’t find a flexible, stable, and type safe library for
<a href="https://en.wikipedia.org/wiki/Aspect-oriented_programming">Aspect Oriented Programming (AOP)</a> in TypeScript.</p>
<p>Fortunately, I managed to roll out my own minimalistic implementation to isolate the logging cross-cutting concerns.</p>
<div id="implementation" />
<h1 id="minimalistic-type-safe-aop">Minimalistic Type Safe AOP</h1>
<p>Let’s start by defining a simple asyncrhonous logging function. For the purposes of this article, we’ll just write to the standard output.
You can obviously have versions of this function which write to a databse, stream, or use a 3rd party logging library.</p>
<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre><span class="kd">const</span> <span class="nx">log</span> <span class="o">=</span> <span class="k">async</span> <span class="p">(</span><span class="nx">message</span><span class="p">:</span> <span class="kr">string</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="c1">// Write to a DB, stream, etc.</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">message</span><span class="p">);</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now let’s implement a small utility function, which converts an arbitrary object to JSON text.
We’ll use the <a href="https://www.npmjs.com/package/json-stringify-safe">json-stringify-safe</a> library to avoid
problems with circular references.</p>
<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="o">*</span> <span class="k">as</span> <span class="nx">stringify</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">json-stringify-safe</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">txt</span> <span class="o">=</span> <span class="p">(</span><span class="nx">o</span><span class="p">:</span> <span class="kr">any</span><span class="p">)</span> <span class="o">=></span> <span class="nx">stringify</span><span class="p">(</span><span class="nx">o</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now let’s define a super type of all operations which we need to audit.<br />
In the most general sense, we want to audit asynchronous functions (e.g. API calls) which take
arbitrary number of parameters. The following type definition of <code class="language-plaintext highlighter-rouge">AsyncFunction</code> represents a <em>super type</em>
for all such functions. In other words, every asyncrhonous function can be assigned to an <code class="language-plaintext highlighter-rouge">AsyncFunction</code> reference.</p>
<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre><span class="kd">type</span> <span class="nx">AsyncFunction</span> <span class="o">=</span> <span class="p">(...</span><span class="nx">args</span><span class="p">:</span> <span class="kr">any</span><span class="p">[])</span> <span class="o">=></span> <span class="nb">Promise</span><span class="o"><</span><span class="kr">any</span><span class="o">></span><span class="p">;</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now we can define a utility higher order function called <code class="language-plaintext highlighter-rouge">auditWrap</code>.
It takes as a parameter the function whose behaviour we need to audit log.
The type parameter <code class="language-plaintext highlighter-rouge">F</code> is sub-type of <code class="language-plaintext highlighter-rouge">AsyncFunction</code> and captures the exact
compile-time type of <code class="language-plaintext highlighter-rouge">fn</code>. Since <code class="language-plaintext highlighter-rouge">auditWrap</code> returns a function of type <code class="language-plaintext highlighter-rouge">F</code>, the types
of <code class="language-plaintext highlighter-rouge">fn</code> and the result are the same. Thus we can preserve the type safety from the caller’s point of view.</p>
<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
</pre></td><td class="rouge-code"><pre><span class="kd">const</span> <span class="nx">auditWrap</span> <span class="o">=</span> <span class="kd">function</span> <span class="o"><</span><span class="nx">F</span> <span class="kd">extends</span> <span class="nx">AsyncFunction</span><span class="o">></span><span class="p">(</span><span class="nx">fn</span><span class="p">:</span> <span class="nx">F</span><span class="p">):</span> <span class="nx">F</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">wrapper</span> <span class="o">=</span> <span class="k">async</span> <span class="p">(...</span><span class="nx">args</span><span class="p">:</span> <span class="kr">any</span><span class="p">[])</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">stringArgs</span> <span class="o">=</span> <span class="nx">args</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">txt</span><span class="p">).</span><span class="nx">join</span><span class="p">(</span><span class="dl">"</span><span class="s2">,</span><span class="se">\n</span><span class="dl">"</span><span class="p">);</span> <span class="c1">// Textualize all arguements</span>
<span class="k">try</span> <span class="p">{</span>
<span class="k">await</span> <span class="nx">log</span><span class="p">(</span><span class="s2">`Attempting to call function: "</span><span class="p">${</span><span class="nx">fn</span><span class="p">.</span><span class="nx">name</span><span class="p">}</span><span class="s2">" with arguements: </span><span class="p">${</span><span class="nx">stringArgs</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">result</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">fn</span><span class="p">(...</span><span class="nx">args</span><span class="p">);</span>
<span class="k">await</span> <span class="nx">log</span><span class="p">(</span><span class="s2">`Call to function "</span><span class="p">${</span><span class="nx">fn</span><span class="p">.</span><span class="nx">name</span><span class="p">}</span><span class="s2">" suceeded with result: </span><span class="p">${</span><span class="nx">txt</span><span class="p">(</span><span class="nx">result</span><span class="p">)}</span><span class="s2">`</span><span class="p">);</span>
<span class="k">return</span> <span class="nx">result</span><span class="p">;</span>
<span class="p">}</span> <span class="k">catch</span> <span class="p">(</span><span class="nx">e</span><span class="p">)</span> <span class="p">{</span>
<span class="k">await</span> <span class="nx">log</span><span class="p">(</span><span class="s2">`Call to function "</span><span class="p">${</span><span class="nx">fn</span><span class="p">.</span><span class="nx">name</span><span class="p">}</span><span class="s2">" failed with message: </span><span class="p">${</span><span class="nx">e</span><span class="p">.</span><span class="nx">message</span><span class="p">}</span><span class="s2">, Details: \n</span><span class="p">${</span><span class="nx">txt</span><span class="p">(</span><span class="nx">e</span><span class="p">)}</span><span class="s2">`</span><span class="p">);</span>
<span class="k">throw</span> <span class="nx">e</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="k">return</span> <span class="nx">wrapper</span> <span class="k">as</span> <span class="kr">any</span><span class="p">;</span>
<span class="p">};</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="usage" />
<h1 id="usage">Usage</h1>
<p>Let’s demonstrate how to use <code class="language-plaintext highlighter-rouge">auditWrap</code> in practice to audit log a couple of API Calls:</p>
<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nx">axios</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">axios</span><span class="dl">'</span><span class="p">;</span>
<span class="k">async</span> <span class="kd">function</span> <span class="nx">example</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">call1Result</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">auditWrap</span><span class="p">(</span><span class="nx">axios</span><span class="p">.</span><span class="kd">get</span><span class="p">)(</span><span class="dl">'</span><span class="s1">https://jsonplaceholder.typicode.com/todos/1</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">call2Result</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">auditWrap</span><span class="p">(</span><span class="nx">axios</span><span class="p">.</span><span class="kd">get</span><span class="p">)(</span><span class="dl">'</span><span class="s1">https://jsonplaceholder.typicode.com/todos/2</span><span class="dl">'</span><span class="p">);</span>
<span class="p">}</span>
<span class="nx">example</span><span class="p">()</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The produced audit log is huge - every axios parameter and response element is printed out. Below is a subset of the output
(<code class="language-plaintext highlighter-rouge">...</code> denotes omited output):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
</pre></td><td class="rouge-code"><pre>Attempting to call function: "wrap" with arguements: "https://jsonplaceholder.typicode.com/todos/1"
Call to function "wrap" suceeded with result: {
"status": 200,
"statusText": "OK",
"headers": {...},
"config": {...},
"request": {...},
"data": {
"userId": 1,
"id": 1,
"title": "delectus aut autem",
"completed": false
}
}
Attempting to call function: "wrap" with arguements: "https://jsonplaceholder.typicode.com/todos/2"
Call to function "wrap" suceeded with result: {
"status": 200,
"statusText": "OK",
"headers": {...},
"config": {...},
"request": {...},
"data": {
"userId": 1,
"id": 2,
"title": "quis ut nam facilis et officia qui",
"completed": false
}
}
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Notice that name of the function in the audit log is <code class="language-plaintext highlighter-rouge">wrap</code>. Indeed, if you print <code class="language-plaintext highlighter-rouge">axios.get.name</code>, you’ll see that this is its actual
function name.</p>
<p>Nevertheless, the above aproach is a significant improvement. We managed to eliminate all <code class="language-plaintext highlighter-rouge">try</code>-<code class="language-plaintext highlighter-rouge">catch</code>-<code class="language-plaintext highlighter-rouge">log</code> boilerplate code and to preserve type safety.
However, we can do a bit better. In the above example, we generated a huge amount of log. What if we want to audit log
only specific parts of the output or have more readable function names?</p>
<p>We can achieve this by introducing higher level function(s), which hide the details of the underlying libraries and return
exactly what we need:</p>
<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre><span class="kd">const</span> <span class="nx">apiCall</span> <span class="o">=</span> <span class="p">(</span><span class="nx">url</span><span class="p">:</span> <span class="kr">string</span><span class="p">,</span> <span class="nx">config</span><span class="p">?:</span> <span class="nx">AxiosRequestConfig</span><span class="p">)</span> <span class="o">=></span> <span class="nx">axios</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="nx">config</span><span class="p">).</span><span class="nx">then</span><span class="p">(</span><span class="nx">d</span> <span class="o">=></span> <span class="nx">d</span><span class="p">.</span><span class="nx">data</span><span class="p">)</span>
<span class="k">async</span> <span class="kd">function</span> <span class="nx">example</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">call1Result</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">auditWrap</span><span class="p">(</span><span class="nx">apiCall</span><span class="p">)(</span><span class="dl">'</span><span class="s1">https://jsonplaceholder.typicode.com/todos/1</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">call2Result</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">auditWrap</span><span class="p">(</span><span class="nx">apiCall</span><span class="p">)(</span><span class="dl">'</span><span class="s1">https://jsonplaceholder.typicode.com/todos/2</span><span class="dl">'</span><span class="p">);</span>
<span class="p">}</span>
<span class="nx">example</span><span class="p">()</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The output of the above is:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
</pre></td><td class="rouge-code"><pre>Attempting to call function: "apiCall" with arguements: "https://jsonplaceholder.typicode.com/todos/1"
Call to function "apiCall" suceeded with result: {
"userId": 1,
"id": 1,
"title": "delectus aut autem",
"completed": false
}
Attempting to call function: "apiCall" with arguements: "https://jsonplaceholder.typicode.com/todos/2"
Call to function "apiCall" suceeded with result: {
"userId": 1,
"id": 2,
"title": "quis ut nam facilis et officia qui",
"completed": false
}
</pre></td></tr></tbody></table></code></pre></div></div>Nikolay Grozevnikolay.grozev@gmail.comIn this article, we will demonstrate how to implement a simple type safe utility function for audit logging in TypeScript.Redux-Form with External Submit Button2018-06-08T05:22:09+00:002018-06-08T05:22:09+00:00https://nikgrozev.com/2018/06/08/redux-form-with-external-submit-button<div id="introduction" />
<h1 id="introduction">Introduction</h1>
<p>Recently, I’ve been working on a <a href="https://reactjs.org/">React</a>/<a href="https://redux.js.org/">Redux</a>
app which uses the <a href="https://redux-form.com/">redux-form</a> library for web forms.
It’s a great library which seamlessly integrates all aspects of web forms, including validation,
into the React/Redux data flow.</p>
<p>Below is a very simple example of a <code class="language-plaintext highlighter-rouge">redux-form</code> based form:</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nx">React</span><span class="p">,</span> <span class="p">{</span> <span class="nx">Component</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">react</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">Form</span><span class="p">,</span> <span class="nx">Field</span><span class="p">,</span> <span class="nx">reduxForm</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">redux-form</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">TextField</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">redux-form-material-ui</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="nx">RaisedButton</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">material-ui/RaisedButton</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">class</span> <span class="nx">SimpleForm</span> <span class="kd">extends</span> <span class="nx">Component</span> <span class="p">{</span>
<span class="nx">render</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">const</span> <span class="p">{</span> <span class="nx">handleSubmit</span><span class="p">,</span> <span class="nx">valid</span><span class="p">,</span> <span class="nx">submitting</span><span class="p">,</span> <span class="nx">onSubmit</span> <span class="p">}</span> <span class="o">=</span> <span class="k">this</span><span class="p">.</span><span class="nx">props</span><span class="p">;</span>
<span class="k">return</span> <span class="p">(</span>
<span class="p"><</span><span class="nc">Form</span> <span class="na">onSubmit</span><span class="p">=</span><span class="si">{</span><span class="nx">handleSubmit</span><span class="p">(</span><span class="nx">onSubmit</span><span class="p">)</span><span class="si">}</span><span class="p">></span>
<span class="p"><</span><span class="nc">Field</span> <span class="na">name</span><span class="p">=</span><span class="s">"firstName"</span> <span class="na">hintText</span><span class="p">=</span><span class="s">"First Name"</span> <span class="na">component</span><span class="p">=</span><span class="si">{</span><span class="nx">TextField</span><span class="si">}</span> <span class="na">fullWidth</span> <span class="p">/></span>
<span class="p"><</span><span class="nc">Field</span> <span class="na">name</span><span class="p">=</span><span class="s">"lastName"</span> <span class="na">hintText</span><span class="p">=</span><span class="s">"Last Name"</span> <span class="na">component</span><span class="p">=</span><span class="si">{</span><span class="nx">TextField</span><span class="si">}</span> <span class="na">fullWidth</span> <span class="p">/></span>
<span class="p"><</span><span class="nc">RaisedButton</span> <span class="na">label</span><span class="p">=</span><span class="s">"Submit"</span> <span class="na">type</span><span class="p">=</span><span class="s">"submit"</span> <span class="na">disabled</span><span class="p">=</span><span class="si">{</span><span class="nx">valid</span> <span class="o">||</span> <span class="nx">submitting</span><span class="si">}</span> <span class="p">/></span>
<span class="p"></</span><span class="nc">Form</span><span class="p">></span>
<span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">validate</span><span class="p">(</span><span class="nx">form</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">errors</span> <span class="o">=</span> <span class="p">{};</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">form</span><span class="p">.</span><span class="nx">firstName</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">errors</span><span class="p">.</span><span class="nx">firstName</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">required</span><span class="dl">"</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nx">errors</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">export</span> <span class="k">default</span> <span class="nx">reduxForm</span><span class="p">({</span> <span class="na">form</span><span class="p">:</span> <span class="dl">'</span><span class="s1">simple-form</span><span class="dl">'</span><span class="p">,</span> <span class="nx">validate</span> <span class="p">})(</span><span class="nx">SimpleForm</span><span class="p">);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>This example uses the <a href="https://material-ui.com/">material-ui</a> library
and its <code class="language-plaintext highlighter-rouge">redux-form</code> bindings. We defined 2 text fields (<code class="language-plaintext highlighter-rouge">firstName</code> and <code class="language-plaintext highlighter-rouge">lastName</code>)
and a submission button, which is only enabled if the form is valid and is not being submitted.
Finally, we register the form with <code class="language-plaintext highlighter-rouge">Redux</code> and bind it to a validation function using the
<code class="language-plaintext highlighter-rouge">reduxForm</code> function.</p>
<p>Behind the scenes, <code class="language-plaintext highlighter-rouge">redux-form</code> invokes the <code class="language-plaintext highlighter-rouge">validate</code> function
on every field change and keeps track of whether the user has touched any field.
This information is then available via the component properties.
You can look at the <a href="https://redux-form.com/">official documentation</a> of <code class="language-plaintext highlighter-rouge">redux-form</code>
to see all relevant properties and quite a few examples.</p>
<p>While this is a very simple and intuitive way to build forms, it has one major problem.
It assumes that the form and the submission button are in the same component.
However, in some cases we need to submit a form from outside of its component.
For example, a form can be embedded as a step/screen within a wizard.
Thus, the “NEXT” button of the parent wizard component must submit the embedded forms.</p>
<p>This setup poses 2 problems:</p>
<ul>
<li>The button in the parent component can not access the form component properties. Therefore, it can not check if the form is ready to be submitted (e.g. all fields are valid);</li>
<li>The button in the parent component can not programmatically submit the form;</li>
</ul>
<div id="solution1" />
<h1 id="solution-1">Solution 1</h1>
<p>A form component can observe when it becomes ready to be submitted and signal
the parent component via a specially provided callback</p>
<p>The following form demonstrates this idea. It does not have a submission button and
accepts a callback function <code class="language-plaintext highlighter-rouge">enabledCallback</code> via its properties. We use the <code class="language-plaintext highlighter-rouge">componentDidUpdate</code>
life cycle method to detect when a component becomes (un)available for submission:</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nx">React</span><span class="p">,</span> <span class="p">{</span> <span class="nx">Component</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">react</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">Form</span><span class="p">,</span> <span class="nx">Field</span><span class="p">,</span> <span class="nx">reduxForm</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">redux-form</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">TextField</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">redux-form-material-ui</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">class</span> <span class="nx">ChildComponentForm</span> <span class="kd">extends</span> <span class="nx">Component</span> <span class="p">{</span>
<span class="nx">componentDidUpdate</span><span class="p">(</span><span class="nx">prevProps</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">enabled</span> <span class="o">=</span> <span class="p">(</span><span class="k">this</span><span class="p">.</span><span class="nx">props</span><span class="p">.</span><span class="nx">valid</span> <span class="o">&&</span> <span class="o">!</span><span class="k">this</span><span class="p">.</span><span class="nx">props</span><span class="p">.</span><span class="nx">submitting</span><span class="p">)</span> <span class="o">||</span> <span class="o">!</span><span class="k">this</span><span class="p">.</span><span class="nx">props</span><span class="p">.</span><span class="nx">anyTouched</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">wasEnabled</span> <span class="o">=</span> <span class="p">(</span><span class="nx">prevProps</span><span class="p">.</span><span class="nx">valid</span> <span class="o">&&</span> <span class="o">!</span><span class="nx">prevProps</span><span class="p">.</span><span class="nx">submitting</span><span class="p">)</span> <span class="o">||</span> <span class="o">!</span><span class="nx">prevProps</span><span class="p">.</span><span class="nx">anyTouched</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">enabled</span> <span class="o">!==</span> <span class="nx">wasEnabled</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Signal to the parent component that the form can be submitted</span>
<span class="k">this</span><span class="p">.</span><span class="nx">props</span><span class="p">.</span><span class="nx">enabledCallback</span><span class="p">(</span><span class="nx">enabled</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nx">render</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">const</span> <span class="p">{</span> <span class="nx">handleSubmit</span><span class="p">,</span> <span class="nx">onSubmit</span> <span class="p">}</span> <span class="o">=</span> <span class="k">this</span><span class="p">.</span><span class="nx">props</span><span class="p">;</span>
<span class="c1">// Standard form but no submit button - it's in the parent</span>
<span class="k">return</span> <span class="p">(</span>
<span class="p"><</span><span class="nc">Form</span> <span class="na">onSubmit</span><span class="p">=</span><span class="si">{</span><span class="nx">handleSubmit</span><span class="p">(</span><span class="nx">onSubmit</span><span class="p">)</span><span class="si">}</span><span class="p">></span>
<span class="p"><</span><span class="nc">Field</span> <span class="na">name</span><span class="p">=</span><span class="s">"firstName"</span> <span class="na">hintText</span><span class="p">=</span><span class="s">"First Name"</span> <span class="na">component</span><span class="p">=</span><span class="si">{</span><span class="nx">TextField</span><span class="si">}</span> <span class="na">fullWidth</span> <span class="p">/></span>
<span class="p"><</span><span class="nc">Field</span> <span class="na">name</span><span class="p">=</span><span class="s">"lastName"</span> <span class="na">hintText</span><span class="p">=</span><span class="s">"Last Name"</span> <span class="na">component</span><span class="p">=</span><span class="si">{</span><span class="nx">TextField</span><span class="si">}</span> <span class="na">fullWidth</span> <span class="p">/></span>
<span class="p"></</span><span class="nc">Form</span><span class="p">></span>
<span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">validate</span><span class="p">(</span><span class="nx">form</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">errors</span> <span class="o">=</span> <span class="p">{};</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">form</span><span class="p">.</span><span class="nx">firstName</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">errors</span><span class="p">.</span><span class="nx">firstName</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">required</span><span class="dl">"</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nx">errors</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">export</span> <span class="k">default</span> <span class="nx">reduxForm</span><span class="p">({</span> <span class="na">form</span><span class="p">:</span> <span class="dl">'</span><span class="s1">child-form</span><span class="dl">'</span><span class="p">,</span> <span class="nx">validate</span> <span class="p">})(</span><span class="nx">ChildComponentForm</span><span class="p">);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>From the parent component, we can pass a callback function which saves in the state
whether the form can be submitted. Based on the state, we can conditionally
disable the submission button.</p>
<p>While rendering the form in its parent, we preserve a reference to its
respective component as a member variable.
We can then use this reference to programmatically submit the form when
the submission button is clicked.</p>
<p>The following demonstrates this approach:</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nx">React</span><span class="p">,</span> <span class="p">{</span> <span class="nx">Component</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">react</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="nx">RaisedButton</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">material-ui/RaisedButton</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="nx">ChildComponentForm</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">./ChildComponentForm</span><span class="dl">'</span><span class="p">;</span>
<span class="k">export</span> <span class="k">default</span> <span class="kd">class</span> <span class="nx">App</span> <span class="kd">extends</span> <span class="nx">Component</span> <span class="p">{</span>
<span class="kd">constructor</span><span class="p">(</span><span class="nx">props</span><span class="p">)</span> <span class="p">{</span>
<span class="k">super</span><span class="p">(</span><span class="nx">props</span><span class="p">);</span>
<span class="c1">// We'll keep in state whether the submission button is enabled</span>
<span class="k">this</span><span class="p">.</span><span class="nx">state</span> <span class="o">=</span> <span class="p">{</span> <span class="na">submitButtonEnabled</span><span class="p">:</span> <span class="kc">true</span> <span class="p">};</span>
<span class="p">}</span>
<span class="nx">render</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// Render the form - pass the callback and obtain a reference</span>
<span class="k">return</span> <span class="p">(</span>
<span class="p"><</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nc">ChildComponentForm</span>
<span class="na">onSubmit</span><span class="p">=</span><span class="si">{</span><span class="k">this</span><span class="p">.</span><span class="nx">onSubmit</span><span class="si">}</span>
<span class="c1">// While rendering - save a reference to the form</span>
<span class="na">ref</span><span class="p">=</span><span class="si">{</span><span class="nx">form</span> <span class="o">=></span> <span class="k">this</span><span class="p">.</span><span class="nx">formReference</span> <span class="o">=</span> <span class="nx">form</span><span class="si">}</span>
<span class="c1">// Pass a callback for when enabled</span>
<span class="na">enabledCallback</span><span class="p">=</span><span class="si">{</span><span class="k">this</span><span class="p">.</span><span class="nx">enabledCallback</span><span class="si">}</span>
<span class="p">/></span>
<span class="p"><</span><span class="nc">RaisedButton</span>
<span class="na">label</span><span class="p">=</span><span class="s">"Submit"</span>
<span class="na">disabled</span><span class="p">=</span><span class="si">{</span><span class="o">!</span><span class="k">this</span><span class="p">.</span><span class="nx">state</span><span class="p">.</span><span class="nx">submitButtonEnabled</span><span class="si">}</span>
<span class="na">onClick</span><span class="p">=</span><span class="si">{</span><span class="k">this</span><span class="p">.</span><span class="nx">onSubmitClick</span><span class="si">}</span>
<span class="p">/></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p">);</span>
<span class="p">}</span>
<span class="c1">// Callback for when the form is submitted</span>
<span class="nx">onSubmit</span> <span class="o">=</span> <span class="p">(</span><span class="nx">form</span><span class="p">)</span> <span class="o">=></span> <span class="nx">alert</span><span class="p">(</span><span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">form</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>
<span class="c1">// Callback for the button - use the saved form reference to submit it </span>
<span class="nx">onSubmitClick</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="k">this</span><span class="p">.</span><span class="nx">formReference</span><span class="p">.</span><span class="nx">submit</span><span class="p">()</span>
<span class="c1">// Callback for when form can be submitted</span>
<span class="nx">enabledCallback</span> <span class="o">=</span> <span class="p">(</span><span class="nx">enabled</span><span class="p">)</span> <span class="o">=></span> <span class="k">this</span><span class="p">.</span><span class="nx">setState</span><span class="p">(</span><span class="nx">prevState</span> <span class="o">=></span> <span class="p">({</span>
<span class="p">...</span><span class="nx">prevState</span><span class="p">,</span>
<span class="na">submitButtonEnabled</span><span class="p">:</span> <span class="nx">enabled</span>
<span class="p">}))</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div id="solution2" />
<h1 id="solution-2">Solution 2</h1>
<p>Another approach is to use the global functions and selectors provided by <code class="language-plaintext highlighter-rouge">redux-form</code>.
They vary from version to version - in the example below we’ll be using version 7.2.0.</p>
<p>Every <code class="language-plaintext highlighter-rouge">redux-form</code> has a unique name, which is used as an identifier in redux.
In the example below, we define a simple form and associate it with a name:</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nx">React</span><span class="p">,</span> <span class="p">{</span> <span class="nx">Component</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">react</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">Form</span><span class="p">,</span> <span class="nx">Field</span><span class="p">,</span> <span class="nx">reduxForm</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">redux-form</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">TextField</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">redux-form-material-ui</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">class</span> <span class="nx">ChildComponentForm</span> <span class="kd">extends</span> <span class="nx">Component</span> <span class="p">{</span>
<span class="nx">render</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">const</span> <span class="p">{</span> <span class="nx">handleSubmit</span><span class="p">,</span> <span class="nx">onSubmit</span> <span class="p">}</span> <span class="o">=</span> <span class="k">this</span><span class="p">.</span><span class="nx">props</span><span class="p">;</span>
<span class="c1">// Standard form but no submit button - it's in the parent</span>
<span class="k">return</span> <span class="p">(</span>
<span class="p"><</span><span class="nc">Form</span> <span class="na">onSubmit</span><span class="p">=</span><span class="si">{</span><span class="nx">handleSubmit</span><span class="p">(</span><span class="nx">onSubmit</span><span class="p">)</span><span class="si">}</span><span class="p">></span>
<span class="p"><</span><span class="nc">Field</span> <span class="na">name</span><span class="p">=</span><span class="s">"firstName"</span> <span class="na">hintText</span><span class="p">=</span><span class="s">"First Name"</span> <span class="na">component</span><span class="p">=</span><span class="si">{</span><span class="nx">TextField</span><span class="si">}</span> <span class="na">fullWidth</span> <span class="p">/></span>
<span class="p"><</span><span class="nc">Field</span> <span class="na">name</span><span class="p">=</span><span class="s">"lastName"</span> <span class="na">hintText</span><span class="p">=</span><span class="s">"Last Name"</span> <span class="na">component</span><span class="p">=</span><span class="si">{</span><span class="nx">TextField</span><span class="si">}</span> <span class="na">fullWidth</span> <span class="p">/></span>
<span class="p"></</span><span class="nc">Form</span><span class="p">></span>
<span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">validate</span><span class="p">(</span><span class="nx">form</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">errors</span> <span class="o">=</span> <span class="p">{};</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">form</span><span class="p">.</span><span class="nx">firstName</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">errors</span><span class="p">.</span><span class="nx">firstName</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">required</span><span class="dl">"</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nx">errors</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// The global name for this form will be 'child-form'</span>
<span class="k">export</span> <span class="k">default</span> <span class="nx">reduxForm</span><span class="p">({</span> <span class="na">form</span><span class="p">:</span> <span class="dl">'</span><span class="s1">child-form</span><span class="dl">'</span><span class="p">,</span> <span class="nx">validate</span> <span class="p">})(</span><span class="nx">ChildComponentForm</span><span class="p">);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>This form is agnostic of its parent which will perform its submission and inspect its validity,
as in this example usage:</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nx">React</span><span class="p">,</span> <span class="p">{</span> <span class="nx">Component</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">react</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">connect</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">react-redux</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="nx">RaisedButton</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">material-ui/RaisedButton</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="nx">ChildComponentForm</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">./ChildComponentForm</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">bindActionCreators</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">redux</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="p">{</span> <span class="nx">isValid</span><span class="p">,</span> <span class="nx">isSubmitting</span><span class="p">,</span> <span class="nx">submit</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">redux-form</span><span class="dl">'</span><span class="p">;</span>
<span class="c1">// A simple component which uses the form</span>
<span class="k">export</span> <span class="kd">class</span> <span class="nx">App</span> <span class="kd">extends</span> <span class="nx">Component</span> <span class="p">{</span>
<span class="nx">render</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">(</span>
<span class="p"><</span><span class="nt">div</span><span class="p">></span>
<span class="p"><</span><span class="nc">ChildComponentForm</span> <span class="na">onSubmit</span><span class="p">=</span><span class="si">{</span><span class="k">this</span><span class="p">.</span><span class="nx">onSubmit</span><span class="si">}</span> <span class="p">/></span>
<span class="p"><</span><span class="nc">RaisedButton</span>
<span class="na">label</span><span class="p">=</span><span class="s">"Submit"</span>
<span class="na">disabled</span><span class="p">=</span><span class="si">{</span><span class="o">!</span><span class="k">this</span><span class="p">.</span><span class="nx">props</span><span class="p">.</span><span class="nx">formEnabled</span><span class="si">}</span>
<span class="na">onClick</span><span class="p">=</span><span class="si">{</span><span class="k">this</span><span class="p">.</span><span class="nx">props</span><span class="p">.</span><span class="nx">submitForm</span><span class="si">}</span>
<span class="p">/></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p">);</span>
<span class="p">}</span>
<span class="c1">// Callback for when the form is submitted</span>
<span class="nx">onSubmit</span> <span class="o">=</span> <span class="p">(</span><span class="nx">form</span><span class="p">)</span> <span class="o">=></span> <span class="nx">alert</span><span class="p">(</span><span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">(</span><span class="nx">form</span><span class="p">,</span> <span class="kc">null</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>
<span class="p">}</span>
<span class="c1">// Use redux-form selectors to check the form's state - e.g. valid, submitting</span>
<span class="kd">function</span> <span class="nx">mapStateToProps</span><span class="p">(</span><span class="nx">state</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">{</span>
<span class="na">formEnabled</span><span class="p">:</span> <span class="nx">isValid</span><span class="p">(</span><span class="dl">'</span><span class="s1">child-form</span><span class="dl">'</span><span class="p">)(</span><span class="nx">state</span><span class="p">)</span> <span class="o">&&</span> <span class="o">!</span><span class="nx">isSubmitting</span><span class="p">(</span><span class="dl">'</span><span class="s1">child-form</span><span class="dl">'</span><span class="p">)(</span><span class="nx">state</span><span class="p">)</span>
<span class="p">};</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">mapDispatchToProps</span><span class="p">(</span><span class="nx">dispatch</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Bind an action, which submit the form by its name</span>
<span class="k">return</span> <span class="nx">bindActionCreators</span><span class="p">({</span>
<span class="na">submitForm</span><span class="p">:</span> <span class="p">()</span> <span class="o">=></span> <span class="nx">submit</span><span class="p">(</span><span class="dl">'</span><span class="s1">child-form</span><span class="dl">'</span><span class="p">)</span>
<span class="p">},</span> <span class="nx">dispatch</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">export</span> <span class="k">default</span> <span class="nx">connect</span><span class="p">(</span><span class="nx">mapStateToProps</span><span class="p">,</span> <span class="nx">mapDispatchToProps</span><span class="p">)(</span><span class="nx">App</span><span class="p">);</span>
</pre></td></tr></tbody></table></code></pre></div></div>Nikolay Grozevnikolay.grozev@gmail.comIn this article, I'll show how to combine a redux-form with an external submission button ...