The Graal Virtual Machine

Oracle are well known for the Java Virtual Machine project (inherited from Sun). They have now released version 1.0 of a general-purpose virtual machine called Graal that supports:

  • Java bytecode (production) - includes Java, Scala, Groovy, Kotlin
  • Javascript (production) - including Node.js applications
  • LLVM bitcode, ie apps compiled from C, C++, Rust and other languages via the LLVM compiler (experimental)
  • Python, Ruby, and R (experimental)

Code in these languages can call into other code running within Graal, regardless of the language it was written in! Arranging for additional libraries (including the language standard libraries) to be available requires some steps, but is possible.

Not only does this allow running apps in a “standalone” environment, it means that any larger software package which embeds the Graal VM and allows user code to run in that VM can support any language that Graal supports. Examples include database servers which embed the VM for stored procedure logic.

With Oracle, it is important to look at the licencing terms-and-conditions. This does initially seem to be OK; the code is completely licensed under the GPL2-with-classpath-exception, like OpenJDK. Oracle does warn that there is “no support” for the open-source code (aka “community edition”) and recommends that a support licence be bought for the “enterprise edition” instead - but OpenJDK is reliable enough, and so the Graal “community edition” will hopefully be so too.

The Graal project website has more information.

The New Oracle Java Release Cycle

Oracle have changed the way they release new versions of the Java JDK and the Oracle Java Virtual Machine.

Google Cloud Functions, BigQuery, and Related Matters

Apache Beam and Google Dataflow Overview

Apache Beam and CSV Headers

As noted in the previous post, I’m working with GCP at the moment - and with Apache Beam on Google Dataflow.

Beam has many cool features - but sometimes things that should be trivial are unexpectedly complicated - like reading the ‘header’ line of a CSV file.

Google Cloud Platform and AppEngine

I’ve been doing a lot of work on the Google Cloud Platform recently. As a result, here are two articles giving an overview of GCP Identities and Resources (which might be a good place to start if you don’t know GCP) and GCP AppEngine. There will be more articles to come..

Accessing HDFS from Spark in Zeppelin

When using a Zeppelin notebook for data analysis (eg on a Hortonworks platform), files stored in HDFS can be manipulated via a paragraph using the shell interpreter (%sh header) and then using the hdfs commandline tool. However sometimes it is nicer to access HDFS from spark/scala code rather than requiring a separate block.

More Spring Quirks

Two more weird problems encountered recently with the Spring framework for Java (v4.3.x)..

The spring bean name “securityProperties” appears to be magic/reserved. I created a class with name SecurityProperties, annotated with @Component as usual for Spring - and at runtime instantiation of a class which should have been injected with the singleton of that type failed with “no such bean”. Specifying an explicit bean-name via @Component(value="someOtherBeanName") fixed the problem; so did changing the class name (ie changing the default derived bean name).

I also had problems with a rest endpoint annotated with @RequestMapping(value="/some/path/{id}") - ie when using a path parameter. When the requested url was of form “/some/path/a.b.c.d” then the annotated method was invoked with param id set to “a.b.c”, ie the last dot and everything following was stripped. Some googling quickly found the answer (ie other people appear to regularly hit this too) - Spring webmvc processes urls like “/some/file.html” by stripping the file-suffix. As noted in this stackoverflow thread, the mapping annotation can be modified to use a regular expression-match eg @RequestMapping(value="/some/path/{id:.*}). However I just wanted to turn off Spring’s suffix-processing completely. It took me some time to figure out how; my solution (for annotation-based configuration) is:

import org.springframework.context.annotation.Bean;
import org.springframework.web.servlet.config.annotation.PathMatchConfigurer;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;
     * Allow rest-entry-points to accept path-params like "a.b.c".
     * <p>
     * By default, given a rest-endpoint mapping of form "/some/endpoint/{id}" and
     * a URL of form "/some/endpoint/aresource.html", Spring sets param id to just
     * "aresource", ie the suffix is stripped. This is not desired in this app, so
     * disable this default behaviour...
     * </p>
    public WebMvcConfigurer disablePathSuffixAdapter() {
        return new WebMvcConfigurerAdapter() {
            public void configurePathMatch(PathMatchConfigurer configurer) {