Intellij Fail with Maven Provided Dependencies

For Java development, I use either Intellij or Eclipse. Both have strengths and weaknesses. I did encounter one very annoying issue with Intellij recently, and apparently it is one that JetBrains have been aware of for over two years, and haven’t shown the slightest interest in fixing. Given that my employer pays for Intellij licences, I feel justified in feeling rather disappointed.

The Intellij/maven integration is generally excellent - much faster and more stable than with Eclipse. However when running Java code from within the IDE, Intellij completely ignores any dependencies declared in the Maven pom with scope=provided. This particularly bites people who (like me) are trying to use Spring Boot, which provides tools that generate maven poms with all necessary dependencies declared. The resulting code compiles fine from the Maven commandline, and can be started via Spring-boot-specific commands. But running the code from the IDE doesn’t work - some required dependencies are not loaded.

This has been reported but is not being taken at all seriously at JetBrains.

It’s a rather tricky problem to work around; a few useful suggestions are made in the above issue-report, but none are perfect.

SSL Failures

Elasticsearch Overview

I haven’t posted much recently, but not because I’ve run out of topics - quite the opposite, it’s been a busy few months. Among other things, I’ve been working on a project involving Elasticsearch. As a side-effect, here is an overview of Elasticsearch.

Java Path Streams Trap

Java1.8 streams and closures can be cool:

  return Files.list(baseDir)
            .filter(path -> path.getFileName().toString().endsWith(suffix))

However there is a nasty bug in the above: Files.list(baseDir) returns a stream object that must be explicitly closed. Not doing so leaks a file-handle to the directory being listed! Sadly, the bugfree version does not look quite so elegant:

  try(Stream<Path> paths = Files.list(baseDir)) {
    return paths
            .filter(path -> path.getFileName().toString().endsWith(suffix))

And by the way, the “toString” is necessary too, as Path.endsWith() and String.endsWith() are not the same thing.

Designing good library APIs is an art - and IMO whoever at Sun approved Files.list and Path.endsWith screwed up. If only they had read Chapter 5, “Candy Machine Interfaces” from “Writing Solid Code” by Steve Maguire - an old book now, but that chapter has some great and still-relevant advice.

Apache HBase Overview

I’ve just completed an article on HBase, in case anyone is interested. Feedback welcome.

Apache Hive v2.0 Released

Apache Hive 2.0 was released in Feb 2016. The most significant changes are:

  • hive metadata can be stored in HBase rather than a relational database (alpha)
  • long-lived worker processes - ie somewhat like Impala (beta)
  • built-in support for HPL/SQL - a “procedural SQL” language mostly compatible with Oracle PL/SQL, Teradata BTEQ etc. See the HPSQL site
  • performance improvements, particularly when using Spark as the back-end execution engine.
  • web-based admin interface for HiveServer2 daemon

Hive 2.0 is not included in Hortonworks HDP2.4 (released 2016-03-01) - maybe next version. It isn’t in the current Cloudera release either.

In related news Kafka 0.9 has a new Java API, and initial support for “native streaming”.

Update 2016-07-03: I’ve added an article on Hive to this site.

Dropwizard and Hystrix

I’ve recently been doing some maintenance work on an existing Java application (a server providing REST APIs to clients, and talking to a database). Two of the libraries used there (Dropwizard, Hystrix) were new to me, and interesting - worth a brief post at least.

The Kafka Message Broker