Java 8 has been released quite a long time, since I come to ruby in 2011, I haven’t work with java. Even I heard that there’re few cool features come out with Java 8, like lambda, stream collection, functional interface, new date api. none of them can attract me, given ruby ship all these features since the date of birth.
While recently I do try to solve a problem with Java, and I found that Java has changed a lot compared with my impression on it. in this post, I’m going to talk about the collectors shipped with Java 8. also I’ll try to give collector example in scala.
The Problem
Reduction, (aka. iterating on a collection, apply reduction computation on each elements, and produce a single result or a smaller collection) is a common problem in any programming language.
Let’s look at a specific example:
Given an collection of employees, grouping these employees by age produce a map between age and list of employees.
if you have been working with Java for quite a long time, you may be sick to write these code. you must have write code in this structure for quite a long time.
to demonstrate the duplication of this structure, let’s rewrite the above code to this format:
all these code did is to collect some information for give collection and apply reduction on the items in this collection and produce a result container.
with Java 8s collector interface, you can simply do
The magic is behind the Collector<T, A, R> interface:
Collectors.groupingBy is a built in collector which acceptting a function with type T -> K which can be group against(in this case, employee.age()).
it will produce a result with type Map<K, List<T>>(aka, the result type R).
Here is the official definition of Collector from its api document:
A mutable reduction operation that accumulates input elements into a mutable result container, optionally transforming the accumulated result into a final representation after all input elements have been processed. Reduction operations can be performed either sequentially or in parallel.
You see from the document, Collector take three type parameters T, A and R, where T is the type of element inside the collection, A is an intermediate type which could be used to do the mutable reduction, R is the type of result.
There four functions in this interface which work together to accumulate entries into a mutable result container.
supplier(), with type () -> A - creation of a new result container.
accumulator(), with type(A, T) -> A - incorprating a new element into the result container.
combiner(), with type (A, A) -> A - combing two result container into one.
finisher(), with type A -> R - a optional final transformation on the result container to get the result. the optional means that in some scenarios, A and R could be same, so the finisher function is not required. but in some other cases, when A and R are different, this function is required to get the final result.
In the previous example, the type of result of Collector.groupingBy is Collector<Employee, ?, Map<Integer, List<Employee>>.
let’s extend this problem a little bit: how about grouping employees by age range(e.g. 20-29 as a group, 30-39 as a group)
this time, you can not find any buitin collector which is suitable to solve this problem, now, you will need a customised collector to do the reduction.
(this blog post)[http://www.nurkiewicz.com/2014/07/introduction-to-writing-custom.html] is a fairly good guide for how to create you own Collector implementation.
Collector in Scala
After found this useful pattern, I wonder if scala’s powerful collection system support this computation. Unfortunately, I can not found a similar api from any collection type. But I do found that we can easily build our own version of collector based on scala.collection.mutable.Builder.
scala.collection.mutable.Builder play the same role with accumulator (the A) in java Collector. Let’s see the following example of how we implement the collect method in scala and how we use it to solve the word count problem:
importscala.collection.mutable.Builder//`T` is the type of element in collection, `Builder` is the type of intermediate result container, `R` is the type of reduction result.defcollect[T, R](ts:List[T],a:Builder[T, R]):R={ts.foreach(a+=_)//invoke the accumulator functiona.result//invoke the finisher function}//word counting builder//from Seq[String] to Map[String, Int]classCounterBuilder[String, Map[String, Int]]extendsBuilder[String, Map[String, Int]]{varcounter=scala.collection.mutable.Map[String, Int]()def+=(el:String)={counter.get(el)match{caseNone=>counter+=el->1caseSome(count)=>counter+=el->(count+1)}this}defclear=counter=scala.collection.mutable.Map[String, Int]()defresult:Map[String, Int]=counter.toMap[String, Int].asInstanceOf[Map[String, Int]]}
and here is the code to use the CounterBuilder
//use casevalxs=List("a","a","a","c","d")//the supplier functionvalbuilder=newCounterBuildervalres=collect[String, Map[String, Int]](xs,newCounterBuilder)//output: Map(d -> 1, a -> 3, c -> 1)
Conclusion
Java 8s Collector Api provide a better way to encapsulate reduction computation - not only some built in reduction(e.g. max, min, sum, average), but also customized reduction(via customised collector), Collector is designed to be composed, which means these reduction logic are much easier to be reused.
Scala dose not have native support for customised mutable reduction, but based on scala’s powerfull collection system, we can create our own version.
micro service getting more attractions in the past few years, more and more orgnizations is moving to this area. many framework were created to make building micro service easier, in java world, we have DropWizard, in ruby world, we have Rails-API, Grape and Lotus. but how the scala guys solve this problem?
in my previous post, I demostrated how to build api with spray, in this post I’ll try to use another framework - unfilter and json4s to build a similar api.
The problems:
you want to build a api which can accept http request and send response in json format.
to achieve this goal, we need:
a server listening on a local port, wrap received http request and pass it to the application.
a request mapping mechanism where you can define how http request should be handled.
a tool which can covert between plan scala object and json.
Unfilter have it’s answer for the first two questions:
Request mapping
in unfilter, the request mapping mechanism was called plan and intent, a accurat name, right?
from unfilter’s document:
An intent is a partial function for matching requests.
A plan binds an intent to a particular server interface.
The biggest difference between spray-json and json4s is serialization/deserialization done implicitly or explicitly.
in spray-json, you can get serialization/deserialization(aka. marshalling) implicitly if you defined your own JsonFormat,
the marshalling mechanism will do their job while extracting information from request and send response implicitly. it’s cool when everything works fine, but if something went wrong it’s really hard to debug. e.g. this one took me years to find out.
Compared with spray, unfilter focused on request mapping and dispatching, json4s focused on json serialization/deserialization, they both did a very good job. I highly recommand you to try it in your next scala api project.
Spray is an open-source toolkit for building REST/HTTP-based integration layers on top of Scala and Akka. Being asynchronous, actor-based, fast, lightweight, modular and testable it’s a great way to connect your Scala applications to the world.
TLDR;
This post will give you a example of how to use spray to build a REST api.
Problem
you want to build a REST api that support the following operations:
GET /users/:id
POST /users
routing
spray-routing gives you a elegant DSL to build routing system, which will accept http request and respond correctly. let’s see the example:
//File: src/scala/com/example/MyService.scalatraitMyServiceextendsHttpService{implicitvalec:ExecutionContext=actorRefFactory.dispatchervaluserRepository:UserRepositoryvalmyRoute=path("users"/Segment){userId=>{get{complete{//must import SprayJsonSupport to get a json mashallerimportspray.httpx.SprayJsonSupport._userRepositoryfetchuserId}}}}}
The MyService trait extends spray.routing.HttpService which includes a bunch of convinient mehtods for creating DSL, such as path, get, complete.
The variable myRoute defines a set of rules:
When a http request matches GET /users/:id, call userRepo.get with extracted userId, and response with the result.
When a http request matches POST /users/, call userRepo.save with extracted userData, and response correponded status code.
Note that there is a field defined as val userRepository: UserRepository not be initialized. This field will be implemened in a acturall actor (MyServiceActor in this example).
The actural business logic was deleagted into this object.
json support
I haven’t find a JSON library in Java/Scala world which providing as good api as Ruby doses. if you konw one please let me know.
spray-json is the most beautiful one I ever found!
spray-json allows you to convert between:
String JSON documents
JSON Abstract Syntax Trees (ASTs) with base type JsValue
instances of arbitrary Scala types
in this post, we just want to convert between JSON documents and Scala case class. To enable JSON serialization/deserialization for your case class, just define a implicit method which returns a instance of RootJsonFormat[YourCaseClass].
This actor mixin MyService, in the receive method,
call runRoute to handle http request with pre-defined route.
A Tips:
put your business logic in plain scala object instead of actor.
I found that it is really hard to test logic in a actor, so I preferred to implate business logic in a pure scala class, then it is much easier to test it.
then inject a instance of this class into a actor.
