Saturday, August 25, 2012

Kiama & macro

Here is a quick and dirty example of using kiama along with the fields macro.
import annotated.{ Field => AnnField, Fields => AnnFields }

object pp {
    private object kpp extends org.kiama.output.PrettyPrinter
    import kpp._

    private def anyToDoc(any : Any) : Doc =
        any match {
            case song : Song        => annotatedToDoc("Song", song, songFields)
            case artist : Artist    => annotatedToDoc("Artist", artist, artistFields)
            case comp : Compilation => annotatedToDoc("Compilation", comp, compilationFields)
            case job : UploaderJob  => annotatedToDoc("Job", job, uploaderJobFields)

            case map : Map[_, _] =>
                    prefix = "Map",
                    elemToDoc = {
                        (pair : (Any, Any)) =>
                            anyToDoc(pair._1) <> " -> " <> nest(anyToDoc(pair._2))

            case seq : Seq[_] =>
                list(seq.toList, prefix = "Sequence", elemToDoc = anyToDoc)

            case _ => value(any)

    private def annotatedToDoc[T <: AnyRef](name : String, t : T, fields : AnnFields[T, FieldArgs]) : Doc = {
            prefix = name,
            elemToDoc = {
                (f : AnnField[T, FieldArgs]) =>
           <> " = " <> anyToDoc(f.get(t))

    def apply(any : Any) : String = pretty(anyToDoc(any))
pp (uploaderJob) easily prints data like this:
    artistMap = Map(
        KeyRef(1x) -> Artist(
                handle = Tester2,
                marks = Set(),
                id = None,
                name = None,
                website = None,
                country = None,
                location = None),
        KeyRef(2u) -> Artist(
                handle = Tester,
                marks = Set(),
                id = None,
                name = None,
                website = None,
                country = None,
                location = None)),
    songMap = Map(
        KeyRef(6t) -> Song(
                relativePath = Path(test7.mp3),
                title = test7,
                artistRefs = Set(KeyRef(2u)),
                tags = Set(),
                id = None,
                sourceId = None,
                mixSongId = None,
                year = Some(1999)),
        KeyRef(5c) -> Song(
                relativePath = Path(test6.mp3),
                title = test6,
                artistRefs = Set(KeyRef(2u)),
                tags = Set(),
                id = None,
                sourceId = None,
                mixSongId = None,
                year = Some(1999)),
        KeyRef(9i) -> Song(
                relativePath = Path(test3.mp3),
                title = test3,
                artistRefs = Set(KeyRef(1x)),
                tags = Set(),
                id = None,
                sourceId = Some(4),
                mixSongId = None,
                year = None),
        KeyRef(3w) -> Song(
                relativePath = Path(test5.mp3),
                title = test5,
                artistRefs = Set(KeyRef(2u)),
                tags = Set(),
                id = None,
                sourceId = None,
                mixSongId = None,
                year = Some(1999)),
        KeyRef(7l) -> Song(
                relativePath = Path(test.mp3),
                title = test,
                artistRefs = Set(),
                tags = Set(),
                id = None,
                sourceId = None,
                mixSongId = None,
                year = None),
        KeyRef(8f) -> Song(
                relativePath = Path(test2.mp3),
                title = test2,
                artistRefs = Set(),
                tags = Set(),
                id = None,
                sourceId = None,
                mixSongId = None,
                year = None),
        KeyRef(4p) -> Song(
                relativePath = Path(test4.mp3),
                title = test4,
                artistRefs = Set(KeyRef(2u)),
                tags = Set(),
                id = None,
                sourceId = None,
                mixSongId = None,
                year = Some(1999)),
    compilationMap = Map(
        KeyRef(11x) -> Compilation(
                title = Compo,
                songRefs = Sequence(KeyRef(6t)),
                marks = Set(),
                year = Some(1999),
                cdOrSide = Some(B)),
        KeyRef(10g) -> Compilation(
                title = Compo,
                songRefs = Sequence(KeyRef(3w), KeyRef(4p), KeyRef(5c)),
                marks = Set(),
                year = Some(1999),
                cdOrSide = Some(A))))

Saturday, August 18, 2012

Scala 2.10: annotated fields macro

Here is a short example of how one can leverage SIP-16 introduced in Scala-2.10.
(The source code you will find below is expected to be compiled on Scala 2.10-M7. Note, that -M6 provides a slightly different set of API for macro.)

Lets define a macro that makes it possible to traverse value fields of a (case) class. First, we import what we will use:
import language.experimental.macros
import scala.reflect.macros.Context
import scala.annotation.Annotation
The macros, we are implementing, will be located in 'annotated' object since Scala allows usage of type aliases inside object (unlike package namespace).
object annotated {
Any field belongs to a class (denoted as I). An annotated field might have useful information given by arguments on annotation. Type of the annotation arguments is denoted as A. Here is the definition of the Field class:
     * An object of this class represents an annotated field.
     * @tparam I type of class the field belongs to
     * @tparam A type of annotation arguments (TupleX or None)
     * @param name name of the field
     * @param get function that returns field value of an instance given as argument to the function
     * @param args list of arguments to the annotation found on the field
    case class Field[I <: AnyRef, A <: Product](name : String, get : I => Any, args : A)
Here is the type alias to save some typing:
     * List of fields belonging to the given type.
     * @tparam I Owner of fields
     * @tparam A type of annotation arguments (TupleX or None)
    type Fields[I <: AnyRef, A <: Product] = List[Field[I, A]]
That is how our macro is supposed to be seen by developers (i.e. it is supposed to be seen as an ordinary function):
     * Macro which inspects class 'I' and returns a list of fields annotated with annotation 'Ann'.
     * @tparam Ann search for field with this annotation
     * @tparam Args type of arguments in the annotation (TupleX or None)
     * @tparam I type of class to scan for annotated fields
    def fields[Ann <: Annotation, Args <: Product, I <: AnyRef] = macro fieldsImpl[Ann, Args, I]
Finally, here is the implementation of the macro itself. The implementation is called by the scala compiler whenever it sees 'fields' macro:
     * Implementation of the fields macro.
    def fieldsImpl[AnnTT <: Annotation : c.AbsTypeTag,
                   Args <: Product : c.AbsTypeTag,
                   ITT <: AnyRef : c.AbsTypeTag](c : Context) : c.Expr[Fields[ITT, Args]] = {
("Args <: Product : c.AbsTypeTag" means "type Args is a subtype of type Product and there is an implicit value of type c.AbsTypeTag[Args]")
Note that here and further below we use types (like AbsTypeTag) which are from the context 'c'. That is a compilation context of the application which the scala compiler will construct for the source code where the macro invocation is faced (not exactly but..).

Now lets import types and values (like Select, Ident, newTermName) from the universe of the application the macro is currently used in:
        import c.universe._
Lets materialize some types as objects for further manipulation:
        val instanceT = implicitly[c.AbsTypeTag[ITT]].tpe
        val annT = implicitly[c.AbsTypeTag[AnnTT]].tpe
Now, some real action: get annotated fields. Note that hasAnnotation doesn't work for a reason I don't know...
        val annSymbol = annT.typeSymbol
        val fields = instanceT.members filter (member => member.getAnnotations.exists(_.atp == annT))
It is convenient to have a helper function. This function will fold given expression sequence into a new expression that creates List of expressions at runtime ;-)
        def foldIntoListExpr[T : c.AbsTypeTag](exprs : Iterable[c.Expr[T]]) : c.Expr[List[T]] =
            exprs.foldLeft(reify { Nil : List[T] }) {
                (accumExpr, expr) =>
                    reify { expr.splice :: accumExpr.splice }
For each field, construct expression that will instantiate Field object at runtime:
        val fieldExprs =
            for (field <- fields) yield {
                val argTrees = field.getAnnotations.find(_.atp == annT).get.args
                val name = // Why is there a space at the end of field name?!
                val nameExpr = c literal name

                // Construct arguments list expression
                val argsExpr =
                    if (argTrees.isEmpty) {
                        c.Expr [Args] (Select(Ident(newTermName("scala")), newTermName("None")))
                    } else {
                        val tupleConstTree = Select(Select(Ident(newTermName ("scala")),
                        c.Expr [Args] (Apply (tupleConstTree, argTrees))
                // Construct expression (x : $I) => x.$name
                val getFunArgTree = ValDef(Modifiers(), newTermName("x"), TypeTree(instanceT), EmptyTree)
                val getFunBodyTree = Select(Ident(newTermName("x")), newTermName(name))
                val getFunExpr = c.Expr[ITT => Any](Function(List(getFunArgTree), getFunBodyTree))
                reify {
                    Field[ITT, Args](name = nameExpr.splice, get = getFunExpr.splice, args = argsExpr.splice)
By this moment, value fieldExprs will contain something like
List (
   reify {Field ('field1', (x => x.field1), (..)},
   reify {Field ('field2', (x => x.field2), (..)}
(where (..) are arguments of annotaiton on that field)
Now we have to lift List construction into expression and we're done!
       // Construct expression list like field1 :: field2 :: Field3 ... :: Nil

And finally lets have some fun. Lets test it in REPL! (beware that scala macros are supposed to be compiled before they are used)
scala> type FormatFun = Any => Any
defined type alias FormatFun

scala> type PrettyArgs = (Option[String], FormatFun)
defined type alias PrettyArgs

scala> class Pretty(aka : Option[String] = None, format : FormatFun = identity) extends annotation.StaticAnnotation
defined class Pretty
scala> :paste
// Entering paste mode (ctrl-D to finish)

def pp[X <: AnyRef](fields : annotated.Fields[X, PrettyArgs])(x : X) = {
    fields map {
        case annotated.Field(fieldName, get, (akaOpt, fmtFun)) =>
            val name = fieldName.replaceAll("([A-Z][a-z]+)", " $1").toLowerCase.capitalize
            val aka = akaOpt map (" (aka " + _ + ")") getOrElse ""
            val value = fmtFun(get(x))
            s"$name$aka: $value"
    } mkString "\n"
// Exiting paste mode, now interpreting.

pp: [X <: AnyRef](fields:[X,(Option[String], Any => Any)])(x: X)String

Still no macro was used. Now here it comes. First, we define case class. Next, we gather annotated fields in the definition of personPrettyFields. When you run it in REPL, it is quite important to use :paste, otherwise annotation of the case class will be lost because subtypes of StaticAnnotation are visible during compilation only (REPL calls a new scala compiler for each expression reusing binary classes compiled during previous steps). So:
scala> :paste
// Entering paste mode (ctrl-D to finish)

case class Person(
    id : Int,
    @Pretty(aka = Some("nickname")) name : String,
    @Pretty firstName : String,
    @Pretty(None, format = _.toString.toUpperCase) secondName : String,
    @Pretty(None, format = { case x : Option[_] => x getOrElse "" }) twitter : Option[String])
val personPrettyFields = annotated.fields[Pretty, PrettyArgs, Person]
// Exiting paste mode, now interpreting.

defined class Person
personPrettyFields: List[[Person,(Option[String], Any => Any)]] =
(I've aligned the output of REPL a bit..)
Lets check field names:
scala> personPrettyFields map (field =>
res0: List[String] = List(name, firstName, secondName, twitter)
Here are getters of each field:
scala> personPrettyFields map (_.get)
res1: List[Person => Any] = List(<function1>, <function1>, <function1>, <function1>)
Now, lets create a Person object:
scala> val person1 = Person(1, "akshaal", "Evgeny", "Chukreev", Some(""))
person1: Person = Person(1,akshaal,Evgeny,Chukreev,Some(
... and a value for each field of this person:
scala> personPrettyFields map (_.get (person1))
res2: List[Any] = List(akshaal, Evgeny, Chukreev, Some(
Some more objects for more fun:
scala> val person2 = Person(2, "BillGates", "Bill", "Gates", Some(""))
person2: Person = Person(2,BillGates,Bill,Gates,Some(

scala> val persons = List(person1, person2)
persons: List[Person] =

scala> val ppPerson = pp(personPrettyFields) _
ppPerson: Person => String = <function1>         
And finally:
scala> persons map ppPerson mkString "\n----------------------------\n"
res5: String =

Name (aka nickname): akshaal
First name: Evgeny
Second name: CHUKREEV
Name (aka nickname): BillGates
First name: Bill
Second name: GATES

That's all ;-) You will find complete source code along with specs2 specification (with one more example) on the gist:

No animals were killed.

No types were casted.

No reflections were used.