Memcached Tomcat Sessions and Grails App Info Plugin

The App Info Plugin for Grails is a great tool. We use it in most of our Grails deployments. One of the great features offered is the ability to view all current sessions and their details. Another tool we are using is Memcached Session Manager which allows shared sessions between our Tomcat servers, great for rolling deploys and high availability.

We found an issue while running both of these in production. Over time the number of sessions reported by App Info would grow extremely large. It turns out that having the App Info plugin track sessions for you when the sessions are shared via memcached can lead to the plugin holding onto references to sessions that have been destroyed on other severs.

So at this time we can’t use the App Info plugin session tracking with the Memcached Session Manager, since it leads to a memory leak as all those sessions have references inside the plugin and can never be garbage collected.

We can simply turn off the session tracking with the following code:

grails.plugins.appinfo.useContextListener = false

But note this config change effect will effect how a war is made so you can’t simply override this in production you will need a new war built.

Below is our memory graph before and after the change:

Graph showing memory usage before and after the config change.

More →

Ratpack Promise

Ratpack has a core class that is the center of the great asynchronous support, Promise.

Ratpack Promises are very easy to work with, there are just a few key points:

  • Only attach to a promise one time
  • If dealing with the error case it must be done before the success case
  • They are Lazy
  • In Groovy we depend on Implicit Closure Coercion to change our closures to an Action.

Happy Path

Consuming Value from Promise

Promise promise = somethingThatReturnsPromise()

promise.then {
  println it
}

What we are doing here is giving a closure to the promise that once the value is ready the closure will be called with the value passed in as a parameter. We can also be very explicit in what we are getting back from the promise.

Explicit Value from Promise

def p = httpClient.get {
  it.url.set(new URI("http://example.com"))
}

p.then { ReceivedResponse receivedResponse ->
  println receivedResponse.statusCode
}

If some error occurs while trying to get the value for the then block the exception will be thrown. Which can be picked up by some error handler down the chain.

Error Callback

So for this works great when dealing with the happy path and wanting exceptions. But we also may want to deal with failures to fulfill the promise. So to do this we start with onError instead of then.

Ratpack Promise with Failure Path

httpClient.get {
    it.url.set(new URI("http://example.com"))
} onError {
    println "Something when wrong: ${it.message}"
} then {
    render "Got a ${it.statusCode} status with body of:\n\n${it.body.text}"
}

onError will pass in a throwable to the closure that you can log or do whatever work you would like in the case of a failure.

Lazy Promises

Ratpack promises won’t actually try to generate the value until the then block is called at the end of the current execution. This is done to allow for deterministic asynchronous operations.

Deterministic Promise

def doWork() {
  httpClient.get {  }.then {  }
  sleep 5000
  throw new RuntimeException("bang!")
}

What will happen in Ratpack is we will always get the exception “bang!”, because the get request will not even get started until the doWork block of execution is finished. Once finished having a then{} will trigger a background thread to start generating the value.

What not to do

You shouldn’t try to attach more than once to a Promise, as what ends up happening is two different promise instances will execute in the background and what we want is only to deal with that value once. So don’t do the following:

Don’t do this


def p = httpClient.get {
  it.url.set(new URI("http://example.com"))
}

p.onError {
  println it
}

p.then {
  println it.statusCode
}

Starting in Ratpack 0.9.9 the above code should actually throw an error.

More →

Cassandra Upsert Everything

Cassandra inserts and updates should always be modeled as upserts when possible. Using the query builder in the Java native driver there isn’t a direct upsert called out, but we can do updates instead of inserts for all cases. The update acts as an upsert and it reduces the number of queries you will need to build.

Statement upsert = QueryBuilder.update("table")
        .with(QueryBuilder.append("visits", new Date())) //Add to a CQL3 List
        .where(QueryBuilder.eq("id", "MyID"));
session.execute(upsert);

Above you can see how we model our “upsert”. If a value isn’t found for the given where clause it will insert it.

You must use all parts of a Primary Key for an updates where cluase given a CQL Table with a compound key:

create table tablex(
     pk1 varchar,
     pk2 varchar,
     colA varchar,
     PRIMARY KEY(pk1,pk2)
);

We can not do the following query:

Statement upsert = QueryBuilder.update("tablex")
                .with(QueryBuilder.set("colA", "2"))
                .where(QueryBuilder.eq("pk1", "1"));

You will get an InvalidQueryException:

com.datastax.driver.core.exceptions.InvalidQueryException: Missing mandatory PRIMARY KEY part pk2
	com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:35)
	com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:256)
	com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:172)

But the following will upsert:

Statement upsert = QueryBuilder.update("tablex")
        .with(QueryBuilder.set("colA", "2"))
        .where(QueryBuilder.eq("pk1", "1"))
        .and(QueryBuilder.eq("pk2", "2"));
More →

Failing @Grab Fixes

If you are working on a Groovy script with @Grab, you will sometimes get download failures for dependencies. Such as the following:

General error during conversion: Error grabbing Grapes -- [download failed: com.google.guava#guava;16.0!guava.jar(bundle), download failed: org.javassist#javassist;3.18.1-GA!javassist.jar(bundle)]

This issues may have nothing to do with the actual dependency but an issue in your local m2 cache. The quick answer is to just delete ~/.groovy/grapes and ~/.m2/repository. But doing this will force you to re-download dependencies.

To only delete the cache for items giving you an issue you just need to delete the correct directories in both m2 and grapes cache. So for our Guava example you would do the following:

rm -r ~/.groovy/grapes/com.google.guava
rm -r ~/.m2/repository/com/google/guava

After that you should be able to run the groovy script normally.

More →

Grails New Relic Detailed Tracing

New Relic with Grails by default will trace most web transactions through the controller but will not trace down into services. While most true work of a request belongs in services or libraries the default tracing leaves something to be desired.

This is easily fixed by adding New Relic annotations to services and libraries.

BuildConfig.groovy Changes

dependencies {
	compile 'com.newrelic.agent.java:newrelic-api:3.4.2'

}

Service Changes

import com.newrelic.api.agent.Trace

class SubscriptionService {

	@Trace
	def save(Subscription subscription) {
    //Work Here
  }

At this point your code is ready to give more detailed transactions, but the agent on the server must also be configured to accept custom tracing. The config option for this is not available from the web so you must update the newrelic.yml file. Set enable_custom_tracing to true.

  #enable_custom_tracing is used to allow @Trace on methods
  enable_custom_tracing: true

Now you will get any custom tracing added to your application as well as custom tracing from libraries.

More →