Vertx3 and Gradle Application Plugin

Vertx 3 is a great step forward you can now work with Vertx without having to install a command line tool to run it. Now you can use standard Maven or Gradle to run your application.

With Gradle the samples given all are based on the use of the great Shadow Plugin. While this is good for deployments and most local development I missed a few of the features of the application plugin.

Set up your build.gradle with the following:

//...

apply plugin: 'java'
apply plugin: 'application'
apply plugin: 'com.github.johnrengelman.shadow'

//...

dependencies {
	compile "io.vertx:vertx-core:${project.vertxVersion}"
	compile "io.vertx:vertx-apex:${project.vertxVersion}"
	compile "io.vertx:vertx-auth-service:${project.vertxVersion}"
}

def mainVerticle = "groovy:ex.main.RestVerticle"

mainClassName = "io.vertx.core.Starter"
run {
	args = ["run", mainVerticle]
}

jar {
	manifest {
		attributes 'Main-Verticle': mainVerticle
	}
}

shadowJar {
	classifier = 'fat'

	mergeServiceFiles {
		include 'META-INF/services/io.vertx.core.spi.VerticleFactory'
	}

	dependencies {
		exclude(dependency('io.vertx:codegen'))
		exclude(dependency('junit:junit'))
		exclude(dependency('org.mvel:mvel2'))
		exclude(dependency('log4j:log4j'))
	}
}

In this case we are deploying Vertx with verticles. So we set the main verticle which does our setup. I define that as variable since we will need to use it in two places once as an argument to the Vertx starter and again in the Jar manifest. You will note the main verticle includes the type in our case it is Groovy but if you are doing JavaScript you would change it there.

Now we can take advantage of the application plugin allowing us to use the simple gradle run instead of depending on the Shadow plugin it is also faster then rebuilding the whole fat jar.

But the main reason I wanted to use the application plugin was the ability to debug easily.

$ gradle run --debug-jvm

That command will allow us to attach a debugger to 5005 default. With these changes Vertx 3 development fits in along side my other Grails 3 and Java 8 projects easily.

More →

Grails 3 and Spring Java Config

The Spring Java config is a great way to work with Spring configuration. It feels similar to Google Guice, which I personally enjoy. Mr. Haki did a great write up on using it with Grails 2.4.

While working with Grails 3 there are a few areas that don’t have plugins yet but do have Spring modules we can leverage. So I started out with the same path given for Grails 2.X.

application.yml

grails:
    profile: web
    codegen:
        defaultPackage: example
    spring:
        bean:
            packages:
                - example.config

I found that while the beans were all wired correctly and @Autowired all worked correctly, but any mappings with @RequestMapping where always returning a 404. Everything looked wired correctly but Grails wasn’t letting any requests through.

So I switched over to using @ComponentScan then the requests were properly mapped and everything started working.

Using ComponentScan with Grails 3 is very easy:

Application.groovy

@ComponentScan("example.config")
class Application extends GrailsAutoConfiguration {
  //...
}

This will scan the package of example.config for any Spring components so your Spring Java configs can all live in this package and easily get picked up.

This post is the blog form of this SO Question.

More →

SpringSecurity Impersonate Users Custom Roles

The Grails SpringSecurity plugin has the ability to allow user impersonation which is a really great tool for support. But many times it makes sense to allow your support user to see different things than a user. We use a special role to achieve this behavior we can also have different roles for tiers of support.

By default even impersonation or user switching is turned off in the plugin by default. You can easily turn it on with a config flag:

grails.plugin.springsecurity.useSwitchUserFilter = true

Spring Security by default will add a new role of ROLE_PREVIOUS_ADMINISTRATOR for the user while impersonating the other user. This will work in most cases but for us it was hard to reason about since we had an idea of admin which was separate from our support team who would mostly be using this tool.

So for our use we wanted our own roles. The ability to do this is exposed in the SwitchUserFilter calling SwitchUserAuthorityChanger. With an implementation of the interface you can add any roles you would like.

Example authority changer:

class RoleImpersonatedSwitchUserAuthorityChanger implements SwitchUserAuthorityChanger {

	@Override
	public Collection<GrantedAuthority> modifyGrantedAuthorities(UserDetails targetUser, Authentication currentAuthentication, Collection<GrantedAuthority> authoritiesToBeGranted) {
		SwitchUserGrantedAuthority roleImpersonatedSwitchAuthority = new SwitchUserGrantedAuthority("ROLE_IMPERSONATED_USER", currentAuthentication);

		Collection<? extends GrantedAuthority> augmentedAuthoritiesToBeGranted = authoritiesToBeGranted + roleImpersonatedSwitchAuthority

		return augmentedAuthoritiesToBeGranted;
	}
}

In Grails we set our authority changer during bootstrap.

BootStrap.groovy


class BootStrap {

	def switchUserProcessingFilter
	def switchUserAuthorityChanger

	def rabbitTemplate

	def init = { servletContext ->  
    switchUserProcessingFilter.setSwitchUserAuthorityChanger(switchUserAuthorityChanger)

    //....
  }

The switch user filter is already exposed as a bean so we can just get that injected and we also exposed the authority changer as a bean so we could just inject it in bootstrap.

There are many other options to think about when allowing impersonation of users, it may even make sense to remove some roles from an impersonated user. That is easily done by simply removing roles from the list retuned from the authority changer.

More →

Cassandra Stale Snapshots

While working with Cassandra we found that sometimes snapshots get left around after repairs fail or have issues. These can cause 100’s of Gigs of space to just be wasted. This can add up quickly and cause issues. On a node with low disk space we can check for stale snapshots the following way.

$ cd /cassandaData/cassandra/data/keyspace/
$ du -h

If you see large or multiple directories under columnFamily/snapshots/ that may indicate there are stale snapshots that can be cleaned up. We will want to check when these snapshots where created so inside the snapshots directory of the column family run

$ ls -lah
total 3.3M
drwxr-xr-x 7 cassandra cassandra 4.0K Mar 11 16:14 .
drwxr-xr-x 3 cassandra cassandra 704K Mar 11 16:31 ..
drwxr-xr-x 2 cassandra cassandra 508K Mar 01 14:24 4f37dcd0-c7fa-11e4-b5ae-5f969a9b23c8
drwxr-xr-x 2 cassandra cassandra 504K Mar 11 16:14 ad70a8e0-c809-11e4-9b55-39152d07d3bf
drwxr-xr-x 2 cassandra cassandra 532K Mar 11 16:03 b5cfd3a0-c807-11e4-bcd5-db76d671c3d5
drwxr-xr-x 2 cassandra cassandra 504K Mar 10 22:50 cfe34050-c777-11e4-b5ae-5f969a9b23c8
drwxr-xr-x 2 cassandra cassandra 536K Mar 11 16:09 edc03dd0-c808-11e4-be46-35521ca10087

Each of those are snapshots but you will notice 4f37dcd0-c7fa-11e4-b5ae-5f969a9b23c8 is stale, we can now clean that up with the nodetool clearsnapshot command.

nodetool clearsnapshot keyspace -t 4f37dcd0-c7fa-11e4-b5ae-5f969a9b23c8

That will take care of removing the stale snapshot you should see disk space recovered immediately.

You can also look in the logs for failed repairs we see something like the following:

2015-02-20 23:37:52,220 RepairSession.java (line 288) [repair #72e69720-b959-11e4-9b55-39152d07d3bf] session completed with the following error
system.log.10-java.io.IOException: Failed during snapshot creation.
system.log.10- at org.apache.cassandra.repair.RepairSession.failedSnapshot(RepairSession.java:323)
system.log.10- at org.apache.cassandra.repair.RepairJob$2.onFailure(RepairJob.java:126)
system.log.10- at com.google.common.util.concurrent.Futures$4.run(Futures.java:1160)
system.log.10- at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
system.log.10- at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
system.log.10- at java.lang.Thread.run(Thread.java:745)

In that case the bad snapshot would be 72e69720-b959-11e4-9b55-39152d07d3bf.

More →

Grails Integration Testing of Complex Transactions

Integration testing complex transactions in grails can be tricky due to the default behavior of wrapping integration tests in transactions and rolling them back when complete.

The simple solution is to simply turn off transactions for integration tests, that solution will work but tends to lead to data pollution in downstream tests. Burt Beckwith has a solution in his post An Alternative Approach for Grails Integration Tests. Using this solution we can rebuild the database for each test.

In general most of the time you can use the default transactional testing behavior, therefore we want to only use this method some of the time. To achieve this effect we will modify Burt’s original solution slightly.

  • Rebuild the database after each test
  • Depend on the configured data source
  • Update to Spock

Updated Solution

NonTransactionalIntegrationSpec.groovy

import org.codehaus.groovy.grails.orm.hibernate.cfg.DefaultGrailsDomainConfiguration
import org.hibernate.SessionFactory
import org.hibernate.cfg.Configuration
import org.hibernate.tool.hbm2ddl.SchemaExport

class NonTransactionalIntegrationSpec extends company.IntegrationSpec {

    @Shared
    private static Configuration _configuration

    @Shared
    def grailsApplication

    static transactional = false

    def setupSpec() {
        if (!_configuration) {
            // 1-time creation of the configuration
            Properties properties = new Properties()
            properties.setProperty 'hibernate.connection.driver_class', grailsApplication.config.dataSource.driverClassName
            properties.setProperty 'hibernate.connection.username', grailsApplication.config.dataSource.username
            properties.setProperty 'hibernate.connection.password', grailsApplication.config.dataSource.password
            properties.setProperty 'hibernate.connection.url', grailsApplication.config.dataSource.url
            properties.setProperty 'hibernate.dialect', 'org.hibernate.dialect.H2Dialect'

            _configuration = new DefaultGrailsDomainConfiguration(grailsApplication: grailsApplication, properties: properties)
        }
    }

    def cleanupSpec() {
      //After spec nuke and pave the test db
      new SchemaExport(_configuration).create(false, true)

      //Clear the sessions
      SessionFactory sf = grailsApplication.getMainContext().getBean('sessionFactory')
      sf.getCurrentSession().clear()
    }

}

You’ll notice we extend a company.IntegrationSpec there isn’t anything special in that just a base integration spec class that can hold our shared testing code such as bootstrapping methods and security methods. Since our data bootstrap logic is set up there we can share the same initial setup across both our normal integration test and our new non-transactional integration specs.

When to Use

The main time we started to need this new testing method was when doing custom propagation of hibernate transactions.

  @Transactional(readOnly = true, propagation = Propagation.REQUIRES_NEW)
  def getSomething(){
    //Do some read only operation.
  }

Since this will require a new transaction we will not be able to see the inserts that have happened in the default transaction. We do many of our reads as read only and force a new transaction when we are more read heavy during an operation.

In general anytime you are working directly with transaction rollbacks or propagation it best to test those things directly without having Grails inject an extra transaction in there for you.

More →