Extending the Neo4j Server with Spring Data Neo4j

On my previous blog post I described how we could use the Neo4j Spatial Plugin to implement a museums search engine.

As a Spring developer, I was really interested in studying Spring Data Neo4j 3.0.1 too. My original intent was to deploy a small Spring MVC based application on a Heroku instance for displaying spatial query results on a Open Street Map on MapQuest.

Queries should have been executed on a remote GrapheneDB instance, but as soon as I switched my configuration from embedded to server mode, I noticed a performance slowdown.

Micheal Hunger, in the official community discussion group for Neo4j, explained me current SDN 3.0.1 “is designed to run against an embedded database, and it is too chatty over the wire” and he suggested me to “create an unmanaged extension using SDN and then deploy it to your server“. That way I would have been able “to combine SDN with SD-REST to provide that extension in a nice, domain level way“.

That sounded really cool and this is what I’m going to talk about on this post.

la-creazione-di-adamo

1. Extending Neo4j Server with SDN

Official documentation about Neo4j Server Extension is available here. We can read “unmanaged extensions are a way of deploying arbitrary JAX-RS code into the Neo4j server“.

Extension can be implemented with Spring Data Neo4j as well: a tiny section on the spring docs is reacheable here.

It quickly became clear, I could have refactor my original code just by:

  1. moving persistence tier – actually built by SDN Repositories – into the graph database, as an extension component (full implementation available here);
  2. refactoring the webapp adding some REST clients to call my server extension API (full implementation available here).

 2. Coding the Unmanaged Extension

Unmanaged extensions essentially implement additional REST API we want to expose in our database server.

They actually are “Jersey resource implementations” and have to be packaged as JAR files we will add to the Neo4j Server’s plugin directory.

2.1 Extension Initialization

Extensions implemented with Spring Data Neo4j need to initialized by telling the server where to find the Spring Context configuration file, and which beans from it to expose:

public class Neo4ArtPluginInitializer extends SpringPluginInitializer
{
  private static final Logger logger = Logger.getLogger(Neo4ArtPluginInitializer.class.getName());

  @SuppressWarnings("unchecked")
  public Neo4ArtPluginInitializer()
  {
    super(new String[] { "META-INF/spring/application-context.xml" },
          expose("neo4jTemplate", Neo4jTemplate.class),
          expose("museumEntityService", MuseumService.class),
          expose("museumRepository", MuseumRepository.class),
          expose("artworkRepository", ArtworkRepository.class));

    logger.info("Spring context configured.");
  }

  // code omitted...
}

This configuration will provide 4 beans as resources injectable where needed – through the @Context annotations:

2.2 Extension Configuration

Neo4j Server Extensions are picked up by the server at startup by providing the necessary /META-INF/services/org.neo4j.server.plugins.ServerPlugin file for Java’s ServiceLoader facility. That’s the content on my own one:

 it.inserpio.neo4art.server.config.Neo4ArtPluginInitializer

Neo4j Server Extensions then need to be configured by adding this line into neo4j-server.properties:

org.neo4j.server.thirdparty_jaxrs_classes=it.inserpio.neo4art=/neo4art

It essentially declares the context through which we are exposing our server extension methods: neo4art server extensions API will be reachable as URI like:

http://{neo4j_server}:{neo4j_port}/neo4art/{extension_path}

2.3 Extension Implementation

As I mentioned, the purpose of this spike was essentially to implement Server Extensions with Spring and indeed I was able to use original services and repositories as they were.

There’s only one thing I’d like to put in evidence on this section: looking at the Spring Data Neo4j configuration file it seems I forgot to define the GraphDatabaseService bean; In reality, the GraphDatabaseService resource is provided, hence injectable where needed, by the graph database server.

2.4 Exposing Extension API as Jersey Resource

There were two main REST services I was really interested to expose:

  • Get Museums Within Distance:
http://{neo4j_server}:{neo4j_port}/neo4art/museums/lon/-0.1283/lat/51.5086/distanceInKm/10.0
  • Get Artworks By Museums:
http://{neo4j_server}:{neo4j_port}/neo4art/artworks/museum/1051

In order to specify the mount point for the first one, it was sufficient to implement this class:

@Controller
@Path("/museums")
public class MuseumRestController
{
  private static final Logger logger = Logger.getLogger(MuseumRestController.class.getName());
 
  @Context
  private MuseumService museumService;
 
  @GET
  @Path("/lon/{lon}/lat/{lat}/distanceInKm/{distanceInKm}")
  @Produces({ MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML })
  @Transactional
  public Response getMuseumsWithinDistance(@PathParam("lon") double longitude,
                                           @PathParam("lat") double latitude,
                                           @PathParam("distanceInKm") double distanceInKm)
  {
    logger.info(String.format("GET /museums/lon/%s/lat/%s/distanceInKm/%s", longitude, latitude, distanceInKm));
 
    List<Museum> museumList = this.museumService.getMuseumsWithinDistance(longitude, latitude, distanceInKm);
 
    logger.info("Number of museums found: " + ((museumList != null) ? museumList.size() : "null"));
 
    GenericEntity<List<Museum>> entity = new GenericEntity<List<Museum>>(museumList) {};
 
    return Response.ok(entity).build();
  }
}

Through the @Context annotation I was able to inject the MuseumService instance managed by the Spring Context (one of my 4 previously exposed bean, as declared in the Neo4ArtPluginInitializer).

Please notice to correctly marshall the retrieved list of Museum it was necessary to implement this generic JSON message body writer:

@Provider
@Produces("application/json")
public class JsonMessageBodyWriter implements MessageBodyWriter
{
  // code omitted...

  @Override
  public void writeTo(Object target, Class type, Type genericType,
                      Annotation[] annotations, MediaType mediaType,
                      MultivaluedMap httpHeaders, OutputStream outputStream) throws IOException
  { 
    new ObjectMapper().writeValue(outputStream, target);
  }
}

3. Calling Server Extension REST API

Once deployed into the Neo4j Server my Neo4Art extension, it was sufficent to implement a simple call though the Spring RestTemplate class:

@Service
public class MuseumRestClientService implements MuseumService
{
  @Value("${database_url}")
  private String databaseURL;
 
  @Value("${neo4art_extension}")
  private String neo4artExtension;
 
  /* (non-Javadoc)
   * @see it.inserpio.neo4art.service.MuseumService#getMuseumsWithinDistance(double, double, double)
   */
  @Override
  @Transactional
  @SuppressWarnings("unchecked")
  public List<Museum> getMuseumsWithinDistance(double longitude, double latitude, double distanceInKm)
  {
    RestTemplate restTemplate = new RestTemplate();
 
    String url = String.format(this.databaseURL + "/" + this.neo4artExtension + "/museums/lon/%s/lat/%s/distanceInKm/%s",
                               longitude, latitude, distanceInKm);
 
    java.util.List<Museum> response = restTemplate.getForObject(url, java.util.List.class);
 
    return response;
  }
}

4. DEPLOYMENT

Accessing Neo4j deployed on a GrapheneDB instance requires autentication. For that reason I added this RestTemplate factory:

@Component
public class Neo4jRestClientFactory
{
  privatestatic String databaseUser;

  privatestatic String databasePassword;

  @Value("${database_usr}")
  public void setDatabaseUser(String databaseUser)
  {
    Neo4jRestClientFactory.databaseUser = databaseUser;
  }

  @Value("${database_pwd}")
  public void setDatabasePassword(String databasePassword)
  {
    Neo4jRestClientFactory.databasePassword = databasePassword;
  }

  public static RestTemplate getInstance()
  {
    CredentialsProvider credsProvider = new BasicCredentialsProvider();

    credsProvider.setCredentials(
            new AuthScope(null, -1),
            new UsernamePasswordCredentials(databaseUser, databasePassword));

    HttpClient httpClient = HttpClients.custom().setDefaultCredentialsProvider(credsProvider).build();

    RestTemplate restTemplate = new RestTemplate();

    restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory(httpClient));

    return restTemplate;
  }
}

Deploying on Heroku requires a Procfile too. This is the content of my own one:

web:    java $JAVA_OPTS -jar target/dependency/webapp-runner.jar --port $PORT target/*.war

 

Here you can check the “Neo4Art Museum Search Engine” deployed atop Heroku + GrapheneDB stack:

http://neo4art-sdn-server-ext.herokuapp.com/artworksRadar

5. REFERENCE

This spike would not have been possible without this work by Bradley Nussbaum. Thanks! 🙂

Thanks to Micheal Hunger and GrapheneDB too!

 

Advertisements

2 thoughts on “Extending the Neo4j Server with Spring Data Neo4j

  1. Pingback: Neo4Art moves to Cloud Foundry at SpringOne 2GX 2014 | Lorenzo Speranzoni's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s