apacheshindig

 

Architectural Overview of Shindig , an OpenSocial Reference Implementation

by Rajdeep Dua
May/08/2008

This article is based on Shindig code available in April,2008 in svn

Introduction

Social Networking has taken the internet from read only mode to a real collaborative mode intertwined with human networks. This has opened up unlimited ways in which the information can be stored, retrieved and used. OpenSocial attempts to reduce the pain for application developers to write applications from scratch for various Social networking sites.

This is a specification which envisions a write once, run anywhere mantra in the Social Networking space. Shindig is an reference implementation of OpenSocial specifications for developers to develop against container providers to use as a reference. The Shindig code base can be used as a starting point and modified for Social networking sites which want to join the OpenSocial revolution.

Shindig Components

The Goal of the Open source project is to be language neutral and cover mulitple languages. Currently Java and PHP implementations are availabe.We will be focussing on the Java based implementation as it is the most mature one. Shindig's Java version is a Java Servlet based application which can sit on top of any servlet container which complies with Servlet 2.3 specifications. By default it is shipped with an embedded jetty servlet container.

On the server side Shindig has 3 major components

  1. Persistent Data Loading Mechanism
  2. Gadget Rendering Infrastructure
  3. OpenSocial server side implementation

Figure 1 describes in more detail these components.All the three components are accessed using HTTP / HTTPs through a servlet. OpenSocial Server side components and Persisitent Data Access Layer share a common Servlet ( GadgetDataServlet ) , while Gadget Rendering components have a separate Servlet entry point ( GadgetRenderingServlet).

Figure 1 : Components of Shindig Server Side container


The container consists of 5 different servlets. We describe how each of these map to server side components listed above

Figure 2 : Class diagram of various servlets within the Shindig container

All the servlets are subclasses of HttpServlet and serve as independent entry points at runtime. This architecture adoption has to do with the evolutionary way in which shindig has evolved.

  • GadgetDataServlet -
    • Takes requests for loading social data from file into DataObjects
    • Takes requests for OpenSocial APIs
  • GadgetRenderingServlet
    • Converts Gadget Xml into HTML
  • Proxy Servlet
    • Takes requests for URL Gadgets connects to the remote URL, fetches the content converts it into JSON and sends it back.
  • JsServlet
    • Used for Loading JS libraries from external sources in URL gadgets
  • RpcServlet
    • Used to handle RPC metadata requests

Persistent Data Loading Mechanism

This layer is responsible for loading the persistent social data which includes- Person details, friends list, activities associated with a Person. In the default implementation this is stored in a XMLfile.

The GadgetDataServlet provides the entry point for this. This layer is pretty much pluggable and can be easily replaced with a more robust persistent mechanisms.

The GadgetDataServlet uses StateFileHandler to load the data for the container into memory. OpenSocialData Handler is used to populate the Open Social objects and query them. StateFileHandler happens to be one of the mechanisms used here as a sample implementation for XML based state persisitence. Actual implementation will replace this depending on the data store chosen to store social data.

Figure 3 : Class diagram of the GadgetDataServlet and its related classes used to reset and access the Open Social state




Figure 4 : Sequence diagram explaining how the state data is reloaded by the Sample container before the Gadget is rendered.


Gadget Rendering Infrastructure

Gadget Rendering infrastructure has client side and server side components. It is responsible for generation of Gadget HTML from the Gadget Xml.

Client Side Components

gadget.js --> Javacript library which calls the GadgetRenderingServlet.

Server Side Components :

  1. GadgetReenderingServlet
  2. GadgetRenderer
  3. GadgetSpec
  4. Gadget

Diagram below explains the sequence in more detail on how HTML is generated from the Gadget Xml and sent back to the browser.


Figure 5 : Collaboration flow on how the Gadget XML gets converted into HTML and by the GadgetRenderingServlet and its downstreamcomponents

OpenSocial Server Side Implementation

We will be covering this scenario in more detail using the collaboration diagram.This diagram will show the call flow from the client side to the Server Side.

Client Side Components:

There are 2 Features for OpenSocial

  1. open-social-0.7
  2. opensocial-reference

All the Java script files within these are added to the HTML Generated.

Server Side Components

All the OpenSocial calls are recieved by GadgetDataServlet and passed on to the OpenSocialDataHandler._This datahandler is called depending on the type of the request which should be one of the following :

  • FETCH_PEOPLE : Get Person data
  • FETCH_PERSON_APP_DATA : Get the data attributes for a person
  • UPDATE_PERSON_APP_DATA : update attributes of a person
  • FETCH_ACTIVITIES : Get List of activities for a Particular Person
  • CREATE_ACTIVITY : create Activity associated with a Person

We will take FETCH_PEOPLE as an example and go through the flow in the figure below. Default People service is a wrapper on XML file based Persistent store which is used in Shindig out of the box.( XML file based implementation is just for demonstration, needs to be replaced by a more robust persistent mechanism)

This service loads, parses and stores the data of Person, Friends, Activities in Memory in Maps. Depending on the nature of the call data is rerieved in memory (6 to 14 )and converted into JSON object(14) and sent back (15 ) to the client side for presentation


 


Figure 6 : Collaboration diagram tracing call to view friends from client side javascript to server side components and back

 

Replacing the default SampleContainer persistent store with a Custom Store.

The diagram below expands the discussion so far into one figure. The OpenSocial Sample Container piece is the actual runtime by defeault. It consists of 3 services, BasicPeopleService, BasicActivitiesService and BasicDataService.

 


Figure 7 : Shindig architecture components and Sample Container Implementation

 

For Custom Persistent implementation the following classes need to be replaced and imlpemented according to the storage mechanism being put in place.


Summary

Shindig's basic implementation is a good starting point for somebody interested in getting started quickly on OpenSocial in their networking websites. We explained how the container has mulitple entry points and our discussion is based on servlet technology. The rendering and social data access are independent compoents and can be replaced depending on the needs for it has to be used. Towards end of the article we talk about the pluggable nature of the data persisitent layer. The pluggability in the architecture is a big help in adapting it quickly to the exact storage model required to be plugged in.

Shindig itself is a moving target with an active community and OpenSocial's evolving spec,but is a very good starting point for container providers looking for a head start.

References

  1. http://www.opensocial.org
  2. http://code.google.com/apis/opensocial/
  3. http://incubator.apache.org/shindig/