OSF Web Service Architecture

From OSF Wiki
Jump to: navigation, search


OSF Web Service is based on what is known as a Web-oriented architecture (WOA) [1], which can be defined as:

WOA = SOA + WWW + REST Concept

WOA is an architectural foundation of the Web, and is a subset of the service-oriented architectural (SOA) style, wherein discrete functions are packaged into modular and shareable elements ("services") that are made available in a distributed and loosely coupled manner. WOA uses the representational state transfer (REST) architectural style defined by Roy Fielding in his 2000 doctoral thesis; Fielding is also one of the principal authors of the Hypertext Transfer Protocol (HTTP) specification.

REST provides principles for how resources are defined and used and addressed with simple interfaces without additional messaging layers such as SOAP or RPC. The principles are couched within the framework of a generalized architectural style and are not limited to the Web, though they are a foundation to it.

REST and WOA stand in contrast to earlier Web service styles that are often known by the WS-* acronym (such as WSDL, etc.). WOA has proven itself to be highly scalable and robust for decentralized users since all messages and interactions are self-contained (convey "state").

OSF Web Service abstracts its WOA services into simple and compound ones (which are combinations of the simple). All Web services (WS) have uniform interfaces and conventions and share the error codes and standard functions of HTTP. We further extend the WOA definition and scope to include linked data, which is also RESTful. Thus, our WOA also sits atop an RDF (Resource Description Framework) database ("triple store") and full-text search engine.

These Web services then become the middleware interaction layer for general access and querying ("endpoints") and for tying in external software ("clients"), portals or content management systems (CMS). This design provides maximum flexibility, extensibility and substitutability of components.

As you can notice in the schema below, the general architecture of this system is split in four main areas:

  1. The OSF Web Services Framework
  2. CMS interacting with the OSF Web Service
  3. External Users interacting with the OSF Web Service
  4. External datastore systems used by the OSF Web Service.
OSF Web Service Architecture

The Web Services Framework Middleware

The core to the system is the Web services middleware layer, or OSF Web Service. This Web services framework (WSF) is the abstraction layer that provides the Web service endpoints and services for external use. It also provides the direct hooks into the underlying RDF triple stores and full-text search engines that drive these services. At initial release, these pre-configured hooks are to the Virtuoso RDF triple store (via ODBC, and later HTTP) and the Solr faceted text search engine (via HTTP). However, the design also allows other systems to be substituted if desired or for other specialized systems to be included (such as an analysis or advanced inference engine).

Authentication/Registration WS

All the web service access is controlled by an API Key shared between the OSF endpoints and the OSF users. This manage the access to dataset contents and refuse unauthorized accesses.

Other Core Web Services

The other core Web services provided with OSF Web Service are the CRUD functional services (create - read - update - delete) and search. These are viewed as core services for any structured dataset. A full list of the available web service endpoints is available here.

All the web service endpoints support the following publishing formats: RDF/XML, RDF/N3, RDF/Turtle, structXML, structJSON and commON.

Datasets, Access and Use Rights

A simple but elegant system guides access and use rights. First, every Web service is characterized as to whether it supports one or more of the CRUD actions. Second, each user is characterized as to whether they first have access rights to a dataset and, if they do, which of the CRUD permissions they have. We can thus characterize the access and use protocol simply as A + CRUD.

Thereafter, a simple mapping of dataset access and CRUD rights determines whether a user sees a given dataset and what Web services ("tools") are presented to them and how they might manipulate that data. When expressed in standard user interfaces this leads to a simple contextual display of datasets and tools.

A Structured Data Foundation

Fundamentally, this "data-driven application" works because of its structured data foundation. OSF Web Service employs an innovative design that exposes all RDF and record aspects to full-text search and is able to present contextual ("faceted") results at all times in the interface [2]. In addition, the Virtuoso universal server provides RDF triple store and related structured data services.

The actual "driver" for the structured data system is one or more schema ("ontologies") setting all of these structured data conditions. These ontologies are also managed by the triple store. The definition of these ontologies is specified in such a way with accompanying documentation to enable new scopes or domains to easily drive the system.

Interactions with CMSs and External Clients

As so far described by the diagram above, all interactions with the system have been mediated either by Web service APIs or external endpoints, such as SPARQL.

For external clients or any HTTP-accessible system, this is sufficient. Programmatically, external clients (software) may readily interact with each OSF Web Service and obtain results via parametric requests.

However, the framework is also designed to be embedded within existing content management systems (CMSs). For this purpose, an additional layer is provided.

The architecture of the system can support interactions with standard open source CMSs or app frameworks such as Django, Drupal, Joomla!, Liferay, Ruby on Rails, or WordPress, as examples. CMS interactions first occur via specific modules or plug-ins written for that system. These are very lightweight wrappers that conform to the registry and hooks of the host CMS system. The actual modules or plug-ins provided are also geared to the management style of the governing CMS and what it offers. Each module or plug-in wrapper is a packaging decision of how to bound the OSF Web Services in a configuration appropriate to the parent CMS.

This design keeps the actual tie-ins to the CMS as a very thin wrapper layer, which can embrace an open source licensing basis consistent with the host CMS. Because all of the underlying functionality has been abstracted in the OSF Web Service framework, licensing integrity across all deployment layers is maintained while allowing broad CMS interoperability. The design also allows networks to be established of multiple portals or nodes with different CMSs, perfect for broad-scale collaboration.

In initial release, OSF Web Services has been linked to the Drupal CMS as the OSF for Drupal distribution.

Benefits of the OSF Web Services Design

This design has some key benefits:

  • Broad suitability to a diversity of deployment architectures, operating systems and scales
  • Substitutability of underlying triple stores and text engines
  • Substitutability of preferred content management systems
  • Access and use of Web service endpoints independent of CMS (external clients)
  • Performant Web-oriented architecture (WOA) design
  • Common, RESTful interface for all Web services and functions in the framework
  • Easy registration of new Web services, inclusion with authorization system
  • Ability to share and interoperate data independent of client CMSs or portals
  • Use of the common lingua franca of RDF and its general advantages [3].


  1. See http://www.mkbergman.com/?cat=153
  2. Frédérick Giasson, 2009. RDF Aggregates and Full Text Search on Steroids with Solr, April 29, 2009. See http://fgiasson.com/blog/index.php/2009/04/29/rdf-aggregates-and-full-text-search-on-steroids-with-solr/
  3. Michael K. Bergman, 2009. Advantages and Myths of RDF, white paper from Structured Dynamics LLC, April 22, 2009, 13 pp. See http://www.mkbergman.com/wp-content/themes/ai3/files/2009Posts/Advantages_Myths_RDF_090422.pdf