Data Federation with OSF Web Services

From OSF Wiki
Jump to: navigation, search

OSF Web Services are a platform-independent means for accessing and exposing structured RDF data. Their central organizing perspective is that of the dataset. These datasets contain instance records, with the structural relationships amongst the data and their attributes and concepts defined via separate ontologies (schema with accompanying vocabularies).[1] OSF Web Services has been designed explicitly to enable data federation (or "mixing") of the widest variety possible of datasets and dataset formats. This article describes that design.

Conceptualized Data Federation with OSF Web Services

There are a number of perspectives and contexts to view OSF Web Services. We'll look at both data formats and data exchange:

OSF Web Service Data Model Relationships

The basic design has two key data considerations. First, all OSF Web Service tools and Web services and schema work from the canonical RDF data model (center in right bubble). It is the hub and common denominator for allOSF Web Service installations. We are able to design and optimize generic tools and services (including converters) around this canonical framework.

Second, we assume most everything in the outside world to be non-compliant with this canonical model, with the data representations often naïve and incomplete. Converters (also known as translators or RDFizers) are an essential bridge to this external world, and need to be designed for re-use and extensibility.

Where the outside world is compliant, they conform to the OSF Web Service APIs or are themselves OSF Web Service installations. In these cases, direct data exchange and access with permission rights occurs at a dataset level (not shown).

The Naïve Part of the Spectrum

Converters are themselves bona fide Web services at the OSF Web Service level. (Only a few are presently shown in the diagram above.) While some may be one-off converters (sometimes off-the-shelf RDFizers), and often devoted to large volume external data sources, it is also helpful to emphasize one or more “standard” naïve external formats. A “standard” external format allows for a more sophisticated converter and enables specific tools to be more easily justified around the standard naïve format.

As noted above, this “standard” is often JSON or a derivative of JSON. But, just as readily, the common ‘naïve’ format could be SQL from relational databases or another format common to the community at hand. In many ways, because the emphasis of data exchange is on the ABox and instance records and assertions (and attribute extensions), the actual format and serialization is pretty much immaterial.

Emphasizing one or a few naïve external formats allows more tools and services to be cost-effectively developed for those formats. And, even though the format(s) chosen for this external standard may lack the expressiveness of RDF (and, ultimately, OWL), because the burden is principally related to data exchange, this layer can be readily optimized for the deployment at hand.

Besides import converters it is also important to have export services for the more broadly used naïve external formats. In fact, some OSF Web Service services can be devoted to data cleanup or attribute (property) or object reconciliation (including disambiguation as a possibility). In this manner, OSF Web Service installations could also improve the authority and trustworthiness of standard data in the wild.

Another common service for this naïve data is to give it unique URI identifiers and to make it Web-accessible, thus turning it into linked data.

The RDF Canonical Data Model

Such generic services are possible because the “highest common denominator” for the system is the canonical RDF model. Because it is the consistent basis for tools and services, once a converter is available and the external information schema is mapped to the internal structure, all existing tools and services are available for re-use. Moreover, this system and its datasets are now ready for sharing with other OSF Web Service instances, within the enterprise or beyond.

Thus, we begin to see a network of canonical “hubs” in a sea of heterogeneity, the interoperation of which is facilitated by a OSF Web Service framework at every network node. This design is discussed more in Distributed Networks with OSF Web Services.

Our choice to use RDF is based on the simplicity and understandability of the data model, plus the richness of languages and standards from the W3C that surround the framework.

Even here, however, the RDF basis of OSF Web Service need not be the final word. Because of a keen intent to keep all designs and ontologies used by OSF Web Service firmly grounded in description logics, it is possible for the OSF Web Service basis to be converted to other languages and frameworks such as Common Logic (CL) that can be expressed in DL.

Bringing it Back to Data Federation

Data mixing — or more preferably, data federation — has as its heart the premise of heterogeneous and distributed data sources. It implicitly acknowledges differences in syntax, semantics and serializations.

The design and architecture of OSF Web Service is similarly premised. While each of us may prefer one model or one format over others, we must interoperate in the real world. And that world, for many understandable and immutable reasons, will retain its diversity. Accepting this reality is a first step to adaptive design.

So, we control what we can control, and we adapt to what else exists. We have chosen RDF as the canonical data model that we can control and have embedded it in a Web services framework that is Web-based and scalable; in other words, a fully compliant Web-oriented architecture. And that for which we cannot control we provide work flows and converters that explicitly allow this data to be incorporated into the system. These are the conceptual data federation foundations to OSF Web Service.


  1. Adapted from M.K. Bergman, 2009. "OSF Web Service: A Framework for Data Mixing," AI3:::Adaptive Information blog, June 30, 2009.