Using the OSF Query Builder

From OSF Wiki
Jump to: navigation, search

The query builder (QB) is a generalized search utility for querying the datasets within a OSF instance. The QB has three purposes:

  1. It is a basis for learning about the structured data within a OSF instance
  2. It is a testbed for exploring the effect of attribute and value boostings on the ranking order of search results sets, and
  3. It is a means to refine queries and then generate query code that can be embedded into Web pages and templates.

General Overview and Operation

The QB consists of a top-line search box, followed below by a series of expand-and-collapse panels. Specialized actions take place within these panels. The operations of each of these panels is described below. Each panel also has a help icon, which when clicked brings up contextual help for that panel.

Upon invoking a search, a results listing appears at the bottom of the standard panels. Please make sure and review the Results Listing section since how results are presented in the query builder is one of its benefits.

The overall screenshot for the query builder is as follows:

Qb main.PNG

Search Box and Input

Any standard site query may be entered into the search box. For legal and available query syntax, see the Search Syntax page.

A search is conducted upon invoking the Search button. The resulting results set is then listed at the bottom of the page in scoring rank order. A Clear causes all settings throughout the QB to be returned to its default (start-up) settings, as if the application were reloaded. The one exception is that if a query had been entered into the top-line search box, it is kept.

Note: these same Search and Clear operations apply when the buttons are encountered on other panels.


The gear icon at far right turns inferencing on or not. By default, inferencing is on. To test the effect of inferencing, conduct the same search with inferencing on or not and select the 'type' attribute in one of the lower panels. When inferencing is on, generally the results set is much larger since matches to related (inferred) concepts are brought into the results.

Note: inferencing is presently limited to the type attribute; an imminent release will expand this to include the subject attribute.


Note: if you do not enter a query within the search entry box, a Search will bring up a results set containing the records of all active datasets.


Dataset Panel

Clicking on the Dataset panel causes it to expand. The panel then shows all datasets available on the default OSF endpoint:

Qb datasets.PNG

Each of the datasets may be included or not in the content basis used by the query builder by checking or unchecking its associated checkbox. Alternatively, all datasets may be selected or not using the global links at the top of the panel.

Each dataset shown also provides a count (in parentheses after the dataset title) for how many records it contains. If a query has already been entered into the top-line search box, these counts pertain to the number of record instances where the search query exists. If there is no query in the search box, the counts shown are for total numbers of records in each dataset.

As datasets are selected, their total number appears in the panel header. Whatever dataset selections are made in this panel remain active during the current QB session or until you actively change them. Upon next startup of the QB, all available datasets are selected as active by default.

Note: for content inspections, make sure to restrict your datasets to the public content, also excluding the underlying ontologies and vocabularies.


Advanced Search Panel

The Advanced Search panel allows quite complicated queries to be built. These extensions are added to whatever query already exists in the top-line search box. If there is no query in that search box, then the specifications in this panel apply on their own.

Like the other panels, clicking on it causes it to expand; clicking on the expanded header causes it to collapse. Also, like all other panels, contextual help is provided via the icon button at the upper right of the panel.

When first expanded, a single line of new query expansion is presented. This line, from left to right, has an operator box, followed by an attribute selection dropdown, and then a means to assign a value to that attribute in the next box. At the end of the line a [+] button allows a new line to be added for additional query expansion. If the [-] button is selected, the current line is removed. The basic aspect of this panel is as shown:

Qb advanced search.png

The operator box enables multiple query expansion lines to be combined together or nested. The available operators are AND, OR or NOT, in combination or not with parentheses if nesting is also desired. They are selected via this dropdown list:

Qb operators.png
Note: the first and last operator boxes shown are limited to the open ("(") and close (")") parentheses, respectively.


Note: an improperly specified or missing operator box will show in red when attempting to invoke a search. The operator must be corrected before the search will be accepted for processing. Here is how this error looks:


Qb bad operator.png

For a single expansion line, no operator needs to be designated in this first operator box. Thereafter, an operator must be specified to "chain together" more query expansion lines.

Once the operator is selected (or not, for a single line), it is next necessary to select one of the available attributes. These appear as a dropdown list in the next box. This attribute box features autocompletion, so beginning to type a few letters will match those same letters in the attribute titles no matter where the matching string may occur (beginning, middle or end of a word). As more letters get typed into the combobox entry, the selections get fewer.

Note: autocompletion typically starts once you place the cursor into the applicable text or combobox.


The available attributes to select from also may be reduced in number based on prior selections or a query already entered into the top-line search box. That is, the attributes available for selection at any given step are limited to those that are active for the query that has been built to that point, with the narrowing occurring each time the 'Search' button is invoked without an intervening 'Clear'.

Note: invoking a 'Search' is necessary to narrow available attribute and value options.


Thus, very complicated query constructions may result in very few attributes being available for further filtering.

The attributes themselves are of two types: There are free text entries and controlled vocabular entries, with these icons appearing in the adjacent values dropdown combobox. For free text entries, any term or string may be entered into this box; for controlled vocabulary entries, you must pick from one of the available options, which are presented to you via a dropddown list, also shown with an autocompletion icon. Like other autocompletion fields, typing more characters causes the possible entries to be narrowed to those that match the string. A good attribute to try this feature with is Subject, since there is a long list of subjects within the available ontologies.

You may construct as many query expansion lines as you would like, again with the caveat that attribute choices will continue to narrow with more filters. If you want to back up a step, you can delete the entire line.

At the bottom of the panel, a search may be invoked via the 'Search' button or all QB settings may be 'Clear' back to original settings. These buttons may be applied here, on this panel, or via the same buttons on the top-line search input.

Attributes Restrictions & Boosting Panel

The Attributes Restrictions and Boosting panel allows you to filter search results by attribute and to change boost values to affect the rankings of the results set. The attribute filters work in conjunction with any other specifications that have been made above, namely simple or advanced search.

Like the other panels, clicking on the Attributes panel causes it to expand; clicking on the expanded header causes it to collapse. Also, like all other panels, contextual help is provided via the icon button at the upper right of the panel. Here is the basic view of this panel:

Qb attributes restrictions.png

The attributes combobox works similarly to what was described for Advanced Search. Via autocompletion and the entry of a few characters, the listing of possible attributes to select from gets narrowed. Further, only those attributes that meet the dataset and query selections made elsewhere appear on this list.

Note: invoking a 'Search' is necessary to narrow available attribute and value options.


Any attribute selected in this panel MUST exist in a given record in order for that result to appear in the results list. Thus, any entry, no matter the boosting weight assigned, must appear for a positive search hit. However, also note that ANY value for that attribute will satisfy the condition and be acceptable. (For actual value filtering, see next section.)

Once a filter is selected, it is then possible to indicate its relative weight in the results set ranking order. You do this by changing the value at the right of the entry line.

Note: by convention, all initial boost weights are set to 1.0. Thus, a value greater than 1.0 means the attribute is to be ranked higher; a value less than 1.0 means the attribute is to be ranked lower than standard.


Note: the actual algorithm to construct a "score" for a given result can be quite complicated in the search function. As a result, it is often hard to see a commensurate change in score via the boost, though a numeric impact will occur and rank order will most likely change.


The plus (+) button allows you to enter another filter or weighting boost. You may add as many of these filters as you wish.

Note: to test the effect of boost differences, first run a search without boost and inspect the resulting search scores as described under Results Listing below. Then, run again with the boosts and inspect score and ranking differences.


At the bottom of the panel, a search may be invoked via the 'Search' button or all QB settings may be 'Clear' back to original settings. These buttons may be applied here, on this panel, or via the same buttons on the top-line search input.

Values Boosting Panel

The Values Boosting panel allows you to boost values to affect the rankings of the results set. It will not act to do any filtering whatsoever; for that purpose, see the Attributes panel above. The values boosts work in conjunction with any other specifications that have been made above, namely simple or advanced search or attribute boosting or filtering.

Like the other panels, clicking on the Values Boosting panel causes it to expand; clicking on the expanded header causes it to collapse. Also, like all other panels, contextual help is provided via the icon button at the upper right of the panel. Here is the basic view of this panel:

Qb values boosting.png

The first combobox presented enables you to select which attribute you want to select for boosting values. The attributes themselves are of two types: There are free text entries and controlled vocabulary entries, with these icons appearing in the adjacent values dropdown combobox. For free text entries, any term or string may be entered into this box; for controlled vocabulary entries, you must pick from one of the available options, which are presented to you via a dropddown list, also shown with an autocompletion icon. Like other autocompletion fields, typing more characters causes the possible entries to be narrowed to those that match the string. A good attribute to try this feature with is Subject, since there is a long list of subjects within the available ontologies.

Once an attribute value is selected, it is then possible to indicate its relative weight in the results set ranking order. You do this by changing the value at the right of the entry line.

Note: by convention, all initial boost weights are set to 1.0. Thus, a value greater than 1.0 means the attribute is to be ranked higher; a value less than 1.0 means the attribute is to be ranked lower than standard.


Note: the actual algorithm to construct a "score" for a given result can be quite complicated in the search function. As a result, it is often hard to see a commensurate change in score via the boost, though a numeric impact will occur and rank order will most likely change.


The plus (+) button allows you to enter another filter or weighting boost. You may add as many of these filters as you wish.

For a given document or result to receive this scoring boost, it must both have the applicable attribute AND the specific value you specified. Without both of those conditions being met, the record receives no scoring boost and its overall score is unaffected. (However, its absolute rank order might be lowered if other records or documents receive the boost causing them to appear higher in the results listing.)

Note: to test the effect of boost differences, first run a search without boost and inspect the resulting search scores as described under Results Listing below. Then, run again with the boosts and inspect score and ranking differences.


At the bottom of the panel, a search may be invoked via the 'Search' button or all QB settings may be 'Clear' back to original settings. These buttons may be applied here, on this panel, or via the same buttons on the top-line search input.

Search Query Code Panel

The Search Query Code panel presents the internal query specification resulting from any of the prior search, dataset selection, filter selection or boosting factors. The code that is presented here may be copied by developers for Web page or layout template use. The code in this panel is also a good way to understand the various components in your queries.

Here is an example screen showing one of these code samples:

Qb search code.PNG

There are two sections available:

  1. OSF-WS-PHP-API Code
    1. This is the OSF-WS-PHP-API code that can be used to query the OSF instance using the Search web service endpoint
  2. Query's JSON Serialized Object
    1. This is an intermediary JSON object representation of the query. This is what is used to create the Search Profiles.

Search Profile

This section let you save the query you created as a OSF for Drupal Search Profile. The only thing you have to do is to define a name for that new search profile, and to click the Save Search Profile button. Once you saved a search profile, it will appears in the list of available search profiles (see below).

Qb search profile.PNG

Results Listing

The results listing appears at the bottom of the expand/collapse panels discussed above. The results listing is one of the major benefits of the query builder in that it provides the full structural detail for each result and also displays its internal search score, which is shown at the upper right of each individual result record.

Note: in an actual results presentation, these structured values may be included or not, snippets might be provided, titles could be made active links, images or maps could be displayed depending on the type of data, etc. The QB merely shows the component building blocks presently available in the datasets for such displays.


The results listing itself begins with a summary of the count for the results set produced by the various filters and query specifications in the query builder. These results are then paginated, in groups of ten, across potentially multiple results pages.

Note: when looking at scoring impacts from boosts, it is often helpful to compare the first and last pages of paginated results.


The basic appearance of the results listing, showing an example first result, appears as follows:

Qb search results.PNG

The results section begins with the standard help icon and the paginator, as well as showing the total number of results for the current query and datasets. Then, each results listing follows a similar display format.

The first line of every result shows the record's title (prefLabel), what type of record it is as shown in italicized parentheses, and its search score at far right. Then, all structured data and metadata are presented below, with the attribute name shown in bold followed by its value. Note that if you mouseover any bolded property label (such as prefLabel) you will see a tooltip showing you the actual URI value for that property.

Most values are string text, but some are URLs, in which case a live link is presented. Some of the attributes or properties, too, also have records within one of the governing ontologies. Subject is a useful property to inspect in this manner. These are shown as a live link as well; clicking on them will take you to a structured data listing.