Published by cce on 2012-03-15
Back in October of last year, we introduced hierarchical headers for our HTML and TEXT output, but they were implemented by convention and not broadly available to other formatters. With the forthcoming 2.3.1 release we add support for nested records to all formatters.
Starting with the 2.3.1 release, we have completely rewritten our internal query output structure and corresponding formatters so that they provide support for nested records. Previously we represented query output as a pair of lists, one with headers and one with row tuples. At that time, most of our formatters followed this same tabular structure, with the exception of the HTML and TEXT formatters — they used an internal trick to present nested headers.
What motivates this update was a desire to have a more formalized mechanism to serialize records and support hierarchical headers. Early ruminations on this change can be read in an October 2011 blog entry, Discussion of JSON/CSV Headers.
A user now has explicit control over how their data is structured for output. A user could group fields together and assign decoration hints to individual fields.
The new /:raw formatter faithfully reproduces this output structure into JSON format so that visual tools could access this detail over HTTP.
Our output structure is a parallel hierarchy: "meta" is a nested data type declaration, while "data" has corresponding instance data. You could understand this by looking at the following trivial query:
{
"meta": {
"domain": {
"type": "list",
"item": {
"domain": {
"type": "integer"
}
}
},
"header": "1",
"syntax": "\/1"
},
"data": [
1
]
}
Since our syntax currently requires / to start every query, our top-level domain is always a list. Above, the structure is a list of integer items. In the example following, we have a list of string values instead.
{
"meta": {
"domain": {
"type": "list",
"item": {
"domain": {
"type": "text"
}
}
},
"header": "name",
"path": "school.name",
"syntax": "\/school.name",
"tag": "name"
},
"data": [
"School of Art & Design",
"School of Business",
"College of Education",
"School of Engineering",
"School of Arts and Humanities",
"School of Music & Dance",
"School of Natural Sciences",
"Public Honorariums",
"School of Continuing Studies"
]
}
The meta part has a "header" attribute that a formatter might use to display column headers. This header attribute can be changed using the :as decorator.
{
"meta": {
"domain": {
"type": "list",
"item": {
"domain": {
"type": "integer"
}
}
},
"header": "output",
"syntax": "\/1 :as output",
"tag": "output"
},
"data": [
1
]
}
In this case, the header applies to the output as a whole, not to either the list nor the integer parts. Values themselves cannot be decorated in this manner. You could see this with the text formatter:
| output |
-+--------+-
| 1 |
Nested records can now be explicitly marked using curly braces {}. For example, the next query returns a list of record having a single integer field:
Slots in a record can be decorated with presentation hints. For example, here we decorate the 1st slot in the record as "one" and the output as a whole "output":
This detail can be seen using a visual formatter:
| output |
+--------+
| one |
-+--------+-
| 1 |
Querying a table produces a list of record items.
Records can be nested, providing very nice visual grouping properties. In this next example, we have a list of record with two slots. The first slot is a nested record having "name" and "campus" strings. The second slot is the department’s "name".
This can be seen using a visual formatter as:
| school | name | |
|---|---|---|
| name | campus | |
| School of Business | south | Accounting |
| School of Arts and Humanities | old | Art History |
| School of Natural Sciences | old | Astronomy |
| School of Engineering | north | Bioengineering |
| Bursar’s Office | ||
When a table link is used, an implicit record is created. For example, this request returns all columns from the department table and the entire corresponding school record.
| school | code | name | school_code | ||
|---|---|---|---|---|---|
| code | name | campus | |||
| bus | School of Business | south | acc | Accounting | bus |
| la | School of Arts and Humanities | old | arthis | Art History | la |
| ns | School of Natural Sciences | old | astro | Astronomy | ns |
| eng | School of Engineering | north | be | Bioengineering | eng |
| bursar | Bursar’s Office | ||||
Notice in the text output how the nested school record is independently visualized. This query is syntax sugar for an equivalent expression using {*} to explicitly request all columns from a school record.
| school | code | name | school_code | ||
|---|---|---|---|---|---|
| code | name | campus | |||
| bus | School of Business | south | acc | Accounting | bus |
| la | School of Arts and Humanities | old | arthis | Art History | la |
| ns | School of Natural Sciences | old | astro | Astronomy | ns |
| eng | School of Engineering | north | be | Bioengineering | eng |
| bursar | Bursar’s Office | ||||
Sometimes it is useful to expand a linked record’s columns inline without creating a nested record. This can be done by using the regular navigation operator:
| campus | name |
|---|---|
| south | Accounting |
| old | Art History |
| old | Astronomy |
| north | Bioengineering |
| Bursar’s Office | |
The wildcard (*) expansion when used in combination with the navigation operator (.) does not create a nested record and is instead expanded inline:
| code | name | campus | code | name | school_code |
|---|---|---|---|---|---|
| bus | School of Business | south | acc | Accounting | bus |
| la | School of Arts and Humanities | old | arthis | Art History | la |
| ns | School of Natural Sciences | old | astro | Astronomy | ns |
| eng | School of Engineering | north | be | Bioengineering | eng |
| bursar | Bursar’s Office | ||||