dstore 1.1 released

We have released dstore version 1.1, which features a new set of stores for local DB storage. This feature provides the ability to store data locally in browsers, and use high-performance querying capabilities through the disparate technologies of IndexedDB and WebSQL (and localStorage), through the consistent dstore interface. Complex queries can be formulated in your application, and data can retrieved from local database storage with the same APIs and query format as is currently supported with other stores. Queries can then be automatically translated to SQL or cursor operations, depending on the underlying technology available.

This local data storage functionality was first introduced in Dojo 1.10 as part of the dojox/store, and you can see our original blog post for more information about local database stores, and specifically for more information on how to configure the LocalDB store. Briefly, we configure the local database storage by defining a database configuration object that specifies which stores will be accessed, and as well as defining the properties on the objects for each store, that will be queried. For example:

var dbConfig = {
    version: 2, // we increment this for every update
    stores: { // specify the set of stores
        product: { // the definition of a store
            // the definition of a property on the objects in the store
            name: {
                // the index preference, indicating
                // how unique these values are
                preference: 10
            },
            price: {
                preference: 5
            }
        },
...

And once we have a database configuration, we can instantiate individual stores by providing the database configuration and the name of the store. The dstore/LocalDB store is the main store that will automatically load the appropriate implementation, whether it be IndexedDB or WebSQL:

require(['dstore/LocalDB'], function(LocalDB){
    var productStore = new LocalDB({
        dbConfig: dbConfig,
        storeName: 'product'
    });

Now, we can interact with the store using the standard dstore API:

productStore.get(id).then(object){
    // get an object by id

    // we could make changes and store the object:
    productStore.put(updatedObject);
});
productStore.add(newObject); // add a new object

Implementations

The various database implementations are available in dstore/db, can be directly referenced if you do not want to use dstore/LocalDB to auto-select the implementation. These include:

• dstore/db/IndexedDB
• dstore/db/SQL
• dstore/db/LocalStorage

Querying

One of the more significant changes in the local database stores as they were migrated from dojox to dstore, is in regards to querying. With the introduction of local database stores in dstore, these stores have adopted the new dstore querying APIs. This provides an important advance, since dstore provides advanced filtering support that can be applied universally across stores. For example, we can do a basic filtering query:

var filteredCollection = store.filter({
        inStock: true
    });

And then we can fetch or iterate through the returned collection (and again, the filter will be translated to the appropriate SQL or IndexedDB index cursor traversals):

filteredCollection.forEach(function (object) {
  // for each item in stock
});

With dstore, we chain query commands to further define the query. For example, to sort on price, we could write:

var sortedCollection = filteredCollection.sort('price');

Advanced Filtering

The advanced filtering support in dstore means that we can use the dstore’s Filter class to create compound filters, using various filter operations, as the filter to be translated for use with the local database store. For example, we could query for items within a certain price range:

var filteredCollection = store.filter(
    new store.Filter().gte('price', 10).lte('price', 20));

And again, this will apply to the IndexedDB and SQL implementations the same as it does to a Memory store. And furthermore, this will utilize the indexed capabilities of the underlying stores to ensure fast, scalable access to the data.

contains Operator

Another addition to dstore 1.1 is a new filter operator, contains. This operator is particularly useful in working with IndexedDB’s multiEntry property functionality, whereby property values can be an array of data, and the contains operator can search for objects where a value needs to match one of the values in the array. For example, we would could search for products with a tag of “spring”:

var filteredCollection = store.filter(
    new store.Filter().contains('tag', 'spring'));

This can also be combined with filters inside the contains, so you can find objects that contain a match for another filter. However, this comes with a distinct caveat for local database stores. Internet Explorer’s implementation of IndexedDB does not support multiEntry property definitions, so this will not work in IE. In the future, we are considering an alternate implementation to support querying by array values in IE.

select Query

We have also added a new query method, select. The select method gives you the ability to select a certain subset of properties to be included in the objects returned from a query. This naturally maps to the SELECT columns in SQL queries, and can be useful for providing a restricted subset of data in query results, with less memory consumption. For example, we could limit results to just the name and price properties by querying:

var filteredCollection = store
    .filter(new store.Filter().lt('price', 20))
    .select(['name', 'price']);

We could also specify a single string (rather than array) as the argument to select() to return the specified property values directly as the query results (rather than inside objects):

store
    .filter(new store.Filter().lt('price', 20))
    .select('price')
    .forEach(function(price){
  // called with the price value itself, rather than an object
});

Relational Queries

In addition to directly using the select function operator, the in operator can receive a select filtered collection, including collections from other stores. This is a powerful combination, as it permits creating queries based on relationships between different stores. This behaves much like an inner SELECT query in SQL. We can query one store for a certain set of data, and use the select to return a set of primary or foreign keys/ids. That set that can be used as an argument for an in or contains operator to filter by primary or foreign keys/ids in another store. For example, let’s imagine we had a “company” store that held a set of companies that is referenced from the products, and we wanted to find all the products sold by companies with less than 30 employees. We could query by company, and than use the set of ids to query by product:

var productsFromSmallCompanies = productStore.filter(
    new productStore.Filter().in('companyId',
        companyStore
          .filter(new companyStore.Filter().lt('employees', 30))
          .select('id')));

This will filter the company store, select the ids, and then use that collection as the possible values for the in filter on companyId in the product store.

Complementary Functionality

The dstore release not only includes new functionality, but also includes a number of fixes, including more consistent and uniform fetch results and total length handling.

Again, the new filter and query operators, along with the new relational capabilities are available on all stores, not just the local database stores. And future releases will continue to expand on advanced querying, with normalization across browsers.

The combination of local database stores, a new filter operator, and a new query operator, should greatly expand the possibilities and capabilities of dstore. And with the continued refinement and fixes of this release, dstore 1.1 provides an unbeatable data foundation.

Learning more

Visit the dstore website to read documentation and tutorials for using dstore.

Or join us in our Dojo workshops offered throughout the US, Canada, and Europe, or at your location to learn more about loading data via object stores and dstore. We also provide expert JavaScript and Dojo support and development services. Contact us for a free 30 minute consultation to discuss how we can help you efficiently manage data within your application.