Working with IndexedDB in HTML5


Reading time: 50 minutes

IndexedDB is a JavaScript-based object-oriented database which lets you store just about anything in the user's browser.

IndexedDB is a low-level approach (or an API) for storing data from a client side. Data can include:

  • raw text data
  • files
  • blobs (binary data)

It uses indexes to search data efficiently. DOM storage is simple and good to use to store small amount of data like login session in a website but it is not good for storing large amounts of data like:

  • an offline version of an interactive site

IndexedDB comes into play in this situation.

Key terms :

Database

The highest level in heiarchy of IndexedDB; the Database contains the data in structures called object stores.

Object stores

An object store is the mechanism by which data is stored in a database. (You can think it as an analogous to tables in SQL Databases.) Unlike tables, the actual JavaScript data types of data within the object store do not need to be consistent. (Suppose we have a store named 'cars' then their manufactured year can be 2000, 'Year 95' and unknown.)

If you're doing video, you could actually download the individual chunks of video and stick them in the database, and then use media source extension to read your database and stream that in your destination.

The records are stored in object store as key-value pairs and are sorted according to the keys in ascending order.

Index

We use an Index as a kind of object store for organizing or retrieving records from another object store (called the reference object store) by an individual property of the data.

For example, when saving books in our object store 'books', we may want to fetch them later by their name, published year or author.

The index is a persistent key-value storage where the value part of its records is the key part of a record in the referenced object store. The records in an index are automatically populated whenever records in the referenced object store are inserted, updated, or deleted. Each record in an index can point to only one record in its referenced object store, but several indexes can reference the same object store. When the object store changes, all indexes that refer to the object store are automatically updated.

Key

A data value by which stored values are organized and retrieved in the object store. The key must be of a data type that has a number that is greater than the one before it. Each record in an object store must have a key that is unique within the same store, so you cannot have multiple records with the same key in a given object store.

  • A key can be one of the following types: string, date, float, a binary blob, and array.
  • The object store can derive the key from one of three sources: a key generator, a key path, or an explicitly specified value.

Key generator

A mechanism for producing new keys in an ordered sequence. If an object store does not have a key generator, then the application must provide keys for records being stored.

Key path

Defines where the browser should extract the key from in the object store or index.

  • A valid key path can include one of the following: an empty string, a JavaScript identifier, or multiple JavaScript identifiers separated by periods or an array containing any of those. It cannot include spaces.

Value

Each record has a value, which could include anything that can be expressed in JavaScript, including boolean, number, string, date, object, array, regexp, undefined, and null.

Transaction

A transaction is wrapper around an operation, or group of operations, that ensures database integrity. If one of the actions within a transaction fail, none of them are applied and the database returns to the state it was in before the transaction began. All read or write operations in IndexedDB must be part of a transaction. This allows for atomic read-modify-write operations without worrying about other threads acting on the database at the same time.

Cursor

A mechanism for iterating over multiple records with a key range. The cursor has a source that indicates which index or object store it is iterating. It has a position within the range, and moves in a direction that is increasing or decreasing in the order of record keys.

How to use IndexedDB?

We will try to cover the basic functions and the concepts of IndexedDB with an example webpage. The webpage I have created will have one object store named books which will have the name of book, author, year published and ISBN as indexes. ISBN will work as the key path, which we consider to be unique for every book.

You can run this demo locally on your machine. This is the code

indexeddb demo 1
  1. We will add two items with the given form:
indexeddb demo 2
  1. And then delete one of the items:
indexeddb demo 3

How to check if IndexedDB is supported?

In case you want to test your code in browsers that still use a prefix, you can use the following code:

// In the following line, you should include the prefixes of
// implementations you want to test.
  window.indexedDB = window.indexedDB || window.mozIndexedDB || window.webkitIndexedDB || window.msIndexedDB;
  
  // DON'T use "var indexedDB = ..." if you're not in a function.
  // Moreover, you may need references to some window.IDB* objects:
  window.IDBTransaction = window.IDBTransaction || window.webkitIDBTransaction || window.msIDBTransaction || {READ_WRITE: "readwrite"};
  // This line should only be needed if it is needed to support the
  // object's constants for older browsers
  
  window.IDBKeyRange = window.IDBKeyRange || window.webkitIDBKeyRange || window.msIDBKeyRange;
  // (Mozilla has never prefixed these objects, so we don't need
  // window.mozIDB*)

You can trigger an alert notification if it is not supported like:

  if (!window.indexedDB) {
    window.alert("Your browser doesn't support a stable version of IndexedDB.");
  }

How to create IndexedDB database and structure the object store?

We will "request" to open the database.

  var request = window.indexedDB.open("MyTestDatabase", 3);
  // The second parameter is the version of the database

Note: The version number is an unsigned long long number. If you use a float, it will be converted to the closest lower integer and the transaction may not start, nor the upgradeneeded event trigger.

The open request doesn't open the database or start the transaction right away. The call to the open() function returns an IDBOpenDBRequest object with a result (success) or error value that you handle as an event. Most other asynchronous functions in IndexedDB do the same thing - return an IDBRequest object with the result or error. The result for the open function is an instance of an IDBDatabase.

  • If the database doesn't already exist, it is created by the open operation, then an onupgradeneeded event is triggered and you create the database schema in the handler for this event.
  • If the database does exist but you are specifying an upgraded version number, an onupgradeneeded event is triggered straight away, allowing you to provide an updated schema in its handler.

Here is the flow diagram to understand the role of 'version' when opening database:
Screenshot-from-2019-06-29-10-56-49
{ Source : Step by Step IndexedDB Crash Course with Javascript }

How to generate Handlers?

If everything succeeds, a success event (that is, a DOM event whose type property is set to "success") is fired with request as its target. Once it is fired, the onsuccess() function on request is triggered with the success event as its argument. Otherwise, if there was any problem, an error event (that is, a DOM event whose type property is set to "error") is fired at request. This triggers the onerror() function with the error event as its argument.

request.onerror = function(event) {
  // Do something with request.errorCode!
  alert("Error faced while opening database");
  };
  
request.onsuccess = function(event) {
    // Do something with request.result!
    db = event.target.result;
    
    // HANDLING ERRORS
    db.onerror = function(event) {
      // Generic error handler for all errors targeted at this 
      // database's equests!
      alert("Database error: " + event.target.errorCode);
    };
};
  • request.result is an instance of the IDBDatabase, where the request was generated with a call to indexedDB.open(). We are storing this instance in the variable named db.

How to create and update version of IndexedDB database?

When you create a new database or increase the version number of an existing database, the onupgradeneeded event will be triggered. In the handler for the upgradeneeded event, you should create the object stores needed for this version of the database:

  request.onupgradeneeded = function(event) { 
    // Save the IDBDatabase interface 
    var db = event.target.result;

    // Create an objectStore for this database to hold info about our
    // books. We'll use "isbn" as our key path because we know its
    // unique for every book
    var objectStore = db.createObjectStore("books", { keyPath: "isbn" });

    // Create an index to search books by name.
    objectStore.createIndex("name", "name", {unique:false});

    // Create an index to search books by author
    objectStore.createIndex("author", "author", {unique:false});

    // Create an index to search books by manufactured year
    objectStore.createIndex("year", "year", {unique:false});

    objectStore.transaction.oncomplete = function(event) {
        // Use transaction oncomplete to make sure the objectStore
        // creation is finished before adding data into it.
    }
  };

(We are done with creating the schema for our object store 'books')

  • If the onupgradeneeded event exits successfully, the onsuccess handler of the open database request will then be triggered.
  • It is the only place where you can alter the structure of the database. In it, you can create and delete object stores and build and remove indices.

How to add, retrieve and remove data in IndexedDB?

  • Before you can do anything with your new database, you need to start a transaction. Transactions come from the database object, and you have to specify which object stores you want the transaction to span.
  • Once you are inside the transaction, you can access the object stores that hold your data and make your requests.
  • Next, you need to decide if you're going to make changes to the database or if you just need to read from it. Transactions have three available modes: readonly, readwrite, and versionchange.

For our web app, we will be adding data to our object store from the values entered in the addForm.

<p>
        <fieldset id="addForm">
            <legend>Add item</legend>
            <label>Name</label>
            <input id="nameInput">
            <label>Author</label>
            <input id="authorInput">
            <label>Year</label>
            <input id="yearInput">
            <label>ISBN</label>
            <input id="isbnInput">
            <button id="addButton">Add</button>
        </fieldset>
</p>
  document.getElementById('addButton').onclick = function(e) {

    // Getting data from the input fields in the form
    var bname = document.getElementById('nameInput').value;
    var bauthor = document.getElementById('authorInput').value;
    var byear = document.getElementById('yearInput').value;
    var bisbn = document.getElementById('isbnInput').value;
    
    // Saving the details in dictionary format
    const book_item = {
      name: bname,
      author: bauthor,
      year: byear,
      isbn: bisbn
    }
    
    // Starting transaction. Scope is set to the object store 'books'
    var transaction = db.transaction(["books"], "readwrite");

    transaction.oncomplete = function(event) {
      console.log("all done with transaction");
    }

    transaction.onerror = function(event){
      console.dir(event);
    }

    var booksObjectStore = transaction.objectStore("books");
    var request = booksObjectStore.add(book_item);

    request.onsuccess = function(event){
      console.log("added item");
    }
    
    // This function updates the table viewed on our web app.
    updatetable();
  }
  • You open such transactions with IDBDatabase.transaction. The method accepts two parameters: the storeNames (the scope, defined as an array of object stores that you want to access) and the mode for the transaction.

  • The method returns a transaction object containing the IDBIndex.objectStore method, which you can use to access your object store.

  • Transactions only let you have an object store that you specified when creating the transaction.
    Note: By default, where no mode is specified, transactions open in readonly mode.

  • The result of a request generated from a call to add() is the key of the value that was added. So in this case, it should equal the ISBN property of the object that was added.

  • The add() function requires that no object already be in the database with the same key.

Now for retrieving data :

In our example, we will retrieve data from the database and show it in a table on our web app.

If you have the key of the record to retrieve, the get() function can be used:

var transaction = db.transaction(["books"]);
var objectStore = transaction.objectStore("books");
var request = objectStore.get("111");
request.onerror = function(event) {
  // Handle errors!
};
request.onsuccess = function(event) {
  // Do something with the request.result!
  alert("Name for ISBN 111 is " + request.result.name);
};

How and why use Cursor in IndexedDB?

We will be using cursor as we have to iterate over multiple records.

The openCursor() function takes several arguments. First, you can limit the range of items that are retrieved by using a key range. Second, you can specify the direction that you want to iterate. In our example, we're iterating over all objects in ascending order.

The success callback for cursors is a little special. The cursor object itself is the result of the request. Then the actual key and value can be found on the key and value properties of the cursor object. If you want to keep going, then you have to call continue() on the cursor.

(We can also use cursor.key instead of cursor.value.isbn, since ISBN is our key for this object store.)

function updatetable(){
    
    // We have to add records to the body of our table.
    document.getElementById("books-table-body").innerHTML = "";

    var request = db.transaction("books").objectStore("books").openCursor();

    request.onerror = function(event){
      console.dir(event);
    };

    request.onsuccess = function(event){

      cursor = event.target.result;

      if(cursor) {
        document.getElementById("books-table-body").innerHTML += 
          "<tr><td>" + cursor.value.name + "</td><td>"
          + cursor.value.author + "</td><td>" + cursor.value.year 
          + "</td><td>" + cursor.value.isbn + "</td></tr>";

          cursor.continue();
      }
    };
  }

Removing data from database

We will be removing a record from our 'books' object store by accepting input from user with the deleteForm.

    <p>
      <fieldset id="deleteForm">
        <legend>Delete item</legend>
        <label>ISBN</label>
        <input id="isbnDelInput">
        <button id="delButton">Delete</button>
      </fieldset>
    </p>
  document.getElementById('delButton').onclick = function(e){

    var isbn_del = document.getElementById('isbnDelInput').value;

    var request = db.transaction(["books"], "readwrite").objectStore("books").delete(isbn_del);

    request.onsuccess = function(event){
      console.log(isbn_del+" deleted");
    }

    updatetable();
  };

Code used for this demo

Here is the full code for the web app I created as an example:
(You can also get it from the git repo. : OpenGenus/indexeddb)

HTML :

<!DOCTYPE html>
<head>
    <script></script>
</head>

<body onload="init()">
  
    <h1>IndexedDB Example</h1>

    <p>
        <fieldset id="addForm">
            <legend>Add item</legend>
            <label>Name</label>
            <input id="nameInput">
            <label>Author</label>
            <input id="authorInput">
            <label>Year</label>
            <input id="yearInput">
            <label>ISBN</label>
            <input id="isbnInput">
            <button id="addButton">Add</button>
        </fieldset>
    </p>

    <p>
      <fieldset id="deleteForm">
        <legend>Delete item</legend>
        <label>ISBN</label>
        <input id="isbnDelInput">
        <button id="delButton">Delete</button>
      </fieldset>
    </p>

    <table id="books-table">
        <thead>
            <tr><th>Name</th><th>Author</th><th>Year</th><th>ISBN</th></tr>
        </thead>
        <tbody id="books-table-body">
        </tbody>
    </table>

</body>
</html>

JS :

// In the following line, you should include the prefixes of
// implementations you want to test.
  window.indexedDB = window.indexedDB || window.mozIndexedDB || window.webkitIndexedDB || window.msIndexedDB;
  
  // DON'T use "var indexedDB = ..." if you're not in a function.
  // Moreover, you may need references to some window.IDB* objects:
  window.IDBTransaction = window.IDBTransaction || window.webkitIDBTransaction || window.msIDBTransaction || {READ_WRITE: "readwrite"};
  // This line should only be needed if it is needed to support the
  // object's constants for older browsers
  
  window.IDBKeyRange = window.IDBKeyRange || window.webkitIDBKeyRange || window.msIDBKeyRange;
  // (Mozilla has never prefixed these objects, so we don't need
  // window.mozIDB*)


  if (!window.indexedDB) {
    window.alert("Your browser doesn't support a stable version of IndexedDB. Such and such feature will not be available.");
  }

function init(){

  // OPENING A DATABASE
  // Let us open our database
  var request = window.indexedDB.open("MyTestDatabase", 3);
  // The second parameter is the version of the database

  // GENERATING HANDLERS
  request.onerror = function(event) {
  // Do something with request.errorCode!
    alert("Error faced while opening database");
  };
  request.onsuccess = function(event) {
    // Do something with request.result!
    db = event.target.result;
    
    // HANDLING ERRORS
    db.onerror = function(event) {
      // Generic error handler for all errors targeted at this database's requests!
      alert("Database error: " + event.target.errorCode);
    };

  };

  // CREATING OR UPDATING THE VERSION OF DATABASE

  request.onupgradeneeded = function(event) { 
    // Save the IDBDatabase interface 
    var db = event.target.result;

    // Create an objectStore for this database to hold info about our books.
    // We'll use "isbn" as our key path because we know its unique for every book
    var objectStore = db.createObjectStore("books", { keyPath: "isbn" });

    // Create an index to search books by name.
    objectStore.createIndex("name", "name", {unique:false});

    // Create an index to search books by author
    objectStore.createIndex("author", "author", {unique:false});

    // Create an index to search books by manufactured year
    objectStore.createIndex("year", "year", {unique:false});

    // Use transaction oncomplete to make sure the objectStore crration
    // is finished before adding data into it.
    objectStore.transaction.oncomplete = function(event) {
      
    }
  };

  document.getElementById('addButton').onclick = function(e) {

    var bname = document.getElementById('nameInput').value;
    var bauthor = document.getElementById('authorInput').value;
    var byear = document.getElementById('yearInput').value;
    var bisbn = document.getElementById('isbnInput').value;
    
    const book_item = {
      name: bname,
      author: bauthor,
      year: byear,
      isbn: bisbn
    }

    var transaction = db.transaction(["books"], "readwrite");

    transaction.oncomplete = function(event) {
      console.log("all done with transaction");
    };

    transaction.onerror = function(event){
      console.dir(event);
    };

    var booksObjectStore = transaction.objectStore("books");
    var request = booksObjectStore.add(book_item);

    request.onsuccess = function(event){
      console.log("added item");
    };

    updatetable();

  };

  document.getElementById('delButton').onclick = function(e){

    var isbn_del = document.getElementById('isbnDelInput').value;

    var request = db.transaction(["books"], "readwrite").objectStore("books").delete(isbn_del);

    request.onsuccess = function(event){
      console.log(isbn_del+" deleted");
    };

    updatetable();
  };

  function updatetable(){

    document.getElementById("books-table-body").innerHTML = "";

    var request = db.transaction("books").objectStore("books").openCursor();

    request.onerror = function(event){
      console.dir(event);
    };

    request.onsuccess = function(event){

      cursor = event.target.result;

      if(cursor) {
        document.getElementById("books-table-body").innerHTML += "<tr><td>" + cursor.value.name + "</td><td>"
          + cursor.value.author + "</td><td>" + cursor.value.year + "</td><td>" + cursor.key + "</td></tr>";

          cursor.continue();
      }
    };
  }
}