Getting Started with Client Side Storage

Tweet about this on TwitterShare on LinkedIn14Share on Google+0Share on Reddit0Buffer this pageFlattr the authorEmail this to someonePrint this page

In this post, we will take a look at various client side storage solutions and how to work with them.

We are going to cover the following

  • Cookies
  • Web Storage API
    • Session Storage
    • Local Storage
    • String Compression
  • IndexedDB
  • Next Steps

You can find the completed code in the below examples here.

Cookies

There was a time when client side storage was all about saving data in cookies. Cookies were designed initially not to save data offline but rather save the established session details and send it with every request to the server.

 … a way to create a stateful session with Hypertext Transfer Protocol (HTTP) requests and responses.  It describes three new headers, Cookie, Cookie2, and Set-Cookie2, which carry state information between participating origin servers and user agents.  The method described here differs from Netscape’s Cookie proposal [Netscape], but it can interoperate with HTTP/1.0 user agents that use Netscape’s method.

You can read more : rfc2965

Back to storage with Cookies. For instance, you have a very long form to be filled in by the user. If the user refreshes the page accidentally while filling the form, you want to save the data and populate it back. And at the same time, you do not want to send this partial data to the server. This should be saved on the client side. And this is how we would implement this using cookies.

You would have a long form like

I have used only 2 form fields for simplicity. Also, I have used Twitter Bootstrap for styling. This form can be how ever long you want and how many ever form elements you want (select, radio, textarea, etc..).

And the JS code to control this would be

Things to notice

Line 5 :  A wrapper API to work with cookies. Create, Read, Delete

Line 38 : When the document is ready, we register for blur events on the form elements. And when the call back is triggered, we create a new cookie passing the cookieName, the entire form data and days to expire to  createCookie().

Line 50 : On page load, we check if there is any pre-saved data available. If yes, we read the same and populate it

Line 66 : And the form is submitted successfully, we delete the saved cookie.

And when you fill the form and “accidentally” refresh, you should the see the saved data populate. And when you inspect the saved cookies and you should see

Screen Shot 2015-04-19 at 8.41.32 am

And when submit, you should see the cookie removed.

Size Limitations :

If you want to support most browsers, then don’t exceed 50 cookies per domain, and don’t exceed 4093** bytes per domain (i.e. total size of all cookies <= 4093 bytes).

You can read more : Browser Cookie Limits.

For me the biggest problem with cookies is that, the browser sends all the cookies related to that domain with every HTTP request. Which does not sound very “Client Side Storage” for me.

Then came along HTML5 with it’s arsenal of storage solutions.

Web Storage API

The Web Storage API provides mechanisms by which browsers can securely store key/value pairs, in a much more intuitive fashion than using cookies.

The two mechanisms within Web Storage are as follows:

  • sessionStorage maintains a separate storage area for each given origin that’s available for the duration of the page session (as long as the browser is open, including page reloads and restores)
  • localStorage does the same thing, but persists even when the browser is closed and reopened.

You can read more about the Web Storage API from MDN here.

As you see from below http://caniuse.com/#feat=namevalue-storage support

Screen Shot 2015-04-19 at 9.18.22 am

there is about 94% support for Web Storage API across all browsers. Which is good as we can consider this to be a reliable solution in almost all cases.

Session Storage

Taking the above example, of saving a long form, we will implement the same using session storage API. The HTML for the page would remain the same. The JS code would look like

Things to notice

Line 1 : A wrapper around sessionStorage API.

Line 8 : We create a method to check sessionStorage support.

Line 12, 16 : Save and retrieve string based data. The Storage API saves all the values in the form of string. So any value as in a string, number, boolean can be saved and retrieved using these methods.

Line 26, 30 : These methods are used to save and retrieve Objects and Arrays. You can read more about this here : Storing Objects in HTML5 Local Storage.

Line 34 : A method to remove an item from the sessionStorage

Line 38 : A  method to clear all items from the sessionStorage

Line 47 : We check if sessionStorage is supported, else we throw an alert

Line 51 : We register a blur event on the form elements. And when the listener fires, we get the form data, convert it to an object and save it

Line 68 : We check if anything is already stored in the sessionStorage. If yes, we populate the form.

Line 76 : When the form saves the data successfully, we clear the sessionStorage.

You can inspect the Session Storage saved data and you should see.

Screen Shot 2015-04-19 at 10.01.37 amIn the above case, if the form fields are dependent on a session then the session storage is perfect. Because if the form changes for the same user for a different session, the old data will not clash with the new data.

Local Storage

What if you are looking for a way to persist the data even after the user’s session expires. The answer is simple, Local Storage API.

The transformation from Session Storage to Local Storage is as simple as replacing the word Session with Local. The updated JS file would look like

Things to notice

Line 1 : A wrapper around localStorage API.

Line 8 : We create a method to check localStorage support.

Line 12, 14 : Save and retrieve string based data. The Storage API saves all the values in the form of string. So any value as in a string, number, boolean can be saved and retrieved using these methods.

Line 26, 30 : These methods are used to save and retrieve Objects and Arrays. You can read more about this Storing Objects in HTML5 Local Storage.

Line 34 : A method to remove an item from the localStorage

Line 38 : A  method to clear all items from the localStorage

Line 47 : We check if localStorage is supported, else we throw an alert

Line 51 : We register a blur event on the form elements. And when the listener fires, we get the form data, convert it to an object and save it

Line 68 : We check if anything is already stored in the localStorage. If yes, we populate the form.

Line 76 : When the form saves the data successfully, we clear the localStorage.

You can inspect the Local Storage saved data and you should see

Screen Shot 2015-04-19 at 10.15.53 amNow, even after rebooting the machine, this data will be persisted.

Size Limitations :

The size limitations for a given domain for the Storage API is dependent on the browser and the OS. If you check out Web Storage Support Test,

Screen Shot 2015-04-19 at 10.34.15 am

The above values are for Chrome 42 64bit. Also based on a lot of solutions on the internet, the storage seems to be around 5 MB.

You can read more : Is 5MB the de facto limit for W3C Web Storage?

String Compression

If you see that your storage needs seem to exceed the 5 MB quota, you can implement lz-string compression. You can checkout the demo page and see the compression factor

Screen Shot 2015-04-19 at 10.44.28 am

The implementation is pretty simple. First we include lz-string.min.js in our page. Then we will update the storage API to compress and decompress the string. The updated Local Storage API would be

On line 13 and line 31, we compress the string before we save it. And on line 19 and line 37, we decompress the string.

Note : On Firefox and IE, localStorage cannot contain invalid UTF16 characters. You need to use  LZString.compressToUTF16() and  LZString.decompressFromUTF16() if you are targeting IE and FF as part of your app. Else you can use  LZString.compress() &  LZString.decompress()

If you observed the save data, you should see the compressed string

Screen Shot 2015-04-19 at 10.51.25 am

You can implement the same API for Session Storage as we did above for Local Storage. Do notice that compressing adds an overhead to the get and set process. So use this only if needed.

IndexedDB

IndexedDB is a way for you to persistently store data inside a user’s browser. Because it lets you create web applications with rich query abilities regardless of network availability, these applications can work both online and offline.

IndexedDB is an alternative to WebSQL Database, which the W3C deprecated on November 18, 2010. While both IndexedDB and WebSQL are solutions for storage, they do not offer the same functionalities. WebSQL Database is a relational database access system, whereas IndexedDB is an indexed table system.

Concepts of IndexedDB

  • IndexedDB databases store key-value pairs.
  • IndexedDB is built on a transactional database model.
  • The IndexedDB API is mostly asynchronous.
  • IndexedDB uses requests all over the place.
  • IndexedDB uses DOM events to notify you when results are available.
  • IndexedDB is object-oriented.
  • IndexedDB does not use Structured Query Language (SQL).
  • IndexedDB adheres to a same-origin policy.

You can read more about the above concepts here.

Before we start implementing IndexedDB, we need to go through its terminology.

  • database : A repository of information, typically comprising one or more object stores.
  • object store : The mechanism by which data is stored in the database.
  • version : When a database is first created, its version is the integer 1.
  • database connection : An operation created by opening a database.
  • transaction : An atomic and durable set of data-access and data-modification operations on a particular database.
  • request : The operation by which reading and writing on a database is done.
  • index : An index is a specialized object store for looking up records in another object store, called the referenced object store.

You can read more here.

Now that we have a basic understanding of what IndexedDB is all about, we will make use of it save our form offline. Yes, this use case is not an ideal example for IndexedDB. It can do a lot more.

If you are planning to use the IndexedDB API as is, you can do so by following this tutorial : Using IndexedDB. To keep things simple and interesting, I am going to use an awesome IndexDB wrapper named Dexie.js.

Dexie has a huge API that helps a lot while interacting with IndexedDB. You can read more about Dexie from its very well documented wiki.

Back to our example. We will be implementing offline storage of our form using IndexedDB via Dexie. The HTML for the form will remain the same. After you include dexie.js in your page, you can write your logic as follows

Things to notice

Line 3 : We create a new database named longForm

Line 5 :  We create a new auto incrementing index. You can read more about Schema definition syntax here.

Line 8 : We connect to the DB. On successful connection, we query the DB for any pre-saved data.

Line 10 : Once we get any pre-saved data, we populate our form on line 13.

Line 20 : On blur of any form fields, we collect the latest form data

Line 32 : We delete any existing data before we save a fresh copy.

Line 39 : When the form is successfully submitted, we will delete the DB.

When you inspect the data, it should look like

Screen Shot 2015-04-19 at 2.15.30 pm

Next Steps

With this, we have completed the most prominent offline storage solution available today. I recommend taking a look at the following as next steps

I will keep adding more resources as I get.


Thanks for reading! Do comment.
@arvindr21

Tweet about this on TwitterShare on LinkedIn14Share on Google+0Share on Reddit0Buffer this pageFlattr the authorEmail this to someonePrint this page
  • Lola

    Hi Arvind, great article!! You book “learning ionic” uses local storage in an example app which gives a good example using local storage in practice. The book is a amazing!! I’ve been programming using ionic for more than a year now. I still read it from time to time. I am pre-ordering the version 2. Regarding local storage, I use local storage in my app as well to store user preferences and session storage to store access tokens(loaded from keychain at run phase). however, the biggest concern of using local Storage is if user uses private browsing. My app will fail. Any advice on persistent data in a more reliable way? Is using a local json a bad idea for example?
    Another question is in practice how to prefex items stored in local storage? If user try to use different accounts on one device, the items might get overwritten, am I correct?

    • / Arvind Ravulavaru

      Thanks for your support Lola. Glad I could be of help.

      Private browsing persistent is always a challenge. In one of the applications I have designed for the web, we detected if local storage was available on page load, if not we threw an alert that either use the website in a “non-private” browsing mode or loose the privilege of persistence.

      Ideally local storage should not be used as a database on the client side rather a lite weight storage till the application process stuff.

      In a mobile app you can always go for sqlite instead of local storage. A more reliable way of storing information.

      Thanks.

  • Sagar

    hi ,please tell me how to have dropdown menu of ID’s while fetching data from mongoDB,

    I need a code sample for it.

    I have 4 collections in my mongoDB database ,I need to have a dropdown menu of ID’s and On selecting one I should be able to fetch data from that particular collection.

    Example: Collection_1,_2,_3,_4 are my 4 collections ID1,ID2,ID3,ID4 are names respectively in dropdown on selecting ID2 from dropdown and clickon submit I should be able to fetch data from collection_2.

    please help me in this matter

  • mugshepherd

    Very helpful!

  • Pierre Grimaud

    I think you might want to update your LZString example to use “compressToUTF16″ instead of “compress”. The “compress” method produces strings that are not valid UTF16 and thus cannot be stored and retrieved from localStorage in IE and Firefox.

    • / Arvind Ravulavaru

      Thanks Pierre! Very valid point. Added the changes to the code example.

  • Hardik Joshi

    Amazing article! Thanks Arvind for refreshing the knowledge about offline storage! :)