Offline / Online Data Synchronization Design (Javascript)

user_160393 0
Viewed 13926

I'm currently in the process of writing an offline webapp using all the html5 goodies for offline support. However I'm starting now to think about writing the sync module that will ensure that any offline data gets sent to the server and server data back to the client. Now I'm sure this has been done before, I mean its a pretty classic design issue that affects mobile devices and a plethora of other things. So I'm wondering can anyone point me to some good design resources for this kind of thing?

Now I really do not need to be too sophisticated with this, I mean I'm not handling multiple users accessing the same data and I'm happy not to merge conflicts (just take the latest) but still I would like a design that will allow me those options in the future.

Also, are there any open source projects implementing this type of thing? I'm not above ripping off someone else's code (if license allows) and I'm happy to port.

pouchdb offline data-synchronization javascript
9 Answers

I had a similar problem. I decided to use a purely JSON in and out approach. The solution I'm taking on form submission is:

  1. catch the form submit event
  2. check whether or not the user is online
  3. if user is online then submit the form as normal form POST
  4. if user is offline then stringify a JSON request and store it locally (I decided to use Web SQL Database). Queue table is simply Uri and Payload.

Then I have global event hooks for the online / offline events. When the user comes back online, it checks the queue, and if the queue has items in it, it then sends them through as JSON POST requests.

If you are primarily interested in getting JSON data and caching it for offline usage, then take a look at jquery.offline.

The challenge with synchronizing in both direction is that you need to update the local cached lists with any CRUD work that you have queued.

I'd like to find a more generic way to do this.

user_1196240

Check out Derby, a Node MVC framework that has some pretty sweet synchronization and conflict resolution features. http://derbyjs.com/

user_1674220

I'm currently working on similar webapp. I've decided to make such workflow:

  1. Form isn't really submitted - "Submit" button actually saves serialized form data to localStorage (in some queue). This saves from troubles with submit capturing and from writing additional error processing code to handle disconnect during form submission.
  2. Transport script is triggered after data saving. It checks online/offline state.
  3. When online, it tries to send latest data from queue to server (AJAX request), and deletes it from queue on success (and continues to send next data from queue after short timeout).
  4. It shedules re-check after some period of time (by setTimeout()).
user_12353940

If you are up for using the potentially heavy Ext JS / Sencha framework, it has a nice data API with offline (e.g. localStorage) support and a proxy approach for write-thru to local then server. I use Sencha Touch (mobile-specific).

For debugging web storage, check out Weinre.

user_12742480
user_13200970

I'd personally recommend you write a wrapper on top of the indexedDB API that checks whether you are online/offline.

  • if offline, just store in indexedDB and set persisted flag to false on all documents
  • if online, get all documents where persisted is false and store them in mongodb or something equivelant on the backend, then store new documents in both indexedDB and on the server with the persisted flag to true.

I've written a small one

You would have to augment the tunnel to set the persisted flag automatically and also tunnel the synchronization of these documents to the backend

user_4199700
Related