Hacker News new | past | comments | ask | show | jobs | submit login
Angular-data: data store and caching for Angular.js (pseudobry.com)
164 points by n8agrin on Sept 10, 2014 | hide | past | favorite | 45 comments



"Angular-data is very optimistic when it comes to caching. If you do DS.find('user', 5) angular-data will first look for that user in the data store. If that user is already in the data store then the promise returned by DS.find('user', 5) will immediately resolve with the user. If no user exists then (if using the http adapter) a GET request will be made, and the user returned by the server will be injected into the data store."

I couldn't find anything in the docs about invalidation. Is anybody aware of how to force invalidation of an object/collection?


you can eject an item or collections of items from the data store with DS.eject and DS.ejectAll. Or you could load angular-cache with angular-data to activate automatic expiry of items (after a timeout or on some interval).


I would probably make an HTTP interceptor to remove the elements from the cache on non-GET methods. The same could probably be done on socket events if you sync your clients with socket.io or something alike.

e.g.: PUT /users/123 => delete from cache the user "123"


This is what I did with angular-cache.


This looks interesting, but I'm not sure whether it's worth tying into a complex project using angular. I have found angular already has a large learning curve for new developers, and adding another layer with its own syntax on top of it adds some complexity that I'm not necessarily convinced that is good in that situation.

It looks alright though in terms of what it offers on first inspection though. It does solve some problems that I have found myself having to solve in a similar manner with homebrewed code & structure.


As someone with a large angular project, this looks to me like it would reduce complexity if it works as well as it says on the tin. In my experience, it takes a ton of work and complexity to make working with data convenient and performant. I'm excited to try this out.


So excited to try this out. An Angular specific smart model layer is something I've been wanting to contribute to the community for a while. Thanks for doing much of the hard work!


This is superb work.

Just one question, does it have the feature to add offline custom methods to the model e.g. I want to add instance calculateAge() for each User ? I couldn't find it in the guides.

Angular-RestMod provides it > https://github.com/platanus/angular-restmod#custom-methods

(basically we use node on the server and many times we just want to reuse the same server side methods on the client)


You can add custom behavior to model instances via the "methods" option when defining a resource. http://angular-data.pseudobry.com/documentation/guide/angula...


From building http://platform.qbix.com/features/streams I know that there are a lot of challenges here. Caching, throttling and batching requests is of course really useful. My biggest question is how do you implement:

  * multi-user access
  * realtime updates to connected clients, and 
  * offline editing and sync?
When multiple users access a resource, you have to consider things like consistency issues (making sure all users see the same changes applied in the same order). Also other stuff would be nice like access control (perhaps integrated with the publishing user's address book) as well as subscriptions notifications when something changes.

Does anyone know of a library that implements that? It took me a couple years to get everything right in my own library.


Multi-user access and realtime updates seem very connected to me. For these you could plug in my upcoming Firebase adapter for angular-data and boom! I actually had a complete GoInstant adapter...

Another option is too use web sockets or a pub/sub messaging service to notify all clients of changes to data, and each client can decide what it wants to do at that point (refresh data, get the changes, etc.). You could implement your own 3-way databinding this way.

Angular-data doesn't support offline editing and sync out of the box right now, but I'm considering writing an adapter that combines the existing http and localStorage adapters into one that supports offline editing and sync. Or you could make your own.


That actually sounds like what Firebase (https://www.firebase.com/) offers, but that's a service not a library.

You might want to have a look at http://pouchdb.com/ but I don't know how (or if) that handles consistency issues.


"Before writing angular-data I used to create fleets of services for my various resources, each performing data management and communicating with a persistence layer."

Can someone give me an example of this? I just use $http and populte the $scope models in callbacks so I don't really recognize the problem.


A simple example would be storing data on your 'logged in user' on a large application that has multiple controllers/services.

A scope only extends as far as a particular controller/service. If you wanted every controller/service to have access to that user's data (eg. that user's ID), you would create a service that would store that data, and then inject that service into each place you need to access the data.


With basic $http or resource:

You have to reload all the data each time you go to this view (if you navigate back to previous page it has to reload).

You have to reload data that is used on multiple pages. eg. products, categories, tags and especially user

The most common solution is to start stuffing things in $rootScope, after that you stuff it in localStorage using angular-cache (which is great).

My own system keeps the user's menu in cache, so it loads near instantly when they come back to the site (still logged in). It invalidates the cache if the user or app version has changed.


For you specific purpose, I don't see why not using the built-in caching mechanism of Angular https://docs.angularjs.org/api/ng/service/$http#caching

The cool thing about angular-data, is that stuff it available, even when the browser is offline and you do a refresh.


the built in $http cache isn't at all suitable for what i described.

I do currently use angular-cache (from the author of angular-data) and build on top of that.

I specifically need to invalidate the storage if the user is different, the server generated session id is different or if the app version has changed since the last page load.

this is a service called UserStorage which can safely be used knowing that it gets dumped if somebody tries to mess with the user or they just logged out/in.

the site I built this for is not a single page app, people load pages often. persistence is essential.

the whole site appears to be server side personalized with menus, saved notes added to property listings etc. but its all frontend enhancement and the server can cache all the pages.


For those writing largish non-trivial apps its essential to separate concerns for maintainability and testability. Controllers and UI logic get separated from domain logic and your Model layer. $scope is no Model layer.


Yet angular-data doesn't seem to deal with model layer per say, but rather synchronization issues between what's somehow in the scope and the web services. To me that seems to be handled quite well with services and maybe factories for wrapping json into custom model classes.

It seems more meant ot be used as a caching layer that would wrap you webservice access services.


I understand. It would be nice to see an example though.


Using $http and callbacks is great to get started but it doesn't scale up to large apps very well.


Can you explain why? A large app will not likely make a far greater number of http calls at the same time. If you must have up to date data then I don't see an alternative in something like angular.


Hey pseudobry, thanks for the hard work first of all. I started using the 2.x branch on a large project and it saved me soooo much time and code over what I was trying to write around localStorage caching and expiration while trying to learn Angular at the same time.

I tried following the advice to upgrade to 3.x when it appeared on the API docs, but it broke my app completely which, after spending 5 minutes reading the 3.x branch's docs made complete sense - you're going for something much larger here.

I haven't personally played with Ember yet, but this seems like it might be inspired by Ember Data? Is that right? From what I hear, that's one of the many thing that Ember has gotten really right.

Thanks again!


I think you're confusing angular-cache with angular-data. Angular-cache (what you describe as 2.x and 3.x) is a drop-in replacement for $cacheFactory. It's a key-value store with extra caching goodies.

Angular-data (at 1.0.0-rc.1 right now) is a full-fledged data store and resource manager.


This looks great! I wrote something similar but Angular-data looks more polished. One of the problems I was trying to solve was synchronizing the same piece of data in multiple places in the client app after the server had validated the data. To do this, I broadcast an event on the rootscope in the http callback. https://github.com/robbyronk/angular-model-sync/blob/master/...


Why does this has to be an angular library ? there's no need for that.

You could provide a generic js library with a way to opt-in angularjs,that way,your data access layer isnt tied to a presentation framework.

Can I use this lib with Backbone or React ? why this fixation on making everything depend on angular ? like jQuery in the past ? are we destined to repeat the same mistakes over and over again in the client js world ?


The "generic js library" is just your browser's localStorage. Open your browser console, type "window.localStorage['test'] = 'whatever'", refresh your browser, then type "window.localStorage.test". Did you want some library that makes it easier to add a key-value pair to an object?

This is just Angular-specific boilerplate to access window/$window and add/retrieve objects without writing a bunch of low-level services. Backbone and Ember already have these model layers but they come with heavy wrappers and getter/setter methods, and the author said he wanted to keep these as POJO objects (which Angular is loved for) without polluting them with ride-along getter/setter methods.


> Plain Old Javascript Objects objects

Plan on visiting the ATM machine later?


Yes, I could generify it, but I'd have implement $q and $http (or introduce other dependencies), and extract angular-specific things like hooking into Angular's $digest cycle for change detection (when the browser doesn't support Object.observe).

This definitely warrants some thought. The next time I write a non-trivial application in a framework other than Angular then this will move up on my priority list. Or perhaps when the community wills it into existence :)



Breeze.js seems like it would be the framework agnostic version of this (I think). Its got client caching, querying, and change tracking.

http://www.breezejs.com/


AFAIK Breeze doesn't support < IE9 which is why I can't use it for certain client projects.


I agree, but it has the convenience of being able to npm install and immediately begin injecting into your app.


This is great. I've written more application-specific angular data stores than I'd care to admit at this point. From what I understand, this is something that Angular 2.0 will handle better, which was enough for me to justify being too lazy to write a generic library. Thanks for doing the grunt work!


I've built something similar recently however also solved client/server synchronisation. You make me want to clean it up and put it on Github. The only difference is that the backend API is HATEOAS.


ELI5 why I'd use this in a regular project using say Rails and angular?


Angular doesn't really provide a data model, you generally have to roll your own for an app.

Think of this as client side ActiveRecord. It handles queries, relations, and syncing with the back end.


Great work, this exactly what I've been looking for. I'd resigned myself to writing it from scratch - so I really appreciate your efforts on this. Will try it out right now.


[deleted]


author here, what browser are you using?


Chrome, latest version. I think it may have been my internet provider since I was unable to connect to several major sites for a bit.

It's working now.


One minor request: if you could add syntax highlighting to the Guide and API, it would be amazing. Otherwise, nice work!


I actually had it in there, upgraded the doc library and highlighting disappeared...

Will get it back though.


Anyone know why these high point submissions aren't on the front page? Is Hacker News ghosting/flagging these posts for any specific reason? Sorry to ramble a bit off topic.


It's on the front page at third place for me. What interface are you using to browse Hacker News?


Currently it's #2 for me, so it's up there.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: