Outgrowing MongoDB

About Me

• Full-Stack Web Developer (Ruby, Javascript)
• Lead-Tech at Quicklizard Ltd.
• Freelance consultant & trainer
• Find me - @zohararad (Twitter / Github)

Outgrowing MongoDB

This isn't a criticism of MongoDB, or a DB competition. MongoDB is great and you should keep using it!

But... sometimes your use-case isn't a perfect fit for MongoDB, and you might be able to employ more modern DBs for your cause.

Outgrowing MongoDB

We're going to look at a few new NoSQL databases and their use-cases.

We're not going to compare performance, just features (well mostly).

When not to Mongo?

• High write load
• Intensive use of M/R (single threadness etc.)
• Highly join-oriented Business-Logic
• ACID is a must

Before we start

When I use the term "Easy Admin / Scaling", I mean it in a subjective way, hinting at the expected complexity of adding more capacity to the cluster by adding nodes, and administering the cluster as a whole.

ArangoDB

ArangoDB is perhaps MongoDBs closest cousin, sharing a very similar feature set and aiming at some of the same use-cases.

ArangoDB uses its own query language called AQL which tries to be closer to SQL

ArangoDB - Notable Features

• Supports multi-model storage - K/V, Document, Graph
• Sharding and replication
• Transactions, joins and ACID
• Multi-threaded (MongoDB is single-threaded)
• Foxx thin application layer for REST APIs
• Admin UI

ArangoDB - Pitfalls

• Not all nodes are created equal (same as MongoDB - complicates admin)
• No transactions in sharded collections
• Not enough examples of cluster deployment scenarios

Choosing ArangoDB

• When you want to mix graph and document data models
• When you need to escape the single-threaded, locking of MongoDB
• When you want a thin REST API layer to serve your data
• When you need joins, transactions or ACID compliance

RethinkDB

RethinkDB is a distributed Document DB with parallel query execution and easy admin. RethinkDB offers a programmatic query language called RQL that is developer-oriented rather than DBA oriented.

RethinkDB is very transparent in it's release process and shares information about performance degradation and feature implementation road-map, making it easy for us to decide if and when to adopt or upgrade.

RethinkDB - Notable Features

• Easy admin (nodes are equal)
• Joins, sub-queries, transactions
• Isolated execution pool & distributed queries
• Lock-free architecture
• Admin UI

RethinkDB - Pitfalls

• No automatic primary re-election
• Query language takes time to get used to
• Tries to always be consistent, at the cost of write availability

Choosing RethinkDB

• When you need an easier deployment setup (i.e. DevOps resources are scarce)
• When you need to escape the single-threaded, locking of MongoDB
• When you need joins or transactions

ArangoDB or RethinkDB?

ArangoDB seems to be a better fit for a multi-model use-case (graphs, K/V, document) or when you want to bind your API layer to your DB (Foxx).

RethinkDB's strongest feature IMHO is easy admin and scaling and would be most suitable when you don't need to worry about the complexities of your cluster.

FoundationDB

FoundationDB is a scalable, fault-tolerant, transactional and ACID compliant Key/Value store.

One of FoudnationDB's most interesting concepts is the separation of the storage layer from the data-model layer, which makes it extremely flexible, without sacrificing performance and scalability.

FoundationDB - Notable Features

• All nodes are created equal (nearly - See SQL Layer) - Easy horizontal scalability & admin
• Can be used as K/V, ANSI SQL, or with custom data-model layer
• High read/write throughput
• Highly-available, Transactional & ACID compliant
• Sorted keys (same as HBase)
• Parallel query execution in SQL Layer

FoundationDB - Pitfalls

• Transactions are limited to 5sec. execution time
• Single thread per query
• 1MB limit of total keys and values size per write
• Slow machines can affect entire cluster throughput

FoundationDB - Some Thoughts

• Useful for K/V & SQL scenarios (think sessions + app data on same DB)
• 5sec. transaction execution limit places more responsibility on developers
• As above can help improve app/data design
• Less suitable for analytics, batch use-cases

ElasticSearch

ElasticSearch is a scalable, fault-tolerant, distributed search-engine and document store

One of ElasticSearch's most compelling features is its aggregation framework which makes it easy to generate real-time analytics from semi-structured data.

ElasticSearch - Notable Features

• All nodes are created equal (nearly - depends on cluster modes) - Simplifies admin & scaling
• Data model is based on JSON documents
• REST API for all operations - Easy integration directly from browser
• Batch operation fo large / long queries (not M/R)
• Full-text search baked in
• High read/write throughput
• Data Aggregation

ElasticSearch - Pitfalls

• Schema changes require re-indexing or recreation of indices
• Query API takes time to get used to (not similar to current query languages)
• Re-sharding requires index recreation
• Schema design can be complicated / requires attention

ElasticSearch - Some Thoughts

• Easy admin and scaling - Great when you need a scalable document store
• Aggregation framework provides excellent entry point for real-time analytical data
• REST API means you can use directly from browser (Kibana does it)
• Extremely large index sizes might affect performance (AFAIK - not verified)

HyperDex

HyperDex is a distributed, fault-tolerant K/V and document store, offering ACID compliance, strong-consistency and transactions.

One of HyperDex's most notable features is it's data-sharding algorithm, which places data in the cluster while retaining its locality, which aids both ACID compliance and search speed.

HyperDex - Notable Features

• All nodes are created equal (nearly - coordinator / daemon) - Easy admin
• K/V with rich data types - Strings, Numbers, Lists, Maps, Sets
• Rich atomic operations of values (eg. add, sub, push, intersect)
• Document storage allows to store JSON objects
• Indexing of document values
• Sync / Async operations
• Mutli-key transactions in commercial version

HyperDex - Pitfalls

• Async operations are concurrent and not ordered
• Fault-tolerance depends on configurable number of coordinators & daemons
• Support for some clients require manual build from source
• Limited querying due to K/V nature

HyperDex - Some Thoughts

• Great for mixed K/V & Document store use cases (eg. session & app data)
• K/V feature set is similar to Redis, but has added benefit of scalability (despite not being in-memory)
• Limited query API means use-cases are mostly in the K/V domain, but with added value of document-indexing
• Documentation is a bit thin - Might take a bit more time to integrate

Closing Thoughts

• Choosing a DB is a matter of requirements coupled with feature-set
• Be cautious of a one-size-fit-all approach
• Don't be afraid to use more than one (two, three) DBs in your app - per use-case
• Don't be fooled by performance benchmarks
• Prototype before you implement (try before you buy)

Links to the DBs we talked about

• ArangoDB - https://www.arangodb.com/
• RethinkDB - http://rethinkdb.com/
• FoundationDB - https://foundationdb.com/
• ElasticSearch - http://www.elasticsearch.org/
• HyperDex - http://hyperdex.org/

If we have time

Replacing MongoDB with ElasticSearch - The story of how Quiclizard replaced MongoDB & HBase with ElasticSearch