Data management scalability needs? Just say "No" as in "NoSQL."

Like many agencies, bloomfield knoble, inc. (bk) is relying more and more on big data to help us deliver on-target messages to a granular target audience. The tremendous rise in interactive applications over the last 15 years has led to a dramatic increase in the data management needs of those apps. To address these needs, many IT departments have begun to adopt NoSQL (also referred to as Not only SQL). Major players, such as Facebook, Yahoo, Google, etc. are adopting different NoSQL solutions for their needs. Horizontal scalability, flexible data modeling and management of big data volumes are only a few advantages of NoSQL.

The IT infrastructure of any large organization is quite complex by default. It usually consists of heterogeneous networks, it resides in different data centers and heavily utilizes databased technology to hold corporate data. An NoSQL database provides a mechanism for storage and retrieval of data that is modeled in means other than the tabular relations used in relational databases. The range of NoSQL solutions varies from key-value stores to graph databases. Even though their advantages are controversial, their penetration into IT infrastructure is obvious. NoSQL databases are finding significant and growing industry use in big data and real-time web applications. My first experience with NoSQL was via CERN and the Compact Muon Solenoid (CMS) experiment and LHC. More than three thousand physicists are involved in a sophisticated research program at CMS that yields a few petabtye of information each year. A petabtye is 1015 bytes of digital information – which is a lot. Application growth, big data scale and changing requirements required CMS developers to bring NoSQL solutions into CMS software stack. Read the story of how this was achieved courtesy of IOPscience express, “Life in extra dimensions of database world or penetration of NoSQL in HEP community.”

The funny part is that NoSQL isn’t really that new (in terms of the speed of technology), but the explosion of mobile devices, combined with the cheap cost of data storage and the sudden interest in utilizing big data, has really brought NoSQL to the forefront of analytics. There are many companies that can implement NoSQL (Oracle, Couchbase, etc.) and just as many who would argue against it. Here at bloomfield knoble, we’re much more interested in advertising than arguing IT. What is clear is that we at bk need an alternative to relational databases in order to truly maximize efficiency (both in development and results). I admit that I, personally, am a fan of NoSQL technology. While I have no direct connection to them, I suggest visiting Couchbase for more information about NoSQL (I found their white paper really informative).

If NoSQL seems like something only IT people have to worry about – you’d be right . . . for now. However, as more agencies begin to compete in the “algorithm” space for new business, understanding NoSQL becomes an important point of difference versus the competition. If I was selling NoSQL, I would promote flexibility, scalability and performance advantages. Martin Fowler has written a book “NoSQL Distilled” to help people understand what they need to know in order to decide if NoSQL is worth serious investigation for their projects. To quote from his site, Martin writes,

The rise of NoSQL databases marks the end of the era of relational database dominance, but NoSQL databases will not become the new dominators. Relational will still be popular and used in the majority of situations. They, however, will no longer be the automatic choice. In the future, organizations will use many data technologies. Data professionals will need to be familiar with these different approaches and know how to match them to different problems. This introduces new ways to think about data modeling, data consistency and evolution. Learning the concepts is an important first step, but to really understand, you’ll need to get the experience of building representative systems using these technologies.

At bloomfield knoble, we are striving to truly understand the vertical axis of big data – not just saying we use it, but truly understanding it and applying it – to achieve results. It’s one of the reasons we say, “It’s a big world. What’s your reach?”