What is this big data hype all about?

Big data has proven to be a bit of a trendy buzzword of late, with lots of use cases, jobs, and technologies focusing in this area. However, what is it all about? With the rise of bigger and greater data storage capacities, we are able to capture and retain ever-growing amounts of source information. The internet has allowed data to be published and distributed far and wide. This is combined with the fact that people as well as organizations and governments are willing to provide more and more information about themselves and their goings-on. So, if you wish to try and find out the correlations between high school results and an area's favorite genre of film, the data is probably out there for you to have look.

However, with so much data, it becomes increasingly impractical for a single machine to store, let alone process, all of it in a tangible amount of time. To combat this, we need to split up and distribute our data around and then process it in...