As we discussed in earlier sections, there exist multiple ways of processing data, however they can be broadly classified into batch and real-time data processing. While there can be scenarios where one of them provides the desired outcomes, there can be additional scenarios that may need data from both batch as well as real-time data processing components. This drives us to a problem of merging batch data with real-time data. This problem is addressed by the Lambda Architecture pattern, which will be discussed in further detail in the next chapter. Here, we are discussing the initial view of a Lambda-Architecture-driven data lake.
Lambda Architecture, as a pattern, provides the ways and means to perform highly scalable and performant distributed computing on large sets of data and yet (eventually) provides consistent data with the required processing, both in batch as well as in near real time. Lambda Architecture defines the ways and means to enable...
