Search icon
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Data Lake for Enterprises

You're reading from  Data Lake for Enterprises

Product type Book
Published in May 2017
Publisher Packt
ISBN-13 9781787281349
Pages 596 pages
Edition 1st Edition
Languages
Authors (3):
Vivek Mishra Vivek Mishra
Profile icon Vivek Mishra
Tomcy John Tomcy John
Profile icon Tomcy John
Pankaj Misra Pankaj Misra
Profile icon Pankaj Misra
View More author details

Table of Contents (23) Chapters

Title Page
Credits
Foreword
About the Authors
About the Reviewers
www.PacktPub.com
Customer Feedback
Preface
Part 1 - Overview
Part 2 - Technical Building blocks of Data Lake
Part 3 - Bringing It All Together
Introduction to Data Comprehensive Concepts of a Data Lake Lambda Architecture as a Pattern for Data Lake Applied Lambda for Data Lake Data Acquisition of Batch Data using Apache Sqoop Data Acquisition of Stream Data using Apache Flume Messaging Layer using Apache Kafka Data Processing using Apache Flink Data Store Using Apache Hadoop Indexed Data Store using Elasticsearch Data Lake Components Working Together Data Lake Use Case Suggestions

When not to use Flume


In some scenarios, usage of Flume is not the ideal choice. There are other options out there which can be employed to solve those use case and not Flume. Do not choose Flume when:

  • You need more data processing as against transfer of data. They are more suited for other stream processing technologies.
  • You need more batch data transfer scenarios (regular batch as against micro-batch).
  • You need a more available setup with no data loss.
  • You need a durable message with very high scalability requirements (there isn't a scientific quantitative figure for that though).
  • You have a huge number of consumers as this has a very high impact on Flume’s scalability.

          Even through Flume can be dynamically configured in many cases, it does incur downtime in certain configuration changes (topology changes).

          lock icon The rest of the chapter is locked
          Register for a free Packt account to unlock a world of extra content!
          A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
          Unlock this book and the full library FREE for 7 days
          Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
          Renews at $15.99/month. Cancel anytime}