Search icon Close icon
Search icon
0
Cart icon
Close icon
You have no products in your basket yet
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Hadoop 2.x Administration Cookbook

You're reading from  Hadoop 2.x Administration Cookbook

Product type Book
Published in May 2017
Publisher Packt
ISBN-13 9781787126732
Pages 348 pages
Edition 1st Edition
Languages
Concepts
Author (1):
Aman Singh Aman Singh
Profile icon Aman Singh
LinkedIn
Gurmukh Singh is a seasoned technology professional with 14+ years of industry experience in infrastructure design, distributed systems, performance optimization, and networks. He has worked in big data domain for the last 5 years and provides consultancy and training on various technologies. He has worked with companies such as HP, JP Morgan, and Yahoo. He has authored Monitoring Hadoop by Packt Publishing
See other products by Aman Singh

Table of Contents (20) Chapters

Hadoop 2.x Administration Cookbook
Credits
About the Author
About the Reviewers
www.PacktPub.com
Customer Feedback
Preface
Hadoop Architecture and Deployment Maintaining Hadoop Cluster HDFS Maintaining Hadoop Cluster – YARN and MapReduce High Availability Schedulers Backup and Recovery Data Ingestion and Workflow Performance Tuning HBase Administration Cluster Planning Troubleshooting, Diagnostics, and Best Practices Security Index

HDFS health and FSCK

The health of the filesystem is very important for data retrieval and optimal performance. In a distributed system, it becomes more critical to maintain the good health of the HDFS filesystem so as to ensure block replication and near-parallel streaming of data blocks.

In this recipe, we will see how to check the health of the filesystem and do repairs, if any are needed.

Getting ready

Make sure you have a running cluster that has already been up for a few days with data. We can run the commands on a new cluster as well, but for the sake of this lab, it will give you more insights if it is run on a cluster with a large dataset.

How to do it...

  1. ssh to the master1.cyrus.com Namenode and change the user to hadoop.

  2. To check the HDFS root filesystem, execute the hdfs fsck / command, as shown in the following screenshot:

  3. We can also check the status of just one file instead of the entire filesystem, as shown in the following screenshot:

  4. The output of the fsck command will show the...

lock icon The rest of the chapter is locked
arrow left Previous Chapter
Chapter 1 of 20
Next Chapter arrow right
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Sign up now
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Start free trial
Renews at $15.99/month. Cancel anytime}

Authors (1)

Profile icon Aman Singh
LinkedIn
Gurmukh Singh is a seasoned technology professional with 14+ years of industry experience in infrastructure design, distributed systems, performance optimization, and networks. He has worked in big data domain for the last 5 years and provides consultancy and training on various technologies. He has worked with companies such as HP, JP Morgan, and Yahoo. He has authored Monitoring Hadoop by Packt Publishing
See other products by Aman Singh

Other recommended products

Related to this chapter
Left arrow icon
Apache Hadoop 3 Quick Start Guide
Apache Hadoop 3 Quick Start Guide
Apache Hadoop is a widely used distributed data platform. It enables large datasets to be efficiently processed instead of using one large computer to store and process the data. This book will get you started with the Hadoop ecosystem, and introduce you to the main technical topics such as MapReduce, YARN and HDFS.
Oct 2018 7 hours 20 minutes
HBase High Performance Cookbook
HBase High Performance Cookbook
Jan 2017 11 hours 40 minutes
Mastering Hadoop 3
Mastering Hadoop 3
This is a comprehensive guide to understand advanced concepts of Hadoop ecosystem. You will learn how Hadoop works internally, and build solutions to some of real world use cases. Finally, you will have a solid understanding of how components in the Hadoop ecosystem are effectively integrated to implement a fast and reliable Big Data pipeline
Feb 2019 18 hours 8 minutes
Apache Hive Essentials
Apache Hive Essentials
Apache Hive helps you deal with data summarization, queries, and analysis for huge amounts of data. This book will give you a background in big data, and familiarize you with your Hive working environment. Next you will cover advanced topics like performance and security in Hive and how to work efficiently to find solutions to big data problems.
Jun 2018 7 hours 0 minutes
Modern Big Data Processing with Hadoop
Modern Big Data Processing with Hadoop
This book presents unique techniques to conquer different Big Data processing and analytics challenges using Hadoop. Practical examples are provided to boost your understanding of Big Data concepts and their implementation. By the end of the book, you will have all the knowledge and skills you need to become a true Big Data expert.
Mar 2018 13 hours 8 minutes
Mastering Apache Storm
Mastering Apache Storm
With real-world examples and clear explanations, this book will ensure you will have a thorough mastery Apache Storm.You’ll get an understanding of deploying Storm on clusters. Introduce yourself to topics such as trident topology, monitoring, Storm Parallelism, scheduler and log processing. Learn how to integrate Storm with other well-known Big Data technologies such as HBase, Redis, Kafka, and Hadoop to realize the full potential of Storm.You will be able to use the knowledge to develop efficient, distributed real-time applications to cater to your business needs.
Aug 2017 9 hours 28 minutes
Data Lake for Enterprises
Data Lake for Enterprises
The term 'Data Lake' has recently emerged as a prominent term in the big data industry. Data scientists can make use of it in deriving meaningful insights which can be used by businesses to redefine or transform the way they operate. Lambda architecture is also emerging as one of the very eminent patterns in the big data landscape, as it helps to derive useful information from not only the historical data but also correlates real-time data to enable business for taking critical decisions. This book tries to bring these two important aspects into one, namely data lake and lambda architecture.
May 2017 19 hours 52 minutes
Right arrow icon

Personalised recommendations for you

Based on your interests and search pattern
Left arrow icon
Designing and Implementing Microsoft Azure Networking Solutions
Designing and Implementing Microsoft Azure Networking Solutions
Designing and Implementing Microsoft Azure Networking Solutions Exam Ref AZ-700 is an all-encompassing guide to the AZ-700 exam and contains all the information you need to succeed in the world of virtual networking with Azure. With this book, you will be fully prepared for the exam and the world of cloud networking.
Aug 2023 17 hours 28 minutes
Microsoft 365 Security, Compliance, and Identity Administration
Microsoft 365 Security, Compliance, and Identity Administration
The Microsoft 365 Security, Compliance, and Identity Administration is a comprehensive guide that helps you employ Microsoft 365's robust suite of features and empowers you to optimize your administrative tasks.
Aug 2023 21 hours 0 minutes
Zero Trust Overview and Playbook Introduction
Zero Trust Overview and Playbook Introduction
Get started on Zero Trust with this step-by-step playbook and learn everything you need to know for a successful Zero Trust journey with tailored guidance for every role, covering strategy, operations, architecture, implementation, and measuring success. This book will become an indispensable reference for everyone in your organization.
Oct 2023 8 hours 0 minutes
The Self-Taught Cloud Computing Engineer
The Self-Taught Cloud Computing Engineer
This self-study book helps you master multiple clouds, including AWS, Azure, and GCP, and serves as a roadmap to becoming a certified cloud computing expert. The book will guide you to develop a professional cloud career by helping you build a broad cloud knowledge base, developing hands-on cloud computing skills, and getting cloud certified.
Sep 2023 15 hours 44 minutes
Technology Operating Models for Cloud and Edge
Technology Operating Models for Cloud and Edge
This book will help you build and create ownership of a technology operating model, as well as connect your leadership with engineering and operations, keeping your internal and external customers in mind. It provides practical tips on why, where, and how to make the cloud and edge platform paradigm sing for you, your team, and your organization.
Aug 2023 7 hours 36 minutes
Azure Architecture Explained
Azure Architecture Explained
Azure is the preferred platform to build mission-critical and secure apps. This book provides comprehensive coverage of essential Azure products, services, and solutions vital for every solution architect's success. Elevate your knowledge and master the critical components of Azure to excel in your role with Azure Architecture Explained.
Sep 2023 14 hours 52 minutes
Pentesting Active Directory and Windows-based Infrastructure
Pentesting Active Directory and Windows-based Infrastructure
This practical guide helps you explore the pentesting of Microsoft infrastructure in detail, and enhances your offensive skillset by showing you the different ways to perform security assessment. This book will help blue teamers and IT engineers get up to speed with possible security issues they may encounter in their Windows environments.
Nov 2023 12 hours 0 minutes
Practical Ansible
Practical Ansible
In Practical Ansible, you'll work with the latest release of Ansible and learn to solve complex issues quickly with the help of task-oriented scenarios. You'll start by installing and configuring Ansible to automate monotonous and repetitive IT tasks and get to grips with concepts such as playbooks, inventories, plugins, collections, and network modules.
Sep 2023 14 hours 0 minutes
Windows 11 for Enterprise Administrators
Windows 11 for Enterprise Administrators
Microsoft’s launch of Windows 11 is a step toward satisfying the enterprise administrator’s needs for better management and enhanced user experience customization. This book provides the enterprise administrator with the knowledge needed to fully utilize the advanced feature set of Windows 11 Enterprise.
Oct 2023 9 hours 32 minutes
The Linux DevOps Handbook
The Linux DevOps Handbook
This book is for software and IT professionals seeking knowledge on Linux systems and DevOps practices. This book will provide you with guidance and tools to learn and gain proficiency in managing Linux-based infrastructures and knowledge of DevOps.
Nov 2023 14 hours 16 minutes
Right arrow icon