The short answer: Hadoop interviews check that you understand how Hadoop stores and processes data across a cluster: HDFS (the NameNode and DataNodes, blocks, replication and rack awareness, and how failures are handled), YARN (how resources are allocated to applications), and MapReduce (map, shuffle and sort, reduce, combiners). Developer roles add data formats, Hive and Spark on YARN, and performance; administrator roles add high availability, capacity, security and troubleshooting. Many teams now run Spark on cloud storage, so expect “When would you still use Hadoop?” too.
HDFS questions
“Describe the HDFS architecture.”
The Apache documentation says “HDFS has a master/slave architecture”: a single NameNode manages the file system namespace and regulates client access to files, while DataNodes, usually one per node, manage the storage attached to their nodes. Files are split into blocks stored on DataNodes; the NameNode knows which blocks make up each file and where they are, but data itself flows directly between clients and DataNodes.
“What is the block size, and why is it so large?”
The docs describe a typical HDFS block size of 128 MB. Large blocks reduce the amount of metadata the NameNode keeps and favor high-throughput sequential reads over low-latency random access. HDFS is designed for write-once-read-many access.
“How does replication work?”
Each block is replicated for fault tolerance; the replication factor is configurable per file and stored by the NameNode, and three is the common case. For a replication factor of three, the documentation describes the placement policy: one replica on the local machine (if the writer is on a DataNode), another on a node in a different rack, and the last on a different node in that same remote rack, which cuts inter-rack write traffic while surviving the loss of a whole rack.
“How does the NameNode know a DataNode has failed?”
It periodically receives a Heartbeat and a Blockreport from each DataNode. A Heartbeat means the DataNode is working; a Blockreport lists all its blocks. When heartbeats stop, the NameNode marks the DataNode dead and re-replicates its blocks elsewhere to restore the replication factor.
“What is safemode?”
At startup, the NameNode enters safemode and doesn’t replicate blocks. It waits for DataNodes to report their blocks; once a configurable percentage of blocks are safely replicated, it leaves safemode and replicates any blocks that are still under-replicated.
“What are the FsImage and EditLog?”
The EditLog is a transaction log of every change to file system metadata; the FsImage stores the whole namespace. At startup, or when a checkpoint is triggered, the NameNode applies the EditLog to the FsImage, saves a new FsImage and truncates the old log. This is a checkpoint. In production, checkpointing is handled by a Secondary or Standby NameNode so the EditLog doesn’t grow without limit.
“How do you make the NameNode highly available?”
Run an active and a standby NameNode sharing the edit log (commonly through JournalNodes), with automatic failover coordinated by ZooKeeper. Without high availability, the NameNode is a single point of failure.
“What is the small files problem?”
Every file, directory and block is an object in NameNode memory, so millions of tiny files exhaust NameNode memory and make jobs slow (one task per small file). Fixes: combine files before loading, use container formats such as SequenceFiles or columnar formats like Parquet and ORC, or compact data periodically.
YARN questions
“What does YARN do?”
According to the Apache docs, “the fundamental idea of YARN is to split up the functionalities of resource management and job scheduling/monitoring into separate daemons”: a global ResourceManager that arbitrates resources among all applications, a NodeManager on each machine that runs and monitors containers, and a per-application ApplicationMaster that negotiates resources and tracks its tasks.
“What happens when you submit a job to YARN?”
The client submits to the ResourceManager, which starts the ApplicationMaster in a container. The ApplicationMaster requests containers for its tasks, NodeManagers launch them, and the ApplicationMaster monitors progress until the job finishes.
“What schedulers does YARN offer?”
The Capacity Scheduler (queues with guaranteed capacity, common in multi-tenant clusters) and the Fair Scheduler (resources shared fairly across running applications).
MapReduce questions
“Explain the phases of a MapReduce job.”
Input is split and processed by mappers, which emit key-value pairs. The Hadoop tutorial describes three reducer phases: shuffle (fetching the relevant partition of every mapper’s output), sort (grouping inputs by key) and reduce. Shuffle and sort happen at the same time.
“What is a combiner?”
A local, mini-reduce that runs on mapper output before the shuffle to reduce data sent over the network, for example summing counts per word on each mapper. It must be safe to apply zero, one or many times, so it works for sums and counts but not, directly, for averages.
“What is a partitioner?”
It decides which reducer each key goes to (by default a hash of the key). A custom partitioner can fix skew or group related keys.
Ecosystem and data questions
- “How do Hive and Spark relate to Hadoop?” Hive provides SQL over data in HDFS; Spark is a general processing engine that can run on YARN and read from HDFS, usually much faster than MapReduce for iterative work. See our Spark interview questions.
- “Which file formats would you use?” Columnar formats (Parquet, ORC) for analytics, because they compress well and let engines read only needed columns; Avro for row-based data exchange with schema evolution.
- “Is Hadoop still relevant?” Many teams have moved storage to cloud object stores and processing to Spark or managed services, but HDFS and YARN still run large on-premises clusters, and the concepts (distribution, replication, data locality) carry over to every modern platform.
Scenario questions
- “A job is much slower than usual.” Check the ResourceManager UI for queued or pending containers, skewed tasks that run far longer than others, input size changes, small files, and failing nodes.
- “A DataNode disk failed.” HDFS detects missing replicas and re-replicates them; replace the disk and check under-replicated block counts until they recover.
- “The NameNode is running out of memory.” Look for small files and excess metadata; compact or archive data and increase NameNode heap as a short-term fix.
- “How would you migrate a Hadoop workload to the cloud?” Inventory data and jobs, move data to object storage, rerun jobs on managed Spark or Hive, validate outputs side by side, then cut over pipeline by pipeline.
Related guides: data engineer interview questions, Kafka interview questions and SSIS interview questions.
Frequently asked questions
What are the most common Hadoop interview questions?
What is the default block size and replication factor in HDFS?
What is the difference between the NameNode and a DataNode?
What are the components of YARN?
What is a combiner in MapReduce?
Sources
- Apache Hadoop, HDFS Architecture. Read on 1 October 2026.
- Apache Hadoop, Apache Hadoop YARN. Read on 1 October 2026.
- Apache Hadoop, MapReduce Tutorial. Read on 1 October 2026.
