Labels (Tags)

Labels in this blog The list of Labels is:-

Comments

Post a Comment

Popular posts from this blog

Parallel Database design, query processing

Image
Parallel Database design, query processing, and case study In this post, we are going to analyze the concepts of Parallel Database design, query processing, and case study. Parallel database design Hardware Architecture Data partitioning Query processing Hardware Architecture Shared Memory Shared Disk> Shared Nothing Data Partitioning Partitioning a relation involves distributing its tuples over several disks Three Kinds – Range Partitioning Round-robin Partitioning Hashing Partitioning Range Partitioning is good for Ideal for point and range queries on the partitioning attribute Hash partitioning Ideal for point queries based on the partitioning attribute Ideal for sequential sca...
Read more

Apache Hadoop Prerequisites and Installation

Image
Apache Hadoop Prerequisites and Installation The following files are needed for any Hadoop project: Please follow the steps given in the following link for seting up Hadoop on Windows 10 machines How to set up Hadoop on Windows 10 hadoop-hdfs-3.3.4.jar (Java Archive File) zookeeper-3.6.4.jar (Java Archive File) log4j-1.2-api-2.19.0.jar (Java Archive File) hadoop-mapreduce-client-core-3.3.4.jar (Java Archive File) hbase-0.92.1.jar (Java Archive File) hadoop-common-3.3.4.jar (Java Archive File) AN IDE(Integrated DEvelopment Environment) like Eclipse Click on the perspective menu in the menu bar and select Mapreduce as the perespective After installing successfully Hadoop, we have a directory structure as given below in the hadoop-3.2.1 directory Eclipse IDE Setup and initialization Configuration The binary executables to start the name node, data nodes, mapreduce sites,...
Read more

Starting with Apache Hadoop

Image
Starting with Apache Hadoop In Hadoop, a single master is managing many slaves The master node consists of a JobTracker , Tasktracker , NameNode , and DataNode . A slave or worker node acts as both DataNode and TaskTracker though it is possible to have data-only worker node, and compute-only workerNodes. NameNode holds the file system metadata. The files are broken up and spread over the DataNode and JobTracker schedules and the manager's job. The TaskTracker executes the individual map and reduced function. If a machine fails, Hadoop continues to operate the cluster by shifting work to the remaining machines. The input file, which resides on a distributed file system throughout the cluster, is split into even-sized chunks replicated for fault tolerance. Haddopp divides each map to reduce jobs into a set of tasks. Each chunk of input is processed by a map task, which outputs a list of key-value pairs. In Hadoop, the shuffle phase o...
Read more