Apache Hadoop | Implementation

-
Starting with Apache Hadoop

System analysis and Design

This article explains how the system is analyzed to carry out the work for the proposed system. System analysis is the process of gathering and interpreting facts, diagnosing problems, and using the facts to improve the system. System analysis does more than just solve the current problem especially when there is no such system exists that is going to be developed. The future needs of the business and the changes required to meet the needs are analyzed. Once the decision is made, the plan is developed to implement the recommendations. The plan includes all system design features, such as new data capture needs(storage system), operating systems, equipment, and personal needs. The system design is like a blueprint: it specifies all the features that are to be in the finished product.

Class diagram

The class diagram is static. It represents the static view of an application. It describes the structure of a system by showing the system's classes, their attributes, and the relationships among the classes. A class diagram is not only used for visualizing, describing, and documenting different aspects of a system but also for constructing an executable code of the software application. The class diagram describes the attributes and operations of a class and also the constraints imposed on the system. Class diagrams are widely used in the modeling of object-oriented systems because they are the only UML diagrams that can be mapped directly with object-oriented languages. The class diagram shows a collection of classes, interfaces, associations, collaborations, and constraints. It is also known as a structural diagram. The purpose of the class diagram is to model the static view of an application, The class diagrams are the only diagrams that can be directly mapped with object-oriented languages and are thus widely sued at the time of construction. So, the purpose of the class diagram can be summarized as :
  • Analysis and design of the static view of an application
  • Describes the responsibilities of a system
  • The base for component diagrams
  • Forward and reverse engineering

Dynamic UML Diagram:

Dynamic diagrams show the dynamic behavior of the objects in a system, which can be described as a series of changes to the system over time. Dynamic diagrams emphasize what must happen in the system being modeled at a given instant in time Since dynamic diagrams illustrate the behavior of a system, they are used extensively used to describe the functionality of software systems. UML state machine diagram: describes the states and state transitions of the system. UML Interaction diagram: describes the dynamic interaction of the different elements of the system. Activity diagram: describes the business and operational step-by-step workflows of components in a system. An activity diagram shows the overall flow of control

Component Diagram

Component diagrams are used to model the physical aspects of a system. Physical elements like executables, libraries, files, documents, etc. reside in a system. A single component diagram cannot represent the entire system but the collection of diagrams is used to represent the whole. So the purpose of the component diagram can be summarized as follows:
  • Visualize the components of a system
  • Construct executable by using forward and reverse engineering
  • Describe the organization and relationships

Deployment diagram

Deployment diagrams are used to visualize the topology of a system where the software components are deployed. So the deployment diagrams are used to describe the static deployment view of the system. Deployment diagrams consist of nodes and their relationships

Technology used

The project can be implemented in any of the following ways:-
  1. Hadoop on a single com[puter
  2. Hardware requirement: 64-bit architecture, minimum 4GB RAM
  3. OPerating system Ubuntu 12.04.4
  4. Software: Hadoop-core0.20.2, Java, VMWare player, Eclipse kepler
  1. Hadoop on a cluster of computers:
  2. Hardware requirement: 64-bit architecture, minimum 4GB RAM
  3. Operatibg system Ubuntuu 12.04.4
  4. Software:Hadoop-core0.20.2, Java, VMWare player, eclipse kepler

Required softwares

Required software for Linux and windows:
  1. Java 1.7.0_51, preferably from Sun, must be installed
  2. HBase 0.90.2 and zookeeper 3.3.6 must be installed
  3. ssh must be installed and sshd must be running using the Hadoop scripts that manage remote Hadoop daemons

Comments

Post a Comment

Popular posts from this blog

XPath for HTML markup

-
XPath Web Scraping Web Scraping is a technique to traverse the DOM (Document Object Model) of an HTML or an HTTP web page Web Scraping is achieved due to XPath XPath nodes: There are seven kinds of nodes Element - It represents any HTML 5 element. For example, <strong></strong> Attribute - It represents any one attribute of any HTML 5 element in the document object model Text - It represents the text between the opening HTML 5 tag and a closing HTML 5 tag Namespace - It represents the pseudo selector of an HTML 5 element Processing-Instructions Comment - It represents any HTML 5 comment It represents the topmost element of the tree is called the root element. For example, the root element for any HTNL 5 document is HTML The first XPath node is \ which Suppose, I have an HTML5 code snippet as follows: Gaurav Shirodkar Then to ...
Read more

Apache Hadoop | Running MapReduce Jobs

-
Image
Apache Hadoop | Running MapReduce Jobs After setting up your environment and running the HDFS and YARN daemons, we can start working on running MapReduce jobs on our local machine. We need to compile our code, produce a JAR file, move our inputs, and run a MapReduce job on Hadoop. Step 1 - Configure extra environment variables As a preface, it is best to setup some extra environment variables to make running jobs from the CLI quicker and easier. You can name these environment variables anything you want, but we will name them HADOOP_CP and HDFS_LOC to not potentially conlict with other environment variables. Open the Start Menu and type in 'environment' and press enter. A new window with System Properties should open up. Click the Environment Variables button near the bottom right. HADOOP_CP environment variable This is used to compile your Java files. The backticks (eg. `some command here`) do not work on Win...
Read more

Laravel | PHP | Basics | Part 2

-
Image
Learning Laravel through Screenshots Caching Laravel by default supports multiple cache stores like memcached , redis ,etc. To install redis as a cache store, you should use composer install predis/predis predis stands for PHP redis The default port number is 6379 Every Cache store requires a prefix which is stored in the ENV file or in the config/cache.php Following are the cache stores supported by laravel Blade Templates Blade is a templating engine for views in laravel. All the files which are of type blade.php are having the extension .blade.php They are stored in the resources/views directory YOu can create suub-folders in the reosurces/views directory like layouts for all the common UI components lke header , footer, sidebar , navbar, menubar, serach bar, etc. To include the parenet ...
Read more