Apache Hadoop | Implementation

-
Starting with Apache Hadoop

System analysis and Design

This article explains how the system is analyzed to carry out the work for the proposed system. System analysis is the process of gathering and interpreting facts, diagnosing problems, and using the facts to improve the system. System analysis does more than just solve the current problem especially when there is no such system exists that is going to be developed. The future needs of the business and the changes required to meet the needs are analyzed. Once the decision is made, the plan is developed to implement the recommendations. The plan includes all system design features, such as new data capture needs(storage system), operating systems, equipment, and personal needs. The system design is like a blueprint: it specifies all the features that are to be in the finished product.

Class diagram

The class diagram is static. It represents the static view of an application. It describes the structure of a system by showing the system's classes, their attributes, and the relationships among the classes. A class diagram is not only used for visualizing, describing, and documenting different aspects of a system but also for constructing an executable code of the software application. The class diagram describes the attributes and operations of a class and also the constraints imposed on the system. Class diagrams are widely used in the modeling of object-oriented systems because they are the only UML diagrams that can be mapped directly with object-oriented languages. The class diagram shows a collection of classes, interfaces, associations, collaborations, and constraints. It is also known as a structural diagram. The purpose of the class diagram is to model the static view of an application, The class diagrams are the only diagrams that can be directly mapped with object-oriented languages and are thus widely sued at the time of construction. So, the purpose of the class diagram can be summarized as :
  • Analysis and design of the static view of an application
  • Describes the responsibilities of a system
  • The base for component diagrams
  • Forward and reverse engineering

Dynamic UML Diagram:

Dynamic diagrams show the dynamic behavior of the objects in a system, which can be described as a series of changes to the system over time. Dynamic diagrams emphasize what must happen in the system being modeled at a given instant in time Since dynamic diagrams illustrate the behavior of a system, they are used extensively used to describe the functionality of software systems. UML state machine diagram: describes the states and state transitions of the system. UML Interaction diagram: describes the dynamic interaction of the different elements of the system. Activity diagram: describes the business and operational step-by-step workflows of components in a system. An activity diagram shows the overall flow of control

Component Diagram

Component diagrams are used to model the physical aspects of a system. Physical elements like executables, libraries, files, documents, etc. reside in a system. A single component diagram cannot represent the entire system but the collection of diagrams is used to represent the whole. So the purpose of the component diagram can be summarized as follows:
  • Visualize the components of a system
  • Construct executable by using forward and reverse engineering
  • Describe the organization and relationships

Deployment diagram

Deployment diagrams are used to visualize the topology of a system where the software components are deployed. So the deployment diagrams are used to describe the static deployment view of the system. Deployment diagrams consist of nodes and their relationships

Technology used

The project can be implemented in any of the following ways:-
  1. Hadoop on a single com[puter
  2. Hardware requirement: 64-bit architecture, minimum 4GB RAM
  3. OPerating system Ubuntu 12.04.4
  4. Software: Hadoop-core0.20.2, Java, VMWare player, Eclipse kepler
  1. Hadoop on a cluster of computers:
  2. Hardware requirement: 64-bit architecture, minimum 4GB RAM
  3. Operatibg system Ubuntuu 12.04.4
  4. Software:Hadoop-core0.20.2, Java, VMWare player, eclipse kepler

Required softwares

Required software for Linux and windows:
  1. Java 1.7.0_51, preferably from Sun, must be installed
  2. HBase 0.90.2 and zookeeper 3.3.6 must be installed
  3. ssh must be installed and sshd must be running using the Hadoop scripts that manage remote Hadoop daemons

Comments

Post a Comment

Popular posts from this blog

Laravel | PHP | Basics | Part 2

-
Image
Learning Laravel through Screenshots Caching Laravel by default supports multiple cache stores like memcached , redis ,etc. To install redis as a cache store, you should use composer install predis/predis predis stands for PHP redis The default port number is 6379 Every Cache store requires a prefix which is stored in the ENV file or in the config/cache.php Following are the cache stores supported by laravel Blade Templates Blade is a templating engine for views in laravel. All the files which are of type blade.php are having the extension .blade.php They are stored in the resources/views directory YOu can create suub-folders in the reosurces/views directory like layouts for all the common UI components lke header , footer, sidebar , navbar, menubar, serach bar, etc. To include the parenet ...
Read more

Parallel Database design, query processing

-
Image
Parallel Database design, query processing, and case study In this post, we are going to analyze the concepts of Parallel Database design, query processing, and case study. Parallel database design Hardware Architecture Data partitioning Query processing Hardware Architecture Shared Memory Shared Disk> Shared Nothing Data Partitioning Partitioning a relation involves distributing its tuples over several disks Three Kinds – Range Partitioning Round-robin Partitioning Hashing Partitioning Range Partitioning is good for Ideal for point and range queries on the partitioning attribute Hash partitioning Ideal for point queries based on the partitioning attribute Ideal for sequential sca...
Read more

Website Hacking Techniques and their workarounds

-
Image
Website Hacking Techniques and their workarounds Cross-site Request Forgery: Sometimes, hackers make an API call to another website. It is called fake identity or false authorization To prevent CSRF, we add an input element with type "hidden" and a _token to the value of a 32 character token that is sent by the server to the client in the first response. After this, the client will send this token in the hidden field every time the client makes HTTP requests to the server Hence, the server matches the token sent by the client to the token stored by the server at every clientHTTP request, and if it matches, it means , that the cliengt is a genuine client. For some pages in a website, We don't want this authorization to take place, hence we disable the token field in the HTTP request for that web page. SQL Injection:- Sometimes, in a form, which has input fields, the user e...
Read more