Web Technology Tutorials

Apache Hadoop | Running MapReduce Jobs After setting up your environment and running the HDFS and YARN daemons, we can start working on running MapReduce jobs on our local machine. We need to compile our code, produce a JAR file, move our inputs, and run a MapReduce job on Hadoop. Step 1 - Configure extra environment variables As a preface, it is best to setup some extra environment variables to make running jobs from the CLI quicker and easier. You can name these environment variables anything you want, but we will name them HADOOP_CP and HDFS_LOC to not potentially conlict with other environment variables. Open the Start Menu and type in 'environment' and press enter. A new window with System Properties should open up. Click the Environment Variables button near the bottom right. HADOOP_CP environment variable This is used to compile your Java files. The backticks (eg. `some command here`) do not work on Win...

Apache Hadoop Prerequisites and Installation The following files are needed for any Hadoop project: Please follow the steps given in the following link for seting up Hadoop on Windows 10 machines How to set up Hadoop on Windows 10 hadoop-hdfs-3.3.4.jar (Java Archive File) zookeeper-3.6.4.jar (Java Archive File) log4j-1.2-api-2.19.0.jar (Java Archive File) hadoop-mapreduce-client-core-3.3.4.jar (Java Archive File) hbase-0.92.1.jar (Java Archive File) hadoop-common-3.3.4.jar (Java Archive File) AN IDE(Integrated DEvelopment Environment) like Eclipse Click on the perspective menu in the menu bar and select Mapreduce as the perespective After installing successfully Hadoop, we have a directory structure as given below in the hadoop-3.2.1 directory Eclipse IDE Setup and initialization Configuration The binary executables to start the name node, data nodes, mapreduce sites,...

Apache Hadoop | Use Case | Association Rule Mapping | Part 1 The classes required to run a Hadoop application: org.apache.zookeeper.*; org.apache.log4j.*; org.apache.hadoop.conf.Configuration; org.apache.hadoop.fs.Path; org.apache.hadoop.hbase.*; org.apache.hadoop.hbase.HBaseConfiguration; org.apache.hadoop.hbase.client.HTable; org.apache.hadoop.hbase.client.Put; org.apache.hadoop.hbase.client.Get; org.apache.hadoop.hbase.util.Bytes; org.apache.hadoop.hbase.client.Result; org.apache.hadoop.hbase.client.ResultScanner; org.apache.hadoop.hbase.client.Scan; org.apache.hadoop.io.IntWritable; org.apache.hadoop.io.LongWritable; org.apache.hadoop.io.ObjectWritable; org.apache.hadoop.io.Text; org.apache.hadoop.mapreduce.Job; org.apache.hadoop.mapreduce.Mapper; org.apache.hadoop.mapreduce.Reducer; org.apache.hadoop.mapreduce.lib.input.FileInputFormat; org.apache.hadoop.mapreduce.lib.input.TextInputFormat; org.apache.hadoop.mapreduce.lib.o...

Starting with Apache Hadoop System analysis and Design This article explains how the system is analyzed to carry out the work for the proposed system. System analysis is the process of gathering and interpreting facts, diagnosing problems, and using the facts to improve the system. System analysis does more than just solve the current problem especially when there is no such system exists that is going to be developed. The future needs of the business and the changes required to meet the needs are analyzed. Once the decision is made, the plan is developed to implement the recommendations. The plan includes all system design features, such as new data capture needs(storage system), operating systems, equipment, and personal needs. The system design is like a blueprint: it specifies all the features that are to be in the finished product. Class diagram The class diagram is static. It represents the static view of an application. It...

Starting with Apache Hadoop In Hadoop, a single master is managing many slaves The master node consists of a JobTracker , Tasktracker , NameNode , and DataNode . A slave or worker node acts as both DataNode and TaskTracker though it is possible to have data-only worker node, and compute-only workerNodes. NameNode holds the file system metadata. The files are broken up and spread over the DataNode and JobTracker schedules and the manager's job. The TaskTracker executes the individual map and reduced function. If a machine fails, Hadoop continues to operate the cluster by shifting work to the remaining machines. The input file, which resides on a distributed file system throughout the cluster, is split into even-sized chunks replicated for fault tolerance. Haddopp divides each map to reduce jobs into a set of tasks. Each chunk of input is processed by a map task, which outputs a list of key-value pairs. In Hadoop, the shuffle phase o...

Handling of Big Data on the Internet When we handle text files that are about 4000-5000 lines long, it is difficult, time-consuming, Input/Output overhead, non-scalable, hardware fault, unnecessary repetition of code, loss of memory space, difficult to process errors and prevent error propagation, etc. The term "Big Data" applies to the above data. Suppose I have a file that is in the English language and is about 4000-5000 lines long with an average of 100 characters per line with a few anomalies as blank lines Heuristics The heuristics we are going to sue are as follows: Articles (a, an, the) Prepositions (of, at, about, around, besides, aside, above, over) Conjunctions (and, between, or, because, hence, since, although, though, not only, but also, but, so, therefore) Adverbs ( adverbs are easy to recognize as they mostly have "ly" as their suffix Pronouns (I, he, we, our, their, he, she, it) ...

Eloquent ORM Eloquent ORM: ORM stands for Ovject Relation Mapping. It converts an object to a database table row. To query the database, Eloquent has a lot of methods For example, we have a school database and inside the school database, we have a table of seventh standard The object contains the following properties. Each of the property is mapped to the column of the database If your model class is User, you should use User::all() to return all the tuples iof the database The where clause: Flight::where('active',1)->orderBy('name')->take(10)->get(); To get the first row of the row result set given by the where clause:- $flight = Flight::where('number','1000')->first(); $flight->number = '1000'; $flight->refresh(); echo $flight->number; foreach($flights as $flight) { echo $flight->name; } Flight::chunk(200, function(Collection $flights) { for...

Learning Laravel through Screenshots Caching Laravel by default supports multiple cache stores like memcached , redis ,etc. To install redis as a cache store, you should use composer install predis/predis predis stands for PHP redis The default port number is 6379 Every Cache store requires a prefix which is stored in the ENV file or in the config/cache.php Following are the cache stores supported by laravel Blade Templates Blade is a templating engine for views in laravel. All the files which are of type blade.php are having the extension .blade.php They are stored in the resources/views directory YOu can create suub-folders in the reosurces/views directory like layouts for all the common UI components lke header , footer, sidebar , navbar, menubar, serach bar, etc. To include the parenet ...

Laravel | PHP | Basics In this post, we will see the basics of laravel Laravel is a frame work on top of PHP (PHP Hypertext Preprocessor) for web artisans. When you download and install composer which is the dependency manager for PHP, you need to run the command composer install Getting started How to create a project in laravel: composer create-project laravel/laravel example-app OR composer global require laravel/installer laravel new example-app The config folder The entry point is in the config folder. In the config/database.php file, the "connection" key contains all the database drivers configuration available or provided by the laravel framework. The supported datavase drivers are sqlite mysql pgsql sqlsrv redis "mysql" => [ "driver" => "mysql", ...

Social Networking sites (SNS) Post no. 100 Gaurav Vinayak Shirodkar Facebook :- Gaurav Shirodkar Twitter :- Gaurav Shirodkar Instagram :- Gaurav Shirodkar Mail :- gaurav111shirodkar@gmail.com, gaurav.cst123@gmail.com LinkedIn :- Gaurav Shirodkar / Whatsapp Contact Number: +91 9869226626 GitHub: Gaurav Shirodkar

Minified files Minified HTML5, CSS3 and Javascript files can save upto 50% of your memory space. So, it is necessary to minify files for faster loading of web pages In this tutorial, we will see some of the important steps to minify the files. Avoid comments Avoid indentations As javascript does not require line-ending tokens, do not use semi colons for .js files Keep variable names as small as possible Declare all variables with their initial values in one line Avoid unneccessary whitespaces Practical Examples: Instead of border:none use border:0 Instead margin:0px use margin:0 Instead of padding:0px use padding:0 Try to create generic class for a set of sub classes which have common properties and values Try to use shorthand properties as far as possible For example, instead of div { margin-top:10px; margin-bottom:15px; margin-left:20px; margin-right:25px...

Creating a trivia in angular.js Create an object with an array of qa objects which stores the question as a string, The 4 options as a string and an answer as the String For e.g., var mdata = { "qa" : [ { "question" : "The compare() method is declared in", "options" : [ "Comparable", "Comparator", "Compare", "Compares" ], "answer" : "Compares" }, { "question" : "Which of the following methods returns List from an array?", "options" : [ "Arrays.asList(Object[] array)", "Arrays.list(Object[] array)", "Arrays.toList(Object[] array)", "Arrays.createList(Object[] array)" ], "answer" : "Arrays.createList(Object[] array)" }, { "questio...

Making a jQuery plugin In this tutorial, we will see how to make a jquery plugin. First step is to create a self executing function. Inside this function we will create our own functions e.g. if we want to create a function then we should pass the jquery object to the self executing function to define a function just add property of the $.fn object e.g if the function name is test the definition should be $.fn.test = function(<optional_parameters>) {}; To call the function, just use $("selector").test(); In this tutorial, we will see how to make a chess borad (8 * 8), First step is to print the board and then place the correct animals in the correct positions. 1.$.fn.board - In this method, we make the design of the board using javascript ECMA script 6 2.$.fn.initilaize - In this method, we place the animals in their appropriate positions <html> <head> <script src="https://ajax....

Previous January Next Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 2 3 4 5 6 ...