Skip to main content

StandAlone Apache Storm Installation in Ubuntu 14.04 LTS

Deploying Apache Storm 


1) If Java 7 is not present, please install it.

2) Zookeeper Installation

Download zookeeper-3.4.6 from Apache site
      $ tar -xvf zookeeper-3.4.6.tar.gz
      $ cd zookeeper-3.4.6/
      $ cp conf/zoo_sample.cfg conf/zoo.cfg
      $ bin/zkServer.sh start

3) Storm Cluster Installation

Download the storm tarball from official Apache Mirror. 
Untar it.

      $ tar xzvf apache-storm-0.9.2-incubating.tar.gz

Move to new directory.

      $ sudo cp -R apache-storm-0.9.2-incubating /usr/lib/

Go to /usr/lib/apache-storm-0.9.2-incubating/ and configure storm.yaml present in conf folder, add the followuing line, this folder must have write permissions too.

storm.zookeeper.servers:
    - "localhost"
storm.zookeeper.port: 2181
nimbus.host: "localhost"
storm.local.dir: "/var/stormtmp"     
java.library.path: "/usr/lib/jvm/java-7-openjdk-amd64"
supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703
worker.childopts: "-Xmx768m"
nimbus.childopts: "-Xmx512m"

supervisor.childopts: "-Xmx256m"

4) Installing Native Dependencies

Before installing ZeroMQ and JZMQ make sure following dependencies are installed on Ubuntu system

libtool
autoconf
pkg-config

In any case to be sure just run following command on terminal

$ sudo apt-get install libtool autoconf pkg-config

(i) ZeroMQ - Download the ZeroMQ tarball from official ZeroMQ site

    $ wget http://download.zeromq.org/zeromq-4.0.4.tar.gz
    $ Untar it tar –xzf zeromq-2.1.7.tar.gz
    $ cd zeromq-2.1.7
    $ ./configure
    $ make
    $ sudo make install

(ii) JZMQ - Clone JZMQ from official git repository

    $ git clone https://github.com/zeromq/jzmq.git
    $ cd jzmq
    $ ./autogen.sh
    $ ./configure
    $ make
    $ sudo make install

5) Starting Storm Cluster and submitting the topology

Then start Storm Cluster by starting master(nimbus) and worker nodes(supervisor).
Go to the ‘bin’ directory of the Storm installation and execute following command. [separate command line window]

    $ sudo ./storm nimbus

To start worker i.e. supervisor go to the ‘bin’ directory of the Storm installation and execute following command. [separate command line window]
    
    $ sudo ./storm supervisor    

6) Upload topology using Storm Client

To upload topology to Storm Cluster go to the ‘bin’ directory of the Storm installation and execute following command. [separate command line window]

    $ sudo ./storm jar <path-to-topology-jar> <class-with-the-main> <arg1> <arg2> ...<argN>

A) <path-to-topology-jar>: is the complete path to the complied jar where your topology code and all your libraries are.
B) <class-with-the-main>: will be the class in jar file having main method where the StormSubmitter is executed.
C) <arg1> <arg2> <argN>:  rest of the arguments will be the parameter that will be received by main method.
Labels:

Comments

Post a Comment

Popular posts from this blog

Multi Node Apache Kafka 0.9 in Linux

Manually Deploying Apache Kafka Please Install Java version 8 before proceeding the Kafka installation. 1) Edit the  /etc/hostname in each  node.         $  vi /etc/hostname                    kafka1 or kafka2 or kafka3 2) Check hostname.         $ hostname 3)  Confirm everything went right.      $ sudo hostname -F /etc/hostname 4) Now edit /etc/hosts to point one node to other nodes and do the same in each node.      $ nano /etc/hosts          127.0.0.1       localhost 127.0.1.1       ubuntu <IP of node1 >   kafka1 <IP of node2>    kafka2 <IP of node3>    kafka3 # The following lines are desirable for IPv6 capable hosts ::1     localhost ip6-localhost ip6-loopba...
Post a Comment
Read more

Google2Ubuntu speech recognition tool for linux Ubuntu 13.10

Google2Ubuntu is a tool that lets you control your computer using voice commands via the Google speech recognition API. Install Google2Ubuntu Google2Ubuntu is available in a PPA for all supported Ubuntu versions. Add the PPA and install it using the following commands: sudo add-apt-repository ppa:benoitfra/google2ubuntu sudo apt-get update sudo apt-get install google2ubuntu I tested the application under Ubuntu 13.10 and 14.04 so I'm not sure if it works properly with older Ubuntu versions. How to configure and use Google2Ubuntu 1. Once installed, you need to set up a keyboard shortcut for triggering Google2Ubuntu. When you use this keyboard shortcut, the Google2Ubuntu speech recognition will be activated, listening for your command (a sound and a notification will be displayed, telling you when to speak). Let's add the keyboard shortcut: - in Unity/GNOME , open System Settings > Keyboard > Shortcuts , then click on Custom Shortcuts on the ...
Post a Comment
Read more