Skip to main content

StandAlone Apache Storm Installation in Ubuntu 14.04 LTS

Deploying Apache Storm 


1) If Java 7 is not present, please install it.

2) Zookeeper Installation

Download zookeeper-3.4.6 from Apache site
      $ tar -xvf zookeeper-3.4.6.tar.gz
      $ cd zookeeper-3.4.6/
      $ cp conf/zoo_sample.cfg conf/zoo.cfg
      $ bin/zkServer.sh start

3) Storm Cluster Installation

Download the storm tarball from official Apache Mirror. 
Untar it.

      $ tar xzvf apache-storm-0.9.2-incubating.tar.gz

Move to new directory.

      $ sudo cp -R apache-storm-0.9.2-incubating /usr/lib/

Go to /usr/lib/apache-storm-0.9.2-incubating/ and configure storm.yaml present in conf folder, add the followuing line, this folder must have write permissions too.

storm.zookeeper.servers:
    - "localhost"
storm.zookeeper.port: 2181
nimbus.host: "localhost"
storm.local.dir: "/var/stormtmp"     
java.library.path: "/usr/lib/jvm/java-7-openjdk-amd64"
supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703
worker.childopts: "-Xmx768m"
nimbus.childopts: "-Xmx512m"

supervisor.childopts: "-Xmx256m"

4) Installing Native Dependencies

Before installing ZeroMQ and JZMQ make sure following dependencies are installed on Ubuntu system

libtool
autoconf
pkg-config

In any case to be sure just run following command on terminal

$ sudo apt-get install libtool autoconf pkg-config

(i) ZeroMQ - Download the ZeroMQ tarball from official ZeroMQ site

    $ wget http://download.zeromq.org/zeromq-4.0.4.tar.gz
    $ Untar it tar –xzf zeromq-2.1.7.tar.gz
    $ cd zeromq-2.1.7
    $ ./configure
    $ make
    $ sudo make install

(ii) JZMQ - Clone JZMQ from official git repository

    $ git clone https://github.com/zeromq/jzmq.git
    $ cd jzmq
    $ ./autogen.sh
    $ ./configure
    $ make
    $ sudo make install

5) Starting Storm Cluster and submitting the topology

Then start Storm Cluster by starting master(nimbus) and worker nodes(supervisor).
Go to the ‘bin’ directory of the Storm installation and execute following command. [separate command line window]

    $ sudo ./storm nimbus

To start worker i.e. supervisor go to the ‘bin’ directory of the Storm installation and execute following command. [separate command line window]
    
    $ sudo ./storm supervisor    

6) Upload topology using Storm Client

To upload topology to Storm Cluster go to the ‘bin’ directory of the Storm installation and execute following command. [separate command line window]

    $ sudo ./storm jar <path-to-topology-jar> <class-with-the-main> <arg1> <arg2> ...<argN>

A) <path-to-topology-jar>: is the complete path to the complied jar where your topology code and all your libraries are.
B) <class-with-the-main>: will be the class in jar file having main method where the StormSubmitter is executed.
C) <arg1> <arg2> <argN>:  rest of the arguments will be the parameter that will be received by main method.

Comments

Post a Comment

Popular posts from this blog

Replace GRUB With BURG to Beautify Boot Menu, Install in Ubuntu via PPA

Burg is boot loader forked of GRUB loader. It uses a new object format which allows it to be built in a wider range of OS, including Linux/Windows/OSX/Solaris/FreeBSD, etc. It also has a highly configurable menu system which works in both text and graphic mode. Additional features like stream support and multiple input/output device are also planned for BURG. BURG features superior theming and has a highly configurable menu system which works at boot time in both text and graphical mode. Since BURG is derived from GRUB, its configuration is similar in most respects. The main configuration file is /boot/burg/burg.cfg  and is usually generated automatically. You can create one by yourself or use  grub-customizer  to modify BRUG entries and other stuff. You can even design theme for BURG checkout this  link  for more details about BURG. All below images are from Noobslab Available for Ubuntu 14.04/14.10/12.04/Linux Mint 17/13 To install Burg with...

How to Re-enable Hibernation in Ubuntu 14.04

1) Open the terminal (Ctrl+alt+T) 2) type: sudo gedit /var/lib/polkit-1/localauthority/10-vend ­or.d/com.ubuntu.desktop.pkla 3) Enter your password when requested. 4) Search for the section  [Disable hibernate by default in upower] Identity=unix-user:* Action=org.freedesktop.upower.hibernate ResultActive=no [Disable hibernate by default in logind] Identity=unix-user:* Action=org.freedesktop.login1.hibernate ResultActive=no 5) Change both value of "ResultActive=no" to "ResultActive=yes" 6) Save and close the editor. 7) Log out and then log back in, or reboot the system. You are now good to go!