Thursday, April 17, 2014

Rack awareness and java in ubuntu

best :---
https://developer.yahoo.com/hadoop/tutorial/module2.html

http://ofirm.wordpress.com/2014/01/09/exploring-the-hadoop-network-topology/

 http://fendertech.blogspot.in/2013/07/hadoop-rack-awareness-104.html
------------------------------------------------------------------------------------------------------------
http://www.michael-noll.com/blog/2011/03/28/hadoop-space-quotas-hdfs-block-size-replication-and-small-files/

http://stackoverflow.com/questions/17799535/hdfs-reduced-replication-factor

http://stackoverflow.com/questions/20119320/when-i-store-files-in-hdfs-will-they-be-replicated

http://architects.dzone.com/articles/how-hdfs-does-replication

http://coenraets.org/blog/2012/10/real-time-web-analytics-with-node-js-and-socket-io/

http://bradhedlund.com/2011/09/10/understanding-hadoop-clusters-and-the-network

https://issues.apache.org/jira/browse/HADOOP-692

------------------------------------------------------------------------------------------------------------
autoreconf error in hadoop build--------------
apt-get -y install maven build-essential autoconf automake libtool cmake zlib1g-dev pkg-config libssl-dev
------------------------------------------------------------------------------------------------------------

    Download Java SE 7 JDK for Linux x86 archive. At the time of writing, the file I'm using is jdk-7u21-linux-i586.tar.gz, but the filename will change as updates are released.
    Apparently there is no longer a jvm folder, so create one.

    sudo mkdir /usr/lib/jvm

    Move the archive to the jvm folder

    sudo mv jdk-7u21-linux-i586.tar.gz /usr/lib/jvm/

    Change to the jvm folder and extract the JDK from the archive

    cd /usr/lib/jvm

    sudo tar zxvf jdk-7u21-linux-i586.tar.gz

    Everything will be extracted to a new jdk1.7.0_21 folder and you can delete
the archive file now.
 
sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/java-6.31-oracle/bin/javac" 1
sudo update-alternatives --config javac
 
similarly for java use java

sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/java-6.31-oracle/jre/bin/java" 1
sudo update-alternatives --config java

Double-check the version
    java -version

from----------- http://www.printandweb.ca/2013/04/manually-install-oracle-jdk-7-for.html


http://www.linuxquestions.org/questions/linux-newbie-8/lost-ubuntu-after-installing-opensuse-932336/

Sunday, April 6, 2014

ssh settings in hadoop...........(passwordless login)

on the host you will connect FROM:
generate the public private keys
> ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
copy the public key to every host you will connect TO:
> scp ~/.ssh/id_dsa.pub my_user_id@1.2.3.4:~/.ssh/id_dsa.pub
* this should prompt you for a password
shell into the remote machine
> ssh my_user_id@1.2.3.4
authorize the key by adding it to the list of authorized keys
> cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
log out of the current shell
> exit
test that you can log in with no password

if problems persist plz refer:-
http://allthingshadoop.com/?s=Hadoop+Cluster+Setup%2C+SSH+Key+Authentication

http://ambari.apache.org/1.2.0/installing-hadoop-using-ambari/content/ambari-chap1-5-2.html