Facing problem: ORA-011033: ORACLE initialization or shutdown in progress

SQL> connect sys/manager as sysdba
Connected.
SQL> shutdown immediate;
ORA-01109: database not open


Database dismounted.
ORACLE instance shut down.

SQL> startup
ORACLE instance started.

Total System Global Area 171966464 bytes
Fixed Size 787988 bytes
Variable Size 145750508 bytes
Database Buffers 25165824 bytes
Redo Buffers 262144 bytes
Database mounted.
ORA-01157: cannot identify/lock data file 6 - see DBWR trace file
ORA-01110: data file 6: 'C:\ORADATA\PRODUCT_DAT1.DBF'

 Solution 1:

shutdown immediate; -- if you have trouble with that, run shutdown abort;
startup mount;
alter database datafile 6 offline;
alter database open;

Further if u get this problem:
While making a datafile offline, got below error.

SQL> ALTER database datafile ‘/export/home/oracle/user02.dbf’ offline;
ALTER database datafile ‘/export/home/oracle/user02.dbf’ offline
*
ERROR at line 1:
ORA-01145: offline immediate disallowed unless media recovery enabled

Solution 2:

To make a datafile offline, archive mode need to be enabled.

SQL> archive log list

Database log mode No Archive Mode

Automatic archival Disabled

Archive destination /oracle/app/oracle/product/12.1.0.2/dbhome_1/dbs/arch

Oldest online log sequence 84

Current log sequence 85

 

Below are steps for enabling archive mode:

 

shutdown immediate;

 

startup mount

 

alter database archivelog;

 

alter database open;

 

SQL> archive log list

Database log mode              Archive Mode

Automatic archival             Enabled

Archive destination            /oracle/app/oracle/product/12.1.0.2/dbhome_1/dbs/arch

Oldest online log sequence     84

Next log sequence to archive   85

Current log sequence           85

SQL>

 

Now try to make the datafile offline.

 

SQL> ALTER database datafile '/export/home/oracle/user02.dbf' offline;

 

Database altered.

 

 

Mooshak – Installation on Ubuntu / Linux ( Fully Tested )

How to install Mooshak in Ubuntu / Linux

Mooshak is an online judge for managing programming contests on the Web/online.

It can be extended for new types of contests. Ya you can also customize mooshak for new programming languages.

But Mooshak installation is not that easy. There are so many problems with the installation of mooshak. So in this post I will try to go Step-by-Step to install mooshak on your system .

Note : This installation guide is for Ubuntu / linux platform works similar to Ubuntu only.

Step 1      :     Install Apache & TCL ( Requirements of Mooshak )

 

$ sudo su

$ sudo apt-get install apache2 apache2-suexec

$ sudo apt-get install tcl xsltproc lpr rsync gcc libxml2-utils

Step 2     :     Enable some modules of Apache.

 

$  a2enmod userdir
$  a2enmod suexec

If everything is okay skip to step 2 else if after command 2 ( $  a2enmod suexec ) you get something like this

ERROR: Config file suexec.conf not properly enabled: /etc/apache2/mods-enabled/suexec.conf is a real file, not touching it

then follow this :

Turns out files shouldn't be stored in mods-enabled, they should rather be stored in mods-available. A symlink should then be created in mods-enabled pointing to the file stored in mods-available.

1. First remove the original:
$ mv /etc/apache2/mods-enabled/suexec.conf /etc/apache2/mods-available/

2. Then create the symbolic link:
$ ln -s /etc/apache2/mods-available/suexec.conf /etc/apache2/mods-enabled/suexec.conf
 

Step 3     :     Download Mooshak : Click Here

 

Go to Directory where Mooshak is downloaded and extract it.

tar -xzvf mooshak-1.5.2.tgz

cd mooshak-1.5.2

Step 4     :     Copy Required files to required places

 

tar xvf source.tgz

cd lib/apache/
sudo cp 67* /etc/apache2/mods-enabled/userdir.conf

sudo cp 69* /etc/apache2/mods-enabled/suexec.conf
sudo cp /etc/apache2/mods-enabled/userdir.conf /etc/apache2/mods-available/userdir.conf
sudo cp /etc/apache2/mods-enabled/suexec.conf /etc/apache2/mods-available/suexec.conf

Step 5     :     Make some changes to your configuration file of apache, i.e., htttpd.conf

 

You need to add the following lines

Options +ExecCGI

Step 6     :     Edit the install script

 

Now you need to edit the install script. Now change the following line (line 151/153, probably)

set fd [ open "| lsof" ]

to

set fd [ open "| /usr/bin/lsof" ]

Step 7     :     Install

 

$ cd mooshak-1.5.2
$ sudo su
$ ./install --config-suexec
$ ./install

Step 8     :     Restart Apache Services

$ service apache2 restart

voila you are done.

Now either use http://127.0.0.1/~mooshak or localhost/~mooshak to run Mooshak

Step 9    :     Login

 

Choose any contest 

Default User : admin

Default Password : admin

Install Linux Without Physical Media With PXE

(If you have an existing DHCP server (CENTOs) and you can configure that use this  )

Recently I picked up an older DellPoweredge 2650 server from Craigslist for $50. Even with the age of the device, I couldn’t turn down such a steal. Loaded with two single core 32bit Xeons, a whopping 2GB of RAM, and a few SCSI drives this 2U server was the perfect development server. There was just one issue, it didn’t have an operating system on it. “Ok, not a problem”, I thought “I’ll just put CentOS 7 on a USB stick and install it”. Only to discover that this server doesn’t support USB booting and that RHEL dropped support for 32bit CPUs. Being that this server only has a CD drive, and I don’t have any CD-R/RWs lying around, a PXE install was the only choice for me. In this tutorial, I will show you how to set up a PXE server on CentOS 7.

---

Ingredients

For this you’re going to need a computer running CentOS 7 (or similar) that will act as the DHCP, TFTP, FTP, and PXE server. I used a virtual machine with 8GB of RAM and a dual-core processor. You’re also going to need a client machine that supports PXE boot. A virtual machine can be used to test your PXE server.

Install the prerequisites

We need the following packages to perform our remote installation:

• dnsmasq
• vsftpd
• syslinux
• tftp-server

You can install all of these (or update if they are out-of-date) using:

yum install -y dnsmasq vsftpd syslinux tftp-server

You’re also going to need an ISO of your favourite Linux distribution. This will be the operating system that will be installed on your client. I chose Fedora Server 21 NetInstall. I recommend that you use a NetInstall image as it reduces the space needed on your server and speeds up the PXE install.

Setting up dnsmasq

DNSMASQ will act as the DHCP and PXE Server for us. The first thing we need to do is configure our DHCP options.

Configure a DHCP Server

1. Run ip addr and note down the IP Address of this server (highlighted in red), and the interface name (highlighted in blue)
   

   ip addr shows you basic information about your network interfaces

2. Make a backup of the default dnsmasq configuration file:

   mv /etc/dnsmasq.conf  /etc/dnsmasq.conf.backup

3. Edit /etc/dnsmasq.conf and add the following lines – make sure to edit the parameters to meet your network:

   interface=,lo
   
   # DHCP range-leases
   dhcp-range= ,,,,1h
   
   # PXE
   dhcp-boot=pxelinux.0,pxeserver,
   
   # Gateway
   dhcp-option=3,
   
   # DNS
   dhcp-option=6, 8.8.8.8
   
   # Broadcast Address
   dhcp-option=28,
   

   If you don’t know your broadcast address you can find it using this Online IP Subnet Calculator
4. Keep that file open, we’re going to keep using it in the next step

Configure a PXE Server

In the same file we were editing for our DHCP server, we’re going to add on this configuration options for our PXE server:

pxe-prompt="Press F8 for menu.", 60
pxe-service=x86PC, "Install Fedora 21", pxelinux
enable-tftp
tftp-root=/var/lib/tftpboot

You can change around the pxe-prompt and the pxe-service but the important one is tftp-root To see all of the possible configuration options, check the DNSMASQ Manual.

Set up Syslinux

Syslinux is a group of smaller projects, one of which is PXELINUX which is a minimal Linux distribution meant to be used with PXE Installations. PXELINUX is not going to be installed on your system, rather just loaded into memory and used to install your actual operating system.

1. Copy over syslinux into the tftpboot directory so that it will be copied onto our clients:

   cp -r /usr/share/syslinux/* /var/lib/tftpboot

2. Create the configuration directory that PXELINUX will use to find its configuration file (made in the next step):

   mkdir /var/lib/tftpboot/pxelinux.cfg

3. Create the file /var/lib/tftpboot/pxelinux.cfg/default and open it
4. Add the following to the file, making sure to edit the IP address to match your servers:

   default menu.c32
   prompt 0
   timeout 100
   ONTIMEOUT local
   
   menu title ########## PXE Boot Menu ##########
   
   label 1
   menu label ^1) Install Fedora 21 Server NetInstall
   kernel fedora21/vmlinuz
   append initrd=fedora21/initrd.img method=ftp:///pub devfs=nomount
   
   label 2
   menu label ^2) Boot from local drive
   

5. Save and close that file

Tip: notice how we are using FTP to transfer the kernal and initrd, if you didn’t want to use FTP, you could use HTTP, HTTPS, or NFS to name a few

Now we have to copy over the linux kernel and the initramdisk to our tftpboot directory, and the rest of the operating system to our FTP directory

1. Start by mounting the ISO:

   mount -o loop  /mnt
   

2. Copy the kernel and the ramdisk:

   mkdir /var/lib/tftpboot/fedora21
   cp /mnt/images/pxeboot/vmlinuz  /var/lib/tftpboot/fedora21
   cp /mnt/images/pxeboot/initrd.img  /var/lib/tftpboot/fedora21
   

3. Copy the entire contents of the ISO to our public ftp directory and make it globally readable:

   cp -r /mnt/*  /var/ftp/pub/ 
   chmod -R 755 /var/ftp/pub
   

Start and Verify the services

Go ahead and fireup dnsmasq and vsftpd:

systemctl restart dnsmasq
systemctl status dnsmasq
systemctl restart vsftpd
systemctl status vsftpd

CentOS 7 has firewalld enabled by default. You can configure the firewall to allow the required incoming connections but in this tutorial we will simple disable the firewall:

systemctl stop firewalld

You should now be able to browse the public FTP directory of your server and see the contents of the ISO:

The Public FTP directory containing the installation files for Fedora 21

Start a PXE Network Install

Start up your client machine and make sure that you eiter configure your boot order so that your network interface comes before your hard drive, or select it in your boot menu. Your client will then obtain an IP address from your DHCP Server and download pxelinux. You’ll be asked to press F8, and once you do you’ll be given a menu with only one option. Press enter to start pxelinux.

After pressing F8, you’ll see this menu with only one option

You will then see the PXE Boot Menu with the two options we configured earlier, you can either wait 10 seconds for the default item to be selected or press enter.

The PXE Boot menu, please note that mine will look different as I have changed some of the values

Your client will then download the kernel and ramdisk from your server, then initialize the installer. There may be moments when the system appears to not be doing anything when it is actually downloading the files from your server over FTP. If everything worked, the installation GUI will appear

Welcome to Fedora-Server 21!

And that’s all there is to it! At this point, the job of the PXE Server is complete and the rest of the installation is completed like any other. You’ve successfully set up a PXE Server and can now install Linux on clients without a DVD, USB, or CD! I hope you enjoyed this tutorial and if you have questions please submit them in the comments below.

Configure JAVAC and JAVA in Linux

Download Java SE 7 JDK for Linux x86 archive. At the time of writing, the file I'm using is jdk-7u21-linux-i586.tar.gz, but the filename will change as updates are released.
    Apparently there is no longer a jvm folder, so create one.

    sudo mkdir /usr/lib/jvm

    Move the archive to the jvm folder

    
 sudo mv jdk-7u21-linux-i586.tar.gz /usr/lib/jvm/

    Change to the jvm folder and extract the JDK from the archive

    cd /usr/lib/jvm

    sudo tar zxvf jdk-7u21-linux-i586.tar.gz

    Everything will be extracted to a new jdk1.7.0_21 folder and you can delete
the archive file now.
  

sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/java-6.31-oracle/bin/javac" 1

sudo update-alternatives --config javac

  
similarly for java use java

sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/java-6.31-oracle/jre/bin/java" 1

sudo update-alternatives --config java

Double-check the version
    java -version

6 Excellent Lightweight Linux Distros for Programming

6 Excellent Lightweight Linux Distros for x86 and ARM

Presenting a nice assortment of lightweight yet fully-functional Linux distros for all occasions. All of these are full distros that do not depend on cloud services; four for x86 and two, count 'em, two for ARM hardware.

Elementary OS

Elementary OS is a beautiful, fast, lightweight Linux for 32- and 64-bit x86. It is built on an Ubuntu core, and Elementary's desktop environment, Pantheon, started out with some stripped-down GNOME 2 elements. But it is more than an Ubuntu respin or GNOME fork-- a lot of custom development goes into Elementary OS, including apps and its development toolkit.

Figure 1: Elementary OS.

A significant aspect of Elementary OS is the inclusion of accessibility features for hearing, sight, and motor-impaired users. The state of accessibility technologies in Linux is far behind where they should be, so it's encouraging to see a distro building them into its core system. Elementary OS has a bit of a Mac-like feel with a sleek, elegant appearance, subtle highlighting cues, minimal clicks to get from one place to another, and lots of useful super key shortcuts. I expect that even inexperienced Linux users could start using Elementary OS and be productive with just a little bit of poking around.

There is currently $5,755 of cash bounties available for bug-fixing some applications and base libraries. If you can't code, putting a few bucks in the bounty kitty is a great way to support Elementary OS.

LXLE

LXLE takes Lubuntu LTS (long-term support), customizes the LXDE desktop, adds proprietary codecs and drivers and a thoughtful selection of default applications, and advertises it as a drop-in replacement for Windows. Me, I think anything is a good replacement for Windows, including an Etch-a-Sketch. But LXLE (Lubuntu eXtra Life Extension) really is an excellent choice for users who want to swap Linux for Windows.

Figure 2: LXLE has the best wallpapers.

LXLE is not amazing new revolutionary technology, but rather an excellently-crafted and refined enhancement of Lubuntu 12.04 and 14.04. The last 5 percent is the hardest, and LXLE goes all the way and finishes that last 5 percent. Installation is fast and simple, and it boots up very quickly. LXLE has five desktop looks to choose from: Unity, Windows XP, GNOME 2, Mac OS X and Netbook. Its most fun feature for me is the 100+ included beautiful wallpapers, and the Random Wallpaper button to cycle them automatically. Windows refugees, or any casual user, will find their way around easily. It also includes the full capabilities of Linux for power users. That is why I love Linux: we can have it all. (32- and 64-bit x86)

Arch Linux ARM

Figure 3: Arch Linux logo.

Arch Linux is the choice of fine nerds everywhere who want a simple yet versatile, up-to-date, lightweight rolling distribution. Arch calls itself simple because it comes with a minimum of bells and whistles, and is for users who want maximum control of their systems with no backtalk from "helpful" utilities.

Arch supports x86, and also has an excellent ARM port. ARM devices are everywhere thanks to single-board computers like Raspberry Pi, Beagleboard, and Arduino, smartphones, tablets, and netbooks like the Samsung Chromebook. Arch is extremely customizable, so you can pare it down to fit even the smallest SBC and make it into a router, a special-purpose server, or even a tiny but useful portable desktop computer. Just like x86 Arch, ARM Arch is well-documented and has active community support.

Point Linux

Point Linux is a baby, barely a year old. It is based on Debian 7 and the MATE desktop, which was originally forked from GNOME 2. So it has a traditional system menu and panels-- nice and clean, and everything easy to find with no dancing icons, no hidden things that appear only when you luck out and hover your cursor over the exactly correct spot, and virtual desktops that stay put. It runs well on old feeble hardware, and comes with a good basic selection of applications. Point Linux is based in Russia, and has good comprehensive localization. If you miss the Ubuntu of old, when it had the best GNOME 2 implementation of any distro, then you might like Point Linux. (32- and 64-bit x86)

Figure 4: Point Linux.

Porteus

Porteus was originally named Slax Remix. Porteus is a combination of "portability" and "Proteus", the god of the sea who could change his form. This is a reference to Porteus' flexibility; it weighs in at less than 300MB, and is optimized to run from a USB stick, CD, Compact Flash, or hard disk. It's a great way to get a prefab version of Slackware all ready to go to work.

Figure 5: Porteus.

You get a choice of five, count 'em, five desktop environments: KDE4, Razor, LXDE, MATE, or Xfce. Porteus includes a package manager, so you can install and remove packages to your heart's content. In my un-humble opinion it is the best portable Linux.

Fedora ARM

Fedora's ARM port has finally been promoted to primary architecture status, as of the Fedora 20 release in December 2013. This is a significant step because it now gets equal priority with the x86 releases, and no packages are pushed into repositories if they fail to build. In typical Fedora fashion, ARM support is broad and pushes into the bleeding edge with support for 64-bit ARM, all the popular ARM SBCs, and a nice selection of unofficial remixes for unsupported devices including the Samsung Chromebook. Which I keep mentioning because it looks like a perfect travel notebook once you clear the Google gunk off and install a good proper Linux on it. Visit the Fedora ARM wiki page to learn everything.

Figure 6: Fedora's Unity desktop.

Dual boot problems in Ubuntu 14.04 and windows 8.1

          Click Here - Installed both OS's but booting onto only ubuntu!!!!!!!!!!!!

For a fresh start to dual boot click on the options below respectively................

1. Click here if you would prefer to dual boot Ubuntu with Windows 7.
2. Click here if you would prefer to dual boot Linux Mint with Windows 8.1.
3. Click here for - How to dual boot Ubuntu and Windows 8.1

An other option is available if unfortunately above methods are not working for u...........

http://askubuntu.com/questions/221835/installing-ubuntu-on-a-pre-installed-windows-8-64-bit-system-uefi-supported

Hadoop Installation

Running Hadoop on Ubuntu Linux (Single-Node Cluster)

  In this tutorial I will describe the required steps for setting up a pseudo-distributed, single-node Hadoop cluster backed by the Hadoop Distributed File System, running on Ubuntu Linux.

Are you looking for the multi-node cluster tutorial? Just head over there.

Hadoop is a framework written in Java for running applications on large clusters of commodity hardware and incorporates features similar to those of the Google File System (GFS) and of the MapReduce computing paradigm. Hadoop’s HDFS is a highly fault-tolerant distributed file system and, like Hadoop in general, designed to be deployed on low-cost hardware. It provides high throughput access to application data and is suitable for applications that have large data sets.
The main goal of this tutorial is to get a simple Hadoop installation up and running so that you can play around with the software and learn more about it.
This tutorial has been tested with the following software versions:

• Ubuntu Linux 12.04 LTS
• Hadoop 1.2.1

Prerequisites

 Sun Java7 or above

$ sudo apt-get update  

$ sudo apt-get install openjdk-7-jdk

$ java -version

Adding a dedicated Hadoop system user

$ sudo addgroup hadoop
$ sudo adduser --ingroup hadoop hduser

Configuring SSH

 Hadoop requires SSH access to manage its nodes, i.e. remote machines plus your local machine if you want to use Hadoop on it (which is what we want to do in this short tutorial). For our single-node setup of Hadoop, we therefore need to configure SSH access to localhost for the hduser user we created in the previous section.
I assume that you have SSH up and running on your machine and configured it to allow SSH public key authentication.

First, we have to generate an SSH key for the hduser user.

user@ubuntu:~$ su - hduser
hduser@ubuntu:~$ ssh-keygen -t rsa -P ""
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hduser/.ssh/id_rsa):
Created directory '/home/hduser/.ssh'.
Your identification has been saved in /home/hduser/.ssh/id_rsa.
Your public key has been saved in /home/hduser/.ssh/id_rsa.pub.
The key fingerprint is:
9b:82:ea:58:b4:e0:35:d7:ff:19:66:a6:ef:ae:0e:d2 hduser@ubuntu
The key's randomart image is:
[...snipp...]
hduser@ubuntu:~$

The second line will create an RSA key pair with an empty password. Generally, using an empty password is not recommended, but in this case it is needed to unlock the key without your interaction (you don’t want to enter the passphrase every time Hadoop interacts with its nodes).
Second, you have to enable SSH access to your local machine with this newly created key.

hduser@ubuntu:~$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

The final step is to test the SSH setup by connecting to your local machine with the hduser user. The step is also needed to save your local machine’s host key fingerprint to the hduser user’s known_hosts file. If you have any special SSH configuration for your local machine like a non-standard SSH port, you can define host-specific SSH options in $HOME/.ssh/config (see man ssh_config for more information).

hduser@ubuntu:~$ ssh localhost The authenticity of host 'localhost (::1)' can't be established. RSA key fingerprint is d7:87:25:47:ae:02:00:eb:1d:75:4f:bb:44:f9:36:26. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'localhost' (RSA) to the list of known hosts. Linux ubuntu 2.6.32-22-generic #33-Ubuntu SMP Wed Apr 28 13:27:30 UTC 2010 i686 GNU/Linux Ubuntu 10.04 LTS [...snipp...] hduser@ubuntu:~$   If the SSH connect should fail, these general tips might help: Enable debugging with ssh -vvv localhost and investigate the error in detail. Check the SSH server configuration in /etc/ssh/sshd_config, in particular the options PubkeyAuthentication (which should be set to yes) and AllowUsers (if this option is active, add the hduser user to it). If you made any changes to the SSH server configuration file, you can force a configuration reload with sudo /etc/init.d/ssh reload. Hadoop Installation Download Hadoop from the Apache Download Mirrors and extract the contents of the Hadoop package to a location of your choice. I picked /usr/local/hadoop. Make sure to change the owner of all the files to the hduser user and hadoop group, for example:   $ cd /usr/local $ sudo tar xzf hadoop-1.0.3.tar.gz $ sudo mv hadoop-1.0.3 hadoop $ sudo chown -R hduser:hadoop hadoop   Update $HOME/.bashrc Add the following lines to the end of the $HOME/.bashrc file of user hduser. If you use a shell other than bash, you should of course update its appropriate configuration files instead of .bashrc. # Set Hadoop-related environment variables export HADOOP_HOME=/usr/local/hadoop # Set JAVA_HOME (we will also configure JAVA_HOME directly for Hadoop later on) export JAVA_HOME=/usr/lib/jvm/open-jdk7 # Some convenient aliases and functions for running Hadoop-related commands unalias fs &> /dev/null alias fs="hadoop fs" unalias hls &> /dev/null alias hls="fs -ls" # If you have LZO compression enabled in your Hadoop cluster and # compress job outputs with LZOP (not covered in this tutorial): # Conveniently inspect an LZOP compressed file from the command # line; run via: # # $ lzohead /hdfs/path/to/lzop/compressed/file.lzo # # Requires installed 'lzop' command. # lzohead () { hadoop fs -cat $1 | lzop -dc | head -1000 | less } # Add Hadoop bin/ directory to PATH export PATH=$PATH:$HADOOP_HOME/bin   Configuration Our goal in this tutorial is a single-node setup of Hadoop. More information of what we do in this section is available on the Hadoop Wiki. hadoop-env.sh The only required environment variable we have to configure for Hadoop in this tutorial is JAVA_HOME. Open conf/hadoop-env.sh in the editor of your choice (if you used the installation path in this tutorial, the full path is /usr/local/hadoop/conf/hadoop-env.sh) and set the JAVA_HOME environment variable to the Sun JDK/JRE 6 directory. Change conf/hadoop-env.sh 1 2 # The java implementation to use. Required. # export JAVA_HOME=/usr/lib/j2sdk1.5-sun to conf/hadoop-env.sh # The java implementation to use. Required. export JAVA_HOME=/usr/lib/jvm/java-6-sun   conf/*-site.xml In this section, we will configure the directory where Hadoop will store its data files, the network ports it listens to, etc. Our setup will use Hadoop’s Distributed File System, HDFS, even though our little “cluster” only contains our single local machine. You can leave the settings below “as is” with the exception of the hadoop.tmp.dir parameter – this parameter you must change to a directory of your choice. We will use the directory /app/hadoop/tmp in this tutorial. Hadoop’s default configurations use hadoop.tmp.dir as the base temporary directory both for the local file system and HDFS, so don’t be surprised if you see Hadoop creating the specified directory automatically on HDFS at some later point. Now we create the directory and set the required ownerships and permissions: $ sudo mkdir -p /app/hadoop/tmp $ sudo chown hduser:hadoop /app/hadoop/tmp # ...and if you want to tighten up security, chmod from 755 to 750... $ sudo chmod 750 /app/hadoop/tmp If you forget to set the required ownerships and permissions, you will see a java.io.IOException when you try to format the name node in the next section). Add the following snippets between the ... tags in the respective configuration XML file. In file conf/core-site.xml:

In file conf/mapred-site.xml:

 


  
 
  In file conf/hdfs-site.xml:

   

Formatting the HDFS filesystem via the NameNode

The first step to starting up your Hadoop installation is formatting the Hadoop filesystem which is implemented on top of the local filesystem of your “cluster” (which includes only your local machine if you followed this tutorial). You need to do this the first time you set up a Hadoop cluster.

Do not format a running Hadoop filesystem as you will lose all the data currently in the cluster (in HDFS)!

To format the filesystem (which simply initializes the directory specified by the dfs.name.dir variable), run the command

hduser@ubuntu:~$ /usr/local/hadoop/bin/hadoop namenode -format

The output will look like this:

hduser@ubuntu:/usr/local/hadoop$ bin/hadoop namenode -format 10/05/08 16:59:56 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = ubuntu/127.0.1.1 STARTUP_MSG: args = [-format] STARTUP_MSG: version = 0.20.2 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010 ************************************************************/ 10/05/08 16:59:56 INFO namenode.FSNamesystem: fsOwner=hduser,hadoop 10/05/08 16:59:56 INFO namenode.FSNamesystem: supergroup=supergroup 10/05/08 16:59:56 INFO namenode.FSNamesystem: isPermissionEnabled=true 10/05/08 16:59:56 INFO common.Storage: Image file of size 96 saved in 0 seconds. 10/05/08 16:59:57 INFO common.Storage: Storage directory .../hadoop-hduser/dfs/name has been successfully formatted. 10/05/08 16:59:57 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1 ************************************************************/ hduser@ubuntu:/usr/local/hadoop$

Starting your single-node cluster

Run the command:

hduser@ubuntu:~$ /usr/local/hadoop/bin/start-all.sh

This will startup a Namenode, Datanode, Jobtracker and a Tasktracker on your machine.
The output will look like this:

hduser@ubuntu:/usr/local/hadoop$ bin/start-all.sh starting namenode, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-namenode-ubuntu.out localhost: starting datanode, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-datanode-ubuntu.out localhost: starting secondarynamenode, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-secondarynamenode-ubuntu.out starting jobtracker, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-jobtracker-ubuntu.out localhost: starting tasktracker, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-tasktracker-ubuntu.out hduser@ubuntu:/usr/local/hadoop$

A nifty tool for checking whether the expected Hadoop processes are running is jps (part of Sun’s Java since v1.5.0). See also How to debug MapReduce programs.

hduser@ubuntu:/usr/local/hadoop$ jps 2287 TaskTracker 2149 JobTracker 1938 DataNode 2085 SecondaryNameNode 2349 Jps 1788 NameNode

You can also check with netstat if Hadoop is listening on the configured ports.

hduser@ubuntu:~$ sudo netstat -plten | grep java tcp 0 0 0.0.0.0:50070 0.0.0.0:* LISTEN 1001 9236 2471/java tcp 0 0 0.0.0.0:50010 0.0.0.0:* LISTEN 1001 9998 2628/java tcp 0 0 0.0.0.0:48159 0.0.0.0:* LISTEN 1001 8496 2628/java tcp 0 0 0.0.0.0:53121 0.0.0.0:* LISTEN 1001 9228 2857/java tcp 0 0 127.0.0.1:54310 0.0.0.0:* LISTEN 1001 8143 2471/java tcp 0 0 127.0.0.1:54311 0.0.0.0:* LISTEN 1001 9230 2857/java tcp 0 0 0.0.0.0:59305 0.0.0.0:* LISTEN 1001 8141 2471/java tcp 0 0 0.0.0.0:50060 0.0.0.0:* LISTEN 1001 9857 3005/java tcp 0 0 0.0.0.0:49900 0.0.0.0:* LISTEN 1001 9037 2785/java tcp 0 0 0.0.0.0:50030 0.0.0.0:* LISTEN 1001 9773 2857/java hduser@ubuntu:~$

If there are any errors, examine the log files in the /logs/ directory.

Stopping your single-node cluster

Run the command

hduser@ubuntu:~$ /usr/local/hadoop/bin/stop-all.sh

to stop all the daemons running on your machine.

Hadoop Web Interfaces

Hadoop comes with several web interfaces which are by default (see conf/hadoop-default.xml) available at these locations:

• http://localhost:50070/ – web UI of the NameNode daemon
• http://localhost:50030/ – web UI of the JobTracker daemon
• http://localhost:50060/ – web UI of the TaskTracker daemon

These web interfaces provide concise information about what’s happening in your Hadoop cluster. You might want to give them a try.

NameNode Web Interface (HDFS layer)

The name node web UI shows you a cluster summary including information about total/remaining capacity, live and dead nodes. Additionally, it allows you to browse the HDFS namespace and view the contents of its files in the web browser. It also gives access to the local machine’s Hadoop log files.
By default, it’s available at http://localhost:50070/.