Sunday, April 7, 2013

Steps to setup apache sqoop


Copy and untar sqoop-1.99.1-bin-hadoop100.tar.gz

Since default port 8080 is running my mongo Query. The listener ports are changed 

Go to /home/hadoop/sqoop-1.99.1-bin-hadoop100
[list]cd server/conf
vi server.xml
Change tomcat's default port to some other values.
[hadoop@someHost-n2 conf]$ diff server.xml server.xml.old
22c22
< <Server port="18005" shutdown="SHUTDOWN">
---
> <Server port="8005" shutdown="SHUTDOWN">
67c67
<          Define a non-SSL HTTP/1.1 Connector on port 18080
---
>          Define a non-SSL HTTP/1.1 Connector on port 8080
69c69
<     <Connector port="18080" protocol="HTTP/1.1"
---
>     <Connector port="8080" protocol="HTTP/1.1"
75c75
<                port="18080" protocol="HTTP/1.1"
---
>                port="8080" protocol="HTTP/1.1"
90c90
<     <Connector port="18009" protocol="AJP/1.3" redirectPort="8443" />
---
>     <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
[/list]

Change configuration to reflect the correct hadoop installation
vi $SQOOPBASE/server/conf/sqoop.properties
[hadoop@someHost-n2 conf]$ diff sqoop.properties sqoop.properties.old  Change the following.
110,111c110
< #org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/etc/hadoop/conf/
< org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/home/hadoop/hadoop-1.0.4/conf
---
> org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/etc/hadoop/conf/


Start the server
[list]cd /home/hadoop/sqoop-1.99.1-bin-hadoop100/
./bin/sqoop.sh server start
[/list]

Watch the log file, make sure no exception
[list]cd /home/hadoop/sqoop-1.99.1-bin-hadoop100/server/logs
tail -f catalina.2013-04-07.log  <--- File name will be different depend on date.[/list]

Start client
[list][hadoop@someHost-n2 sqoop-1.99.1-bin-hadoop100]$ bin/sqoop.sh client
..
sqoop:000> set server --host someHost-n2 --port 18080
Server is set successfully
[/list]

Show server version 
sqoop:000> show version -s
server version:
  Sqoop 1.99.1 revision Unknown
  Compiled by jarcec on Fri Dec 21 12:11:06 PST 2012

No comments:

Post a Comment