hadoop - Creating Hive Metastore Database Tables Error -

i'm running through cloudera manager (free edition) , reached point wizard creating hive metastore database. error shown , halts configuration process.

using /var/run/cloudera-scm-agent/process/40-hive-metastore-create-tables/hadoop-conf hadoop_conf_dir

i cant seem find information might cause this?

every thing has been configured correctly point, installed , user names , passwords correct.

has seen error before? thoughts?

error log:

at sun.reflect.delegatingconstructoraccessorimpl.newinstance(delegatingconstructoraccessorimpl.java:27) @ java.lang.reflect.constructor.newinstance(constructor.java:513) @ com.mysql.jdbc.util.handlenewinstance(util.java:411) @ com.mysql.jdbc.sqlerror.createcommunicationsexception(sqlerror.java:1116) @ com.mysql.jdbc.mysqlio.readpacket(mysqlio.java:688) @ com.mysql.jdbc.mysqlio.dohandshake(mysqlio.java:1094) @ com.mysql.jdbc.connectionimpl.coreconnect(connectionimpl.java:2337) @ com.mysql.jdbc.connectionimpl.connectonetryonly(connectionimpl.java:2370) @ com.mysql.jdbc.connectionimpl.createnewio(connectionimpl.java:2154) @ com.mysql.jdbc.connectionimpl.<init>(connectionimpl.java:792) @ com.mysql.jdbc.jdbc4connection.<init>(jdbc4connection.java:49) @ sun.reflect.nativeconstructoraccessorimpl.newinstance0(native method) @ sun.reflect.nativeconstructoraccessorimpl.newinstance(nativeconstructoraccessorimpl.java:39) @ sun.reflect.delegatingconstructoraccessorimpl.newinstance(delegatingconstructoraccessorimpl.java:27) @ java.lang.reflect.constructor.newinstance(constructor.java:513) @ com.mysql.jdbc.util.handlenewinstance(util.java:411) @ com.mysql.jdbc.connectionimpl.getinstance(connectionimpl.java:381) @ com.mysql.jdbc.nonregisteringdriver.connect(nonregisteringdriver.java:305) @ java.sql.drivermanager.getconnection(drivermanager.java:582) @ java.sql.drivermanager.getconnection(drivermanager.java:185) @ com.cloudera.enterprise.dbutil.sqlrunner.open(sqlrunner.java:109) @ com.cloudera.enterprise.dbutil.sqlrunner.runsinglequery(sqlrunner.java:80) @ com.cloudera.cmf.service.hive.hivemetastoredbutil.counttables(hivemetastoredbutil.java:191) ... 2 more  caused by: java.io.eofexception: can not read response server. expected read 4 bytes, read 0 bytes before connection unexpectedly lost. @ com.mysql.jdbc.mysqlio.readfully(mysqlio.java:2540) @ com.mysql.jdbc.mysqlio.readpacket(mysqlio.java:612) ... 20 more 

ok cloudera using version hive 0.10, doesn't support remote login

but need go around bug, logging server getting error cloudera manager tell ip

1) create login in server fails install hive 2) create $haddop_home

$hadoop_home="/usr/lib/hadoop/" 

3) install postgres in server fails

`$ sudo apt-get install postgresql`  $ cat /etc/postgresql/9.1/main/postgresql.conf | grep -e listen -e standard_conforming_strings 

modify lines in file

listen_addresses = '*' standard_conforming_strings = off 

you need configure authentication network in pg_hba.conf. need make sure postgresql user create in next step have access server remote host. this, add new line pg_hba.con has following information:

host    <database>         <user>         <network address>         <mask>               password 

start postgresql server

$ sudo service postgresql start 

use chkconfig utility ensure postgresql server start @ boot time:

chkconfig postgresql on 

you can use chkconfig utility verify postgresql server started @ boot time, example:

chkconfig --list postgresql 

step 2: install postgres jdbc driver

before can run hive metastore remote postgresql database, must configure jdbc driver remote postgresql database, set initial database schema, , configure postgresql user account hive user.

to install postgresql jdbc driver on debian/ubuntu system:

install libpostgresql-jdbc-java , symbolically link file /usr/lib/hive/lib/ directory.

$ sudo apt-get install libpostgresql-jdbc-java $ ln -s /usr/share/java/postgresql-jdbc4.jar /usr/lib/hive/lib/postgresql-jdbc4.jar 

step 3: create metastore database , user account

bash# sudo –u postgres psql bash$ psql postgres=# create user hiveuser password 'mypassword'; postgres=# create database metastore; postgres=# \c metastore; connected database 'metastore'. postgres=# \i /usr/lib/hive/scripts/metastore/upgrade/postgres/hive-schema-0.10.0.postgres.sql set set ... 

now need grant permission metastore tables user hiveuser. postgresql not have statements grant permissions tables @ once; you'll need grant permissions 1 table @ time. automate task following sql script:

bash# sudo –u postgres psql metastore=#         \o /tmp/grant-privs metastore=#         select 'grant select,insert,update,delete on "'  || schemaname || '"."' || tablename || '" hiveuser ;' metastore-#           pg_tables metastore-#           tableowner = current_user , schemaname = 'public'; metastore=#         \o metastore=#         \i /tmp/grant-privs 

you can verify connection machine you'll running metastore service follows:

psql –h myhost –u hiveuser –d metastore metastore=# 

step 4: configure metastore service communicate postgresql database

change ip of aws amazon master server, or master server, don't use dns name

$find / -name hive-site.xml $nano /run/cloudera-scm-agent/process/27-hive-metastore-create-tables/hive-site.xml 

in file search for:

<property>   <name>javax.jdo.option.connectionurl</name>   <value>jdbc:postgresql://myhost/metastore</value> </property> 

and change correct ip master hadoop server u running cloudera manager every link in file not correctly write hadoop master cloudera manager connector , have change correct ip

after autoinstall of cloudera manager , run again , :)

that installation have work around our contract cloudera support (that's business) :)

all works fine me when have problem in de cloudera cdh 4.x sorl

regards