1. 程式人生 > >load資料到hive分割槽表報錯: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTas

load資料到hive分割槽表報錯: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTas

在啟動hive建立表的時候,興許你們回碰到這個錯誤:http://blog.csdn.net/qq_35732963/article/details/54139581;百度一下,說是字元編碼的問題,那個好解決,但是下面這個錯,卻不一樣,歸根到底是資料庫出了問題!

1、上傳資料到hive表時報錯:

hive> load data local inpath '/home/hadoop/data1.txt' into table partition_tablespartition(gender='M');
Loading data to table hive1.partition_tables partition (gender=M)


Failed with exception MetaException(message:javax.jdo.JDODataStoreException: Error(s) were found while auto-creating/validating the datastore for classes. The errors are printed in the log, and are attached to this exception.
    at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
    at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
    at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
    at org.apache.hadoop.hive.metastore.ObjectStore.addPartition(ObjectStore.java:1431)
。。。。

    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask

2、說到底還是改下資料庫方面的:刪除原先自動建立的hive資料庫,新建metastore資料庫

mysql> create database metastore;
Query OK, 1 row affected (0.01 sec)

mysql> show create database metastore;
+-----------+--------------------------------------------------------------------+
| Database  | Create Database                                                    |
+-----------+--------------------------------------------------------------------+
| metastore | CREATE DATABASE `metastore` /*!40100 DEFAULT CHARACTER SET utf8 */ |
+-----------+--------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> alter database metastore  character set latin1;
Query OK, 1 row affected (0.00 sec)

3、去hadoop使用者下修改hive/conf裡面的hive-site.xml

這裡原本是hive的資料庫,但由於報錯,把原來的hive'刪了,新建一個metastore

[[email protected] conf]$ vi hive-site.xml

<!-- metadata database connection configuration -->
<property>
  <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true</value>   ----這個引數使用來設定元資料連線字串
    <description>JDBC connect string for a JDBC metastore</description>
</property>

4、 ---給建立的2個檔案 許可權

[[email protected] hadoop]# ll  
total 76
drwxrwxr-x.  4 hadoop hadoop 4096 Dec 26 19:30 app
-rw-r--r--   1 hadoop hadoop   55 Jan  9 19:12 data1.txt
-rw-r--r--   1 root   root     44 Jan  9 19:13 data2.txt

[[email protected] hadoop]# chgrp hadoop data2.txt
[[email protected] hadoop]# chown hadoop data2.txt

5、啟動hive

[[email protected] conf]$ hive
Logging initialized using configuration in file:/home/hadoop/hive-0.14.0/conf/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.6.2/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hive-0.14.0/lib/hive-jdbc-0.14.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

hive> create database hive1;
OK
Time taken: 0.415 seconds
hive> create table partition_tables
    > (tid int,tname string, age int)
    > partitioned by (gender string)
    > row format delimited fields terminated by ',';
OK
Time taken: 0.328 seconds

6、再次上傳檔案到分割槽表,顯示上傳成功

hive> load data local inpath '/home/hadoop/data1.txt' into table partition_tables partition(gender='M');
Loading data to table default.partition_tables partition (gender=M)
Partition default.partition_tables{gender=M} stats: [numFiles=1, numRows=0, totalSize=55, rawDataSize=0]
OK
Time taken: 1.138 seconds

hive>  load data local inpath '/home/hadoop/data2.txt' into table partition_tables partition(gender='F');
Loading data to table default.partition_tables partition (gender=F)
Partition default.partition_tables{gender=F} stats: [numFiles=1, numRows=0, totalSize=44, rawDataSize=0]
OK
Time taken: 0.661 seconds

7、進入http://master:50070/檢視partition_table分割槽是否成功


相關推薦

load資料到hive分割槽表報 FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTas

在啟動hive建立表的時候,興許你們回碰到這個錯誤:http://blog.csdn.net/qq_35732963/article/details/54139581;百度一下,說是字元編碼的問題,那

Hive on spark 報FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.spark

cp /opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/lib/spark/lib/spark-assembly.jar /opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/lib/hive/lib

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/util/JSON

uri object red handle column got 文件夾 tid tails 問題: 將MongoDB數據導入Hive,按照https://blog.csdn.net/thriving_fcl/article/details/51471248文章,在

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask:28:27'

呼叫hive查詢資料時報錯 原因:hive叢集棧資源不夠 報錯日誌: TExecuteStatementResp(status=TStatus(errorCode=1, errorMessage=u'Error while processing statement: FAILED: Ex

程式中執行hive sql出錯[FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask]

執行SQL時出了個錯: SQL: INSERT OVERWRITE DIRECTORY 'result/testConsole' select count(1) from nutable;  錯誤資訊: Failed with exception Unable to rename: hdfs://in

hiveExecution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(me

建立hive表時報如下錯 Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: java.net.ConnectExcept

hive Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:For direct MetaStore DB connections,

pre mysq rom ges character base ddltask for latin 學習hive 使用mysql作為元數據 hive創建數據庫和切換數據庫都是可以的 但是創建表就是出問題 百度之後發現 是編碼問題 特別記錄一下~~~ 1.報錯前如圖:

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask

out line man 轉換 numa 文件 cat 報錯 參數 我們采用亞馬遜emr構建的集群,用hive查詢的時候報錯,FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesi

Hive error log :FAILED: Execution Error, return code 137 from org.apache.hadoop.hive.ql.exec.mr.Mapr

From below log is not easy to find the root reason ,any body know that ,thx 2018-10-22 03:45:41 INF

Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/

!!!!!!!Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/HBaseConfiguration!!!!!!!! **hive表關聯HBa

Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask問題解決

發現這個報錯後,就網上查資料嘍。目前看下來兩種情況比較常見,不排除其他可能。一般排查方式也是檢視hive日誌,job日誌在你的HIVEHOME/conf的hive?log4j.properties下的hive.log.dir={java.io.tmpdir}/u

Error, return code 1 from org.apache.hadoop.hive.

lin 創建 latin color base creat break round 建表 Hive創建表格報【Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException】這個

hive中刪除表的錯誤Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException

成了 ret jar tor java-5 drop meta org -c hive使用drop table 表名刪除表時報錯,return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException

Hive建立表格報Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException的錯誤

1.首先保證hive正常配置, 修改$HIVE_HOME/conf/hive-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?

hive MapJoin 異常 return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask

今天寫了一個hive sql,A表往B表插入資料,如果公共欄位id相同,則不插入,即不存在則插入,否則不插入,這樣一個sql,可是執行時報了記憶體異常, 具體資訊是: 2018-08-14 13:45:17 Starting to launch local task to pro

排查Hiveorg.apache.hadoop.hive.serde2.SerDeException: java.io.IOException: Start of Array expected

arr .json span 問題 catalog pan 不支持 led open CREATE TABLE json_nested_test ( count string, usage string, pkg map<string

解決kylin報 ClassCastException org.apache.hadoop.hive.ql.exec.ConditionalTask cannot be cast to org.apache.hadoop.hive.ql.exec.mr.MapRedTask

conf lan exe hive oop ann 關於 .exe map 方法:去掉參數SET hive.auto.convert.join=true; 從配置文件$KYLIN_HOME/conf/kylin_hive_conf.xml刪掉 或 kylin-gui的cu

shematool -initschema -dbtype mysql error org.apache.hadoop.hive.metastore.hivemetaexception:Failed to get schema version

hang my.cnf blog address com rest chang init edit 命令:schematool -initSchema -dbType mysql Fix the issue: edit /etc/mysql/my.cnf change b

啟動hiveUnable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStor