Problem in importing all tables from MySQL to Hive using Apache Sqoop-1.4.7

user_12825713 0
Viewed 365

I'm trying to import all tables from MySQL database into Hive database using Apache Sqoop CLI, when i execute the following import command:

[hadoop@localhost bin]$ sqoop import-all-tables --connect jdbc:mysql://localhost/mysql --username root --password root

Somehow the import fails and i get the following error message at the end of the output 

20/05/09 23:06:27 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
.
.
.
20/05/09 23:06:38 ERROR tool.ImportAllTablesTool: Encountered IOException running import job:
java.io.IOException: Generating splits for a textual index column allowed only in case of
"-Dorg.apache.sqoop.splitter.allow_text_splitter=true" property passed as a parameter

Prior to running Sqoop-1.4.7 i had the following installed and up & running

[hduser@localhost ~]$ java -version
openjdk version "1.8.0_252" 
OpenJDK Runtime Environment (build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

[hadoop@localhost ~]$ hadoop version
Hadoop 2.7.2
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
b165c4fe8a74265c792ce23f546c64604acf0e41
Compiled by jenkins on 2016-01-26T00:08Z
Compiled with protoc 2.5.0
From source with checksum d0fda26633fa762bff87ec759ebe689c
This command was run using /opt/hadoop/share/hadoop/common/hadoop-common-2.7.2.jar 

MySQL Server version: 5.6.48
Hive-1.2.2

Based on this configuration, how to import all tables to hive successfully?

sqoop hive
1 Answers

It is possible to use a character attribute as split-by attribute. From the message of your error

Generating splits for a textual index column allowed only in case of
"-Dorg.apache.sqoop.splitter.allow_text_splitter=true" property passed as a parameter

You only need to add -Dorg.apache.sqoop.splitter.allow_text_splitter=true after your 'sqoop job' statement like this:

sqoop import-all-tables -Dorg.apache.sqoop.splitter.allow_text_splitter=true \
--connect jdbc:mysql://localhost/mysql \ 
--username root \
--password root
user_85714980
Related