Adapt Your Job to Connect to Kerberos

When working with Kerberized data lakes, you must adapt your jobs to connect to Kerberos.

Saagie technologies have the required libraries to connect to Kerberos before your job runs. Add the following kinit command in the command line of your job:

echo $MY_PASSWORD | kinit $MY_LOGIN

Where:

$MY_PASSWORD must be replaced with your password.
$MY_LOGIN must be replaced with your login.

The vertical bar (|) avoids you from being prompted for your password.

With this command, some technologies, such as Spark, work transparently with Kerberos and do not require additional code updates. Other technologies, however, will require to update their WebHDFS port or their Hive JDBC (Java Database Connectivity) URL. The following articles show you how to do this for each technology:

Technology	Article
Java/Scala	Read and Write Files From HDFS With Java/Scala Read and Write Tables From Hive With Java/Scala Read and Write Tables From Impala With Java/Scala
Python	Read and Write Files From HDFS With Python Read and Write Tables From Hive With Python Read and Write Tables From Impala With Python
R	Read and Write Files From HDFS With R Read and Write Tables From Hive With R Read and Write Tables From Impala With R
Talend	Read, Write, and List Files From HDFS With Talend Read and Write Tables From Hive With Talend Read and Write Tables From Impala With Talend
WebHDFS	Read and Write Files From HDFS, WebHDFS, and HTTPFS With HDFS

Technology

Article

Java/Scala