Adapt Your Job to Connect to Kerberos

When working with kerberized data lakes, you must adapt your jobs to connect to Kerberos.

Saagie technologies have the required libraries to connect to Kerberos before your job runs. Add the following kinit command in the command line of your job:

echo $MY_PASSWORD | kinit $MY_LOGIN

Where:

  • $MY_PASSWORD must be replaced with your password.

  • $MY_LOGIN must be replaced with your login.

The vertical bar (|) avoids you from being prompted for your password.

With this command, some technologies, such as Spark, work transparently with Kerberos and do not require additional code updates. Other technologies, however, will require to update their WebHDFS port or their Hive JDBC (Java Database Connectivity) URL. The following articles show you how to do this for each technology:

Technology Article

Java/Scala

Python

R

Talend

WebHDFS