Securely Providing ADLS Credentials

You can provide your ADLS credentials securely by hiding the open, readable configuration on the command line using the Hadoop credential provider.

  1. Generate a jceks file for ADLS authorization:
    hadoop credential create -provider jceks://hdfs/user/USER_NAME/adlskeyfile.jceks -value client ID
    hadoop credential create dfs.adls.oauth2.credential -provider jceks://hdfs/user/USER_NAME/adlskeyfile.jceks -value client secret
    hadoop credential create dfs.adls.oauth2.refresh.url -provider jceks://hdfs/user/USER_NAME/adlskeyfile.jceks -value refresh URL
  2. Run the DistCp example using the jceks file:
    hadoop distcp
    hdfs://<NameNode Hostname>:9001/user/foo/007020615
    adl://<Account Name>
  3. Configure the core-site.xml file to use the jceks file:
      <description>Path to interrogate for protected credentials.</description>