Apache Oozie - Part 11: Java API for interfacing with oozie workflows

What's covered in the blog?

1. Documentation on the Oozie java API
2. Code of a sample java program that calls a oozie workflow with a java action to process some syslog generated log files.  Instructions on loading sample data, workflow files and running the workflow are provided, along with some notes based on my learnings.

Version:
Oozie 3.3.0;

Related blogs:

Blog 1:  Oozie workflow with hdfs and email actions
Blog 2: Oozie workflow with hdfs, email and hive actions
Blog 3: Oozie workflow with sqoop action (Hive-mysql; sqoop export)
Blog 4: Oozie workflow with java map-reduce (new API) action
Blog 5: Oozie workflow with streaming map-reduce action
Blog 6: Oozie workflow with java main action
Blog 7: Oozie workflow with Pig action
Blog 8: Oozie Java API for interfacing with oozie workflows
Blog 9: Oozie Web Service API for interfacing with oozie workflows
Blog 10: Oozie bundle & coordinator jobs

Your thoughts/updates:
If you want to share your thoughts/updates, email me at airawat.blog@gmail.com.

1.0. About the Oozie java API

Oozie provides a Java Client API that simplifies integrating Oozie with Java applications. This Java Client API is a convenience API to interact with Oozie Web-Services API.
The following code snippet shows how to submit an Oozie job using the Java Client API.

import org.apache.oozie.client.OozieClient;
import org.apache.oozie.client.WorkflowJob;
.
import java.util.Properties;
.
    ...
.
    // get a OozieClient for local Oozie
    OozieClient wc = new OozieClient("http://bar:11000/oozie");
.
    // create a workflow job configuration and set the workflow application path
    Properties conf = wc.createConfiguration();
    conf.setProperty(OozieClient.APP_PATH, "hdfs://foo:8020/usr/tucu/my-wf-app");
.
    // setting workflow parameters
    conf.setProperty("jobTracker", "foo:8021");
    conf.setProperty("inputDir", "/usr/tucu/inputdir");
    conf.setProperty("outputDir", "/usr/tucu/outputdir");
    ...
.
    // submit and start the workflow job
    String jobId = wc.run(conf);
    System.out.println("Workflow job submitted");
.
    // wait until the workflow job finishes printing the status every 10 seconds
    while (wc.getJobInfo(jobId).getStatus() == Workflow.Status.RUNNING) {
        System.out.println("Workflow job running ...");
        Thread.sleep(10 * 1000);
    }
.
    // print the final status o the workflow job
    System.out.println("Workflow job completed ...");
    System.out.println(wf.getJobInfo(jobId));
    ...

Source of the documentation, above:

http://archive.cloudera.com/cdh/3/oozie/DG_Examples.html#Java_API_Example

2.0. Exercise

The java program below calls the workflow built in my blog 6 - include java code, workflow related files and sample data.

2.0.1. Sample data and sample workflow

Blog 6: Oozie workflow with java action

2.0.2. Sample Java program to call workflow

Note: Ensure you replace configuration highlighted in yellow ochre with your cluster specific configuration.

import java.util.Properties;

import org.apache.oozie.client.OozieClient;

import org.apache.oozie.client.WorkflowJob;

public class myOozieWorkflowJavaAPICall {

public static void main(String[] args) {

    OozieClient wc = new OozieClient("http://cdh-dev01:11000/oozie");

    Properties conf = wc.createConfiguration();

    conf.setProperty(OozieClient.APP_PATH, "hdfs://cdh-nn01.hadoop.com:8020/user/airawat/oozieProject/javaApplication/workflow.xml");

    conf.setProperty("jobTracker", "cdh-jt01:8021");

    conf.setProperty("nameNode", "hdfs://cdh-nn01.hadoop.com:8020");

    conf.setProperty("queueName", "default");

    conf.setProperty("airawatOozieRoot", "hdfs://cdh-nn01.hadoop.com:8020/user/airawat/oozieProject/javaApplication");

    conf.setProperty("oozie.libpath", "hdfs://cdh-nn01.hadoop.com:8020/user/oozie/share/lib");

    conf.setProperty("oozie.use.system.libpath", "true");

    conf.setProperty("oozie.wf.rerun.failnodes", "true");

    try {

        String jobId = wc.run(conf);

        System.out.println("Workflow job, " + jobId + " submitted");

        while (wc.getJobInfo(jobId).getStatus() == WorkflowJob.Status.RUNNING) {

            System.out.println("Workflow job running ...");

            Thread.sleep(10 * 1000);

        }

        System.out.println("Workflow job completed ...");

System.out.println(wc.getJobInfo(jobId));

    } catch (Exception r) {

        System.out.println("Errors " + r.getLocalizedMessage());

    }

}

}

2.0.3. Program output

Workflow job, 0000081-130613112811513-oozie-oozi-W submitted

Workflow job running ...

Workflow job running ...

Workflow job running ...

Workflow job running ...

Workflow job running ...

Workflow job running ...

Workflow job running ...

Workflow job completed ...

Workflow id[0000081-130613112811513-oozie-oozi-W] status[SUCCEEDED]

2.0.4. Oozie web console

http://YourOozieServer:TypicallyPort11000/oozie