Executing external command from within JVM often causes problems- be it in terms of the code to write and manage or in the ease of implementation. I had similar requirement in my Major project for my Under Graduate Degree, where in I had to launch a C program from the Java code. I ran into different issues like- the Main thread getting blocked, the GUI freezing, or reading the output streams and so on. Finally I had to give up the idea and stick with launching the external command externally :P Had I found the Exec library from Apache Commons then, my work would have been lot easier. Anyways better late then never. I will quickly go through how one can useĀ  Exec library to launch external programs from JVM- Its a wrapper over Java’s ProcessBuilder, Runtime.getRuntime().exe(). The Javadoc shows lots of classes- but important one among them- which the end user would mostly be using are:

  • DefaultExecutor (A default implementation for Executor Interface)
  • DefaultExecuteResultHandler (A default implementation for ExecuteResultHandler interface)
  • ExecuteWatchdog
  • PumpStreamHandler
  • CommandLine
  • EnvironmentUtils

There are other CommandLauncher classes which are internally called by when one uses DefaultExecutor, though the methods of these classes are available for one to use them directly.

Using Exec helps us to:

  • Set environment variables- Current Java implementation provides- System.getEnv() or System.getEnv(String) but no System.setEnv(). Yet the exec() method of Runtime class is overloaded to accept the Environment variables. This can at time lead to confusion. Exec provides an EnvironmentUtils class which takes care of environment variable related tasks.
  • Easily set the Stream handlers- There’s a class PumpStreamHandler which can take- Output Stream, Input Stream and error stream. It captures the data in different streams via creating different Threads for each stream. So one need not worry about creating threads for each stream.
  • Build the Command using CommandLine class.
  • Execute the command- Synchronously or Asynchronously.
  • Associate watchdogs for each command- So that these commands dont take too much of time.
  • Destroying the process when the JVM exits.

Lets have a look at the implementation of the library.

Building the Command-Uses CommandLine class.

//Just the Executable, No arguments
CommandLine command = new CommandLine("ls");
command.addArgument(String) //Adding a String argument
command.addArgument("${arg1}"); //Expands the arg1 value from the Map

//map used to expand the arguments like above
command.setSubstitutionMap(Map);

command.addArguments(String[]) //Adding all the arguments at one go

Now we need to set the Stream handler- PumpStreamHandler


//Takes System.out for dumping the output and System.err for Error
PumpStreamHandler streamHandler = new PumpStreamHander();

//Create a new FileOutputStream instance
//Sends the output and error to the file
streamHandler = new PumpStreamHandler(FileOutputStream);

In this way PumpStreamHandler provides overloaded Constructors. Also note that internally it creates different Threads for each Stream viz Output, Input, Error.

Now we should be executing the command- DefaultExecutor- There are lot of variations in this. Lets start with the Simple-

DefaultExecutor executor = new DefaultExecutor();
executor.setStreamHandler(PumpStreamHandler); //Sets the stream handler
executor.execute(CommandLine);//Executes the command

Lets apply the above to create a simple “ls” on Linux systems.


CommandLine command = new CommandLine("ls");
PumpStreamHandler streamHandler = new PumpStreamHandler();
DefaultExecutor executor = new DefaultExecutor();
executor.setStreamHandler(streamHandler);
executor.execute(command);

Now lets execute the same Asynchronously- uses DefaultExeuteResultHandler class. I would be using the sleep command as that will clearly show the difference


CommandLine command = new CommandLine("sleep");
command.addArgument("10");//Number of seconds to sleep
System.out.println("Before Sleep");
DefaultExecuteResultHandler resultHandler
         = new DefaultExecuteResultHandler();
DefaultExecutor executor = new DefaultExecutor();
PumpStreamHandler streamHandler = new PumpStreamHandler();
executor.setStreamHandler(streamHandler);
executor.execute(command,resultHandler);
System.out.println("After Sleep");
//Use of resultHandler makes this a Asynch process

You can see that there’s now time interval between the two print statements. If we add this line after Line 09-

//Wait for the command to finish execution.
resultHandler.waitFor();
//Obtain the exit value of the command.
int exitValue = resultHandler.getExitValue();

Then there’s a delay- which means the main thread waits for the sub process to complete before proceeding- waitFor() internally uses Thread.sleep(time). Also the exitValue is captured which can be used for:


//testing if the exitValue indicates failure
if(executor.isFailure(exitValue){

System.out.println("Command execution failed");
}else{

System.out.println("Command execution Successful");

}

ifFailure() checks if the exitValue obtained after executing the command denotes a Success (exitValue=0) or a Failure (any other exitValue). One can also set exitValue for the executor using setExitValue(int) or setExitValues(int[]) in that case the exitValue for success will not be “0” but the value specified. One can use waitFor() if the command has to be executed before the program continues execution. Otherwise in cases where the user wants to execute the command and store the result in a File waitFor() will not be required.

Sometimes the command might continue executing for a long time and we need to keep a check to see that it doesnt exceed the time. In those cases ExecuteWatchdog class can be used. Whenever the process terminates due to the timeout specified in WatchDog, the exit value returned will be for that of a failure.


ExecuteWatchdog watchDog = new ExecuteWatchdog(timeout);

exeuctor.setWatchDog(watchDog);//setting the watchdog for the executor

//The time out can also be ExecuteWatchdog.INFINITE_TIMEOUT

Summarizing:

  • DefaultExecutor- Class for launching/executing the commands. Can set TIME_OUT, StreamHandler, ResultHandler, Working Directory for the command.
  • DefaultExecuteResultHandler- Monitors the process when its launched Asynchronously. Can be used to get the Exception, Exit_Value for the process.
  • PumpStreamHandler- Manager the Output,Error,Input Stream of the Process- Each of them in a separate thread.
  • ExecuteWatchdog- Used for setting the timeout for the command.
  • EnvironmentUtils- Utility class to work with the Environment variables
  • CommandLine- For setting up the executable, and its arguments. Can take the argument values from a Map.

Note: Most of the classes are default implementations of their interfaces

Let me give a sample code which uses all of the above:


//Command to be executed
 CommandLine command = new CommandLine("ls");

 //Adding its arguments
 command.addArguments(new String[]{"-l","-t","-R"});

 //Infinite timeout
 ExecuteWatchdog watchDog = new ExecuteWatchdog(ExecuteWatchdog.INFINITE_TIMEOUT);

 //Result Handler for executing the process in a Asynch way
 DefaultExecuteResultHandler resultHandler = new DefaultExecuteResultHandler();

 //Using Std out for the output/error stream
 PumpStreamHandler streamHandler = new PumpStreamHandler();

 //This is used to end the process when the JVM exits
 ShutdownHookProcessDestroyer processDestroyer = new ShutdownHookProcessDestroyer();

 //Our main command executor
 DefaultExecutor executor = new DefaultExecutor();

 //Setting the properties
 executor.setStreamHandler(streamHandler);
 executor.setWatchdog(watchDog);
 //Setting the working directory
 //Use of recursion along with the ls makes this a long running process
 executor.setWorkingDirectory(new File("/home"));
 executor.setProcessDestroyer(processDestroyer);

 //Executing the command
 executor.execute(command,resultHandler);

 //The below section depends on your need
 //Anything after this will be executed only when the command completes the execution
 resultHandler.waitFor();
 int exitValue = resultHandler.getExitValue();
 System.out.println(exitValue);
 if(executor.isFailure(exitValue)){
    System.out.println("Execution failed");
 }else{
    System.out.println("Execution Successful");
 }

Note: The Exec library has to be in your classpath- Either add it in CLASSPATH env variable or use java -cp

Please drop in any corrections, additional information as comments. I will update the post accordingly

Tagged with:
 
About The Author

Mohamed Sanaulla

12 Responses to Execute external process from within JVM using Apache Commons Exec library

  1. Anthony says:

    I don’t see any obvious advantages over using the standard Java SE classes.

    Moreover, you CAN set environment variables, using the java.lang.ProcessBuilder#environment() method, which returns a Map. Simply add extra variables to this map.

    • Mohamed Sanaulla says:

      If we explore the Exec API – We can find lot of advantages. There are features like-
      >> Associate watchdogs for each command- Kill the process if its taking too much of time or time more than what we would be expecting
      >> Easily set the Stream handlers which internally are multi threaded- So we need not worry about handling these stream readers in a multi threaded way.
      >> Execute the commands asynchronously

      These are just few which I could list from what I have implemented.

      And this article: http://www.javaworld.com/jw-12-2000/jw-1229-traps.html explains certain pitfalls associated with Runtime.exec(). So according to me- Exec libraries tries to overcome few of these.

  2. Anthony says:


    Mohamed Sanaulla:

    And this article: http://www.javaworld.com/jw-12-2000/jw-1229-traps.html explains certain pitfalls associated with Runtime.exec(). So according to me- Exec libraries tries to overcome few of these.

    You shouldn’t need Runtime.exec() anymore anyway. Just use Process & ProcessBuilder.

    Moreover, in 98% of the cases, you don’t need those few “extras” the library has to offer.

    Sorry, but you can’t convince me to try this out :)

  3. reza says:

    i think groovy adds similar features to Java. How do you compare them ?

    • Mohamed Sanaulla says:

      I haven’t tried Groovy. Do add in if you have worked on Groovy. If I can- I will try to learn and see how it compares with Exec library.

  4. Thank you for this really nice breakdown of the apache’s commons-exec libraries. I was wonderign if you know about and wouldn’t mind enhancing your article to cover inputStreamPumper?

    • Mohamed Sanaulla says:

      Thanks for reading the article.

      I dont know about InputStreamPumper, I will look into it and try to enhance the article. Thanks for the suggestion.

  5. Jit B says:

    How can I get a java.lang.String from the PumpStreamHandler?? or even a InputStream?

    the issue is that i cannot directly get a java.io.InputStream object from this. And to set one up, i need to implement the abstract read() method.

    I need to get the output of a process to parse it with regex….

  6. JavaPins says:

    Execute external process from within JVM using Apache Commons Exec library | Experiences Unlimited…

    Thank you for submitting this cool story – Trackback from JavaPins…

  7. Folarin says:

    int exitValue = resultHandler.waitFor() returns void.

    It should be resultHandler.getExitValue()

  8. Mohamed:

    Great article. Does a great job of explaining this useful library. Thanks for sharing it!

Leave a Reply