How to properly handle the output of the ProcessBuilder

It’s time for a rewrite of my old post https://thilosdevblog.wordpress.com/2011/11/21/proper-handling-of-the-processbuilder, because since Java 9 there are some new nice features, which make things easier.

There are many topics around the ProcessBuilder. In this post the focus is how to handle the output of executed commands correctly!

The following two things are important:

• Consuming STDOUT and STDERR is necessary to avoid freezing!
• If the process writes a large amount of output, it must be consumed.
  This can be done by calling #redirectErrorStream, which redirects STDERR to STDOUT!

Have look at my ProcessBuilderWrapper.

/**
 * GPL
 */
public class ProcessBuilderWrapper {
	private String[] command;
	private File workingDirectory;
	
	// exit value of the process
	private int status;

	private boolean redirectToStdout;
	private boolean redirectErrorStream;

	private ByteArrayOutputStream outStd = new ByteArrayOutputStream();
	private ByteArrayOutputStream outError = new ByteArrayOutputStream();

	// environment variables, that are visible to the process
	private Map<String, String> environment = null;

	public ProcessBuilderWrapper(String... command) {
		this.command = command;
	}

	public void run() throws IOException, InterruptedException {
		ProcessBuilder pb = new ProcessBuilder(command);
		if(workingDirectory != null)
			pb.directory(workingDirectory);
		if(environment != null && environment.size() > 0)
			pb.environment().putAll(environment);

		pb.redirectErrorStream(redirectErrorStream);

		Process process = pb.start();
		try (var infoStream = process.getInputStream(); var errorStream = process.getErrorStream()) {
			if(redirectToStdout) {
				infoStream.transferTo(System.out);
				errorStream.transferTo(System.out);
			} else {
				infoStream.transferTo(this.outStd);
				errorStream.transferTo(this.outError);
			}
		}
		status = process.waitFor();
	}

	public void redirectToStdOut() {
		this.redirectToStdout = true;
	}

	public void redirectErrorStream() {
		this.redirectErrorStream = true;
	}

	public void setWorkingDirectory(String dir) {
		this.workingDirectory = Paths.get(dir).toFile();
	}

	public void setEnvironment(Map<String, String> environment) {
		this.environment = environment;
	}

	public int getStatus() {
		return status;
	}

	public String getOutput() {
		return (redirectToStdout) ? "n/a" : outStd.toString();
	}

	public String getError() {
		return (redirectToStdout) ? "n/a" : outError.toString();
	}

	public boolean hasErrors() {
		return getStatus() != 0;
	}
}

Usage for regular commands

To run a regular command, you just have to do something like this:

ProcessBuilderWrapper pbw = new ProcessBuilderWrapper("ls", "-al");
pbw.setWorkingDirectory("/tmp");
pbw.redirectErrorStream();
pbw.run();
if(!pbw.hasErrors())
	System.out.println(pbw.getOutput());

Time consuming commands

For time consuming commands it could be important to see some progress information or output of the command. In this case, the inputstreams of the Process can be redirected to standard out.

To test my ProcessBuilderWrapper, I used the following time consuming bash script (sleep.sh):

#!/bin/bash
for i in {1..10} ; do
   echo "Step: ${i}"
   sleep 3s
done

The call of the ProcessBuilderWrapper:

ProcessBuilderWrapper pbw = new ProcessBuilderWrapper("bash", "sleep.sh");
pbw.setWorkingDirectory([path_to_the_script]);
pbw.redirectToStdOut();
pbw.run();
if(!pbw.hasErrors())
	System.out.println(pbw.getOutput());

Error case

To test the error case, I used the following bash script (error.sh):

#!/bin/bash

echo "Text on Stdtout"
echo "Text on StdtErr1" >/dev/stderr
echo "Text on StdtErr2" >&2
exit 1

And the call of the ProcessBuilderWrapper:

ProcessBuilderWrapper pbw = new ProcessBuilderWrapper("bash", "error.sh");
pbw.setWorkingDirectory([path_to_the_script]);
pbw.redirectToStdOut();
pbw.run();
if(!pbw.hasErrors())
	System.out.println(pbw.getOutput());

It’s tested with mascOS and Linux.

Proper handling of the ProcessBuilder

This post is outdated! A newer Version can bei found here: How to handle the output of the ProcessBuilder!

This article describes the proper handling of the streams which are provided by the ProccessBuilder. If you are interested in the common usage and the difference between Runtime and Processbuilder, refere from Runtime.exec to ProcessBuilder.

I had good experiences with the ProcessBuilder to call native commands. That’s why I would like to use it for another project. But testing it in the new project environment, it often hung.

But why and what’s the difference to the other project? It was the quantity of output.

In the JDK documentation of the ProcessBuilder I’ve found right hint:

The parent process uses these streams (#getInputStream(), #getErrorStream()) to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.

That means: The streams must be read!!! Depending on the native platform, the quantity of output (info or error, what ever) can cause a buffer overflow and therefore an hanging Process.

The code above shows is a wrapper to the ProcessBuilder which boozes the streams:

/**
 * LGPL
 */
public class ProcessBuilderWrapper {
	private StringWriter infos;
	private StringWriter errors;
	private int status;

	public ProcessBuilderWrapper(File directory, List command) throws Exception {
		infos = new StringWriter();
		errors = new StringWriter();
		ProcessBuilder pb = new ProcessBuilder(command);
		if(directory != null)
			pb.directory(directory);
		Process process = pb.start();
		StreamBoozer seInfo = new StreamBoozer(process.getInputStream(), new PrintWriter(infos, true));
		StreamBoozer seError = new StreamBoozer(process.getErrorStream(), new PrintWriter(errors, true));
		seInfo.start();
		seError.start();
		status = process.waitFor();
		seInfo.join();
		seError.join();
	}

	public ProcessBuilderWrapper(List command) throws Exception {
		this(null, command);
	}

	public String getErrors() {
		return errors.toString();
	}

	public String getInfos() {
		return infos.toString();
	}

	public int getStatus() {
		return status;
	}

	class StreamBoozer extends Thread {
		private InputStream in;
		private PrintWriter pw;

		StreamBoozer(InputStream in, PrintWriter pw) {
			this.in = in;
			this.pw = pw;
		}

		@Override
		public void run() {
			BufferedReader br = null;
			try {
				br = new BufferedReader(new InputStreamReader(in));
				String line = null;
	            while ( (line = br.readLine()) != null) {
	            	pw.println(line);
	            }
			} catch (Exception e) {
				e.printStackTrace();
			} finally {
				try {
					br.close();
				} catch (IOException e) {
					e.printStackTrace();
				}
			}
		}
	}

Example:
List cmd = new ArrayList();
cmd.add("ls");
cmd.add("-al");
ProcessBuilderWrapper pbd = new ProcessBuilderWrapper(new File("/tmp"), cmd);
System.out.println("Command has terminated with status: " + pbd.getStatus());
System.out.println("Output:\n" + pbd.getInfos());
System.out.println("Error: " + pbd.getErrors());