Debugging resource leaks through Java Streams

Any Java programmer worth their salt will know about the Stream<T> interface and how to use it for data processing. However, incorrect use can lead to resource leaks if the source of the stream is some resource that needs to be closed.

Stream extends the Autocloseable interface, and also allows for onClose callbacks to be added to do cleanup work.

In this example a Stream<String> is returned which reads lines from a File. The onClose handler closes the underlying file handle. If the stream is not closed properly, the handler will not be called and there will be a resource leak.

Stream<String> readLinesAsStream(String filename) throws IOException {
    File file = new File(filename);
	BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(file)));
	Iterator<String> iterator = new Iterator<String>() {
		public String next() {
		    try {
			    return reader.readLine();
		    } catch (IOException e) {
		        throw new RuntimeException(e);
		    }
		}

		public boolean hasNext() {
		    try {
			    return reader.ready();
		    } catch (IOException e) {
		        throw new RuntimeException(e);
		    }
		}
	};
	return StreamSupport.stream(Spliterators.spliteratorUnknownSize(iterator, 0), false)
        .onClose(() -> {
            try {
                reader.close();
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        });
}

void correctUsage() throws IOException {
	try (Stream<String> stream = readLinesAsStream("/tmp/file.txt")) {
		stream.forEach(System.out::println);
	}
}

void incorrectUsageWhichLeaks() throws IOException {
	readLinesAsStream("/tmp/file.txt").forEach(System.out::println);
}

This was a problem that caught out my team. We had a stream-based API on top of a JDBC layer which interacted with our database; akin APIs like jOOQ and JPA . This gave the advantage of a fluent API for accessing data, but the disadvantage of leaking database connections if not closed correctly.

When I arrived on the project I was told about this problem and set about trying to solve it.

At first I tried using IDE settings and static analysis tools to try and find the leaks. However, they didn’t seem to track resources managed through aStream especially as they were passed back through long call chains (where various methods acted on the stream).

However, the Errorprone compiler wrapper had a nifty solution in the form of the @MustBeClosed annotation. This can be added to methods which return a Stream which must be closed, and warnings will be given if its not handled correctly. This seemed to work through long call chains as expected and added very little compile time overhead. After introducing this annotation in the right places in the API, I was able to track down the leaks and fix them.

Here is an example of the annotation applied to a method.

@MustBeClosed
Stream<String> readLinesAsStream(File file) {
	...
}