Catching the Uncaught: How We Uncovered Java Bugs Lurking in Prod

Writing code that's both robust and bug-free is a complex challenge. Developers often focus on creating tests for the scenarios they believe will occur in production while overlooking less common, unforeseen use cases. Now imagine a situation where a portion of the code, written and forgotten, never gets executed in production—until one day it does, throwing a bug. How did this happen? Why was there no unit test to catch it? It's a familiar scenario for many developers: writing minimal test cases for the given production scenario, then moving on. But what if production scenarios change over time? What if the untested code suddenly starts executing, and the bug goes undetected?

This isn't just hypothetical; it's a problem we recently faced here at Sapient.ai. We discovered that human error, lack of time, and shifting priorities led to latent bugs lurking in our codebase. As production use cases evolved, these hidden faults were waiting to surface. We fell victim to this human error, but through the diligent use of our own product, Sapient, we were able to catch those hidden Java bugs before they wreaked havoc in production.

In this post, we'll jump into this very real challenge by sharing two specific examples of how Sapient.ai helped us identify and rectify issues that would otherwise have gone unnoticed. It's a lesson in the dangers of kicking the can down the road, assuming someone else will catch the error, or that it will become evident in production. Read on to learn how we uncovered these hidden pitfalls and what you can do to avoid a similar fate.

‍

Example 1: Null-Pointer Exception in getFieldsAsList()

Consider a class where we generate a test from a simple method named getFieldsAsList():

We ran our own Sapient-generated test, and immediately found the following null-pointer exception thrown:

The root cause of this exception lies in the constructor, where the fields are allowed to be null:

‍

This lack of null-checking means that when the getFieldsAsList() function is called, there is no guarantee that the fields are not null.

To address this issue, a straightforward fix is creating an empty set in the constructor when the input classFields is null or adding a null check within functions like getFieldsAsList(). After implementing this fix, the generated unit tests run without issues, ensuring the method is more robust and resistant to potential null-pointer exceptions.

‍

Example 2: String Index Out of Range Exception

In the second example, the generated unit test uncovers an exception within the following code segment:

Again, we ran another Sapient-generated test, and the execution of the test results in a String index out of range exception:

A close examination reveals confusion between two similar variables: variableName (an input to the function) and validVariableName. The error occurs because the check for a non-empty string is performed on the wrong variable (variableName instead of validVariableName), leading to the exception.

Fixing this bug requires adjusting the code to perform the check on the correct variable (validVariableName). Once corrected, the generated test runs successfully, showcasing the utility of Sapient.ai in detecting and addressing even subtle coding errors.

‍

Conclusion

The journey of uncovering hidden bugs within our own code at Sapient.ai illustrates a universal challenge faced by Java developers and QA engineers worldwide. It's not just about the obvious errors and production scenarios; it's about those unexpected, hidden faults that can lurk undetected for months or even years.

We learned from this experience that no matter how diligent or experienced we are, human error and oversight can allow faults to slip through the cracks. The issue is not isolated to our team; it's a systemic problem prevalent in software development.

The true power of Sapient lies in its ability to help us navigate these murky waters. By automating unit test generation and rigorously scrutinizing our code, it detected errors we might have otherwise missed. It's a vivid reminder of why comprehensive testing, augmented by intelligent tools like Sapient.ai, is not just a best practice but a necessity.

These examples are not just lessons learned; they are calls to action for all developers to rethink their approach to testing. Let's not wait for the ticking time bomb to explode in production. Let's catch those hidden bugs, address those unseen issues, and build more resilient software. With Sapient.ai at our side, we're not just hoping for error-free code but actively ensuring it. The future of bug-free Java programming is here, powered by intelligence, diligence, and the innovative technology of Sapient.ai.

‍