ALM Best Practices

4 Software Quality Predictions for 2012

This week, Parasoft Evangelist / Solutions Architect Arthur Hicken shares four software quality/testing trends to watch in 2012…

 

1. Something big will force the issue on secure application development

I've previously blogged about a simple yet effective way to prevent the SQL injection attacks that became increasingly common in 2011. I'd like to hope that 2012 will be the year that people start taking better precautions to safeguard their applications against known security issues. Although it might not be feasible to prevent every possible internal breach, there are proven methods to protect applications against the most common external attacks. If done properly, input validation takes care of not only SQL injection, but a host of other common security issues. By taking advantage of such proven techniques, I expect that some day we'll reach the point where SQL injection and other common attacks are a thing of the past—like Polio in the US.

Nevertheless, I predict that such attacks will continue unabated until something really nasty scares people into taking action. I think that 2012 might be the year for this. What needs to occur for people to get serious about application security? Probably either a surge in the number of security attacks or some truly devastating attack.

If the "anonymous" hacking continues to permeate a large number of fairly low-profile organizations, it might be sufficient to scare people out of the "it won't happen to me" mindset. Another possibility is that the government could get involved: either because government sites continue to get hacked or because some industry segment, such as consumer protection or banking, decides that it needs to take action because the industry is not policing itself. I could see this happening if, for example, some high profile attacks on financial institutions manage to steal not just passwords, but actual dollars. Maybe the industry will decide to regulate itself, or perhaps the government will step in.

 

2. The cloud will continue to breed a dangerous quality mindset

When a quality problem is discovered with an application deployed in the cloud, it's much easier to deploy fixes than when software is installed on individual machines (or embedded within devices). This makes it all too easy to adopt the mindset of "'let's throw it against the wall and see what sticks—if there are problems, it's not too painful to fix them later."

The fact that the cloud makes it easier to fix problems is no excuse to slight testing. If it's an in-house application for a small shop, you might get away with it. But if it's an externally-facing app, you're exposing the organization to tremendous risk. Many people assume that crashes are the worst case scenario. It's important to realize that untested software might also expose sensitive data and enable hackers to access to perform unauthorized actions—resulting in extensive damage to the business or organization that lingers long after the "fix" is deployed in the cloud.

The assumption that you can go light on testing because users will find and report problems is also a slippery slope. Slapping the "beta" label on something you toss into the marketplace does not mean you'll get the testing you need—even if you have thousands or even millions of users. As anyone who's ever run a beta program can attest, there's a very slim chance that beta users will actually exercise the software…an even slighter chance that they will notice the issues that a professional tester would…and virtually no chance that they'll report them to you (especially in a way that would allow you to reproduce and debug them). 

 

3. Testing components in isolation will be increasingly critical

End-to-end testing on true cloud applications—where instead of relying on a single server, you have a series of services that are intertwined in funny ways that are perhaps even non-deterministic—is truly painful. Debugging any problems that are exposed by that testing can be even more daunting.

People often overlook this. They think, "It's going to take a couple months to build a proper set of tests for this. Let's just let it fail, and we'll fix it later." What they're missing is that the fact that without proper testing, a monstrous amount of man hours will probably be required to find and fix the root cause of some of these problems.

How can you reduce the pain? Ideally, your system will already be architected so that each component can be tested well by itself. If not (e.g., the system is so tightly coupled and poorly designed that you can't see the discrete components), the development team should really step back and rebuild it.

Assuming that you can identify the different components, you want to test each of those components in isolation to make sure that they work correctly with a very discrete set of data that pushes all the proper boundaries. You also need to test interactions between any pairs that talk to each other. Testing trios and quads and beyond might not be necessary. But if any components might interact, they absolutely need to be tested together. A system that allows you to swap components in and out (a pair, everything except a pair, individual components, etc.) and emulate the interactions with any components you cannot readily access for testing really helps you simplify this process.

 

4. Kindle Fire will force testing for caching issues

Kindle Fire's heavy use of caching technology is likely to impact all sorts of applications that seemed to be running fine when tested in other environments.

Amazon is trying to improve performance while simultaneously cutting down network traffic. When you make a browser request through a Kindle Fire, it automatically goes to a caching server. As Erick Schonfeld wrote in TechCrunch,

The so-called split browser essentially has two homes: on the Kindle Fire itself, and in Amazon’s EC2. Basically, when a user pulls up a webpage on their Kindle Fire, EC2 handles all the rendering to optimize it for the Fire’s screen. Images are resized on the fly, and what’s more, it tracks user’s behavior. Users who visit TechCrunch all the time, will notice quicker load times because Silk detects that pattern of activity and pre-caches the site. Or similarly, if a lot of people going to the New York Times’ homepage today then click on a set of particular stories, those subsequent pages can be predictively pre-cached and delivered faster.

Not all applications handle this level of caching well, and this can have some interesting repercussions. For instance, what if a user changes his zip code to a different state—and then this doesn't end up changing the sales tax rate accordingly before a purchase?

Caching issues could come from a couple different sources. Maybe the application isn't built to handle the level of caching Amazon is doing. Or maybe there are quirks with the caching servers.

As a tester, you'll want to ensure that your application plays well with caching servers. If you're going to test an app for Kindle Fire (or somewhere else where you expect a caching server to sit between the browser and the actual server), ensure that your tests include something that permutes data from a previous request so that you should get a different answer back. And make sure to verify that a different response is actually delivered.  If needed, developers can use various tricks (like those used at amazon.com) to ensure that the caching server doesn’t just regurgitate the same old information, but rather retrieves new data from the server itself.