Posted on Thu, Jan 26, 2012
By Wayne Ariola, VP of Strategy and Corporate Development
Being in the software testing and quality space, I'm always hesitant to deliver predictions for the coming year. Personally, it turns out to be a mental battle between what I want to see and what I think will actually occur.
From a software quality and software development lifecycle perspective, the market moves extremely slowly. Trends and technologies that seem like “no-brainers” for adoption slowly eek their way into an organization's infrastructure at a pace that is beyond conservative.
Instead of making actual technical predictions for 2012, let me entertain a couple of “concepts” that I feel will become broader discussion topics.
1. Switching costs for software solutions will continue to decline.
In the past, when an organization decided to invest in a specific vendor’s software solution, the organization was forging a marriage that could end only in a costly divorce. Software vendors were particularly keen to architect various methods to lock in the client by making it very difficult to separate the vendor’s software from the client’s process. The cost of change far exceeded the cost to customize or extend the currently-implemented solution.
With integration and message communication standards very reliable, browser-based technologies becoming more malleable, and software-as-a-service becoming more acceptable, we are entering an era when switching costs are in decline. Of course, we won’t leave out the impact of change on people as the primary hurdle. However, even people are becoming more amenable to technical change.
For example, let's take a look at mobile phones, mobile apps and internet-based services. If there's a better site to find and purchase airline tickets, then I’m gone to the new site—zero switching costs. If there's a better ninety-nine cent app for my smart phone, then I’m gone to the new app—very, very low switching costs. If a new SaaS-based, subscription-based business service is better than my old application, then I’m gone to the new service—some OpEx and training costs.
In summary, development productivity and software quality count now more than ever! Don’t give your clients a reason to check out the competition.
2. Compliance standards for safety-critical and life-critical software will become imperative.
Device manufacturers that embed software for safety-critical and life-critical applications have been on-notice for years now about process and development techniques to better verify and validate their software. This will make the penalties for non-compliance increase significantly as well as introduce the potential for punitive measures.
The cost of quality is a battle that has been waged for years. However there are some industries where academic conversation must be thrown out the window because the true costs of poor quality have life-critical or safety-critical circumstances. The maturity of testing tools and quality methods—as well as standard software development processes—means that there is no excuse for non-compliance.
No longer can we complain that unit testing is too tedious or debate the ROI of defect prevention techniques. We must take the stance that if developers and engineers fail to embrace the industry-standard quality processes and write code in a way that reduces risk, they are making potentially critical business decisions.
3. With software presenting a much broader business risk, time to market will shift (slightly) in favor of market-ready.
Ok, I admit that this is more of a dream than a prediction. But the bottom line here is that there's no longer an excuse to not invest in QA. Conversely, we can no longer afford to compress quality efforts into a tiny post-development cycle—we need to look at software quality as a continuous process.
This is where mature software products can assist the development and QA teams to better manage the quality lifecycle when faced with incomplete or evolving software components. Technologies such as Application Behavior Virtualization (or Service Virtualization) can significantly impact the overall quality of an application while assisting the organization to meet market-imposed deadlines.
Furthermore, more collaborative test management frameworks will allow for the true reuse of testing assets throughout the software development lifecycle. Major vendors who have been trying to “lock-out” competition (I’m sure you can’t imagine who I’m talking about) will find it pretty difficult to keep up with the market demand for more ubiquitous collaboration.
Posted on Thu, Jan 19, 2012
IMA Reduces the Time and Cost of Regulatory Compliance through Static Analysis
Since 1961, IMA has been designing and producing implants and equipment for packaging in the food and pharmaceutical industries. IMA products are characterized by a high level of personalization to meet the client's specific needs—in both single packaging machines and complete production plants. In 2011, IMA celebrated its 50th year of business. Today, it has over 3400 employees; 22 production plants spread throughout Italy, Germany, U. K., U.S.A., India, and China; and a large network of agents and affiliates spanning virtually the entire globe.
By working with Parasoft , IMA significantly increased the efficiency and auditability of the strict quality process they adopted to comply with pharmaceutical industry regulations.
The Challenge: Reducing the Burden of Complying with Pharmaceutical Industry Standards
In order to develop solutions for the pharmaceutical market, a company must not only follow very strict requirements, but also demonstrate that they actually satisfied these rigorous expectations. To achieve this, they must provide evidence that the system is designed, constructed, and tested according to best practices implemented throughout the various phases of the lifecycle. These demands apply to software—which is becoming an increasingly critical and complex component of such systems—as well as system hardware.
To reduce the burden of complying with these strict guidelines, IMA started to research products that could help them manage the software development lifecycle—especially the testing and verification of the software that drives IMA machinery functionality. Specifically, they were looking to streamline the complementary processes of a) writing code according to predefined standards and b) verifying whether the code successfully follows those standards.
Beni Fricano, IMA Quality Assurance Manager, explained, "This is a process that, if done manually, would be arduous, expensive, difficult to document, and produce contestable results. After all, how can we demonstrate and document that among thousands of lines of code, not one line of dead code has escaped checking? These challenges are exacerbated by the fact that the manual processes used to achieve compliance on the software for one registered machine cannot readily be reused to check the software for another machine."
The Solution: Static Analysis with a Robust, Easily-Configurable Flexible Rule Set
To tackle this challenge, IMA turned to Parasoft. Years ago, they started using Parasoft CodeWizard, a static analysis product that is the predecessor of the current Parasoft C/C++ test product. The product shipped with a preconfigured set of rules that could be easily customized to suit the needs of a given project and/or industry.
They liked how simple it was to use and configure the tool in order to check the guidelines that were important for them. Fricano noted that "the icing on the cake" was the RuleWizard tool, which allowed them to modify the built-in rules as well as graphically define additional rules for any custom coding guidelines they decided to check. This tool was instrumental in enabling IMA to automate checking of the specialized set of standards they needed to follow for compliance purposes.
Since then, IMA has migrated to Parasoft C/C++test, which extends the basic Parasoft CodeWizard coding standard checking with comprehensive static analysis (static code analysis, flow analysis, metrics), peer code review, unit testing, and runtime error detection.
Results: Faster, Easier, and Better-Documented Compliance Efforts
According to Fricano, the solution delivered value immediately. "Right away, previously arduous, boring and difficult-to-document tasks were transformed into tasks that we could perform systematically—in an automated and rapid manner. Parasoft's reports can be attached to the documentation supplied to the client to prove that a task was completed. This is definitely less contestable than a manually-created report."
As soon as they started working with Parasoft's static analysis, the developers appreciated having an easy-to-use tool that provided many features (static analysis as well as code review and rich dynamic analysis capabilities introduced by Parasoft C/C++test) in a single integrated environment. They could apply it as required for compliance—with virtually no extra effort.
***
Posted on Thu, Jan 12, 2012
This week, Parasoft Evangelist / Solutions Architect Arthur Hicken shares four software quality/testing trends to watch in 2012…
1. Something big will force the issue on secure application development
I've previously blogged about a simple yet effective way to prevent the SQL injection attacks that became increasingly common in 2011. I'd like to hope that 2012 will be the year that people start taking better precautions to safeguard their applications against known security issues. Although it might not be feasible to prevent every possible internal breach, there are proven methods to protect applications against the most common external attacks. If done properly, input validation takes care of not only SQL injection, but a host of other common security issues. By taking advantage of such proven techniques, I expect that some day we'll reach the point where SQL injection and other common attacks are a thing of the past—like Polio in the US.
Nevertheless, I predict that such attacks will continue unabated until something really nasty scares people into taking action. I think that 2012 might be the year for this. What needs to occur for people to get serious about application security? Probably either a surge in the number of security attacks or some truly devastating attack.
If the "anonymous" hacking continues to permeate a large number of fairly low-profile organizations, it might be sufficient to scare people out of the "it won't happen to me" mindset. Another possibility is that the government could get involved: either because government sites continue to get hacked or because some industry segment, such as consumer protection or banking, decides that it needs to take action because the industry is not policing itself. I could see this happening if, for example, some high profile attacks on financial institutions manage to steal not just passwords, but actual dollars. Maybe the industry will decide to regulate itself, or perhaps the government will step in.
2. The cloud will continue to breed a dangerous quality mindset
When a quality problem is discovered with an application deployed in the cloud, it's much easier to deploy fixes than when software is installed on individual machines (or embedded within devices). This makes it all too easy to adopt the mindset of "'let's throw it against the wall and see what sticks—if there are problems, it's not too painful to fix them later."
The fact that the cloud makes it easier to fix problems is no excuse to slight testing. If it's an in-house application for a small shop, you might get away with it. But if it's an externally-facing app, you're exposing the organization to tremendous risk. Many people assume that crashes are the worst case scenario. It's important to realize that untested software might also expose sensitive data and enable hackers to access to perform unauthorized actions—resulting in extensive damage to the business or organization that lingers long after the "fix" is deployed in the cloud.
The assumption that you can go light on testing because users will find and report problems is also a slippery slope. Slapping the "beta" label on something you toss into the marketplace does not mean you'll get the testing you need—even if you have thousands or even millions of users. As anyone who's ever run a beta program can attest, there's a very slim chance that beta users will actually exercise the software…an even slighter chance that they will notice the issues that a professional tester would…and virtually no chance that they'll report them to you (especially in a way that would allow you to reproduce and debug them).
3. Testing components in isolation will be increasingly critical
End-to-end testing on true cloud applications—where instead of relying on a single server, you have a series of services that are intertwined in funny ways that are perhaps even non-deterministic—is truly painful. Debugging any problems that are exposed by that testing can be even more daunting.
People often overlook this. They think, "It's going to take a couple months to build a proper set of tests for this. Let's just let it fail, and we'll fix it later." What they're missing is that the fact that without proper testing, a monstrous amount of man hours will probably be required to find and fix the root cause of some of these problems.
How can you reduce the pain? Ideally, your system will already be architected so that each component can be tested well by itself. If not (e.g., the system is so tightly coupled and poorly designed that you can't see the discrete components), the development team should really step back and rebuild it.
Assuming that you can identify the different components, you want to test each of those components in isolation to make sure that they work correctly with a very discrete set of data that pushes all the proper boundaries. You also need to test interactions between any pairs that talk to each other. Testing trios and quads and beyond might not be necessary. But if any components might interact, they absolutely need to be tested together. A system that allows you to swap components in and out (a pair, everything except a pair, individual components, etc.) and emulate the interactions with any components you cannot readily access for testing really helps you simplify this process.
4. Kindle Fire will force testing for caching issues
Kindle Fire's heavy use of caching technology is likely to impact all sorts of applications that seemed to be running fine when tested in other environments.
Amazon is trying to improve performance while simultaneously cutting down network traffic. When you make a browser request through a Kindle Fire, it automatically goes to a caching server. As Erick Schonfeld wrote in TechCrunch,
The so-called split browser essentially has two homes: on the Kindle Fire itself, and in Amazon’s EC2. Basically, when a user pulls up a webpage on their Kindle Fire, EC2 handles all the rendering to optimize it for the Fire’s screen. Images are resized on the fly, and what’s more, it tracks user’s behavior. Users who visit TechCrunch all the time, will notice quicker load times because Silk detects that pattern of activity and pre-caches the site. Or similarly, if a lot of people going to the New York Times’ homepage today then click on a set of particular stories, those subsequent pages can be predictively pre-cached and delivered faster.
Not all applications handle this level of caching well, and this can have some interesting repercussions. For instance, what if a user changes his zip code to a different state—and then this doesn't end up changing the sales tax rate accordingly before a purchase?
Caching issues could come from a couple different sources. Maybe the application isn't built to handle the level of caching Amazon is doing. Or maybe there are quirks with the caching servers.
As a tester, you'll want to ensure that your application plays well with caching servers. If you're going to test an app for Kindle Fire (or somewhere else where you expect a caching server to sit between the browser and the actual server), ensure that your tests include something that permutes data from a previous request so that you should get a different answer back. And make sure to verify that a different response is actually delivered. If needed, developers can use various tricks (like those used at amazon.com) to ensure that the caching server doesn’t just regurgitate the same old information, but rather retrieves new data from the server itself.
Posted on Thu, Jan 05, 2012
Live Webinar – January 19
Join our Load & Performance Testing Made Easy webinar on Thursday January 19 to learn how to simplify load testing for today's complex, distributed, and rapidly-changing SOA/composite applications.
This webinar will demonstrate how Parasoft SOAtest and Parasoft Load Test can:
- Create realistic traffic distributions to simulate real-world load scenarios
- Virtualize missing components to test in incomplete environments
- Leverage JUnit and "pluggable" code components to load test systems via custom APIs
Best Practices eBook
In the meantime, if you’d like to read about general SOA performance & load testing strategies, we invite you to browse our Load Testing Web Services eBook.
This 15-page eBook explains how to:
- Overcome the problems with the traditional and "test early, test often" approaches
- Automate continuous test execution & QoS analysis
- Evolve performance test profiles as the application changes
- Reuse functional tests for load testing
- Use data sources & dynamically-generated data to increase test scope
- Zero in on the root cause of performance problems
- Emulate real-world request streams and use case scenarios
Posted on Wed, Dec 28, 2011
Here are the 10 most-read white papers and eKits from the Parasoft
Software Testing and Software Quality Resource Center in 2011:
-
Agile Quality: Increase Velocity While Controlling Defects Aggregates 10 tips for ensuring that each new feature is truly "done"...and defects don't derail your Agile project.
-
Safety-Critical Development Starter Kit Introduces proven strategies for controlling safety risks and complying with DO-178B, MISRA, JSF, ISO 26262, IEC 61508 or other guidelines.
- The Next Generation of Test Environment Management Introduces Application Behavior Virtualization, which gives teams the freedom to exercise their composite applications in incomplete, constantly evolving, and/or difficult-to-access environments.
-
Static Analysis for FDA Software Validation Compliance Explains what the FDA software validation guidance expects in terms of static analysis and medical device software compliance.
-
Unit Testing Starter Kit Provides practical tips on how adopt unit testing without all the traditional hassles—so you can obtain real value from unit testing.
-
FDA Software Validation Guidance Learn how one company cut compliance costs 50%— and get tips for your own compliance.
-
Load Testing Web Services Presents practical tips on how to automate web services load testing as part of the SOA testing process.
-
End-to-End Functional Testing and Validation Demonstrates how to leverage automated end-to-end testing and application behavior virtualization to execute and validate all critical aspects of transactions directly from the test environment— even when some components are unavailable.
-
Unit Testing for Java Enterprise Applications: Extend Beyond JUnit Explains why a more comprehensive regression test suite is required for ensuring the continued functionality of Java EE apps, then covers how to build and maintain such a test suite.
- Increase Agile Productivity Offers 4 tips for achieving greater efficiency in Agile development projects...up to 10X.
Posted on Wed, Dec 21, 2011
Here are the 10 most-read
Parasoft Blog articles from 2011:
- How Can Test-Driven Development (TDD) Be Automated? Explores how automation can make Test-Driven Development (TDD) less painful and more valuable.
- Static Analysis for FDA: 3 Keys to Success Presents 3 keys to success with static analysis for FDA/medical device software validation.
- Test Code Coverage Metrics Explores code coverage metrics including statement coverage (block coverage), branch coverage, and path coverage.
- Find More Bugs in Embedded C Software Outlines how developers can use pattern-based static code analysis, runtime error detection, unit testing, and flow analysis together to find more bugs in an embedded C application.
- Static Analysis: One Piece of the FDA Compliance Puzzle Introduces what the FDA software validation guidance expects in terms of static analysis and medical device software compliance.
- Medical Device Software Validation & Compliance: A Case Study Discusses how Inomed was able to rapidly automate their established processes for IEC 62304 medical device software compliance—as well as introduce full requirements traceability—to significantly reduce the work required to achieve and sustain the mandated medical device software compliance certification.
- Agile Quality Tip #4 - Use Defect Detection for Defect Prevention Discusses how to use each error detected as an opportunity to immunize your software development process against similar software defects.
- Agile Quality Tip #8 - Reduce End-to-End Testing Complexity Explains how Application-Behavior Virtualization removes end-to-end testing constraints.
- New .NET Static Analysis Article Introduces static analysis techniques (static code analysis, data flow analysis, metrics), as well as provides tips on how to apply them to .NET (C#, VB.NET, etc.).
- Agile Quality Tip #5 - Manage Code Complexity Discusses how managing code complexity reduces the time and effort required to modify and extend code in Agile development processes.
Posted on Thu, Dec 15, 2011
In practice, not many software projects have the luxury of building and maintaining a decent and up-to-date unit test suite.
This may change, especially for embedded systems, as the demand for delivering quality software continues to grow. International standards such as IEC-61508, ISO-26262, and DO-178B require module testing for a given functional safety level. Unit testing at the module level helps to satisfy this requirement.
Read the complete "Making Unit Testing Practical for Embedded Development" article by Parasoft's Marek Kucharski to learn how to start verifying code as soon as it is completed—even if the target hardware is not yet built or available for testing. With this strategy, the majority of the problems with the application logic are exposed early and target testing can focus on verifying the interface between the hardware and the software.
***
To explore additional embedded testing white papers, articles, and videos/webinars, visit our Embedded Testing Resource Center. Or, visit the Parasoft Embedded page.
Posted on Thu, Dec 08, 2011
Want to learn about a new strategy for eliminating the hassles of test environment management?
Join us on for the Service Virtualization: Test Environments on Demand webinar on Wednesday December 14 (1pm ET, 10am PT).
Constraints inherent in today's complex, highly integrated systems limit access to realistic test environments—wreaking havoc on functional and performance testing efforts. Hardware/OS virtualization eliminates some environment constraints, yet it doesn't solve the problem for large mainframes, ERPs or 3rd-party systems.
Application-Behavior Virtualization (an extension of Service Virtualization) is the missing piece of the puzzle. This ground-breaking technology provides developers & testers access to any environment needed to develop, test, or validate an application...faster and at a lower cost.
Reseve your spot for next week's live webinar if you want to learn about how Application-Behavior Virtualization works and how you can apply it for performance testing, extended functional testing, and agile/parallel development.
Posted on Mon, Dec 05, 2011
Wednesday, December 7, 2011 — 10:00 a.m. PT / 1:00 p.m. ET
Join us this Wednesday as Parasoft's Arthur Hicken explores the Top 10 Mistakes with Static Analysis.
Hicken has been helping software development organizations implement and optimize static analysis since 1996. Draw on his experience to learn the top 10 reasons why static analysis initiatives fail to deliver the expected results—and how you can steer clear of these common pitfalls.
The webinar will cover mistakes related to:
- Workflow integration
- Number of rules
- The audit approach
- Management involvement
- Training
- Process evolution
- and more
The webinar is THIS WEEK... sign up today.
Posted on Thu, Dec 01, 2011
Although software verification techniques can help developers find specific types of defects in embedded C software, they can overlook some errors. To rest assured that defects aren’t slipping through the cracks, developers should apply complementary techniques in concert.
Read the Integrating error-detection techniques to find more bugs in embedded C software article by Parasoft's Marek Kucharski and Miroslaw Zielinski to learn how automated techniques such as pattern-based static code analysis, runtime memory monitoring, unit testing, and flow analysis can be used together to find bugs in an embedded C application.
The article demonstrates the recommended bug-finding strategies in the context of a simple sensor application running on an ARM Cortex-M3 board.
***
To explore additional embedded testing white papers, articles, and videos/webinars, visit our Embedded Testing Resource Center. Or, visit the Parasoft Embedded page.