National Institute of Statistical Sciences
19 T. W. Alexander Drive
P.O. Box 14006
Research Triangle Park, NC 27709-4006
Tel: 919.685.9300
FAX: 919.685.9310
admin@niss.org

PROGRESS REPORT (SBR--9529926)

Code Decay in Legacy Software Systems:
Measurement, Models and Statistical Strategies

Alan F. Karr

March 15, 1998

1. Summary

Work during the period March, 1997 -- February, 1998, focused on

• Tools for analysis and visualization of large-scale software data: Section 2.
• Code decay indices defined from version management data: Section 3.
• Effects of code decay on software quality: Section 4.
• Organizational issues: Section 5.

Some additional threads of the research are noted in Section 6. Research thrusts for the coming year are summarized in Section 7.

Further information is available from the project's own Web site (www.bell-labs.com/org/11259/projects/decay/).

2. Data Analysis

The handling of large-scale software data, especially that in version management systems, is a continuing theme of the research. Tools have been developed to extract and summarize data using Web browsers to present the results. Aspects of this work appears in [2, 3, 5]; a paper collecting the tools in one place is in preparation [9].

3. Code Decay Indices

A key thread of the research has been to define code decay indices (CDIs) that quantify and predict three key responses: (1) effort and (2) interval required to make changes and (3) quality of the software products. This work appears in [4]. A principal finding has been that code decay exists and can be predicted. This is not a tautology: despite strong anecdotal evidence from developers, tools and methods did not exist previously to study code decay as a scientific phenomenon. Two key manifestations of decay are in Figures 1 and 2.

Figure 1. The Span of Changes Increases Over Time

.

1988	1989	1996 (magnified)

Table 2. Modualrity Breaks Down Over Time

Figure 1 shows that the span - number of modules touched - of changes increases over time, which (see below is clear evidence that changes are becoming more difficult. Figure 2 demonstrates that modularity of the software breaks down over time. This means that the software architecture is not able to withstand repeated changes.

Models are presented in [4] that predict the effort to implement a change (In this case, a change is a new feature in the software.) as a function of various characteristics, among them its span. One such model is

In equation (1), FILES(c) is the span of change c (The other variables represent the numbers of lines added [ADD(c)] and deleted [DEL}(c)], the interval [INT(c)] and the number of editing changes [DELTAS}(c)].) Thus, the increasing span of changes discussed above (Figure 1) matters: changes do indeed take more effort when their span is greater.

4. Quality: Fault Potential Modeling

As noted in Section 3, quality is a primary response variable for code decay; because the final product must be as fault-free as possible, quality is measured primarily by by appearance of faults during the maintenance process. Thus, the thrust of fault potential models is to predict the distribution of future faults over modules in the subsystem from the modules' change history. The best models predicted numbers of faults using numbers of changes to the module in the past, the dates of these changes, and their sizes:

FPOT_WTD(m) ~ Sum_{c "touches" m} e^{0.75 x DATE(c)}log[{ADD}(c,m) + DEL}(c,m)],

with the parameter a = .75 determined by statistical analysis. Thus, large, recent changes add the most to fault potential, and the number of times a code unit has been changed is a better predictor of the number of faults it will suffer in the future than is its size. That a is not zero is the primary (and direct) evidence that changes induce faults: were a = 0, past changes would be indistinguishable from one another, and hence none could be posited to have any specific effect.

Equally important is that other variables did not improve the predictions, once size and time of changes are taken into account. These include (1) module size or other metrics of software complexity, (2) the number of developers touching a module (One explanation is that strong organization programming standards attenuate any such effects.) and (3) concurrent changes with large numbers of other modules. This work is reported in [6].

5. Organizational Issues

Two major data collection efforts were completed:

1. A critical event history for Lucent for the same the period 1980-98 --- the same period for which version management data are available. As noted in Section 7, a key thrust in Year 3 is to link organizational and version management data. A summary of the critical event history appears in [11]. Almost needless to say, this period was turbulent: it includes not only the 1984 AT&T divestiture but also the 1996 further breakup of AT&T. Even locally within Bell Labs, changes were significant, especially the relationships of organizations developing domestic and international versions of the same software. Thus, the critical event history stands as a significant contribution in its own right.
2. A large-scale survey of software developers at Lucent, designed --- consistent with the goal to link organizational and version management data --- not only to confirm and refine the critical event history but also to probe developer perceptions of code decay and its relation to organizational events.

6. Additional Items

These include

• Further research on software maintenance [1, 8, 10].
• New methodology to evaluation regression testing techniques [10].

7. Plans for Year 3

Principal emphases will be:

• Visualization of changes,using Java applets. The goal is better understanding of software changes from both the software engineering and the organizational perspective. Two preliminary versions are shown in Figures 3 and 4. The first (various interactive features of which are not shown) enables exploration of various characteristics of changes. The second is an animation (using a VCR metaphor) that illuminates phenomena such as the increasing span of changes noted in Section 3. The customers for such visualizations are not only researchers but also managers of software development organizations.
• Effort Modeling. It has become clear that effort is the response variable of greatest interest: not only is it the primary driver of cost, but also it reflects organizational structure and practice most directly. However, the most readily available effort data (equation (1) ) are too aggregated (At the feature level.) to allow prediction of the effects of software engineering or organizational strategies to reduce effort. Currently, a statistical study is under way to disaggregate effort data using models such as equation (1); it raises new and intriguing issues having to do with the (venerable but still not definitively resolved) problem of reconstructing contingency tables from their marginal totals.
• Transferability of results. The code decay tools [9] and change visualization methods will be applied in settings other than where they were developed. (Lucent Technologies' 5ESS telephone switch.} Transfer with 5ESS -the tools were actually developed using only one of more that 50 subsystems - is being addressed currently. Application within Lucent's Visual Insights subsidiary is planned. Finally, data from other organizations are being assembled; a crucial question to be answered with the latter is how well the tools perform on less complete data (than Lucent's).
• Relating code decay to organizational events. Currently, the "statistics" and "organization" sub-teams of the project are conducting a major study (and an absolute first) relating the organization data (Section 5) to change data from the version management system. One paper is nearly complete ([11] is a condensed version); others are in earlier stages.

Figure 3. A Java Applet for Exploring Software Changes

Figure 4. A Java Animation of Key Aspects of Software Changes

References

1. Ball, T., Kim, J.-M., Porter, A. A., and Siy, H. (1997). If your version control system could talk. In Proceedings of the Workshop on Process Modeling and Empirical Studies Software Evolution.
2. Eick, S. G., Graves, T. L., Karr, A. F., and Mockus, A. (1997a). A Web laboratory for software data analysis. World Wide Web (To appear).
3. Eick, S. G., Mockus, A., Graves, T. L., and Karr, A. F. (1997b). Web-based text visualization. In Bandilla, W., and Faulbaum, F., eds., Softstat '97 Advances in Statistical Software 6 3-10.
4. Eick, S. G., Graves, T. L., Karr, A. F., Marron, J. S., and Mockus, A. (1998). Does Code Decay? Assessing the evidence from change management data. Submitted to IEEE Trans. Software Engrg.
5. Graves, T. L., Karr, A. F., and Mockus, A. (1997). Modeling software changes. In Minder, C.~E., and Friedl, H., eds., Proceedings of the 12th International Workshop on Statistical Modelling.
6. Graves, T. L., Karr, A. F., Marron, J. S., and Siy, H. (1997). Predicting software faults using change history. Submitted to IEEE Trans. Software Engrg.
7. Graves, T. L., Harrold, M. J., Kim, J.-M., Porter, A. A., and Rothermel, G. (1998). An empirical study of regression test selection techniques. International Conference on Software Engineering 1998. (To appear)
8. Karr, A. F., Porter, A. A., and Votta, L. G.. (1996). An empirical exploration of code evolution. In Proceedings of the International Workshop on Empirical Studies of Software Maintenance.
9. Mockus, A., Eick, S. G., Graves, T. L., and Karr, A. F. (1998). Tools for software data. (In preparation).
10. Porter, A. A. (1997). Fundamental laws and assumptions of software maintenance. Empirical Software Engrg. J. 2.
11. Staudenmayer, N. A., Graves, T. L., Marron, J. S., Mockus, A., Perry, D., Siy, H., and Votta, L. G. (1998). Adapting to a new environment: how a legacy software organization copes with volatility and change. In Proceedings of 5th International Product Development Management Conference. (To appear)

NISS Home Page	Help