Return on Investment (ROI)

DV8 uses multiple methods to detect high-maintenance file groups, such as anti-patterns, hotspots, or roots. An anti-pattern filegroup is not necessarily a real debt. If these files are no longer active, they will not necessarily result in excessive maintenance costs, even if they have design problems. In contrast, the hotspot file groups are active during the evaluation period, and their design problems are more likely to become real debts.

Treating these file groups as design debt, DV8 can quantify their maintenance costs and estimate the return on investment of refactoring these debts, to help the user decide where to refactor and if it is worthwhile to refactor. The anti-patterns detected among these files provide guidance on how to refactor. 

Suppose the user executes the "Analyze Software" function or uses the arch-report command in DV8  console to scan the project automatically. In that case, both the automatically generated "Root" folder and "Hotspot" folder will contain return on investment analysis spreadsheets. You can also use the arch-debt-roi command from the DV8 console to generate an ROI spreadsheet for any file groups in the form of DSM files.

Here we illustrate how to understand the return on investment (ROI) spreadsheet. Each automatically generated ROI spreadsheet has two worksheets. We first take a look at the second worksheet, as shown in the example below, that contains basic information about the project:

This row summarizes the overall project data. In this example, the project has 1304 files in total. As recorded in its revision history, there are 173 commits related to bug-fixing; These bug-fixing revisions consumed 4541 lines of code. In total, there are 11,159 commits in the given time range, consuming 568,465 lines of code, including both bug-related changes and other changes.

Column B also shows the average maintenance costs of each file: in this project, each file, on average, has 0.13 defects, requiring 3.48 lines of code to fix. Each file is changed 8.56 times on average, and each file consumes 435.94 lines of code to maintain.

The following shows the columns A to D in the first worksheet of this ROI spreadsheet. In this sample project, the eight root spaces detected by DV8 are considered as design debts, and this spreadsheet calculates how much maintenance costs can be saved if these files are refactored.  Here maintenance costs are quantified using four measures: the number of bug commits, the total number of lines of code related to bug commits, the total number of commits, and the total number of committed lines of code.

Column C contains the number of files within each root space. We normalized the sizes of the debts, as shown in Column D. Since a file can belong to more than one design debt, if we do not normalize them, we will be over-counting the impact of files in more than one design space. The normalized sizes are shown in column D.  These debts represent a total of 329 distinct files, which is 25.2% of all 1304 files in the project.

The following shows the columns E to L in the first worksheet. Columns E, G, I, and K display the maintenance costs of each debt in terms of the number of bug commits, the number of lines code fixed for bugs, the total number of changes, and the number of lines of code committed for these changes.

 

Their normalized numbers are in columns F, H, J, L, And the totals are shown in cells F12, H12, J12, and L12.  Note that although the 329 files involved in these debts account for only 25.2% of the entire project, the number of bugs related to these debt files covers 74.6% of the total bugs in the project, and the bug-related lines of code cover more than 55.5% of all lines of code. Similarly, their normalized change count is 4829, accounting for about 43.3% of the total number of changes,  and the maintenance costs consumed by these debts account for about 41.8% of all project maintenance costs in terms of lines of code.

Next, we introduce how DV8 estimates the expected benefits of removing these debts. We use project averages as the basis for estimation. The assumption is that refactoring these debts will bring their maintenance costs down to the project average. This is a conservative assumption for the following reasons: first, the project averages already contain these defective files, which will inflate the average values; Second, refactoring may result in a better design and hence lower maintenance costs than the current project averages.

Based on this assumption, columns M, N, O, and P, as shown below, list the expected costs associated with each file group after refactoring. That is, the refactored files are expected to exhibit average project behavior.

Finally, the expected "savings" are given in cells M14, N14, O14, and P14, that is, the differences between the expected maintenance costs after refactoring and the current average numbers. The percentage of saving is shown in the cells below. 

We ignore time and reputation loss due to avoidable defects and only focus on the number of lines of code that the project can save after refactoring. In this project, by refactoring these file groups with design problems, the project is expected to save 94,144 lines of code.

In case studies with industrial collaborators, we transformed lines of code into saved person-months of effort based on current productivity rates. Since each debt reported by DV8 pinpoints which files need to be refactored, and the anti-patterns within these files suggest possible refactoring strategies, a user can easily estimate the person-months required to implement the factoring and thus calculate return on investment more precisely, in units of person-months.