A Survey of Tool Support for Working with Design Decisions in Code (ACM CSUR 2023)

Whenever developers choose among alternative technical approaches, they make a design decision. Collectively, design decisions shape how software implements its requirements and shape non-functional quality attributes such as maintainability, extensibility, and performance. Developers work with design decisions both when identifying, choosing, and documenting alternatives and when later work requires following and understanding previously made design decisions. Design decisions encompass design rationale, describing the alternatives and justification for a design choice, as well as design rules, describing the constraints imposed by specific alternatives. This article summarizes and classifies research on these activities, examining different approaches through which tools may support developers in working with design decisions in code. We focus both on the technical aspects of tools as well as the human aspects of how tools support developers. Our survey identifies goals developers have in working with design decisions throughout the lifecycle of design decisions. We also examine the potential support tools may offer developers in achieving these goals and the challenges in offering better support.

Sahar Mehrpour, Thomas D. LaToza.

DOI, Pre-Print, Supplementary Materials

Can static analysis tools find more defects? (EMSE 2023)

Static analysis tools find defects in code, checking code against rules to reveal potential defects. Many studies have evaluated these tools by measuring their ability to detect known defects in code. But these studies measure the current state of tools rather than their future potential to find more defects. To investigate the prospects for tools to find more defects, we conducted a study where we formulated each issue raised by a code reviewer as a violation of a rule, which we then compared to what static analysis tools might potentially check. We first gathered a corpus of 1323 defects found through code review. Through a qualitative analysis process, for each defect we identified a violated rule and the type of Static Analysis Tool (SAT) which might check this rule. We found that SATs might, in principle, be used to detect as many as 76% of code review defects, considerably more than current tools have been demonstrated to successfully detect. Among a variety of types of SATs, Style Checkers and AST Pattern Checkers had the broadest coverage of defects, each with the potential to detect 25% of all code review defects. We found that static analysis tools might be able to detect more code review defects by better supporting the creation of project-specific rules. We also investigated the characteristics of code review defects not detectable by traditional static analysis techniques, which to detect might require tools which simulate human judgements about code.

Sahar Mehrpour, Thomas D. LaToza.

DOI, Pre-Print, Dataset

Programming Tools for Working with Design Decisions in Code (PLATEAU 2021)

When writing code, developers make design decisions by choosing between alternatives. Subsequent work with code requires reasoning about these design decisions, ensuring their code is consistent and answering rationale questions about why they were made. While documentation might help, it is rarely updated and often incomplete and untrustworthy. We propose a new form of documentation which is active, making design decisions checkable and offering immediate feedback on violations as they occur. Active documentation helps developers reason about design decisions by offering explanations and linked code snippets illustrating how to follow a design decision. To ensure active documentation is easy to create and maintain, new ways to create and edit checkable design decisions are needed. We offer a vision for active documentation, offer evidence for its potential, describe several techniques to achieve it, and suggest future directions for programming tools to better support it.

Sahar Mehrpour, Thomas D. LaToza.

DOI, Pre-Print, Slides.

RulePad (ESEC/FSE 2020)

Good documentation offers the promise of enabling developers to easily understand design decisions. Unfortunately, in practice, design documents are often rarely updated, becoming inaccurate, incomplete, and untrustworthy. A better solution is to enable developers to write down design rules which may be checked against code for consistency. But existing rule checkers require learning specialized query languages or program analysis frameworks, offering a barrier to writing project-specific rules. We introduce two new authoring techniques for design rules: snippet-based authoring and semi-natural-language authoring. In snippet-based authoring, developers specify characteristics of elements to match by writing partial code snippets. In semi-natural language authoring, a textual representation offers a representation for understanding design rules and resolving ambiguities, which is bidirectionally synchronized. We implemented these approaches in RulePad.

RulePad enables novice developers to write design rules in a code-based template by specifying the elements and conditions on each element. While authoring rules in the tool, RulePad shows the written format of the design rule along with extracted snippets from the actual codebase. The former feature helps developers learn the language used in the tool and later use the semi-natural language interface for authoring design rules. The latter feature allows developers to validate the design rules and make changes if necessary.

Sahar Mehrpour, Thomas D. LaToza, Hamed Sarvari.

DOI, Pre-Print, Replication Package
Demo (video), Teaser (2-minute video), Presentation (17-minute video)
Tool Playground

Active Documentation (VL/HCC 2019)

Good documentation has long been argued to be key to helping developers write code more quickly and consistently with design decisions, but is left largely disconnected from code. We propose a method for active documentation, where design decisions are made explicit as design rules and checked against code. Developers can discover how to follow a design rule by navigating to examples in their codebase. After editing code, developers receive immediate feedback about which design rules are satisfied and which are violated, notifying developers who miss design decisions about the existence of these design decisions.

ActiveDocumentation is a tool that helps developers follow design decisions by making the constraints of decisions explicit and checkable as AST queries. ActiveDocumentation checks code for conformance constantly and notifies the users about violations of decisions. It also extracts example code snippets that follow the decisions enabling developers to use them as references and learn how to follow the decisions.

Sahar Mehrpour, Thomas D. LaToza, Rahul K. Kindi.

DOI, Pre-Print, Slides, Replication Package, Demo