keepcalculator

06 Numbers and Powers

Duplicate Code Detector: Repeated Structure vs Rule

Card No. 06·71430 · Updated · Standard formula, shown below

Duplicate code detectors flag repeated structure, not the underlying rule or intent. They identify syntactically or semantically similar code blocks, but they cannot determine whether that repetition represents a violation of the DRY principle or a legitimate, intentional pattern.

What Duplicate Code Detectors Actually Do

Duplicate code detectors work by comparing code fragments for similarity. They use algorithms like token-based comparison, abstract syntax tree (AST) matching, or metric-based fingerprinting to find blocks of code that look alike. The output is a list of locations where similar code appears, along with a similarity score.

For example, a detector might flag two functions that both calculate the area of a rectangle, even if one is in a billing module and the other in a shipping module. It sees the same structure: multiply width by height. It does not see that both modules legitimately need this calculation and that extracting a shared utility would introduce an unnecessary dependency.

The Rule vs. Structure Distinction

The “rule” in this context refers to the business logic or domain invariant that the code implements. When you have repeated structure, the underlying rule might be the same, or it might be different. Consider two loops that iterate over a list and sum the elements: one might sum prices, the other might sum quantities. The structure is identical, but the rules are different.

Detectors cannot read intent. They only see patterns. A human reviewer must decide whether the repetition is accidental (a copy-paste error that should be refactored) or intentional (a deliberate choice to keep two implementations independent). This is why static analysis tools are called “detectors”—they flag potential issues, but they do not make judgments.

Why Detectors Flag Repeated Structure

Detectors are designed to find duplication because duplication is a common source of bugs. When you copy-paste code and forget to change a variable name, you introduce a subtle error. When you fix a bug in one copy but not the other, the bug persists. Detectors help developers find these spots so they can consolidate the logic.

However, the detector’s job ends at flagging. It does not know whether the duplication is harmful. In fact, some duplication is beneficial: it can improve performance (by avoiding function calls), reduce coupling, or make code easier to read in small, isolated contexts. The detector cannot weigh these trade-offs.

How to Interpret Detector Results

When a detector flags a block, you should manually review it and ask:

  • Does this repeated code implement the same rule? If yes, consider extracting it into a function or class.
  • Are the inputs and outputs identical? If they differ, the rule may be different even if the structure looks the same.
  • Would refactoring introduce risk? If the code is in a critical path, maybe the duplication is acceptable.
  • Is there a reason the code must remain separate? For example, if the two copies are in different layers of the architecture, merging them might violate layering.

This manual step is essential because detectors are heuristic. They produce false positives. A good developer uses the detector as a starting point, not as an authority.

Common Problems with Duplicate Detection

One common problem is that detectors flag code that is similar but not identical. Two functions that differ only in a constant value might be flagged as duplicates, even though they serve different purposes. Another problem is that detectors may miss duplication that is not textually similar—for example, two functions that use different algorithms to achieve the same result.

Detectors also struggle with language-specific idioms. In some languages, certain repetitive patterns are idiomatic and should not be refactored. For instance, in Java, it is common to write getter and setter methods that follow a standard template. A detector might flag all of them as duplicates, but they are not—they are boilerplate that is expected and necessary.

Choosing the Right Detector

Different detectors use different algorithms, and the choice depends on your needs. Token-based detectors are fast and language-agnostic, but they may produce many false positives. AST-based detectors are more accurate but slower and may require language-specific parsers. Some detectors use machine learning to identify semantic similarities, but these are less common and may be overkill for most projects.

You should also consider the granularity. Some detectors find duplicated lines, others find duplicated functions or classes. The right granularity depends on your refactoring goals. If you want to find copy-paste errors, line-level detection is useful. If you want to find design issues, function-level detection is better.

Alternatives to Duplicate Detection

Instead of relying solely on detectors, you can use other techniques to manage duplication. Code reviews are a human-based approach that can catch duplication that tools miss. Design patterns, such as the Template Method pattern, can help you avoid duplication by design. And good documentation can make it clear why a piece of code is duplicated, reducing the risk of accidental changes.

Another alternative is to use a linter or style guide that enforces rules about duplication. For example, you can set a threshold for the maximum number of lines that can be duplicated before a warning is issued. This gives you a consistent policy, but it still requires human judgment to decide whether the warning is valid.

Frequently Asked Questions

Can duplicate code detectors find semantic duplication?

Some advanced detectors use AST or data-flow analysis to find code that is semantically similar, even if the syntax differs. However, they still cannot understand the business rule. They might find two functions that compute the same mathematical formula, but they cannot tell you if the formula is the right one for both contexts.

Why does my detector flag code that is not duplicated?

Detectors often flag code that is structurally similar but not truly duplicated. This is a false positive. It happens because the detector uses a similarity threshold, and code that is above that threshold is flagged. You can adjust the threshold or ignore the warning if the code is intentionally similar.

Should I always refactor duplicated code?

No. Refactoring duplicated code can introduce bugs, especially if the code is in a critical path. You should only refactor when the duplication is harmful, such as when it leads to inconsistent changes. If the duplication is harmless, leaving it as is may be the better choice.

What is the difference between a duplicate code detector and a linter?

A linter checks for style and common errors, such as unused variables or missing semicolons. A duplicate code detector specifically looks for code that is repeated. Some linters include duplicate detection as a rule, but they are not the same thing.

Can duplicate code detectors be used in CI/CD pipelines?

Yes, many duplicate code detectors can be integrated into CI/CD pipelines. They can run on every commit and fail the build if the duplication ratio exceeds a threshold. This helps maintain code quality, but it can also cause false failures if the threshold is too low.

Related

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →