Do Code-Generating Models Recognize When They Should Not Make a Change?

Abstract

Code-generating models are increasingly used to modify existing software repositories rather than produce isolated code snippets. In this setting, an important but relatively underexplored capability is recognizing when a requested change should not be implemented directly. A task may be underspecified, inconsistent with repository assumptions, or likely to introduce behavior outside its intended scope. This paper studies abstention in repository-level code modification. We construct development tasks containing ambiguous requirements, incompatible API assumptions, insufficient contextual information, and well-specified control cases, and examine whether code-generation models can distinguish actionable requests from those requiring clarification. We further investigate signals available before code generation, including uncertainty across candidate patches, disagreement between repository evidence and task descriptions, and predicted change scope. The results reveal a recurring failure mode: models frequently produce syntactically plausible patches even when essential implementation assumptions cannot be established from the repository. Based on these observations, we introduce a selective modification strategy that allows a model to request additional information or decline unsafe edits. The study suggests that reliable coding assistance requires evaluating not only whether generated code is correct, but whether generating a patch was appropriate in the first place.