9 Aug
2021
9 Aug
'21
4:42 p.m.
Dear all,
I would like to inform you that the following paper
A Comparative Study of Automatic Software Repair Techniques for Security Vulnerabilities
Eduard Pinconschi, Rui Abreu, and Pedro Adão
will be published in the conference (core-A ranking)
The 32nd International Symposium on Software Reliability Engineering (ISSRE 2021)
http://2021.issre.net/ <http://2021.issre.net/>
Oct 25 - 28, 2021, Wuhan, China
and will acknowledge SPARTA.
I will make the paper available as soon as we have the camera ready version (28th of
August).
Do let me know if you need a draft in advance.
Best regards,
Pedro
Abstract:
In the past years, research on automatic program repair (APR), in particular on
test-suite-based approaches, has significantly attracted the attention of researchers.
Despite the advances in the field, it remains unclear how these techniques fare in the
context of security---most approaches are evaluated using benchmarks of bugs that do not
(\textit{only}) contain security vulnerabilities.
In this paper, we present our observations using 10 state-of-the-art test-suite-based
automatic program repair tools on the DARPA Cyber Grand Challenge benchmark of
vulnerabilities in C/C++. Our intention is to have a better understanding of the current
state of automatic program repair tools when addressing security issues.
In particular, our study is guided by the hypothesis that the efficiency of repair tools
may not generalize to security vulnerabilities. We found that the 10 analyzed tools can
only fix 30 out of 55 vulnerable programs---54.5\% of the considered issues. In
particular, we found that APR tools with atomic change operators and brute-force search
strategy (\emph{AE} and \emph{GenProg}) and brute-force functionality deletion
(\emph{Kali}) overall perform better at repairing security vulnerabilities (considering
both efficiency and effectiveness). \emph{AE} is the tool that individually repairs most
programs with 20 out of 55 programs (36.4\%).
The causes for failing to repair are discussed in the paper, which can help repair tool
designers to improve their techniques and tools.