News
- 4 April 2016: Prize winners announced
- 2 April 2016: competition day. 14 teams participating, using 9 different tools: Dafny (4 x), Why3 (2 x), KIV, KeY, CIVL, VerCors, Viper, mCRL2, and VeriFast. There are 4 student-only teams, and also 4 non-developers teams.
- 9 March 2016: program available, Dafny tutorial and challenge open to all ETAPS participants
- 22 February 2016: travel grants available
- 13 November: 2015 a call for problems is published. Submission deadline is December 4th, 2015.
- 22 October 2015: website online
About
VerifyThis 2016 is a program verification competition taking place as part of the European Joint Conferences on Theory and Practice of Software (ETAPS 2016) on April 2-3, 2016 in Eindhoven, the Netherlands. It is the 5th event in the VerifyThis competition series.
The aims of the competition are:
- to bring together those interested in formal verification, and to provide an engaging, hands-on, and fun opportunity for discussion
- to evaluate the usability of logic-based program verification tools in a controlled experiment that could be easily repeated by others.
The competition will offer a number of challenges presented in natural language and pseudo code. Participants have to formalize the requirements, implement a solution, and formally verify the implementation for adherence to the specification.
There are no restrictions on the programming language and verification technology used. The correctness properties posed in problems will have the input-output behavior of programs in focus. Solutions will be judged for correctness, completeness and elegance.
Program
| Saturday April 2 |
|
| 9 :00 – 9:45 | Dafny tutorial - Rustan Leino, MSR research (open to everybody) |
| 9:45 – 10:30 | Dafny challenge (open to everybody) |
| 10:30 – 11:00 | coffee break |
| Start of the competition | |
| 11:00 – 12:30 | challenge 1 (90 minutes) |
| 12:30 – 14:00 |
lunch |
| 14:00 – 15:30 | challenge 2 (90 minutes) |
| 15:30 – 16:00 | coffee break |
| 16:00 – 17:30 | challenge 3 (90 minutes) |
| 17:30 – 19:00 | participants that are leaving already discuss their answers with the judges |
| 19:30 | dinner for all competition participants |
| Sunday April 3 | |
| whole day |
informal discussions among participants and judging |
| Monday April 4 | |
| during reception | award ceremony |
Dafny Tutorial
The competition will start with a tutorial on Dafny by Rustan Leino, MSR Research. Dafny has been especially popular in previous verification competitions. The tutorial will explain the basic ideas of program verification in Dafny, which can then immediately be tried out on a small challenge problem.
The Dafny tutorial is open to all ETAPS participants.
Travel Grants
The competition has funds for a limited number of travel grants. A grant covers the incurred travel and accommodation costs up to a certain amount. The amount is EUR 250 for those coming from Europe and EUR 500 for those coming from outside Europe.
To apply for a travel grant, send an email to etaps2016@verifythis.org. The application should include:
- your name
- your affiliation
- the academic degree you are seeking resp. your career stage
- the verification system(s) you plan to use at the competition
- the planned composition of your team
- a short letter of motivation explaining your involvement with formal verification so far
- if you are a student or PhD student, please have your supervisor send a brief letter of support by email to the same address.
Evaluation criteria are qualifications (for the applicant's career level), need (please explain briefly in your application), and diversity (technical, geographical, etc.).
Prizes
Prizes will be awarded in the following categories:
- best team
- best student team
- best challenge submitted
- distinguished user-assistance tool feature
Participation / Registration
Participation is open for anybody interested. Teams of up to two
people are allowed. Physical presence on site is required. We
particularly encourage participation of:
- student teams (this includes PhD students)
- non-developer teams using a tool someone else developed
- several teams using the same tool
Call for Problems
To extend the problem pool and tender better to the needs of
the participants, we are now soliciting verification problems for
the competition:
- a problem should contain an informal statement of the algorithm to be implemented (optionally with complete or partial pseudocode) and the requirement(s) to be verified
- a problem should be suitable for a 60-90 minute time slot
- submission of reference solutions is strongly encouraged
- problems with an inherent language- or tool-specific bias should be clearly identified as such
- problems that contain several subproblems or other means of difficulty scaling are especially welcome
- the organizers reserve the right (but no obligation) to use the problems in the competition, either as submitted or with modifications
- submissions from (potential) competition participants are allowed
The best submission will receive a prize.
Related Events and Activities
VerifyThis 2016 is the 5th event in the VerifyThis competition series. Related events are the Verified Software Competition (VSComp: 1st, 2nd, latest) held online and the Competition on Software Verification (SV-COMP) focusing on evaluating systems in a way that does not require user interaction. SV-COMP is associated with TACAS.
VerifyThis is also a collection of verification problems (and solutions). Its counterpart is VerifyThus - a distribution of deductive verification tools for Java-like languages, bundled and ready to run in a VM. Both were created with support from COST Action IC0701.
A workshop on comparative empirical evaluation of reasoning systems (COMPARE2012) was held on June 30th at IJCAR 2012 in Manchester. Competitions were one of the main topics of the workshop.
Organizers
- Marieke Huisman, U Twente, NL
- Rosemary Monahan, NUI Maynooth, IE
- Peter Müller, ETH Zürich, CH
The organizers can be reached per email at etaps2016@verifythis.org.