VerifyThis Verification Competition
to be held at ETAPS 2016 in Eindhoven
Saturday, 2 April 2016

News

  • 4 April 2016: Prize winners announced
  • 2 April 2016: competition day. 14 teams participating, using 9 different tools: Dafny (4 x), Why3 (2 x), KIV, KeY, CIVL, VerCors, Viper, mCRL2, and VeriFast. There are 4 student-only teams, and also 4 non-developers teams.
  • 9 March 2016: program available, Dafny tutorial and challenge open to all ETAPS participants
  • 22 February 2016: travel grants available
  • 13 November: 2015 a call for problems is published. Submission deadline is December 4th, 2015.
  • 22 October 2015: website online

About

VerifyThis 2016 is a program verification competition taking place as part of the European Joint Conferences on Theory and Practice of Software (ETAPS 2016) on April 2-3, 2016 in Eindhoven, the Netherlands. It is the 5th event in the VerifyThis competition series.

The aims of the competition are:
  • to bring together those interested in formal verification, and to provide an engaging, hands-on, and fun opportunity for discussion 
  • to evaluate the usability of logic-based program verification tools in a controlled experiment that could be easily repeated by others. 
The competition will offer a number of challenges presented in natural language and pseudo code. Participants have to formalize the requirements, implement a solution, and formally verify the implementation for adherence to the specification. 

There are no restrictions on the programming language and verification technology used. The correctness properties posed in problems will have the input-output behavior of programs in focus. Solutions will be judged for correctness, completeness and elegance.

Program

Saturday April 2
 
9 :00 – 9:45  Dafny tutorial - Rustan Leino, MSR research (open to everybody)
9:45 – 10:30  Dafny challenge (open to everybody)
10:30 – 11:00  coffee break
Start of the competition  
11:00 – 12:30  challenge 1 (90 minutes)
12:30 – 14:00
 lunch
14:00 – 15:30  challenge 2 (90 minutes)
15:30 – 16:00  coffee break
16:00 – 17:30  challenge 3 (90 minutes)
17:30 – 19:00  participants that are leaving already discuss their answers with the judges
19:30  dinner for all competition participants
   
Sunday April 3  
whole day
 informal discussions among participants and judging
   
Monday April 4  
during reception  award ceremony

Dafny Tutorial

The competition will start with a tutorial on Dafny by Rustan Leino, MSR Research. Dafny has been especially popular in previous verification competitions. The tutorial will explain the basic ideas of program verification in Dafny, which can then immediately be tried out on a small challenge problem.

The Dafny tutorial is open to all ETAPS participants.

Travel Grants

The competition has funds for a limited number of travel grants. A grant covers the incurred travel and accommodation costs up to a certain amount. The amount is EUR 250 for those coming from Europe and EUR 500 for those coming from outside Europe.

To apply for a travel grant, send an email to etaps2016@verifythis.org.  The application should include:

  • your name
  • your affiliation
  • the academic degree you are seeking resp. your career stage
  • the verification system(s) you plan to use at the competition
  • the planned composition of your team
  • a short letter of motivation explaining your involvement with formal verification so far
  • if you are a student or PhD student, please have your supervisor send a brief letter of support by email to the same address.

Evaluation criteria are qualifications (for the applicant's career level), need (please explain briefly in your application), and diversity (technical, geographical, etc.).

Prizes

Prizes will be awarded in the following categories:
  • best team
  • best student team
  • best challenge submitted
  • distinguished user-assistance tool feature

Participation / Registration

Participation is open for anybody interested. Teams of up to two people are allowed. Physical presence on site is required. We particularly encourage participation of:
  • student teams (this includes PhD students)
  • non-developer teams using a tool someone else developed
  • several teams using the same tool
VerifyThis is an official one-day satellite event of ETAPS. Please register with ETAPS. To facilitate planning, please also send an email to etaps2016@verifythis.org stating your planned team composition and verification system(s) you plan to use. Informal inquiries are welcome at the same address.

Call for Problems

To extend the problem pool and tender better to the needs of the participants, we are now soliciting verification problems for the competition:
  • a problem should contain an informal statement of the algorithm to be implemented (optionally with complete or partial pseudocode) and the requirement(s) to be verified
  • a problem should be suitable for a 60-90 minute time slot
  • submission of reference solutions is strongly encouraged
  • problems with an inherent language- or tool-specific bias should be clearly identified as such
  • problems that contain several subproblems or other means of difficulty scaling are especially welcome
  • the organizers reserve the right (but no obligation) to use the problems in the competition, either as submitted or with modifications
  • submissions from (potential) competition participants are allowed
Submissions are to be sent per email to etaps2016@verifythis.org. Submission deadline is 4 December, 2015.
The best submission will receive a prize.

Related Events and Activities

VerifyThis 2016 is the 5th event in the VerifyThis competition series. Related events are the Verified Software Competition (VSComp: 1st, 2nd, latest) held online and the Competition on Software Verification (SV-COMP) focusing on evaluating systems in a way that does not require user interaction. SV-COMP is associated with TACAS.

VerifyThis is also a collection of verification problems (and solutions). Its counterpart is VerifyThus - a distribution of deductive verification tools for Java-like languages, bundled and ready to run in a VM. Both were created with support from COST Action IC0701.

A workshop on comparative empirical evaluation of reasoning systems (COMPARE2012) was held on June 30th at IJCAR 2012 in Manchester. Competitions were one of the main topics of the workshop.

Organizers

The organizers can be reached per email at etaps2016@verifythis.org.