The RERS Challenge

Text? RERS 2014

Co-located with ISoLA 2014,
Corfu, Greece, October 8th -11th 2014







Rigorous Examination of Reactive Systems (RERS)

Reactive systems appear everywhere, e.g. as Web services, decision support systems, or logical controllers. Their validation techniques are as diverse as their appearance and structure. They comprise various forms of static analysis, model checking, symbolic execution and (model-based) testing, often tailored to quite extreme frame conditions. Thus it is almost impossible to compare these techniques, let alone to establish clear application profiles as a means for recommendation. The RERS Challenges aim at overcoming this situation by providing a forum for experimental profile evaluation based on specifically designed Benchmark suites. These benchmarks are automatically synthesized to exhibit chosen properties, and then enhanced to include dedicated dimensions of difficulty, ranging from conceptual complexity of the properties (e.g. reachablity, full safety, liveness), over size of the reactive systems (a few hundred lines to millions of them), to exploited language features (arrays and arithmetic at index pointer).

Characteristic for RERS is its wide scope, which addresses not only source code analyzers (White-Box problems), but also (model-based) testers and (test-based) modelers (Black-Box problems), and in particular Free stylers (Grey-Box problems). Currently, RERS focuses on functional properties only, but non-functional properties like time, performance, and stochastical behavior are envisaged. In addition, it is planned to feature a number of case studies provided by industry.

A leading goal is to investigate the power of and synergy potential between source code-based (White-Box) approaches and purely testing-based (Black-Box) approaches:

  • How much does the source code of historically grown legacy applications help?
  • How far carries a purely testing-based investigation?
  • What is a good way to combine the two?

Testing-based approaches are independent of language features, but, to quote Dijkstra, they can only proof the presence of errors, never their absence. Source code analysis has the power to proof the absence of errors, but each new language feature may requires an enormous effort. Realistic problems will most probably always require both approaches, which, today, are typically applied in isolation.

The RERS Challenge provides a wealth of problems of increasing complexity, the more involved of which will probably be beyond any individual state-of-the-art method or tool. In order to encourage cooperation and the consideration of alternative solutions the RERS Challenges follow a clear free-style philosophy: apply whatever you have or you can get hold of, you are willing to specifically construct, and what you consider valuable.


Text? RERS 2014

The challenge had two sections, namely White-Box and Extended White-Box. The challenge was colocated with ISoLA 2014 which took place in Corfu, Greece. More details about RERS 2014 can be found here.



Text? RERS 2013

The challenge had different sections including separate Black-Box, White-Box and Grey-Box challenges. The challenge was colocated with ASE 2013 which took place in Silicon Valley, California, USA. More details about RERS 2013 can be found here.



Text?RERS 2012

The challenge focussed on the analysis of Event Condition Action systems. The challenge was colocated with ISoLA 2012 which took place in Crete, Greece. More details about RERS 2012 can be found here.



Text? RERS 2010

The challenge focussed on active automata learning. The challenge was colocated with ISoLA 2010 which took place in Crete, Greece.