Pseudo Boolean Evaluation 2006

The second Pseudo Boolean evaluation is organized as a special event of the SAT 2006 conference. It follows the first Pseudo Boolean evaluation which gathered together 8 solvers. The goal  of this evaluation is to assess the state of the art in the field of pseudo-Boolean solvers.

For questions, comments and any other issue regarding the PB Evaluation, please get in touch with pbeval@cril.univ-artois.fr.

Deadlines

The timetable of the 2006 evaluation is detailed below. Deadlines will be strict.

newThe results of the evaluation are now publicly available here.new

Opening of registration site March 13, 2006
Pre-registration (solvers and benchmarks) expected on
April 2nd, 2006
Solvers must be submitted before
April 30, 2006
Benchmarks must be submitted before
April 30, 2006
Conformance test and bug fixes
May 1-14, 2006
Solvers running May-June 2006
Solvers description must be provided by
June 26, 2006new
Results will be given during the SAT 2006 conference August 12-15, 2006

Noticeable changes

We report here the most important changes in the evaluation rules (in comparison to the PB'05 evaluation). As some other minor points were also modified,  you are invited to read completely and carefully the call for solvers and benchmarks.

Call for solvers

There are a few simple requirements  to ensure  that a submitted solver will  correctly operate in the evaluation  environment. It is expected that any pseudo-Boolean solver can be easily modified to conform to these requirements. People interested in submitting a solver can get in touch with the organizers to get some help.

All the details about the input format, the output of the solver and its running environment are available on a separate page.

Sample code, parsers for a few languages and a few programming tips are also available

A preliminary set of benchmarks is available for testing the solvers

Some conformance tests are available.

Solver submitters implicitly accept that the results of their solver be publicly available on the evaluation web site. Solver submitters are strongly encouraged to make their solver publicly available, either in source code or in executable format. However, this is not a requirement for entering the evaluation.

Call for benchmarks

Benchmarks may be submitted in any reasonable format. The organizers will do their best to translate the submitted benchmarks to the input format adopted for the evaluation. A benchmark submitted in the format of the evaluation will still be normalized to ensure consistency among all the input files.

We would particularly appreciate the submission of
Submitted benchmarks will be made available on the evaluation web site shortly after the beginning of the evaluation. We cannot accept benchmarks which are not publicly available (because anyone must be able to reproduce the experiments of the evaluation).

Benchmarks submitters may provide a description of their benchmarks set to be made available on the evaluation web site.

A preliminary set of benchmarks is available for testing the solvers

Organization

To reach the organizers, send an email to pbeval@cril.univ-artois.fr.

Vasco MANQUINHO
INESC-ID, Lisboa, Portugal
vmm at sat.inesc.pt / home page
Olivier ROUSSEL
CRIL, Université d'Artois, France
olivier.roussel at cril.univ-artois.fr / home page