Trace number 238080

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds, and are wall clock time (not CPU time).

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
galac 1? (TO) 1800.05 1812

General information on the benchmark

Namehaystacks/
haystacks_20.xml
MD5SUM5947a972166e8262f434c28cfd58ec68
Bench Category2-ARY-INT (binary constraints in intension)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark7.1919
SatisfiableNO
(Un)Satisfiability was proved
Number of variables400
Number of constraints3819
Maximum constraint arity2
Maximum domain size20
Number of constraints which are defined in extension0
Number of constraints which are defined in intension3819
Global constraints used (with number of constraints)

Solver Data (download as text)

0.03	c --- Preproc Stats ---
2.30	c start processing
2.31	c start relation processing
2.31	c start cnf exporting
5.86	c parsing time : 2.138
5.86	c preprocessing time : 5.696
5.86	
5.86	c Nary constraints - conflicts : 0 supports : 0
5.86	c Binary constraints - conflicts : 3800 supports :19
5.86	c max arity constraint : 2
5.86	c Max Tuples generated 400
5.86	c relations : 4
5.86	c relations BDD: 0
5.86	c vars : 8001 - clauses :  153882
5.86	
5.91	 
5.91	c --- Minisat Stats ---
5.91	 

Verifier Data (download as text)

ERROR: Unexpected answer ! (SAT/UNSAT expected)
Got answer: 

Watcher Data (download as text)

runsolver version 3.1.3 (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w ROOT/results/node55/watcher-238080-1168353465 -o ROOT/results/node55/solver-238080-1168353465 -C 1800 -M 900 /tmp/evaluation/238080-1168353465/galac.sh /tmp/evaluation/238080-1168353465/unknown.xml 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 921600 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 972800 KiB
Current StackSize limit: 10240 KiB

/proc/loadavg: 2.03 2.03 2.00 5/81 22942
/proc/meminfo: memFree=1556920/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=5228 CPUtime=0
/proc/22941/stat : 22941 (galac.sh) R 22939 22941 20921 0 -1 4194304 130 0 0 0 0 0 0 0 21 0 1 0 183744780 5353472 99 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 4425608 0 0 4096 0 0 0 0 17 1 0 0
/proc/22941/statm: 1307 99 75 79 0 111 0

[startup+0.104634 s]
/proc/loadavg: 2.03 2.03 2.00 5/81 22942
/proc/meminfo: memFree=1556920/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=0.01
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 762 1704 0 0 0 0 0 1 18 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 47580

[startup+0.516671 s]
/proc/loadavg: 2.03 2.03 2.00 5/81 22942
/proc/meminfo: memFree=1556920/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=0.01
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 762 1704 0 0 0 0 0 1 18 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 47580

[startup+1.33979 s]
/proc/loadavg: 2.03 2.03 2.00 3/89 22962
/proc/meminfo: memFree=1517368/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=0.01
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 762 1704 0 0 0 0 0 1 18 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22955] ppid=22941 vsize=860984 CPUtime=1.28
/proc/22955/stat : 22955 (java) R 22941 22941 20921 0 -1 0 11966 0 1 0 120 8 0 0 17 0 8 0 183744784 881647616 11717 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4072943745 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/22955/statm: 215246 11717 1994 14 0 201504 0
[pid=22955/tid=22956] ppid=22941 vsize=860984 CPUtime=0.12
/proc/22955/task/22956/stat : 22956 (java) S 22941 22941 20921 0 -1 64 3377 0 0 0 11 1 0 0 16 0 8 0 183744788 881647616 11717 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22957] ppid=22941 vsize=860984 CPUtime=0
/proc/22955/task/22957/stat : 22957 (java) S 22941 22941 20921 0 -1 64 12 0 0 0 0 0 0 0 15 0 8 0 183744788 881647616 11717 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22958] ppid=22941 vsize=860984 CPUtime=0
/proc/22955/task/22958/stat : 22958 (java) S 22941 22941 20921 0 -1 64 7 0 0 0 0 0 0 0 15 0 8 0 183744788 881647616 11717 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22959] ppid=22941 vsize=860984 CPUtime=0
/proc/22955/task/22959/stat : 22959 (java) S 22941 22941 20921 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183744792 881647616 11717 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22960] ppid=22941 vsize=860984 CPUtime=0.3
/proc/22955/task/22960/stat : 22960 (java) R 22941 22941 20921 0 -1 64 1047 0 0 0 29 1 0 0 21 0 8 0 183744792 881647616 11717 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22961] ppid=22941 vsize=860984 CPUtime=0
/proc/22955/task/22961/stat : 22961 (java) S 22941 22941 20921 0 -1 64 2 0 0 0 0 0 0 0 19 0 8 0 183744792 881647616 11717 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22962] ppid=22941 vsize=860984 CPUtime=0
/proc/22955/task/22962/stat : 22962 (java) S 22941 22941 20921 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183744792 881647616 11717 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 1.29
Current children cumulated vsize (KiB) 908564

[startup+2.97691 s]
/proc/loadavg: 2.03 2.03 2.00 3/89 22962
/proc/meminfo: memFree=1493688/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=0.01
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 762 1704 0 0 0 0 0 1 18 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22955] ppid=22941 vsize=860984 CPUtime=2.9
/proc/22955/stat : 22955 (java) R 22941 22941 20921 0 -1 0 19577 0 1 0 278 12 0 0 25 0 8 0 183744784 881647616 19326 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4073419803 0 4 0 23759 18446744073709551615 0 0 17 1 0 0
/proc/22955/statm: 215246 19326 2015 14 0 201504 0
[pid=22955/tid=22956] ppid=22941 vsize=860984 CPUtime=0.12
/proc/22955/task/22956/stat : 22956 (java) S 22941 22941 20921 0 -1 64 3494 0 0 0 11 1 0 0 16 0 8 0 183744788 881647616 19326 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 0 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22957] ppid=22941 vsize=860984 CPUtime=0
/proc/22955/task/22957/stat : 22957 (java) S 22941 22941 20921 0 -1 64 12 0 0 0 0 0 0 0 16 0 8 0 183744788 881647616 19326 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22958] ppid=22941 vsize=860984 CPUtime=0
/proc/22955/task/22958/stat : 22958 (java) S 22941 22941 20921 0 -1 64 7 0 0 0 0 0 0 0 15 0 8 0 183744788 881647616 19326 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22959] ppid=22941 vsize=860984 CPUtime=0
/proc/22955/task/22959/stat : 22959 (java) S 22941 22941 20921 0 -1 64 1 0 0 0 0 0 0 0 18 0 8 0 183744792 881647616 19326 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22960] ppid=22941 vsize=860984 CPUtime=0.39
/proc/22955/task/22960/stat : 22960 (java) S 22941 22941 20921 0 -1 64 1111 0 0 0 37 2 0 0 16 0 8 0 183744792 881647616 19326 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22961] ppid=22941 vsize=860984 CPUtime=0
/proc/22955/task/22961/stat : 22961 (java) S 22941 22941 20921 0 -1 64 2 0 0 0 0 0 0 0 19 0 8 0 183744792 881647616 19326 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
[pid=22955/tid=22962] ppid=22941 vsize=860984 CPUtime=0
/proc/22955/task/22962/stat : 22962 (java) S 22941 22941 20921 0 -1 64 1 0 0 0 0 0 0 0 15 0 8 0 183744792 881647616 19326 18446744073709551615 134512640 134570532 4294956112 18446744073709551615 4294960144 0 4 0 23759 0 0 0 -1 1 0 0
Current children cumulated CPU time (s) 2.91
Current children cumulated vsize (KiB) 908564

[startup+6.26723 s]
/proc/loadavg: 2.03 2.03 2.00 3/89 22962
/proc/meminfo: memFree=1484024/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
Current children cumulated CPU time (s) 5.79
Current children cumulated vsize (KiB) 47580

[startup+12.7449 s]
/proc/loadavg: 2.03 2.03 2.00 3/82 22963
/proc/meminfo: memFree=1518064/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=37192 CPUtime=6.76
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 9133 0 0 0 672 4 0 0 25 0 1 0 183745372 38084608 9115 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529430 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 9298 9115 54 142 0 9154 0
Current children cumulated CPU time (s) 12.55
Current children cumulated vsize (KiB) 84772

[startup+25.5881 s]
/proc/loadavg: 2.02 2.03 2.00 3/82 22963
/proc/meminfo: memFree=1505648/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=50172 CPUtime=19.48
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 12367 0 0 0 1941 7 0 0 25 0 1 0 183745372 51376128 12349 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529430 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 12543 12349 54 142 0 12399 0
Current children cumulated CPU time (s) 25.27
Current children cumulated vsize (KiB) 97752

[startup+51.2706 s]
/proc/loadavg: 2.01 2.02 2.00 3/82 22963
/proc/meminfo: memFree=1489264/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=65988 CPUtime=44.94
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 16311 0 0 0 4483 11 0 0 25 0 1 0 183745372 67571712 16293 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 16497 16293 54 142 0 16353 0
Current children cumulated CPU time (s) 50.73
Current children cumulated vsize (KiB) 113568

[startup+102.513 s]
/proc/loadavg: 2.00 2.02 2.00 3/82 22963
/proc/meminfo: memFree=1471600/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=84176 CPUtime=95.71
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 20755 0 0 0 9556 15 0 0 25 0 1 0 183745372 86196224 20737 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 21044 20737 54 142 0 20900 0
Current children cumulated CPU time (s) 101.5
Current children cumulated vsize (KiB) 131756

[startup+162.516 s]
/proc/loadavg: 2.00 2.01 2.00 3/82 22963
/proc/meminfo: memFree=1453744/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=101876 CPUtime=155.16
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 25176 0 0 0 15496 20 0 0 25 0 1 0 183745372 104321024 25158 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529430 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 25469 25158 54 142 0 25325 0
Current children cumulated CPU time (s) 160.95
Current children cumulated vsize (KiB) 149456

[startup+222.519 s]
/proc/loadavg: 2.00 2.01 2.00 3/82 22963
/proc/meminfo: memFree=1438320/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=117620 CPUtime=214.62
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 29033 0 0 0 21438 24 0 0 25 0 1 0 183745372 120442880 29015 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529430 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 29405 29015 54 142 0 29261 0
Current children cumulated CPU time (s) 220.41
Current children cumulated vsize (KiB) 165200

[startup+282.6 s]
/proc/loadavg: 2.00 2.00 2.00 4/82 22963
/proc/meminfo: memFree=1430128/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=126008 CPUtime=274.16
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 31046 0 0 0 27389 27 0 0 25 0 1 0 183745372 129032192 31028 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529446 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 31502 31028 54 142 0 31358 0
Current children cumulated CPU time (s) 279.95
Current children cumulated vsize (KiB) 173588

[startup+342.68 s]
/proc/loadavg: 2.04 2.01 2.00 3/82 22963
/proc/meminfo: memFree=1422896/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=133112 CPUtime=333.69
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 32866 0 0 0 33340 29 0 0 25 0 1 0 183745372 136306688 32848 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 33278 32848 54 142 0 33134 0
Current children cumulated CPU time (s) 339.48
Current children cumulated vsize (KiB) 180692

[startup+402.681 s]
/proc/loadavg: 2.01 2.01 2.00 3/82 22963
/proc/meminfo: memFree=1414960/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=141084 CPUtime=393.14
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 34833 0 0 0 39284 30 0 0 25 0 1 0 183745372 144470016 34815 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529430 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 35271 34815 54 142 0 35127 0

################
# More data... #
################

/proc/loadavg: 2.07 2.03 2.00 3/82 22963
/proc/meminfo: memFree=1338032/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218356 CPUtime=1107.95
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 53933 0 0 0 110742 53 0 0 25 0 1 0 183745372 223596544 53915 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 54589 53915 54 142 0 54445 0
Current children cumulated CPU time (s) 1113.74
Current children cumulated vsize (KiB) 265936

[startup+1183.47 s]
/proc/loadavg: 2.03 2.02 2.00 3/82 22963
/proc/meminfo: memFree=1337968/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218356 CPUtime=1167.74
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 53937 0 0 0 116720 54 0 0 25 0 1 0 183745372 223596544 53919 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529417 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 54589 53919 54 142 0 54445 0
Current children cumulated CPU time (s) 1173.53
Current children cumulated vsize (KiB) 265936

[startup+1243.54 s]
/proc/loadavg: 2.01 2.02 2.00 3/82 22963
/proc/meminfo: memFree=1337968/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218356 CPUtime=1227.52
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 53940 0 0 0 122698 54 0 0 25 0 1 0 183745372 223596544 53922 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 54589 53922 54 142 0 54445 0
Current children cumulated CPU time (s) 1233.31
Current children cumulated vsize (KiB) 265936

[startup+1303.61 s]
/proc/loadavg: 2.06 2.03 2.00 3/82 22963
/proc/meminfo: memFree=1337968/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218356 CPUtime=1287.31
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 53942 0 0 0 128677 54 0 0 25 0 1 0 183745372 223596544 53924 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 54589 53924 54 142 0 54445 0
Current children cumulated CPU time (s) 1293.1
Current children cumulated vsize (KiB) 265936

[startup+1363.67 s]
/proc/loadavg: 2.07 2.04 2.00 3/82 22963
/proc/meminfo: memFree=1337904/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218356 CPUtime=1347.09
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 53948 0 0 0 134655 54 0 0 25 0 1 0 183745372 223596544 53930 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 54589 53930 54 142 0 54445 0
Current children cumulated CPU time (s) 1352.88
Current children cumulated vsize (KiB) 265936

[startup+1423.75 s]
/proc/loadavg: 2.14 2.06 2.01 3/88 23071
/proc/meminfo: memFree=1336568/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218356 CPUtime=1406.77
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 53958 0 0 0 140623 54 0 0 25 0 1 0 183745372 223596544 53940 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 54589 53940 54 142 0 54445 0
Current children cumulated CPU time (s) 1412.56
Current children cumulated vsize (KiB) 265936

[startup+1483.84 s]
/proc/loadavg: 2.05 2.04 2.00 3/88 23137
/proc/meminfo: memFree=1336632/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218356 CPUtime=1466.5
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 53965 0 0 0 146596 54 0 0 25 0 1 0 183745372 223596544 53947 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529450 0 0 4096 3 0 0 0 17 1 0 0
/proc/22963/statm: 54589 53947 54 142 0 54445 0
Current children cumulated CPU time (s) 1472.29
Current children cumulated vsize (KiB) 265936

[startup+1543.91 s]
/proc/loadavg: 2.02 2.03 2.00 2/70 23266
/proc/meminfo: memFree=1540176/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218356 CPUtime=1526.26
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 53969 0 0 0 152570 56 0 0 25 0 1 0 183745372 223596544 53951 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 0 0 0
/proc/22963/statm: 54589 53951 54 142 0 54445 0
Current children cumulated CPU time (s) 1532.05
Current children cumulated vsize (KiB) 265936

[startup+1603.99 s]
/proc/loadavg: 1.37 1.84 1.94 2/66 23344
/proc/meminfo: memFree=1546584/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218356 CPUtime=1586.32
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 53972 0 0 0 158574 58 0 0 25 0 1 0 183745372 223596544 53954 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 0 0 0
/proc/22963/statm: 54589 53954 54 142 0 54445 0
Current children cumulated CPU time (s) 1592.11
Current children cumulated vsize (KiB) 265936

[startup+1664.06 s]
/proc/loadavg: 1.13 1.69 1.87 2/66 23344
/proc/meminfo: memFree=1546584/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218356 CPUtime=1646.38
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 54006 0 0 0 164580 58 0 0 25 0 1 0 183745372 223596544 53988 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 0 0 0
/proc/22963/statm: 54589 53988 54 142 0 54445 0
Current children cumulated CPU time (s) 1652.17
Current children cumulated vsize (KiB) 265936

[startup+1724.13 s]
/proc/loadavg: 1.05 1.56 1.82 2/66 23344
/proc/meminfo: memFree=1546584/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218356 CPUtime=1706.43
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 54006 0 0 0 170585 58 0 0 25 0 1 0 183745372 223596544 53988 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529304 0 0 4096 3 0 0 0 17 0 0 0
/proc/22963/statm: 54589 53988 54 142 0 54445 0
Current children cumulated CPU time (s) 1712.22
Current children cumulated vsize (KiB) 265936

[startup+1784.21 s]
/proc/loadavg: 1.02 1.45 1.76 2/66 23344
/proc/meminfo: memFree=1546648/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218436 CPUtime=1766.49
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 54017 0 0 0 176591 58 0 0 25 0 1 0 183745372 223678464 53999 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529316 0 0 4096 3 0 0 0 17 0 0 0
/proc/22963/statm: 54609 53999 54 142 0 54465 0
Current children cumulated CPU time (s) 1772.28
Current children cumulated vsize (KiB) 266016



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1811.99 s]
/proc/loadavg: 1.01 1.41 1.74 2/66 23344
/proc/meminfo: memFree=1546584/2055920 swapFree=4192192/4192956
[pid=22941] ppid=22939 vsize=47580 CPUtime=5.79
/proc/22941/stat : 22941 (galac.sh) S 22939 22941 20921 0 -1 4194304 800 21417 0 1 0 0 563 16 17 0 1 0 183744780 48721920 363 18446744073709551615 4194304 4520092 548682069344 18446744073709551615 248761215834 0 2 4096 73728 18446744071563181037 0 0 17 1 0 0
/proc/22941/statm: 11895 363 245 79 0 218 0
[pid=22963] ppid=22941 vsize=218436 CPUtime=1794.26
/proc/22963/stat : 22963 (minisat) R 22941 22941 20921 0 -1 4194304 54018 0 0 0 179368 58 0 0 25 0 1 0 183745372 223678464 54000 18446744073709551615 134512640 135096742 4294956480 18446744073709551615 134529417 0 0 4096 3 0 0 0 17 0 0 0
/proc/22963/statm: 54609 54000 54 142 0 54465 0
Current children cumulated CPU time (s) 1800.05
Current children cumulated vsize (KiB) 266016

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Child ended because it received signal 15 (SIGTERM)

!!! problem with CPU time !!!
wait4(...,&childrusage) returns 22941 and gives childrusage.ru_utime.tv_sec=5 childrusage.ru_utime.tv_usec=638142 childrusage.ru_stime.tv_sec=0 childrusage.ru_stime.tv_usec=172973
CPU time returned by wait4() is 5.81112
while last known CPU time is 1800.05

Solver probably didn't/couldn't wait for its children
Using last known CPU time as value...

Real time (s): 1812
CPU time (s): 1800.05
CPU user time (s): 1799.31
CPU system time (s): 0.74
CPU usage (%): 99.3407
Max. virtual memory (cumulated for all children) (KiB): 908564

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5.63814
system time used= 0.172973
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 22217
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1385
involuntary context switches= 1461

runsolver used 2.34664 s user time and 5.42018 s system time

The end

Launcher Data (download as text)

Begin job on node55 on Tue Jan  9 14:37:45 UTC 2007


IDJOB= 238080
IDBENCH= 6981
FILE ID= node55/238080-1168353465

PBS_JOBID= 3503632

Free space on /tmp= 66559 MiB

BENCH NAME= HOME/pub/bench/CPAI06/haystacks/haystacks_20.xml
COMMAND LINE= /tmp/evaluation/238080-1168353465/galac.sh /tmp/evaluation/238080-1168353465/unknown.xml
RUNSOLVER COMMAND LINE= runsolver  --timestamp  -w ROOT/results/node55/watcher-238080-1168353465 -o ROOT/results/node55/solver-238080-1168353465 -C 1800 -M 900  /tmp/evaluation/238080-1168353465/galac.sh /tmp/evaluation/238080-1168353465/unknown.xml

META MD5SUM SOLVER= d8bf20eb49fe33822f00bab08f33d7fc
MD5SUM BENCH=  5947a972166e8262f434c28cfd58ec68

RANDOM SEED= 221216547

TIME LIMIT= 1800 seconds

MEMORY LIMIT= 900 MiB


/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.218
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5931.00
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.218
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1557336 kB
Buffers:         29556 kB
Cached:         191448 kB
SwapCached:        268 kB
Active:         316032 kB
Inactive:       125996 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1557336 kB
SwapTotal:     4192956 kB
SwapFree:      4192192 kB
Dirty:            3284 kB
Writeback:           0 kB
Mapped:         237904 kB
Slab:            41512 kB
Committed_AS:  4368540 kB
PageTables:       2192 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66559 MiB



End job on node55 on Tue Jan  9 15:08:59 UTC 2007