Trace number 1737736

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
HydraSAT-Multi 2009-03-22-Multi? (TO) 1200.1 1200.71

General information on the benchmark

NameCRAFTED/parity-games/
instance_n8_i8_pp_ci_ce.cnf
MD5SUM68c51e190d82ac762156471d1f7373da
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark70.8762
Satisfiable
(Un)Satisfiability was proved
Number of variables21640
Number of clauses257551
Sum of the clauses size850944
Maximum clause length56
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 244056
Number of clauses of size 3125040
Number of clauses of size 468272
Number of clauses of size 515680
Number of clauses of size over 54503

Solver Data 

0.00/0.00	c Trying configuration 1
700.15/700.52	HOME/hydrasat-multi.sh: line 48: 21627 CPU time limit exceeded (core dumped) $SOLVER1 $BENCHNAME $OPTIONS1
700.15/700.52	c Trying configuration 2

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-1737736-1240832152/watcher-1737736-1240832152 -o /tmp/evaluation-result-1737736-1240832152/solver-1737736-1240832152 -C 1200 -W 1800 -M 1800 --output-limit 1,15 HOME/hydrasat-multi.sh HOME HOME/instance-1737736-1240832152.cnf 1800 1653146632 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1200 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1230 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 1843200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 1894400 KiB
Solver output will be limited to a maximum of 15728640 bytes. The first 1048576 bytes and the last 14680064 bytes will be preserved
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.83 1.71 1.37 3/64 21626
/proc/meminfo: memFree=1697984/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) R 21624 21626 21558 0 -1 4194304 270 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 238 1992294400 4194304 4889804 548682068720 18446744073709551615 236437695135 0 65538 4100 65536 0 0 0 17 0 0 0
/proc/21626/statm: 1340 238 196 169 0 51 0
[pid=21627] ppid=21626 vsize=2460 CPUtime=0
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 166 0 0 0 0 0 0 0 18 0 1 0 111242806 2519040 148 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 615 148 117 317 0 295 0

[startup+0.083691 s]
/proc/loadavg: 1.83 1.71 1.37 3/64 21626
/proc/meminfo: memFree=1697984/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=4968 CPUtime=0.07
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 841 0 0 0 7 0 0 0 19 0 1 0 111242806 5087232 823 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 1242 824 117 317 0 922 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 10328

[startup+0.101691 s]
/proc/loadavg: 1.83 1.71 1.37 3/64 21626
/proc/meminfo: memFree=1697984/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=5628 CPUtime=0.09
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 995 0 0 0 9 0 0 0 19 0 1 0 111242806 5763072 977 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 1407 977 117 317 0 1087 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 10988

[startup+0.301713 s]
/proc/loadavg: 1.83 1.71 1.37 3/64 21626
/proc/meminfo: memFree=1697984/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=12888 CPUtime=0.29
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 2944 0 0 0 28 1 0 0 21 0 1 0 111242806 13197312 2926 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 3222 2927 117 317 0 2902 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 18248

[startup+0.701759 s]
/proc/loadavg: 1.83 1.71 1.37 3/64 21626
/proc/meminfo: memFree=1697984/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=23736 CPUtime=0.68
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 5935 0 0 0 66 2 0 0 25 0 1 0 111242806 24305664 5619 1992294400 134512640 135813592 4294956160 18446744073709551615 134521357 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 5934 5619 124 317 0 5614 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 29096

[startup+1.50185 s]
/proc/loadavg: 1.83 1.71 1.37 2/66 21628
/proc/meminfo: memFree=1673904/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=26752 CPUtime=1.49
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 6732 0 0 0 146 3 0 0 25 0 1 0 111242806 27394048 6164 1992294400 134512640 135813592 4294956160 18446744073709551615 134520738 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 6688 6164 126 317 0 6368 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 32112

[startup+3.10103 s]
/proc/loadavg: 1.83 1.71 1.37 2/66 21628
/proc/meminfo: memFree=1662768/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=38324 CPUtime=3.08
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 9962 0 0 0 304 4 0 0 25 0 1 0 111242806 39243776 9053 1992294400 134512640 135813592 4294956160 18446744073709551615 134665314 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 9581 9053 149 317 0 9261 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 43684

[startup+6.3024 s]
/proc/loadavg: 1.77 1.70 1.37 2/66 21628
/proc/meminfo: memFree=1661424/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=38720 CPUtime=6.28
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 10077 0 0 0 624 4 0 0 25 0 1 0 111242806 39649280 9168 1992294400 134512640 135813592 4294956160 18446744073709551615 134592202 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 9680 9168 149 317 0 9360 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 44080

[startup+12.7011 s]
/proc/loadavg: 1.71 1.69 1.36 2/66 21628
/proc/meminfo: memFree=1660024/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=40440 CPUtime=12.68
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 10501 0 0 0 1263 5 0 0 25 0 1 0 111242806 41410560 9592 1992294400 134512640 135813592 4294956160 18446744073709551615 134665314 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 10110 9592 149 317 0 9790 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 45800

[startup+25.5016 s]
/proc/loadavg: 1.55 1.65 1.36 2/66 21628
/proc/meminfo: memFree=1658680/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=41596 CPUtime=25.47
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 10775 0 0 0 2542 5 0 0 25 0 1 0 111242806 42594304 9866 1992294400 134512640 135813592 4294956160 18446744073709551615 134594191 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 10399 9866 149 317 0 10079 0
Current children cumulated CPU time (s) 25.47
Current children cumulated vsize (KiB) 46956

[startup+51.1015 s]
/proc/loadavg: 1.36 1.60 1.35 2/66 21628
/proc/meminfo: memFree=1654904/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=45436 CPUtime=51.05
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 11742 0 0 0 5099 6 0 0 25 0 1 0 111242806 46526464 10833 1992294400 134512640 135813592 4294956160 18446744073709551615 134665984 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 11359 10833 149 317 0 11039 0
Current children cumulated CPU time (s) 51.05
Current children cumulated vsize (KiB) 50796

[startup+102.305 s]
/proc/loadavg: 1.15 1.51 1.33 2/66 21628
/proc/meminfo: memFree=1650040/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=50264 CPUtime=102.24
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 12919 0 0 0 10217 7 0 0 25 0 1 0 111242806 51470336 12010 1992294400 134512640 135813592 4294956160 18446744073709551615 135206894 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 12566 12010 149 317 0 12246 0
Current children cumulated CPU time (s) 102.24
Current children cumulated vsize (KiB) 55624

[startup+162.301 s]
/proc/loadavg: 1.06 1.41 1.31 2/66 21628
/proc/meminfo: memFree=1645624/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=54808 CPUtime=162.21
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 14039 0 0 0 16213 8 0 0 25 0 1 0 111242806 56123392 13130 1992294400 134512640 135813592 4294956160 18446744073709551615 134662852 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 13702 13130 149 317 0 13382 0
Current children cumulated CPU time (s) 162.21
Current children cumulated vsize (KiB) 60168

[startup+222.301 s]
/proc/loadavg: 1.02 1.33 1.28 2/66 21630
/proc/meminfo: memFree=1641272/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=59520 CPUtime=222.17
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 15123 0 0 0 22208 9 0 0 25 0 1 0 111242806 60948480 14214 1992294400 134512640 135813592 4294956160 18446744073709551615 134665314 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 14880 14214 149 317 0 14560 0
Current children cumulated CPU time (s) 222.17
Current children cumulated vsize (KiB) 64880

[startup+282.302 s]
/proc/loadavg: 1.00 1.27 1.26 2/66 21630
/proc/meminfo: memFree=1638136/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=63008 CPUtime=282.15
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 15898 0 0 0 28205 10 0 0 25 0 1 0 111242806 64520192 14989 1992294400 134512640 135813592 4294956160 18446744073709551615 134665303 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 15752 14989 149 317 0 15432 0
Current children cumulated CPU time (s) 282.15
Current children cumulated vsize (KiB) 68368

[startup+342.302 s]
/proc/loadavg: 1.00 1.22 1.24 2/66 21630
/proc/meminfo: memFree=1637048/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=64108 CPUtime=342.13
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 16165 0 0 0 34203 10 0 0 25 0 1 0 111242806 65646592 15256 1992294400 134512640 135813592 4294956160 18446744073709551615 134665314 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 16027 15256 149 317 0 15707 0

################
# More data... #
################

/proc/21627/statm: 18651 17714 149 317 0 18331 0
Current children cumulated CPU time (s) 582.01
Current children cumulated vsize (KiB) 79964

[startup+642.302 s]
/proc/loadavg: 1.00 1.07 1.16 2/66 21630
/proc/meminfo: memFree=1624824/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5360 CPUtime=0
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 289 0 0 0 0 0 0 0 18 0 1 0 111242806 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1340 239 197 169 0 51 0
[pid=21627] ppid=21626 vsize=77112 CPUtime=641.99
/proc/21627/stat : 21627 (hydrasat_cache4) R 21626 21626 21558 0 -1 4194304 19185 0 0 0 64185 14 0 0 25 0 1 0 111242806 78962688 18276 1992294400 134512640 135813592 4294956160 18446744073709551615 134665314 0 0 4096 1 0 0 0 17 0 0 0
/proc/21627/statm: 19278 18276 149 317 0 18958 0
Current children cumulated CPU time (s) 641.99
Current children cumulated vsize (KiB) 82472

[startup+702.302 s]
/proc/loadavg: 1.00 1.06 1.15 2/66 21631
/proc/meminfo: memFree=1601720/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5364 CPUtime=700.18
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 321 20525 0 0 0 0 69974 44 16 0 1 0 111242806 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1341 245 202 169 0 52 0
[pid=21631] ppid=21626 vsize=23828 CPUtime=1.76
/proc/21631/stat : 21631 (hydrasat_cachew) R 21626 21626 21558 0 -1 4194304 6336 0 0 0 175 1 0 0 25 0 1 0 111312859 24399872 5428 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 0 0 0
/proc/21631/statm: 5957 5428 171 362 0 5592 0
Current children cumulated CPU time (s) 701.94
Current children cumulated vsize (KiB) 29192

[startup+762.306 s]
/proc/loadavg: 1.00 1.04 1.14 2/66 21631
/proc/meminfo: memFree=1595896/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5364 CPUtime=700.18
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 321 20525 0 0 0 0 69974 44 16 0 1 0 111242806 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1341 245 202 169 0 52 0
[pid=21631] ppid=21626 vsize=30028 CPUtime=61.71
/proc/21631/stat : 21631 (hydrasat_cachew) R 21626 21626 21558 0 -1 4194304 7847 0 0 0 6168 3 0 0 25 0 1 0 111312859 30748672 6939 1992294400 134512640 135995732 4294956016 18446744073709551615 134807804 0 0 4096 1 0 0 0 17 0 0 0
/proc/21631/statm: 7507 6939 171 362 0 7142 0
Current children cumulated CPU time (s) 761.89
Current children cumulated vsize (KiB) 35392

[startup+822.301 s]
/proc/loadavg: 1.00 1.03 1.13 2/66 21631
/proc/meminfo: memFree=1590776/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5364 CPUtime=700.18
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 321 20525 0 0 0 0 69974 44 16 0 1 0 111242806 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1341 245 202 169 0 52 0
[pid=21631] ppid=21626 vsize=35468 CPUtime=121.68
/proc/21631/stat : 21631 (hydrasat_cachew) R 21626 21626 21558 0 -1 4194304 9230 0 0 0 12164 4 0 0 25 0 1 0 111312859 36319232 8322 1992294400 134512640 135995732 4294956016 18446744073709551615 134807701 0 0 4096 1 0 0 0 17 0 0 0
/proc/21631/statm: 8867 8323 171 362 0 8502 0
Current children cumulated CPU time (s) 821.86
Current children cumulated vsize (KiB) 40832

[startup+882.301 s]
/proc/loadavg: 1.00 1.02 1.12 2/66 21631
/proc/meminfo: memFree=1587384/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5364 CPUtime=700.18
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 321 20525 0 0 0 0 69974 44 16 0 1 0 111242806 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1341 245 202 169 0 52 0
[pid=21631] ppid=21626 vsize=38908 CPUtime=181.66
/proc/21631/stat : 21631 (hydrasat_cachew) R 21626 21626 21558 0 -1 4194304 10012 0 0 0 18161 5 0 0 25 0 1 0 111312859 39841792 9104 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 0 0 0
/proc/21631/statm: 9727 9104 171 362 0 9362 0
Current children cumulated CPU time (s) 881.84
Current children cumulated vsize (KiB) 44272

[startup+942.301 s]
/proc/loadavg: 1.00 1.02 1.11 2/66 21631
/proc/meminfo: memFree=1584760/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5364 CPUtime=700.18
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 321 20525 0 0 0 0 69974 44 16 0 1 0 111242806 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1341 245 202 169 0 52 0
[pid=21631] ppid=21626 vsize=41600 CPUtime=241.63
/proc/21631/stat : 21631 (hydrasat_cachew) R 21626 21626 21558 0 -1 4194304 10696 0 0 0 24157 6 0 0 25 0 1 0 111312859 42598400 9788 1992294400 134512640 135995732 4294956016 18446744073709551615 134807701 0 0 4096 1 0 0 0 17 0 0 0
/proc/21631/statm: 10400 9788 171 362 0 10035 0
Current children cumulated CPU time (s) 941.81
Current children cumulated vsize (KiB) 46964

[startup+1002.3 s]
/proc/loadavg: 1.00 1.01 1.09 2/66 21631
/proc/meminfo: memFree=1583032/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5364 CPUtime=700.18
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 321 20525 0 0 0 0 69974 44 16 0 1 0 111242806 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1341 245 202 169 0 52 0
[pid=21631] ppid=21626 vsize=43332 CPUtime=301.6
/proc/21631/stat : 21631 (hydrasat_cachew) R 21626 21626 21558 0 -1 4194304 11131 0 0 0 30154 6 0 0 25 0 1 0 111312859 44371968 10223 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 0 0 0
/proc/21631/statm: 10833 10223 171 362 0 10468 0
Current children cumulated CPU time (s) 1001.78
Current children cumulated vsize (KiB) 48696

[startup+1062.3 s]
/proc/loadavg: 1.00 1.01 1.09 2/66 21631
/proc/meminfo: memFree=1580280/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5364 CPUtime=700.18
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 321 20525 0 0 0 0 69974 44 16 0 1 0 111242806 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1341 245 202 169 0 52 0
[pid=21631] ppid=21626 vsize=46144 CPUtime=361.57
/proc/21631/stat : 21631 (hydrasat_cachew) R 21626 21626 21558 0 -1 4194304 11836 0 0 0 36150 7 0 0 25 0 1 0 111312859 47251456 10928 1992294400 134512640 135995732 4294956016 18446744073709551615 134807654 0 0 4096 1 0 0 0 17 0 0 0
/proc/21631/statm: 11536 10928 171 362 0 11171 0
Current children cumulated CPU time (s) 1061.75
Current children cumulated vsize (KiB) 51508

[startup+1122.3 s]
/proc/loadavg: 1.00 1.00 1.08 2/66 21631
/proc/meminfo: memFree=1575352/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5364 CPUtime=700.18
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 321 20525 0 0 0 0 69974 44 16 0 1 0 111242806 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1341 245 202 169 0 52 0
[pid=21631] ppid=21626 vsize=51064 CPUtime=421.54
/proc/21631/stat : 21631 (hydrasat_cachew) R 21626 21626 21558 0 -1 4194304 13022 0 0 0 42146 8 0 0 25 0 1 0 111312859 52289536 12114 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 0 0 0
/proc/21631/statm: 12766 12114 171 362 0 12401 0
Current children cumulated CPU time (s) 1121.72
Current children cumulated vsize (KiB) 56428

[startup+1182.3 s]
/proc/loadavg: 1.00 1.00 1.07 2/66 21631
/proc/meminfo: memFree=1571448/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5364 CPUtime=700.18
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 321 20525 0 0 0 0 69974 44 16 0 1 0 111242806 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1341 245 202 169 0 52 0
[pid=21631] ppid=21626 vsize=54980 CPUtime=481.51
/proc/21631/stat : 21631 (hydrasat_cachew) R 21626 21626 21558 0 -1 4194304 14002 0 0 0 48143 8 0 0 25 0 1 0 111312859 56299520 13094 1992294400 134512640 135995732 4294956016 18446744073709551615 134843101 0 0 4096 1 0 0 0 17 0 0 0
/proc/21631/statm: 13745 13094 171 362 0 13380 0
Current children cumulated CPU time (s) 1181.69
Current children cumulated vsize (KiB) 60344



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1200.7 s]
/proc/loadavg: 1.00 1.00 1.07 2/66 21631
/proc/meminfo: memFree=1570040/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5364 CPUtime=700.18
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 321 20525 0 0 0 0 69974 44 16 0 1 0 111242806 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1341 245 202 169 0 52 0
[pid=21631] ppid=21626 vsize=56552 CPUtime=499.92
/proc/21631/stat : 21631 (hydrasat_cachew) R 21626 21626 21558 0 -1 4194304 14354 0 0 0 49983 9 0 0 25 0 1 0 111312859 57909248 13446 1992294400 134512640 135995732 4294956016 18446744073709551615 134806670 0 0 4096 1 0 0 0 17 0 0 0
/proc/21631/statm: 14138 13446 171 362 0 13773 0
Current children cumulated CPU time (s) 1200.1
Current children cumulated vsize (KiB) 61916

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1200.7 s]
/proc/loadavg: 1.00 1.00 1.07 2/66 21631
/proc/meminfo: memFree=1570040/2055920 swapFree=4192956/4192956
[pid=21626] ppid=21624 vsize=5364 CPUtime=700.18
/proc/21626/stat : 21626 (hydrasat-multi.) S 21624 21626 21558 0 -1 4194304 321 20525 0 0 0 0 69974 44 16 0 1 0 111242806 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 236437693252 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/21626/statm: 1341 245 202 169 0 52 0
[pid=21631] ppid=21626 vsize=56552 CPUtime=499.92
/proc/21631/stat : 21631 (hydrasat_cachew) R 21626 21626 21558 0 -1 4194304 14354 0 0 0 49983 9 0 0 25 0 1 0 111312859 57909248 13446 1992294400 134512640 135995732 4294956016 18446744073709551615 134806670 0 0 4096 1 0 0 0 17 0 0 0
/proc/21631/statm: 14138 13446 171 362 0 13773 0
Current children cumulated CPU time (s) 1200.1
Current children cumulated vsize (KiB) 61916

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 21626 and gives
#  childrusage.ru_utime.tv_sec=699
#  childrusage.ru_utime.tv_usec=746622
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=446932
# CPU time returned by wait4() is 700.194
# while last known CPU time is 1200.1
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1200.71
CPU time (s): 1200.1
CPU user time (s): 1199.57
CPU system time (s): 0.53
CPU usage (%): 99.949
Max. virtual memory (cumulated for all children) (KiB): 84500

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 699.747
system time used= 0.446932
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 20846
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6
involuntary context switches= 2241

runsolver used 0.938857 second user time and 3.43748 second system time

The end

Launcher Data 

Begin job on node62 at 2009-04-27 13:35:52
IDJOB=1737736
IDBENCH=71794
IDSOLVER=518
FILE ID=node62/1737736-1240832152
PBS_JOBID= 9187526
Free space on /tmp= 66380 MiB

SOLVER NAME= HydraSAT-Multi 2009-03-22-Multi
BENCH NAME= SAT09/CRAFTED/parity-games/instance_n8_i8_pp_ci_ce.cnf
COMMAND LINE= DIR/hydrasat-multi.sh DIR BENCHNAME MEMLIMIT RANDOMSEED
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-1737736-1240832152/watcher-1737736-1240832152 -o /tmp/evaluation-result-1737736-1240832152/solver-1737736-1240832152 -C 1200 -W 1800 -M 1800 --output-limit 1,15  HOME/hydrasat-multi.sh HOME HOME/instance-1737736-1240832152.cnf 1800 1653146632

TIME LIMIT= 1200 seconds
MEMORY LIMIT= 1800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 68c51e190d82ac762156471d1f7373da
RANDOM SEED=1653146632

node62.alineos.net Linux 2.6.9-22.EL.rootsmp #1 SMP Mon Oct 3 08:59:52 CEST 2005

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.220
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.220
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1698472 kB
Buffers:         53444 kB
Cached:         214000 kB
SwapCached:          0 kB
Active:         131184 kB
Inactive:       151668 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1698472 kB
SwapTotal:     4192956 kB
SwapFree:      4192956 kB
Dirty:            9488 kB
Writeback:           0 kB
Mapped:          25180 kB
Slab:            60428 kB
Committed_AS:   282112 kB
PageTables:       1456 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264960 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66304 MiB
End job on node62 at 2009-04-27 13:55:55