Trace number 1723783

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
HydraSAT-Multi 2009-03-22-Multi? (TO) 1200 1200.51

General information on the benchmark

Namevelev-vliw-uns-4.0-9C1.cnf
MD5SUM0c9a15d0f561e9d1bb438b04ae3c31af
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark324.148
Satisfiable
(Un)Satisfiability was proved
Number of variables96177
Number of clauses1814189
Sum of the clauses size5280501
Maximum clause length2216
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 21178665
Number of clauses of size 3560157
Number of clauses of size 45491
Number of clauses of size 54739
Number of clauses of size over 565136

Solver Data 

0.00/0.00	c Trying configuration 1
701.15/701.55	HOME/hydrasat-multi.sh: line 48: 18956 CPU time limit exceeded (core dumped) $SOLVER1 $BENCHNAME $OPTIONS1
701.15/701.55	c Trying configuration 2

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-1723783-1240688905/watcher-1723783-1240688905 -o /tmp/evaluation-result-1723783-1240688905/solver-1723783-1240688905 -C 1200 -W 1800 -M 1800 --output-limit 1,15 HOME/hydrasat-multi.sh HOME HOME/instance-1723783-1240688905.cnf 1800 1977676429 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1200 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1230 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 1843200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 1894400 KiB
Solver output will be limited to a maximum of 15728640 bytes. The first 1048576 bytes and the last 14680064 bytes will be preserved
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.00 1.00 1.00 2/64 18955
/proc/meminfo: memFree=1207408/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=8540 CPUtime=0
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 160 0 0 0 0 0 0 0 23 0 1 0 96924462 8744960 142 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 2135 142 117 317 0 1815 0

[startup+0.04428 s]
/proc/loadavg: 1.00 1.00 1.00 2/64 18955
/proc/meminfo: memFree=1207408/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=10520 CPUtime=0.03
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 675 0 0 0 3 0 0 0 23 0 1 0 96924462 10772480 657 1992294400 134512640 135813592 4294956160 18446744073709551615 135267678 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 2630 658 117 317 0 2310 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 15880

[startup+0.101286 s]
/proc/loadavg: 1.00 1.00 1.00 2/64 18955
/proc/meminfo: memFree=1207408/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=13292 CPUtime=0.09
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 1423 0 0 0 9 0 0 0 23 0 1 0 96924462 13611008 1405 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 3323 1406 117 317 0 3003 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 18652

[startup+0.301308 s]
/proc/loadavg: 1.00 1.00 1.00 2/64 18955
/proc/meminfo: memFree=1207408/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=22924 CPUtime=0.29
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 3987 0 0 0 27 2 0 0 25 0 1 0 96924462 23474176 3969 1992294400 134512640 135813592 4294956160 18446744073709551615 135207783 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 5731 3969 117 317 0 5411 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 28284

[startup+0.701352 s]
/proc/loadavg: 1.00 1.00 1.00 2/64 18955
/proc/meminfo: memFree=1207408/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=41392 CPUtime=0.69
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 8929 0 0 0 65 4 0 0 25 0 1 0 96924462 42385408 8911 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 10348 8911 117 317 0 10028 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 46752

[startup+1.50144 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=1156128/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=77940 CPUtime=1.49
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 18704 0 0 0 139 10 0 0 25 0 1 0 96924462 79810560 18686 1992294400 134512640 135813592 4294956160 18446744073709551615 135207862 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 19485 18686 117 317 0 19165 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 83300

[startup+3.10162 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=1106080/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=158356 CPUtime=3.09
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 40818 0 0 0 288 21 0 0 25 0 1 0 96924462 162156544 37887 1992294400 134512640 135813592 4294956160 18446744073709551615 134676958 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 39589 37887 124 317 0 39269 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 163716

[startup+6.30198 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=1056096/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=158356 CPUtime=6.29
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 40818 0 0 0 608 21 0 0 25 0 1 0 96924462 162156544 37887 1992294400 134512640 135813592 4294956160 18446744073709551615 134676846 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 39589 37887 124 317 0 39269 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 163716

[startup+12.7017 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=1047136/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=174900 CPUtime=12.68
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 45015 0 0 0 1245 23 0 0 25 0 1 0 96924462 179097600 40312 1992294400 134512640 135813592 4294956160 18446744073709551615 134520740 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 43725 40312 126 317 0 43405 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 180260

[startup+25.5011 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=1045408/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=176112 CPUtime=25.48
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 45325 0 0 0 2524 24 0 0 25 0 1 0 96924462 180338688 40622 1992294400 134512640 135813592 4294956160 18446744073709551615 134521381 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 44028 40622 126 317 0 43708 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 181472

[startup+51.102 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=967456/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=258712 CPUtime=51.07
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 67324 0 0 0 5075 32 0 0 25 0 1 0 96924462 264921088 60112 1992294400 134512640 135813592 4294956160 18446744073709551615 134664943 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 64678 60112 149 317 0 64358 0
Current children cumulated CPU time (s) 51.07
Current children cumulated vsize (KiB) 264072

[startup+102.306 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=959520/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=267180 CPUtime=102.26
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 69312 0 0 0 10192 34 0 0 25 0 1 0 96924462 273592320 62100 1992294400 134512640 135813592 4294956160 18446744073709551615 135202737 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 66795 62100 149 317 0 66475 0
Current children cumulated CPU time (s) 102.26
Current children cumulated vsize (KiB) 272540

[startup+162.301 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=953120/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=273616 CPUtime=162.23
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 70898 0 0 0 16188 35 0 0 25 0 1 0 96924462 280182784 63686 1992294400 134512640 135813592 4294956160 18446744073709551615 135224171 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 68404 63686 149 317 0 68084 0
Current children cumulated CPU time (s) 162.23
Current children cumulated vsize (KiB) 278976

[startup+222.301 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=945056/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=282020 CPUtime=222.22
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 72908 0 0 0 22186 36 0 0 25 0 1 0 96924462 288788480 65696 1992294400 134512640 135813592 4294956160 18446744073709551615 134661569 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 70505 65696 149 317 0 70185 0
Current children cumulated CPU time (s) 222.22
Current children cumulated vsize (KiB) 287380

[startup+282.302 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=942560/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=284616 CPUtime=282.2
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 73519 0 0 0 28183 37 0 0 25 0 1 0 96924462 291446784 66307 1992294400 134512640 135813592 4294956160 18446744073709551615 134662668 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 71154 66307 149 317 0 70834 0
Current children cumulated CPU time (s) 282.2
Current children cumulated vsize (KiB) 289976

[startup+342.301 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=941600/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=285904 CPUtime=342.19
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 73748 0 0 0 34182 37 0 0 25 0 1 0 96924462 292765696 66536 1992294400 134512640 135813592 4294956160 18446744073709551615 134664957 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 71476 66536 149 317 0 71156 0

################
# More data... #
################

/proc/18956/statm: 84050 78433 149 317 0 83730 0
Current children cumulated CPU time (s) 582.11
Current children cumulated vsize (KiB) 341560

[startup+642.302 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18957
/proc/meminfo: memFree=883168/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5360 CPUtime=0
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 289 0 0 0 0 0 0 0 20 0 1 0 96924461 5488640 239 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 1 0 0
/proc/18955/statm: 1340 239 197 169 0 51 0
[pid=18956] ppid=18955 vsize=347448 CPUtime=642.1
/proc/18956/stat : 18956 (hydrasat_cache4) R 18955 18955 18548 0 -1 4194304 88325 0 0 0 64163 47 0 0 25 0 1 0 96924462 355786752 81113 1992294400 134512640 135813592 4294956160 18446744073709551615 134665320 0 0 4096 1 0 0 0 17 1 0 0
/proc/18956/statm: 86862 81113 149 317 0 86542 0
Current children cumulated CPU time (s) 642.1
Current children cumulated vsize (KiB) 352808

[startup+702.302 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18958
/proc/meminfo: memFree=862688/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5364 CPUtime=701.21
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 321 91857 0 0 0 0 69942 179 16 0 1 0 96924461 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18955/statm: 1341 245 202 169 0 52 0
[pid=18958] ppid=18955 vsize=38412 CPUtime=0.73
/proc/18958/stat : 18958 (hydrasat_cachew) R 18955 18955 18548 0 -1 4194304 8276 0 0 0 69 4 0 0 23 0 1 0 96994617 39333888 8259 1992294400 134512640 135995732 4294956016 18446744073709551615 135369318 0 0 4096 1 0 0 0 17 1 0 0
/proc/18958/statm: 9603 8259 142 362 0 9238 0
Current children cumulated CPU time (s) 701.94
Current children cumulated vsize (KiB) 43776

[startup+762.305 s]
/proc/loadavg: 1.07 1.03 1.01 2/66 18963
/proc/meminfo: memFree=729688/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5364 CPUtime=701.21
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 321 91857 0 0 0 0 69942 179 16 0 1 0 96924461 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18955/statm: 1341 245 202 169 0 52 0
[pid=18958] ppid=18955 vsize=163320 CPUtime=60.72
/proc/18958/stat : 18958 (hydrasat_cachew) R 18955 18955 18548 0 -1 4194304 44405 0 0 0 6050 22 0 0 25 0 1 0 96994617 167239680 37194 1992294400 134512640 135995732 4294956016 18446744073709551615 134807763 0 0 4096 1 0 0 0 17 1 0 0
/proc/18958/statm: 40830 37194 172 362 0 40465 0
Current children cumulated CPU time (s) 761.93
Current children cumulated vsize (KiB) 168684

[startup+822.302 s]
/proc/loadavg: 1.02 1.02 1.00 2/66 18963
/proc/meminfo: memFree=717976/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5364 CPUtime=701.21
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 321 91857 0 0 0 0 69942 179 16 0 1 0 96924461 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18955/statm: 1341 245 202 169 0 52 0
[pid=18958] ppid=18955 vsize=175504 CPUtime=120.7
/proc/18958/stat : 18958 (hydrasat_cachew) R 18955 18955 18548 0 -1 4194304 47342 0 0 0 12046 24 0 0 25 0 1 0 96994617 179716096 40131 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 1 0 0
/proc/18958/statm: 43876 40131 172 362 0 43511 0
Current children cumulated CPU time (s) 821.91
Current children cumulated vsize (KiB) 180868

[startup+882.302 s]
/proc/loadavg: 1.01 1.01 1.00 2/66 18963
/proc/meminfo: memFree=717912/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5364 CPUtime=701.21
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 321 91857 0 0 0 0 69942 179 16 0 1 0 96924461 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18955/statm: 1341 245 202 169 0 52 0
[pid=18958] ppid=18955 vsize=175504 CPUtime=180.68
/proc/18958/stat : 18958 (hydrasat_cachew) R 18955 18955 18548 0 -1 4194304 47342 0 0 0 18044 24 0 0 25 0 1 0 96994617 179716096 40131 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 1 0 0
/proc/18958/statm: 43876 40131 172 362 0 43511 0
Current children cumulated CPU time (s) 881.89
Current children cumulated vsize (KiB) 180868

[startup+942.301 s]
/proc/loadavg: 1.00 1.01 1.00 2/66 18963
/proc/meminfo: memFree=717976/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5364 CPUtime=701.21
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 321 91857 0 0 0 0 69942 179 16 0 1 0 96924461 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18955/statm: 1341 245 202 169 0 52 0
[pid=18958] ppid=18955 vsize=175504 CPUtime=240.66
/proc/18958/stat : 18958 (hydrasat_cachew) R 18955 18955 18548 0 -1 4194304 47342 0 0 0 24042 24 0 0 25 0 1 0 96994617 179716096 40131 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 1 0 0
/proc/18958/statm: 43876 40131 172 362 0 43511 0
Current children cumulated CPU time (s) 941.87
Current children cumulated vsize (KiB) 180868

[startup+1002.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18963
/proc/meminfo: memFree=717912/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5364 CPUtime=701.21
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 321 91857 0 0 0 0 69942 179 16 0 1 0 96924461 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18955/statm: 1341 245 202 169 0 52 0
[pid=18958] ppid=18955 vsize=175504 CPUtime=300.64
/proc/18958/stat : 18958 (hydrasat_cachew) R 18955 18955 18548 0 -1 4194304 47345 0 0 0 30040 24 0 0 25 0 1 0 96994617 179716096 40134 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 1 0 0
/proc/18958/statm: 43876 40134 172 362 0 43511 0
Current children cumulated CPU time (s) 1001.85
Current children cumulated vsize (KiB) 180868

[startup+1062.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18963
/proc/meminfo: memFree=717984/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5364 CPUtime=701.21
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 321 91857 0 0 0 0 69942 179 16 0 1 0 96924461 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18955/statm: 1341 245 202 169 0 52 0
[pid=18958] ppid=18955 vsize=175504 CPUtime=360.62
/proc/18958/stat : 18958 (hydrasat_cachew) R 18955 18955 18548 0 -1 4194304 47345 0 0 0 36038 24 0 0 25 0 1 0 96994617 179716096 40134 1992294400 134512640 135995732 4294956016 18446744073709551615 134589569 0 0 4096 1 0 0 0 17 1 0 0
/proc/18958/statm: 43876 40134 172 362 0 43511 0
Current children cumulated CPU time (s) 1061.83
Current children cumulated vsize (KiB) 180868

[startup+1122.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18963
/proc/meminfo: memFree=717856/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5364 CPUtime=701.21
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 321 91857 0 0 0 0 69942 179 16 0 1 0 96924461 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18955/statm: 1341 245 202 169 0 52 0
[pid=18958] ppid=18955 vsize=175504 CPUtime=420.61
/proc/18958/stat : 18958 (hydrasat_cachew) R 18955 18955 18548 0 -1 4194304 47363 0 0 0 42036 25 0 0 25 0 1 0 96994617 179716096 40152 1992294400 134512640 135995732 4294956016 18446744073709551615 135385625 0 0 4096 1 0 0 0 17 1 0 0
/proc/18958/statm: 43876 40152 172 362 0 43511 0
Current children cumulated CPU time (s) 1121.82
Current children cumulated vsize (KiB) 180868

[startup+1182.3 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18963
/proc/meminfo: memFree=716000/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5364 CPUtime=701.21
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 321 91857 0 0 0 0 69942 179 16 0 1 0 96924461 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18955/statm: 1341 245 202 169 0 52 0
[pid=18958] ppid=18955 vsize=177388 CPUtime=480.59
/proc/18958/stat : 18958 (hydrasat_cachew) R 18955 18955 18548 0 -1 4194304 47823 0 0 0 48034 25 0 0 25 0 1 0 96994617 181645312 40612 1992294400 134512640 135995732 4294956016 18446744073709551615 134681370 0 0 4096 1 0 0 0 17 1 0 0
/proc/18958/statm: 44347 40612 172 362 0 43982 0
Current children cumulated CPU time (s) 1181.8
Current children cumulated vsize (KiB) 182752



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1200.5 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18963
/proc/meminfo: memFree=714848/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5364 CPUtime=701.21
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 321 91857 0 0 0 0 69942 179 16 0 1 0 96924461 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18955/statm: 1341 245 202 169 0 52 0
[pid=18958] ppid=18955 vsize=178448 CPUtime=498.79
/proc/18958/stat : 18958 (hydrasat_cachew) R 18955 18955 18548 0 -1 4194304 48101 0 0 0 49853 26 0 0 25 0 1 0 96994617 182730752 40890 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 1 0 0
/proc/18958/statm: 44612 40890 172 362 0 44247 0
Current children cumulated CPU time (s) 1200
Current children cumulated vsize (KiB) 183812

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1200.5 s]
/proc/loadavg: 1.00 1.00 1.00 2/66 18963
/proc/meminfo: memFree=714848/2055920 swapFree=4192956/4192956
[pid=18955] ppid=18953 vsize=5364 CPUtime=701.21
/proc/18955/stat : 18955 (hydrasat-multi.) S 18953 18955 18548 0 -1 4194304 321 91857 0 0 0 0 69942 179 16 0 1 0 96924461 5492736 245 1992294400 4194304 4889804 548682068720 18446744073709551615 215418987332 0 65536 4100 65538 18446744071563356171 0 0 17 0 0 0
/proc/18955/statm: 1341 245 202 169 0 52 0
[pid=18958] ppid=18955 vsize=178448 CPUtime=498.79
/proc/18958/stat : 18958 (hydrasat_cachew) R 18955 18955 18548 0 -1 4194304 48101 0 0 0 49853 26 0 0 25 0 1 0 96994617 182730752 40890 1992294400 134512640 135995732 4294956016 18446744073709551615 134807726 0 0 4096 1 0 0 0 17 1 0 0
/proc/18958/statm: 44612 40890 172 362 0 44247 0
Current children cumulated CPU time (s) 1200
Current children cumulated vsize (KiB) 183812

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 18955 and gives
#  childrusage.ru_utime.tv_sec=699
#  childrusage.ru_utime.tv_usec=424671
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=795727
# CPU time returned by wait4() is 701.22
# while last known CPU time is 1200
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1200.51
CPU time (s): 1200
CPU user time (s): 1197.95
CPU system time (s): 2.05
CPU usage (%): 99.9572
Max. virtual memory (cumulated for all children) (KiB): 352808

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 699.425
system time used= 1.79573
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 92178
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6
involuntary context switches= 1653

runsolver used 1.34579 second user time and 3.00254 second system time

The end

Launcher Data 

Begin job on node9 at 2009-04-25 21:48:27
IDJOB=1723783
IDBENCH=71736
IDSOLVER=518
FILE ID=node9/1723783-1240688905
PBS_JOBID= 9186664
Free space on /tmp= 66296 MiB

SOLVER NAME= HydraSAT-Multi 2009-03-22-Multi
BENCH NAME= SAT_RACE06/velev-vliw-uns-4.0-9C1.cnf
COMMAND LINE= DIR/hydrasat-multi.sh DIR BENCHNAME MEMLIMIT RANDOMSEED
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-1723783-1240688905/watcher-1723783-1240688905 -o /tmp/evaluation-result-1723783-1240688905/solver-1723783-1240688905 -C 1200 -W 1800 -M 1800 --output-limit 1,15  HOME/hydrasat-multi.sh HOME HOME/instance-1723783-1240688905.cnf 1800 1977676429

TIME LIMIT= 1200 seconds
MEMORY LIMIT= 1800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 0c9a15d0f561e9d1bb438b04ae3c31af
RANDOM SEED=1977676429

node9.alineos.net Linux 2.6.9-22.EL.rootsmp #1 SMP Mon Oct 3 08:59:52 CEST 2005

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.240
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.240
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1207888 kB
Buffers:         77592 kB
Cached:         674964 kB
SwapCached:          0 kB
Active:         154920 kB
Inactive:       611588 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1207888 kB
SwapTotal:     4192956 kB
SwapFree:      4192956 kB
Dirty:           39856 kB
Writeback:           0 kB
Mapped:          23696 kB
Slab:            67536 kB
Committed_AS:   330124 kB
PageTables:       1388 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 65976 MiB
End job on node9 at 2009-04-25 22:08:30