Trace number 298823

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds, and are wall clock time (not CPU time).

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
CMUSAT BASE 2007-02-08? (TO) 1200.09 1200.66

General information on the benchmark

Namerandom/OnTreshold/3SAT/v360/
unif-k3-r4.26-v360-c1533-S1370059435-05.UNSAT.shuffled.cnf
MD5SUM727fc359ee7c9f3d412b57a84acf2e25
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark15.4926
Satisfiable
(Un)Satisfiability was proved
Number of variables360
Number of clauses1533
Sum of the clauses size4599
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 31533
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data (download as text)

Verifier Data (download as text)

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: 

Watcher Data (download as text)

runsolver version 3.2.0 (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w ROOT/results/node71/watcher-298823-1171861236 -o ROOT/results/node71/solver-298823-1171861236 -C 1200 -M 1800 --output-limit 1,15 /tmp/evaluation/298823-1171861236/cmusat_base /tmp/evaluation/298823-1171861236/instance-298823-1171861236.cnf 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1200 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1230 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 1843200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 1894400 KiB
Solver output will be limited to a maximum of 15728640 bytes. The first 1048576 bytes and the last 14680064 bytes will be preserved
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.92 0.98 0.99 3/64 28085
/proc/meminfo: memFree=945936/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=2308 CPUtime=0
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 180 0 0 0 0 0 0 0 18 0 1 0 206081464 2363392 162 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134547998 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 577 162 93 241 0 333 0

[startup+0.106925 s]
/proc/loadavg: 0.92 0.98 0.99 3/64 28085
/proc/meminfo: memFree=945936/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=2572 CPUtime=0.1
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 260 0 0 0 10 0 0 0 18 0 1 0 206081464 2633728 242 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559804 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 643 242 109 241 0 399 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 2572

[startup+0.515013 s]
/proc/loadavg: 0.92 0.98 0.99 3/64 28085
/proc/meminfo: memFree=945936/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=3036 CPUtime=0.51
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 349 0 0 0 51 0 0 0 21 0 1 0 206081464 3108864 331 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559989 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 759 331 109 241 0 515 0
Current children cumulated CPU time (s) 0.51
Current children cumulated vsize (KiB) 3036

[startup+1.33519 s]
/proc/loadavg: 0.92 0.98 0.99 2/65 28086
/proc/meminfo: memFree=944712/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=4068 CPUtime=1.33
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 571 0 0 0 133 0 0 0 25 0 1 0 206081464 4165632 553 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134549387 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 1017 553 109 241 0 773 0
Current children cumulated CPU time (s) 1.33
Current children cumulated vsize (KiB) 4068

[startup+2.97155 s]
/proc/loadavg: 0.92 0.98 0.99 2/65 28086
/proc/meminfo: memFree=943880/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=4204 CPUtime=2.96
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 645 0 0 0 296 0 0 0 25 0 1 0 206081464 4304896 627 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134560049 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 1051 627 109 241 0 807 0
Current children cumulated CPU time (s) 2.96
Current children cumulated vsize (KiB) 4204

[startup+6.25026 s]
/proc/loadavg: 0.93 0.98 0.99 2/65 28086
/proc/meminfo: memFree=941960/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=6196 CPUtime=6.24
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 1107 0 0 0 624 0 0 0 25 0 1 0 206081464 6344704 1053 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559878 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 1549 1053 109 241 0 1305 0
Current children cumulated CPU time (s) 6.24
Current children cumulated vsize (KiB) 6196

[startup+12.7037 s]
/proc/loadavg: 0.93 0.98 0.99 2/65 28086
/proc/meminfo: memFree=941208/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=6800 CPUtime=12.69
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 1454 0 0 0 1269 0 0 0 25 0 1 0 206081464 6963200 1286 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134549167 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 1700 1286 109 241 0 1456 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 6800

[startup+25.5074 s]
/proc/loadavg: 0.95 0.98 0.99 2/71 28112
/proc/meminfo: memFree=937320/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=9752 CPUtime=25.47
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 2434 0 0 0 2545 2 0 0 25 0 1 0 206081464 9986048 1891 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134560149 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 2438 1891 109 241 0 2194 0
Current children cumulated CPU time (s) 25.47
Current children cumulated vsize (KiB) 9752

[startup+51.128 s]
/proc/loadavg: 0.96 0.98 0.99 2/71 28188
/proc/meminfo: memFree=934120/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=12420 CPUtime=51.06
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 3495 0 0 0 5103 3 0 0 25 0 1 0 206081464 12718080 2654 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134560595 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 3105 2654 109 241 0 2861 0
Current children cumulated CPU time (s) 51.06
Current children cumulated vsize (KiB) 12420

[startup+102.364 s]
/proc/loadavg: 0.98 0.98 0.99 2/71 28254
/proc/meminfo: memFree=929056/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=18420 CPUtime=102.23
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 5562 0 0 0 10219 4 0 0 25 0 1 0 206081464 18862080 3907 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134560509 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 4605 3907 109 241 0 4361 0
Current children cumulated CPU time (s) 102.23
Current children cumulated vsize (KiB) 18420

[startup+162.392 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=927328/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=21020 CPUtime=162.21
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 6580 0 0 0 16216 5 0 0 25 0 1 0 206081464 21524480 4643 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559940 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 5255 4643 109 241 0 5011 0
Current children cumulated CPU time (s) 162.21
Current children cumulated vsize (KiB) 21020

[startup+222.41 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=916832/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=34232 CPUtime=222.21
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 10310 0 0 0 22214 7 0 0 25 0 1 0 206081464 35053568 7265 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559860 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 8558 7265 109 241 0 8314 0
Current children cumulated CPU time (s) 222.21
Current children cumulated vsize (KiB) 34232

[startup+282.426 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=916256/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=34440 CPUtime=282.21
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 10668 0 0 0 28213 8 0 0 25 0 1 0 206081464 35266560 7428 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134551769 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 8610 7428 109 241 0 8366 0
Current children cumulated CPU time (s) 282.21
Current children cumulated vsize (KiB) 34440

[startup+342.446 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=915744/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=34796 CPUtime=342.2
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 11003 0 0 0 34212 8 0 0 25 0 1 0 206081464 35631104 7549 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559860 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 8699 7549 109 241 0 8455 0
Current children cumulated CPU time (s) 342.2
Current children cumulated vsize (KiB) 34796

[startup+402.469 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=915104/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=35024 CPUtime=402.2
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 11405 0 0 0 40212 8 0 0 25 0 1 0 206081464 35864576 7716 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134560152 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 8756 7716 109 241 0 8512 0
Current children cumulated CPU time (s) 402.2
Current children cumulated vsize (KiB) 35024

[startup+462.486 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=914208/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=35520 CPUtime=462.19
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 11878 0 0 0 46210 9 0 0 25 0 1 0 206081464 36372480 7930 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134560110 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 8880 7930 109 241 0 8636 0
Current children cumulated CPU time (s) 462.19
Current children cumulated vsize (KiB) 35520

[startup+522.503 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=911904/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=37560 CPUtime=522.19
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 12475 0 0 0 52209 10 0 0 25 0 1 0 206081464 38461440 8527 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559940 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 9390 8527 109 241 0 9146 0
Current children cumulated CPU time (s) 522.19
Current children cumulated vsize (KiB) 37560

[startup+582.519 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=911456/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=37988 CPUtime=582.18
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 12845 0 0 0 58208 10 0 0 25 0 1 0 206081464 38899712 8613 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134560023 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 9497 8613 109 241 0 9253 0
Current children cumulated CPU time (s) 582.18
Current children cumulated vsize (KiB) 37988

[startup+642.546 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=908832/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=40236 CPUtime=642.18
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 13822 0 0 0 64207 11 0 0 25 0 1 0 206081464 41201664 9278 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559940 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 10059 9278 109 241 0 9815 0
Current children cumulated CPU time (s) 642.18
Current children cumulated vsize (KiB) 40236

[startup+702.564 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=908448/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=40644 CPUtime=702.18
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 13906 0 0 0 70207 11 0 0 25 0 1 0 206081464 41619456 9362 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559860 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 10161 9362 109 241 0 9917 0
Current children cumulated CPU time (s) 702.18
Current children cumulated vsize (KiB) 40644

[startup+762.581 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=907168/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=41964 CPUtime=762.17
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 14261 0 0 0 76206 11 0 0 25 0 1 0 206081464 42971136 9717 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134551171 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 10491 9717 109 241 0 10247 0
Current children cumulated CPU time (s) 762.17
Current children cumulated vsize (KiB) 41964

[startup+822.604 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=904288/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=44380 CPUtime=822.17
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 15301 0 0 0 82205 12 0 0 25 0 1 0 206081464 45445120 10414 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559940 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 11095 10414 109 241 0 10851 0
Current children cumulated CPU time (s) 822.17
Current children cumulated vsize (KiB) 44380

[startup+882.628 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=904288/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=44540 CPUtime=882.17
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 15321 0 0 0 88205 12 0 0 25 0 1 0 206081464 45608960 10434 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559860 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 11135 10434 109 241 0 10891 0
Current children cumulated CPU time (s) 882.17
Current children cumulated vsize (KiB) 44540

[startup+942.645 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=903840/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=44964 CPUtime=942.16
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 15442 0 0 0 94204 12 0 0 25 0 1 0 206081464 46043136 10555 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559881 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 11241 10555 109 241 0 10997 0
Current children cumulated CPU time (s) 942.16
Current children cumulated vsize (KiB) 44964

[startup+1002.66 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=888288/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=66156 CPUtime=1002.16
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 21251 0 0 0 100201 15 0 0 25 0 1 0 206081464 67743744 14419 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559932 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 16539 14419 109 241 0 16295 0
Current children cumulated CPU time (s) 1002.16
Current children cumulated vsize (KiB) 66156

[startup+1062.68 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=888032/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=66416 CPUtime=1062.16
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 21312 0 0 0 106201 15 0 0 25 0 1 0 206081464 68009984 14480 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559860 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 16604 14480 109 241 0 16360 0
Current children cumulated CPU time (s) 1062.16
Current children cumulated vsize (KiB) 66416

[startup+1122.7 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=888032/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=66416 CPUtime=1122.15
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 21320 0 0 0 112200 15 0 0 25 0 1 0 206081464 68009984 14488 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559860 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 16604 14488 109 241 0 16360 0
Current children cumulated CPU time (s) 1122.15
Current children cumulated vsize (KiB) 66416

[startup+1182.72 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=887328/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=66804 CPUtime=1182.15
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 21523 0 0 0 118200 15 0 0 25 0 1 0 206081464 68407296 14658 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559860 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 16701 14658 109 241 0 16457 0
Current children cumulated CPU time (s) 1182.15
Current children cumulated vsize (KiB) 66804



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1200.65 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=886880/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=66968 CPUtime=1200.06
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 22038 0 0 0 119991 15 0 0 25 0 1 0 206081464 68575232 14759 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559860 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 16742 14759 109 241 0 16498 0
Current children cumulated CPU time (s) 1200.06
Current children cumulated vsize (KiB) 66968

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1200.65 s]
/proc/loadavg: 0.99 0.98 0.99 2/65 28284
/proc/meminfo: memFree=886880/2055920 swapFree=4174780/4192956
[pid=28085] ppid=28083 vsize=66968 CPUtime=1200.06
/proc/28085/stat : 28085 (cmusat_base) R 28083 28085 26854 0 -1 4194304 22038 0 0 0 119991 15 0 0 25 0 1 0 206081464 68575232 14759 18446744073709551615 134512640 135500113 4294956704 18446744073709551615 134559860 0 0 4096 35 0 0 0 17 1 0 0
/proc/28085/statm: 16742 14759 109 241 0 16498 0
Current children cumulated CPU time (s) 1200.06
Current children cumulated vsize (KiB) 66968

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1200.66
CPU time (s): 1200.09
CPU user time (s): 1199.92
CPU system time (s): 0.167974
CPU usage (%): 99.9522
Max. virtual memory (cumulated for all children) (KiB): 66968

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1199.92
system time used= 0.167974
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 22038
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6
involuntary context switches= 2899

runsolver used 0.806877 second user time and 2.48362 second system time

The end

Launcher Data (download as text)

Begin job on node71 on Mon Feb 19 05:00:37 UTC 2007

IDJOB= 298823
IDBENCH= 19995
IDSOLVER= 120
FILE ID= node71/298823-1171861236

PBS_JOBID= 3839751

Free space on /tmp= 66561 MiB

SOLVER NAME= CMUSAT BASE 2007-02-08
BENCH NAME= HOME/pub/bench/SAT07/random/OnTreshold/3SAT/v360/unif-k3-r4.26-v360-c1533-S1370059435-05.UNSAT.shuffled.cnf
COMMAND LINE= /tmp/evaluation/298823-1171861236/cmusat_base /tmp/evaluation/298823-1171861236/instance-298823-1171861236.cnf            
RUNSOLVER COMMAND LINE= runsolver  --timestamp  -w ROOT/results/node71/watcher-298823-1171861236 -o ROOT/results/node71/solver-298823-1171861236 -C 1200 -M 1800 --output-limit 1,15  /tmp/evaluation/298823-1171861236/cmusat_base /tmp/evaluation/298823-1171861236/instance-298823-1171861236.cnf            

META MD5SUM SOLVER= af990fb12f8926fdd756b397adb3b5cf
MD5SUM BENCH=  727fc359ee7c9f3d412b57a84acf2e25

RANDOM SEED= 337195178

TIME LIMIT= 1200 seconds
MEMORY LIMIT= 1800 MiB

Linux node71.alineos.net 2.6.9-22.EL.rootsmp #1 SMP Mon Oct 3 08:59:52 CEST 2005 x86_64 x86_64 x86_64 GNU/Linux

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.266
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.266
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:        946408 kB
Buffers:         50920 kB
Cached:         918064 kB
SwapCached:      12392 kB
Active:         758028 kB
Inactive:       229468 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:        946408 kB
SwapTotal:     4192956 kB
SwapFree:      4174780 kB
Dirty:            2468 kB
Writeback:           0 kB
Mapped:          15808 kB
Slab:           108052 kB
Committed_AS:   668816 kB
PageTables:       1448 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66561 MiB

End job on node71 on Mon Feb 19 05:20:39 UTC 2007